Carbon coverage: User testing rd 2

[RichKummer]

Plan and test a prototype of the Carbon coverage Figma plugin with volunteer designers across IBM.

Volunteers:

• Laura Marshall
• Jacki Walkin
• Meme Betadam
• Shaun Lynch
• Tyler Watson
• Luke Firth *

Tasks

Preview Give feedback

1. Schedule discussions
2. Plan user testing sessions
3. Testing with prototype
4. Package functional prototype and send to slack channel/volunteers
5. Send local Figma plugin quickstart guide
6. Refine design specs based on feedback
7. Schedule installation meeting (30min)

[RichKummer]

Session 02: Laura Marshall

Recording

Key takeaways

• Question of what is the plugin testing against. Might consider making it clear which libraries the plugin checks against.
• Worried that designers would see notification that a component is at issue, even if it's from the C4IP libraries. Over time, once that's added, it would be better. If this can't be fixed in the short term, probably should not use a percentage/bar. The most important piece is the issue list at the bottom.
• Designers with less Carbon knowledge might find more use out of this. Unsure if someone's rushing to hand-off that they would use the plugin.
• Laura was concerned that designers new to Carbon would not really know how to resolve an issue without suggestions.
• Wondering if there's enough value for designers new to Carbon – if there's no suggestion, how will the update the design to properly follow Carbon guidance.
• Worried that phased approach could mean that designers using the plugin early may not get enough value, and would write off any updates that come later with fixes (and avoid using the updated version).
• If people are looking for something to make their workload faster, and it does not, then they won't use the tool. People get used to certain techniques to move at speed.
• Favors a clear action/button to target the issue node in the canvas. Was not sure that the contained list was enough to indicate that there was a way to select the node. The version with a button is more similar to how it's done in Figma. "A lot clearer with the icon..."
• Wondered if there was a way to click on an issue to learn more about what's wrong.
• Echoed Sam and Cameron feedback that there could be cases with very similar issues appearing and wanting a solution that handles all at once. Ex.) the multiple tabs with disconnected type tokens - is there a way to see one issue instead of many and then resolve all at once.
• Definitely prefer the plugin updating the list of issues dynamically as the designer fixes issues. If that's not possible, could we indicate that the issue (at least) has been clicked on. Think notification in Slack. This way the user knows that they have viewed this one if the issues are similar.
• Liked tabbed approach – "Like I could imagine probably if it was me, I'd want to go and fix all the colour things in one go and then move on to the category." Would like to see number of issues in tab label.
• Would it be possible to collect the data on what areas designers struggling?
• Worries that if the MVE does not solve issues, then it may not be used widely: "But I just know that if something doesn't work very well first time and it may discourage people from using it and I would hate that to be the case."

I like that as well as potentially a learning moment, because ideally everyone is perfect and their files are perfect first time and dev don't have any issues and I do feel like this kind of builds in very softly into that learning moment of OK yeah, I can see that I'm making the same error over and over again, or like someone that I'm mentoring someone and this is their file and I clearly I'll just tell them like they maybe they've missed something because I can look and they can see every single piece of text isn't using a type token and every single color using a hex code instead of a token.

Considerations

• Review Styles and variables organizer Figma plugin – "I liked that it offered suggestions, allowed mass changes, as well as the ability to select individual things, as well as grouping things with the same issue. It is a LOT slower than I remember it being now though..."
• Consider a plugin initial screen or tooltip to clarify which libraries the plugin is testing against. More important for first-time users.
• Definite desire to bundle similar issues together and have a single click to resolve them quickly. Looking for suggested ways to resolve the issue.
• Look into collecting high level data on what types of issues designers are facing. Is it mostly color? Type?

[RichKummer]

Session 03: Jacki Wakin, Meme Betadam, Shaun Lynch, Tyler Watson

Recording

Key takeaways

• Were interested in tagging certain components or libraries as "good" so that they would not show up in the issues. For example, tearsheet would show up for now since it lives in the C4IP library. Wish they could just flag it once so it does not appear in issues in this file.
• Some preference for the three-up display for tokens (either number or percentage). Said that color and type tokens were much more important than number of components. Also liked the percentage since that was a single element to look at rather than multiple.
• Brought up an issue with nesting – what if I slot a non-v11 component into something like a data table? Would that show up?
• UX Designers are often building designs in mid-fi, this seems more useful for visual designers.
• Interested in the ability to batch hide/ignore certain issues. Would be frustrated without it to have to hide each item.
• They are often designing in files that might be 50% Carbon v11 components, but 50% others. This is also where tokens are more important than components.
• Prefer the contained list version without a target action button. Felt users would understand after they used the tool. Were concerned that the space for the issue would get crowded out and harder to understand.
• Would like to know what is a detached component specifically.
• Would also like spacing guidance – for example, highlighting a spacing between paragraphs that's 16px in multiple areas, but then it's 24px in another. Could be useful for consistency.
• Felt there might be value in prompting the designer to use spacing variables instead of flat values. However, only one designer had started to use the spacing variables of the four.
• Developers will sometimes/often ignore spacing in a spec, and speccing/redlining takes time. Might be interested in Figma's dev tools for this. Wondering if devs would use the plugin.
• Wondering if the plugin will cause memory or performance issues in large files, such as complex prototypes of dashboards.

Considerations

• Designers are asking for a way to select multiple of the same types of issues to resolve all at once. We should consider an update to the UI to help enable this, beyond individual item fixes. Design Lint has the functionality to group similar issues.
  
  
• Designers are looking for guidance in the plugin – especially around spacing.
• Designers may also look for a way to batch hide multiple non-relevant issues, instead of hiding one-at-a-time.