Carbon coverage: User testing rd 1

[RichKummer]

Plan and test a prototype of the Carbon coverage Figma plugin with volunteer designers across IBM.

Volunteers:

• Cameron Calder
• Samuel Ting
• Jacki Walkin
• Laura Marshall
• Meme Betadam
• Shaun Lynch
• Tyler Watson
• Luke Firth *

Tasks

Preview Give feedback

1. Schedule discussions
2. Plan user testing sessions
3. Testing with prototype
4. Set up and introduce slack channel

[RichKummer]

Session 01: Cameron Calder and Sam Ting

Recording
Full notes

Key takeaways

• Both interested in automating updates to the design file from the plugin, similar to Design Lint. This could be a second phase addition, but may cause feature creep.
• Both felt linking to docs would be fantastic. Could either be a general link to docs or component specific. We need to consider how much space this takes up and where.
• Cameron was concerned about putting too much content in the snapshot area, which could make the designer take more time scrolling to the individual issues below.
• For the percentage, both were skeptical. Sam noted that some designers may think the percentage readout could be sent to their manager. If not, he said that designers may feel disappointed or frustrated with a low score. For now, I think we should create an exploration that does not rely on a percentage.
• If we did include a percentage, Sam noted that he wasn't sure if the bar was decorative or not. We could add motion to the bar and displayed percentage to help bridge that gap. Sam also felt a donut chart might be easier to understand instead of the bar, and liked the Lighthouse example.
• Also for progress bar, they were a little confused by showing red until we showed the different colors. Felt that the percent of Carbon was good, not bad. Could cause confusion if we did go the percentage route.
• For the individual issues, were not sure that each issue was clickable at first (this has come up before.
• Were also interested in selecting more than one issue to hide. We could consider a checkbox approach similar to the Access checker.
• Should drop the "plugin" from the title since that is redundant.
• Cameron suggested a way to connect the plugin to the design file visually by adding a hover effect on the issue element/node when you hover on the issue in the plugin. This might help indicate that you can then jump to the issue.
• Plugin design should be updated to reflect the rounded borders (this is already reflected in the coded version).
• Sam indicated that designers detach components every day. Even he does as an expert if it means he can save time and create assets and artifacts. They detach to move quickly for client work.
• Both were interested in grouping issues more. Liked the tabs, but wondered if they could somehow group different issues. For example, Sam brought up an example of an accordion with edited labels on each item – could you select all labels and edit them at once?
• When asked about the value to their team, Sam was worried that a score might reflect poorly on the designer or be sent to their manager. They may not use it if they get demoralized by a score. Also worried if there are too many issues, it may turn them off from the tool. Cameron sees two main use cases for it:

One is like something like very specific to one of our teams and another one is like more just general for like product use the product use one would be like look this is the part of like the quality we checks we do before. Releasing for development to make sure that we are moving forward with as much quality in our execution as possible and making sure that we have a good understanding of like what it is that we're delivering and how it incorporates the system. Also see as an opportunity to up level people who are new to IBM junior designers to get more fluent with like how they should be like engaging with this system. Real feedback, real training.

The other one, and this is kind of like a unique edge case. Is some of the some of the work that our team produces?It's meant for the community. It's meant for scale for the masses, so we gotta make sure that like what we are putting out there is of like the utmost quality and representative of our system. And this would be a great tool to go through and just make sure that it's like we're completely checked and everything is is, you know, above board really great kind of like standards before we put it out there for community consumption.

So those are the those are the two ways that I would think about using it and recommending our teams using it.

Considerations

• Update design file with rounded edges
• Consider the hover interaction on issue item to hint at the connection to the canvas (this may be lower priority, or an update post-launch).
• Consider another selection indicator than contained list – some designers have said they weren't sure you could click to go to the issue node at a glance.
• Remove the percentage and instead find an more general or abstract way to display the snapshot.
• If we do go with a percentage, we should consider a donut chart instead. Should be trim so it doesn't take up too much vertical space (make title and donut chart horizontally aligned).