6-2.md

Responsiveness

CS 349 - User interfaces, LEC 001

6-10-2016

Elvin Yung

Slides

Elevator story

• A bunch of people were sick of the waiting times for the elevator in a building.
• Adding another elevator was unfeasible because it was too expensive.
• They installed mirrors inside and near the elevators - and the complaints were drastically decreased.
• Basically, it had to do with people's perception of the delay, not the delay itself.
• Now that there were mirrors, people would look at themselves and fix themselves using the mirror - it was no longer just pure waiting time.

Responsive Applications

You can make your software faster by doing these things:

• making the UI to fit human deadline requirements, and
• actually making it faster.

Responsiveness

• If we know how long an operation takes to be perceived, we can design our UI around that.

• What affects this?

• User expectations: Your expectations change how you think about the software.

• Example: You expect the ATM to be slow, to process your card, update you account, etc.

• Another example: We already expect desktop apps to be fast, but no one's surprised when it takes web apps longer to load.

• The system itself also matters. If something is slow, you still want to give an indication that something is being done.

• Responsiveness is the single most important factor in satisfying users with your UI.

How do we make a system responsive?

• If something is slow, we still want to provide feedback.
• Confirmation feedback: Let the user know that their input is being processed.
• Progress feedback: Show what's currently being done, and what the progress is.
• The operation should be asynchronous: The user should not be blocked if the operation takes an hour.

A system has poor responsiveness if:

• There's delayed feedback. People can't stand that!
• Ignoring user input: You never, ever, want to do that.
• The wait-cursor effect: You don't want to completely ignore user input when something is loading/waiting.
• Things like jerky animation is bad - the animation should be easy to follow.

Human deadlines

// TODO: add table

• Interesting: Humans can detect sound changes the fastest. We can detect a gap of silence that's 0.001s long!
• Saccade: When your eye moves, you actually can't see well for a short time.

0.14 second

• 0.14 second is the most important time. It's the basic perception threshold for our our eyes.
• After 0.14s without displaying feedback to a user action, the interface no longer feels responsive.
• If an operation takes longer than that, you need a progress indicator.

// TODO: add others from slides

Responsive UI tricks

• e.g. the hourglass on Windows, the beach ball on Macs
• Busy indicators are bad. They block the entire system!
• You would rather provide a progress indicator: let the user see an estimate of how much time (and probably how much work) is left to perform the task.

Progress Indicator best practices:

• Show work remaining
• Show total progress: Don't say that you're done 90% of step 3, say that you're done 2% of all the updates.
• Start at 1%, not 0%. People will think you've done nothing if it stays at 0% for extended periods of time.
• Don't show 100% for a long time - if it's done, it should be done.
• Show smooth, linear progress
• Use human scale precision - use human times, like 4 minutes instead of 240 seconds.
  • 4 minutes is easier conceptually.
  • Also: for most operations with progress dialogs, the application isn't actually really good at how long it's actually going to take. 4 minutes seems like a rougher estimate)
  • It's hard to estimate the time to copy a file!

More tricks

• Render or display more important information first.

• For example, PDFs can load and display pages sequentially, rather than all one burst.

• Load images progressively: Show a lower resolution image first, and then re-render at higher resolution. This is pretty common on the web.

• Fake heavyweight computations during hand-eye coordination tasks.

• Example: Scaling something in Photoshop: when the user is dragging the corner, Photoshop only scales out the border, and snaps to the new size when the user is done.

• Working ahead: Use downtime to do things that you know you'll do later.

• Example: A browser that prefetches pages that are linked from the current page.

• Weirder example: ML optimizations that guesses what you're probably going to do, and then precomputes it.

Responsiveness in a Java App

Handling long-running tasks

• People expect feedback from user interfaces in 10-100 ms, but this isn't always easy to do. How do you manage this?
• Our goal is to maintain the interactions that we have while the application is still doing some intensive work.
• While the task is running, users should still be able to work with other elements of the UI.
• They should also be able to pause or cancel the task.

Slow things

Usually, tasks like these are slow:

• Grabbing things from the internet
• Searching a large data structure
• Image/video processing
• Factoring big numbers
• etc.

Completely uninteractive ("hanging") applications are really bad!

TL;DR for the rest of this section

• When you have to do intensive work in a Swing app, you basically have two real choices on where to do it: on the Swing event dispatch thread, or on a separate background thread.
• Putting it on the event thread means that you don't have to deal with synchronization problems, but it could potentially block the Swing event loop unpredictably (making the app choppy), and interfere with the UI.
• Putting it on a separate thread means that you don't block the Swing event loop, but comes with all the standard drawbacks of using multiple threads (risk of race conditions, deadlocks, etc.)

Demos

• Consider a program that calculates all the prime numbers within some interval.
• The application is implemented using a basic MVC pattern.
• The UI looks like this:

• We'll change the model and view iteratively to progressively improve the program's responsiveness.

Demo1

• When the user clicks the button, we calculate all the primes at once.
• The calculate loop takes multiple seconds to run, and doesn't update any view until it's finished.
• It's completely unoptimized.
• Clearly, that's not great.

What's wrong?

• The Swing event loop is basically a single thread. It runs your code, performs event dispatch, runs your code again, and so on...
• You can think of it like this: There's a queue where each item represents a task to be processed, and a loop that keeps running tasks from the queue. It can only do one thing at a time.
• It doesn't preempt anything you're executing.
• This means that when you're doing a lot of processing in a single task, it blocks the thread and doesn't let anything happen.
• The golden rule of event loops: Don't block the event loop.

Threads

• We could move calculation into multiple threads. In Java, multiple threads can run concurrently (unlike CPython or MRI).
• This seems great because it lets multiple things happen at the same time - you can use more than one core.
• But it could also get messy because all the threads share the same addressing space, which means that you might have to deal with things like deadlocks and race conditions.
• Concurrency is hard!

In Java, there are three "types" of threads:

• Initial/main thread
• Event Dispatch Thread (or UI Thread, Swing thread)
• Worker/background threads

For our prime calculation program, we basically have two strategies:

• Strategy A: Split up our work into multiple tasks, so that it doesn't have to be run all at once in the event loop.
• Strategy B: Have worker threads that do the work, and communicate with the UI thread.

Demo2

• Demo2 implements strategy A.

• We split up calculation into multiple tasks (Runnable objects) that get added to the Swing event queue.

• We periodically add tasks to the event queue, until all the processing's done.

• Each task is responsible for adding the next task to the event queue, unless we're done or the user cancelled.

• This demo is jittery.

• The problem is that we're technically still breaking the golden rule. Each task still blocks the event loop when it runs.

• We can manually try to tune the size of each task, but it's still pretty clunky.

• Also, it's hard to break up certain tasks, like blocking I/O.

• (Some event loop implementations, e.g. Node.js or EventMachine, have evented/non-blocking equivalents of all the I/O operations, which lets the event loop handle other tasks while waiting for I/O. Swing doesn't do this, unfortunately.)

Demo3

• Strategy B - Delegate actual work to background worker threads.
• We still have a Swing thread that does all the UI updates, but we added a worker thread that does the actual calculations.
• Using more than one thread lets us scale to more than one core.
• As expected, Demo3 is much more responsive than any of the previous methods.

Demo4

• Why do we use SwingUtilities.invokeLater() to update the UI, instead of updating it directly?
• Demo4 has two worker threads: One that uses invokeLater, and one that updates it directly.
• As it turns out, the thread that uses invokeLater updates much more smoothly.
• The problem is that since Swing isn't thread-safe, calling the Swing methods directly could lead to lots of synchronization issues.
• On the other hand, using invokeLater means that the UI thread performs synchronization for you.
• Race conditions are bad.

More stuff

• SwingWorker was added to Java SE 6, and is now the standard way to create worker threads.

• For Android, there's AsyncTask.

• Java 8 added lambda functions. Lambdas are Runnable, which means that we can more concisely run asynchronous tasks, like this:

SwingUtilities.invokeLater(() -> {
  // ...
});

Responsiveness in a Web App

Loading data efficiently

• The real problem with webpages is that we need to manage web latency.

A Brief History of Web Architectures

Gen 1

• Thin clients.
• Your browser makes a request to the server, which returns a complete HTML page - everything is computed and rendered serverside.
• Whenever we make an interaction, the webserver sends back a completely new page for the browser to load.

Gen 2

• Fat clients.
• The server is only responsible for delivering the data. It exposes some API over HTTP, probably in some JSON or XML based format.
• The browser gets this data asynchronously, and renders it dynamically on the client side.
• This is called a single-page app, where clients are dynamic, and can do a lot on the frontend using JavaScript.

Gen 3

• Previously, our backend webserver was a monolith - everything was on one process.
• This is hard to scale, since different features do different things, and can usually be scaled up differently.
• We can break up our logic into multiple services.
• The webserver that talks to the frontend can transparently grab the data from all the services it needs to.