Flutter's TextInputClient and related APIs are a growing problem and pain point

[matthew-carroll]

Recent Context

Super Editor is currently working on IME support for desktop. This work comes after the Flutter team extended their delta support to all desktop platforms, thus making it possible for Super Editor to do the same.

We've recently run into issues with special character entry on desktop. The first issue was resolved here, and the second issue is supposedly resolved here. We're continuing to validate Flutter delta capabilities.

Super Editor broke on master

Super Editor tries to stay on the stable branch because that's where our customers operate. However, to verify the desktop IME fixes, I had to jump to the master branch.

As soon as I switched to master, all of Super Editor's TextInputClient implementations broke because new methods were added to the base interface:

• showToolbar()
• insertTextPlaceholder()
• removeTextPlaceholder()

This change, which as far as I know was not socialized anywhere, has now caused a business problem for us, and I believe it continues to exacerbate and enlarge an existing problem with TextInputClient and related APIs.

The Business Problem

Super Editor, and all of its users now have to make a choice. Do we add support for Desktop IME AND force everyone to move to master, or do we allow people to stay on stable but suffer without Desktop IME until Flutter releases the next stable version?

We're stuck with this choice because Flutter altered an interface, which resulted in compile-time breakages across every existing implementation (as is always expected when altering an interface). We can't adopt the new desktop IME fixes without moving to master, but if we move to master then we must adjust our code to work with the master-only version of TextInputClient.

If Super Editor stays on stable, then Superlist has to wait for this important feature. If Super Editor moves to master then Super Editor can't cut a new release until Flutter releases a new stable version, which prevents us from fulfilling our recent promise to release new versions more often.

The Communication Problem

Unannounced Interface breakage of that level reduces the trust in the framework. I think that I myself, as well as Superlist tried to make the significance of text editing clear to many members of the Flutter team (see also below Treating Flutter apps as dumb clients). I am unsure how much else we can signal that those changes should at least be discussed beforehand with a stakeholder like Superlist, that tries to significantly increase the capabilities of a Framework that you publish.

After hitting this breakage, a member of the Superlist team discovered a breaking change notice:
https://docs.flutter.dev/release/breaking-changes/scribble-text-input-client

The members of the Flutter Text Working Group may want to reflect on the utility, or lack of utility of such a notice. This notice may provide some context, after Flutter breaks a project, but the message appears after the damage has been done. In my opinion, this change should not have been made, and I go into some of those details below.

The Technical Problem

One of the technically frustrating things about this API change is that it continues down this monolithic road for text input. Based on many experiences over the past year, Super Editor has expressed and explained concerns and discontent with this monolith approach. Nonetheless, we continue to see random APIs thrown into core interfaces, creating an ever-expanding Frankenstein's monster for text input.

This approach doesn't adhere to Flutter's stated principles, it doesn't make sense across diverse platforms, and it makes it difficult or impossible for Flutter developers to make effective software engineering decisions (read: establish encapsulation boundaries). TextInputClient is not the only monolithic API in Flutter related to text, but it's quickly becoming yet another such interface.

I strongly encourage the Flutter team to remove these new methods before they roll to stable. They do not belong in TextInputClient and the damage caused by forever leaving them in will far exceed the momentary damage from moving them to a more appropriate place.

Here is the current interface for TextInputClient:

abstract class TextInputClient {
  const TextInputClient();
  void connectionClosed();

  // Fundamental IME text input
  TextEditingValue? get currentTextEditingValue;
  void updateEditingValue(TextEditingValue value);
  void performAction(TextInputAction action);
  void showAutocorrectionPromptRect(int start, int end);

  // I believe this is Android only, but maybe it's fundamental for all platforms.
  AutofillScope? get currentAutofillScope;

  // Arbitrary control. Unclear which platforms actually use this, or why. Unclear what an implementer needs to do.
  void performPrivateCommand(String action, Map<String, dynamic> data);

  // iOS only
  void updateFloatingCursor(RawFloatingCursorPoint point);

  // New methods for some "scribble" system.

  /// Requests that the client show the editing toolbar, for example when the
  /// platform changes the selection through a non-flutter method such as
  /// scribble.
  void showToolbar() {}

  /// Requests that the client add a text placeholder to reserve visual space
  /// in the text.
  ///
  /// For example, this is called when responding to UIKit requesting
  /// a text placeholder be added at the current selection, such as when
  /// requesting additional writing space with iPadOS14 Scribble.
  void insertTextPlaceholder(Size size) {}

  /// Requests that the client remove the text placeholder.
  void removeTextPlaceholder() {}
}

// Here is the new "scribble client" - it seems to only apply to iOS
/// An interface to receive focus from the engine.
///
/// This is currently only used to handle UIIndirectScribbleInteraction.
abstract class ScribbleClient {
  /// A unique identifier for this element.
  String get elementIdentifier;

  /// Called by the engine when the [ScribbleClient] should receive focus.
  ///
  /// For example, this method is called during a UIIndirectScribbleInteraction.
  void onScribbleFocus(Offset offset);

  /// Tests whether the [ScribbleClient] overlaps the given rectangle bounds.
  bool isInScribbleRect(Rect rect);

  /// The current bounds of the [ScribbleClient].
  Rect get bounds;
}

I'll say again, this is not the only place where platform-specific APIs are tossed into an interface that everyone has to worry about. For example, the other side of this equation, called TextInputConnection, does the same thing in the other direction. This particular interface just happens to be the latest place where this is a problem.

Platform specifics

The most obvious problem with these new APIs is that they were introduced for a platform-specific purpose. The new methods in TextInputClient describe themselves in a generic way, but I don't see any reason to believe that they're generic. It seems that these methods were introduced specifically to facilitate the ScribbleClient, so why would we assume that those methods will ever be used for anything else? And without strong evidence to the contrary, why would Flutter allow those methods to be added to the fundamental text input API for all Flutter apps and packages? Clearly, Flutter made it to this point without those methods, so the evidence seems rather strong that they aren't generic.

Treating Flutter apps as dumb clients

Arguably, an even bigger problem is that Flutter, more and more, treats apps and packages as a dumb, passive text client. But this couldn't be further from the truth.

Super Editor is doing far more with text input than Flutter, itself. Flutter builds little text fields, and that's about it. We're building an entire document editing experience. Flutter's APIs tend to "delegate" or "dictate" specific actions to clients, rather than just notify clients about what happened.

The perfect example in this case is showToolbar().

Who said that there's a toolbar? And why, exactly, should we show it? Are we expected to mindlessly toggle something on/off in our app or package? We have dozens of competing interactions. We have multiple types of toolbars in different places. We have overlay handles, carets, and magnifiers. We have multi-user, concurrent selections. So, what if showing the toolbar isn't appropriate right now, regardless of what happened on the OS side? What we need to know is why the OS wants to show the toolbar. That's the information that allows our package or app to make the decision about whether or not to show a toolbar.

One might argue that the app can just ignore the call. But that's not accurate. We may not know whether we should ignore the call, and that's the point. The app needs to know what happened. The app needs the information with which it can make a decision, instead of Flutter making a decision on our behalf and then telling us what to do. Unless Flutter plans to implement universal document editing support, this approach is untenable and inappropriate.

By the way, where's the hideToolbar() method? Surely if the OS can show a toolbar, the OS can hide a toolbar, right?

Now a mixin?

Lastly, notice in the TextInputClient that the new methods are implemented as stubs. Meaning, the methods are given empty implementations.

Perhaps the implementer believed that stubs would mitigate compile-time breakages. But TextInputClient didn't have stubs defined anywhere else. It has always been a true interface. It's unlikely that anyone was using TextInputClient as a mixin, because there was no behavior to mix in. As a result, those stubs didn't solve anything, but they have now altered the implied use of TextInputClient to become a partial implementation, rather than a true interface. Was this a conscious decision? Is it the right decision? Is that path likely to lead to a more maintainable or less maintainable place? Personally, I would worry very much about Flutter framework developers installing any behavior at all in this critical, core interface that already operates as an unavoidable choke point for every app and package that takes text input.

Why isn't Super Editor submitting tests

The Flutter team might suggest that Super Editor hasn't done its part in this situation because we aren't submitting 3rd party tests to Flutter. We are on a path to do that. Up to this point our clients' needs for speedy delivery has made it difficult to match the relatively strict requirements for 3rd party tests. Additionally, in areas where we're pushing Flutters boundaries, like text editing deltas, Flutter itself doesn't provide corresponding testing tools, greatly increasing the test implementation burden on super_editor. However, we are actively decomposing super_editor into smaller projects that we hope to wrap with more stable tests, which we will then submit to Flutter.

Conclusion

Flutter's most fundamental text input API, the API that every single Flutter app must utilize for text input, has now increased its API surface by nearly 30% overnight, with methods that may not have any relevance for most platforms, and which fail to express intent. This is a continuation of what I believe to be a lack of rigorous thought and planning around critical text APIs in Flutter. Now, it seems that we're stuck with this change, forever, unless the Flutter team quickly reverses course before cutting the next stable release.

I strongly encourage the Flutter team to remove these new methods before they roll to stable. They do not belong in TextInputClient and the damage caused by forever leaving them in will far exceed the momentary damage from moving them to a more appropriate place.

This particular change broke Super Editor, and created a business problem for Superlist.

This issue could have been easily avoided with a brief conversation in the Flutter Text Working Group.