diff --git a/README.md b/README.md index 2b3fcb7..99aa224 100644 --- a/README.md +++ b/README.md @@ -175,6 +175,73 @@ console.log(summary); // will be in Chinese If the `outputLanguage` is not supplied, the default behavior is to produce the output in "the same language as the input". For the multilingual input case, what this means is left implementation-defined for now, and implementations should err on the side of rejecting with a `"NotSupportedError"` `DOMException`. For this reason, it's strongly recommended that developers supply `outputLanguage`. +### Too-large inputs + +It's possible that the inputs given for summarizing and rewriting might be too large for the underlying machine learning model to handle. The same can even be the case for strings that are usually smaller, such as the writing task for the writer API, or the context given to all APIs. + +Whenever any API call fails due to too-large input, it is rejected with a `TooManyTokensError`. This is a new type of exception, which subclasses `DOMException`, and has the following additional properties: + +* `tokenCount`: how many tokens the input consists of +* `tokensAvailable`: how many tokens were available (which will be less than `tokenCount`) + +("[Tokens](https://arxiv.org/abs/2404.08335)" are the way that current language models process their input, and the exact mapping of strings to tokens is implementation-defined. However, we believe this API is relatively future-proof, since even if the technology moves away from current tokenization strategies, at the limit we can reinterpret tokens to mean code units, i.e., normal JavaScript string length.) + +This allows detecting failures due to overlarge inputs and giving clear feedback to the user, with code such as the following: + +```js +const summarizer = await ai.summarizer.create(); + +try { + console.log(await summarizer.summarize(potentiallyLargeInput)); +} catch (e) { + if (e.name === "TooManyTokensError") { + console.error(`Input too large! You tried to summarize ${e.tokenCount} tokens, but only ${e.tokensAvailable} were available.`); + + // Or maybe: + console.error(`Input too large! It's ${e.tokenCount / e.tokensAvailable}x as large as the maximum possible input size.`); + } +} +``` + +Note that all of the following methods can reject (or error the relevant stream) with this exception: + +* `ai.summarizer.create()`, if `sharedContext` is too large; + +* `ai.summarizer.summarize()`/`summarizeStreaming()`, if the combination of the creation-time `sharedContext`, the current method call's `input` argument, and the current method call's `context` is too large; + +* Similarly for writer creation / writing, and rewriter creation / rewriting. + +In some cases, instead of providing errors after the fact, the developer needs to be able to communicate to the user how close they are to the limit. For this, they can use the `tokensAvailable` property and `countTokens()` methods on the summarizer/writer/rewriter objects: + +```js +const rewriter = await ai.rewriter.create(); +meterEl.max = rewriter.tokensAvailable; + +textbox.addEventListener("input", () => { + meterEl.value = await rewriter.countTokens(textbox.value); + submitButton.disabled = meterEl.value > meterEl.max; +}); + +submitButton.addEventListener("click", () => { + console.log(rewriter.rewrite(textbox.value)); +}); +``` + +Developers need to be cautious not to over-use this API, however, as it requires a round-trip to the language model. That is, the following code is bad, as it performs two round trips with the same input: + +```js +// DO NOT DO THIS + +const tokenCount = await rewriter.countTokens(input); +if (tokenCount < rewriter.tokensAvailable) { + console.log(await rewriter.rewrite(input)); +} else { + console.error(`Input too large!`); +} +``` + +If you're planning to call `rewrite()` anyway, then using a pattern like the one that opened this section, which catches `TooManyTokensError`s, is more efficient than using `countTokens()` plus a conditional call to `rewrite()`. + ### Testing available options before creation All APIs are customizable during their `create()` calls, with various options. In addition to the language options above, the others are given in more detail in [the spec](https://webmachinelearning.github.io/writing-assistance-apis/). diff --git a/index.bs b/index.bs index 95ffbd4..e5c80a1 100644 --- a/index.bs +++ b/index.bs @@ -59,6 +59,19 @@ interface AICreateMonitor : EventTarget { attribute EventHandler ondownloadprogress; }; +[Exposed=(Window,Worker), SecureContext, Serializable] +interface TooManyTokensError : DOMException { + constructor(optional DOMString message = "", TooManyTokensErrorOptions options); + + readonly attribute unsigned long long tokenCount; + readonly attribute unsigned long long tokensAvailable; +}; + +dictionary TooManyTokensErrorOptions { + required [EnforceRange] unsigned long long tokenCount; + required [EnforceRange] unsigned long long tokensAvailable; +}; + callback AICreateMonitorCallback = undefined (AICreateMonitor monitor); enum AICapabilityAvailability { "readily", "after-download", "no" }; @@ -101,6 +114,48 @@ The following are the [=event handlers=] (and their corresponding [=event handle
This could be the same value as given by [=counting the tokens=] in |options|["{{AISummarizerCreateOptions/sharedContext}}"], or it could be larger. For example, if other options are encoded using prompt engineering, then the prompt would be included when computing |tokenCount|.
+ + 1. Let |tokensAvailable| be the maximum number of tokens that the user agent supports. + + 1. [=Assert=]: |tokensAvailable| is less than |tokenCount|. (That is how we reached this error branch.) + + 1. [=Queue a global task=] on the [=AI task source=] given |promise|'s [=relevant global object=] to [=reject=] |promise| with a {{TooManyTokensError}} whose [=TooManyTokensError/token count=] is |tokenCount| and [=TooManyTokensError/tokens available=] is |tokensAvailable|. + + 1. If initialization failed for any other reason, then: 1. [=Queue a global task=] on the [=AI task source=] given |promise|'s [=relevant global object=] to [=reject=] |promise| with an "{{OperationError}}" {{DOMException}}. @@ -328,6 +401,10 @@ The summarizer getter steps are to return [=this=]This check is necessary in case any code running on the [=agent/event loop=] caused the {{AbortSignal}} to become [=AbortSignal/aborted=] before this [=task=] ran. + 1. Let |tokensAvailable| be the total number of tokens available to the user agent for future [=summarize|summarization=] operations. + +
This will generally vary for each {{AISummarizer}} instance, depending on how many tokens were taken up by encoding |options|. See the earlier note on this encoding.
+ 1. Let |summarizer| be a new {{AISummarizer}} object, created in |promise|'s [=relevant realm=], with{{FrozenArray}}<{{DOMString}}>
or null, set during creation.
Every {{AISummarizer}} has an expected context languages, a {{FrozenArray}}<{{DOMString}}>
or null, set during creation.
@@ -624,6 +706,8 @@ The format getter steps are to return [=
The length getter steps are to return [=this=]'s [=AISummarizer/summary length=].
+The tokensAvailable getter steps are to return [=this=]'s [=AISummarizer/tokens available=].
+
The expectedInputLanguages getter steps are to return [=this=]'s [=AISummarizer/expected input languages=].
The expectedContextLanguages getter steps are to return [=this=]'s [=AISummarizer/expected context languages=].
@@ -637,10 +721,10 @@ The outputLanguage getter steps are to r
1. If [=this=]'s [=relevant global object=] is a {{Window}} whose [=associated Document=] is not [=Document/fully active=], then return [=a promise rejected with=] an "{{InvalidStateError}}" {{DOMException}}.
- 1. If [=this=]'s [=AISummarizer/destroyed=] is true, then return [=a promise rejected with=] [=this=]'s [=AISummarizer/destruction reason=].
-
1. If |options|["{{AISummarizerSummarizeOptions/signal}}"] [=map/exists=] and is [=AbortSignal/aborted=], then return [=a promise rejected with=] |options|["{{AISummarizerSummarizeOptions/signal}}"]'s [=AbortSignal/abort reason=].
+ 1. If [=this=]'s [=AISummarizer/destroyed=] is true, then return [=a promise rejected with=] [=this=]'s [=AISummarizer/destruction reason=].
+
1. Let |abortedDuringSummarization| be false.
This variable will be written to from the [=event loop=], but read from [=in parallel=]. @@ -703,9 +787,7 @@ The outputLanguage getter steps are to r 1. Abort these steps. - 1. Let |exception| be the result of [=exception/creating=] a {{DOMException}} with name given by |errorInfo|'s [=summarization error information/error name=], using |errorInfo|'s [=summarization error information/error information=] to populate the message appropriately. - - 1. [=Reject=] |promise| with |exception|. + 1. [=Reject=] |promise| with the result of [=converting error information into an exception object=] given [=this=] and |errorInfo|. 1. Let |stopProducing| be the following steps: @@ -721,10 +803,10 @@ The outputLanguage getter steps are to r 1. If [=this=]'s [=relevant global object=] is a {{Window}} whose [=associated Document=] is not [=Document/fully active=], then throw an "{{InvalidStateError}}" {{DOMException}}. - 1. If [=this=]'s [=AISummarizer/destroyed=] is true, then throw [=this=]'s [=AISummarizer/destruction reason=]. - 1. If |options|["{{AISummarizerSummarizeOptions/signal}}"] [=map/exists=] and is [=AbortSignal/aborted=], then throw |options|["{{AISummarizerSummarizeOptions/signal}}"]'s [=AbortSignal/abort reason=]. + 1. If [=this=]'s [=AISummarizer/destroyed=] is true, then throw [=this=]'s [=AISummarizer/destruction reason=]. + 1. Let |abortedDuringSummarization| be false.
This variable tracks web developer aborts via the |options|["{{AISummarizerSummarizeOptions/signal}}"] {{AbortSignal}}, which are surfaced as errors. It will be written to from the [=event loop=], but sometimes read from [=in parallel=]. @@ -799,9 +881,7 @@ The outputLanguage getter steps are to r 1. Abort these steps. - 1. Let |exception| be the result of [=exception/creating=] a {{DOMException}} with name given by |errorInfo|'s [=summarization error information/error name=], using |errorInfo|'s [=summarization error information/error information=] to populate the message appropriately. - - 1. [=ReadableStream/Error=] |stream| with |exception|. + 1. [=ReadableStream/Error=] |stream| with the result of [=converting error information into an exception object=] given [=this=] and |errorInfo|. 1. Let |stopProducing| be the following steps: @@ -814,12 +894,72 @@ The outputLanguage getter steps are to r 1. Return |stream|. +
See this note for more detail on what |inputToModel| might consist of.
+ + 1. Let |tokenCount| be the result of [=counting the tokens=] given |inputToModel|. + + 1. [=Queue a global task=] on the [=AI task source=] given [=this=]'s [=relevant global object=] to perform the following steps: + + 1. If |options|["{{AISummarizerSummarizeOptions/signal}}"] [=map/exists=] and is [=AbortSignal/aborted=], then: + + 1. [=Reject=] |promise| with |options|["{{AISummarizerSummarizeOptions/signal}}"]'s [=AbortSignal/abort reason=]. + + 1. Abort these steps. + + 1. If [=this=]'s [=AISummarizer/destroyed=] is true, then: + + 1. [=Reject=] |promise| with [=this=]'s [=AISummarizer/destruction reason=]. + + 1. Abort these steps. + + 1. If |tokenCount| is null, then: + + 1. [=Reject=] |promise| with an "{{UnknownError}}" {{DOMException}}. + + 1. Abort these steps. + + 1. [=Resolve=] |promise| with |tokenCount|. + + 1. Return |promise|. +This might be something similar to the concatenation of |input| and |context|, if all of the previous options were loaded into the model during initialization, and so the tokens used to express them are already accounted for via the {{AISummarizer/tokensAvailable}} property's current value. Or it might require more tokens, if the options are sent along with every summarization call, or if there is a per-summarization wrapper prompt of some sort.
+ + 1. Let |tokenCount| be the result of [=counting the tokens=] given |inputToModel|. + + 1. If |tokenCount| is greater than the number of tokens the language model can accept, then: + + 1. Let |errorInfo| be a [=too many tokens error information=] with a [=too many tokens error information/token count=] of |tokenCount|. + + 1. Perform |error| given |errorInfo|. + + 1. Return. + + 1. In an [=implementation-defined=] manner, subject to the following guidelines, begin the processs of summarizing |input| into a string. If they are non-null, |sharedContext| and |context| should be used to aid in the summarization by providing context on how the web developer wishes the input to be summarized. @@ -856,6 +996,20 @@ The outputLanguage getter steps are to r 1. [=iteration/Break=]."{{QuotaExceededError}}" - |
- The input to be summarized was too large for the user agent to handle. |
"{{UnknownError}}" |
All other scenarios, or if the user agent would prefer not to disclose the failure reason. |
This table does not give the complete list of exceptions that can be surfaced by {{AISummarizer/summarize()|summarizer.summarize()}} and {{AISummarizer/summarize()|summarizer.summarizeStreaming()}}. It only contains those which can come from the [=implementation-defined=] [=summarize=] algorithm. +
This table does not give the complete list of exceptions that can be surfaced by {{AISummarizer/summarize()|summarizer.summarize()}} and {{AISummarizer/summarize()|summarizer.summarizeStreaming()}}. It only contains those which can come from the key step of the [=summarize=] algorithm, where [=implementation-defined=] errors can occur.