CFGFSM
+
+
+
+ Bases: FSM
FSM to generate text that is in the language of a context-free grammar.
+ +outlines/fsm/fsm.py230 +231 +232 +233 +234 +235 +236 +237 +238 +239 +240 +241 +242 +243 +244 +245 +246 +247 +248 +249 +250 +251 +252 +253 +254 +255 +256 +257 +258 +259 +260 +261 +262 +263 +264 +265 +266 +267 +268 +269 +270 +271 +272 +273 +274 +275 +276 +277 +278 +279 +280 +281 +282 +283 +284 +285 +286 +287 +288 +289 +290 +291 +292 +293 +294 +295 +296 +297 +298 +299 +300 +301 +302 +303 +304 +305 +306 +307 +308 +309 +310 +311 +312 +313 +314 +315 +316 +317 +318 +319 +320 +321 +322 +323 +324 +325 +326 +327 +328 +329 +330 +331 +332 +333 +334 +335 +336 +337 +338 +339 +340 +341 +342 +343 +344 +345 +346 +347 +348 +349 +350 +351 +352 +353 +354 +355 +356 +357 +358 +359 +360 +361 +362 +363 +364 +365 +366 +367 +368 +369 +370 +371 +372 +373 +374 +375 | |
allowed_token_ids(state)
+
+Generate a list of allowed tokens for the next step.
+Upon initialization, the CFG incremental parser is used to determine the +first regex and construct the first FSM to generate the first terminal.
+This FSM is used for proposals until either:
+The CFG incremental parser is allowed to propose the EOS token from any accepting state, +and once it is generated, the FSM will continue to always generate the EOS token.
+state + The current state of the FSM.
+A list that contains the tokens to mask.
+ +outlines/fsm/fsm.py261 +262 +263 +264 +265 +266 +267 +268 +269 +270 +271 +272 +273 +274 +275 +276 +277 +278 +279 +280 +281 +282 +283 +284 +285 +286 +287 +288 +289 +290 +291 +292 +293 +294 +295 +296 +297 +298 +299 +300 +301 +302 +303 +304 +305 +306 +307 +308 +309 +310 +311 +312 +313 +314 +315 +316 +317 +318 +319 +320 +321 +322 +323 +324 +325 +326 +327 +328 +329 +330 +331 +332 +333 +334 +335 +336 +337 | |
copy()
+
+next_state(state, token_id)
+
+Update the state of the FSM.
+Transitions the underlying regex FSM to its next state. +If at max tokens or EOS token, transition permanently to the final state. +Update stored partial generations for subsequent incremental parsing.
+state + The current state of the FSM. +token_id + The id of the token that was just generated.
+The new state of the FSM.
+ +outlines/fsm/fsm.pyFSM
+
+
+
+ Bases: Protocol
outlines/fsm/fsm.pyis_final_state(state)
+
+RegexFSM
+
+
+
+ Bases: FSM
FSM to generate text that is in the language of a regular expression.
+ +outlines/fsm/fsm.py91 + 92 + 93 + 94 + 95 + 96 + 97 + 98 + 99 +100 +101 +102 +103 +104 +105 +106 +107 +108 +109 +110 +111 +112 +113 +114 +115 +116 +117 +118 +119 +120 +121 +122 +123 +124 +125 +126 +127 +128 +129 +130 +131 +132 +133 +134 +135 +136 +137 +138 +139 +140 +141 +142 +143 +144 +145 +146 +147 +148 +149 +150 +151 +152 +153 +154 +155 +156 +157 +158 +159 +160 +161 +162 +163 +164 +165 +166 +167 +168 +169 +170 +171 +172 +173 +174 +175 +176 +177 +178 +179 +180 +181 +182 +183 +184 +185 +186 +187 +188 +189 +190 +191 +192 +193 +194 +195 +196 +197 +198 +199 +200 +201 +202 +203 +204 +205 +206 +207 +208 +209 +210 +211 +212 +213 +214 +215 +216 +217 +218 +219 +220 +221 +222 +223 +224 +225 +226 +227 | |
__init__(regex_string, tokenizer)
+
+outlines/fsm/fsm.pyallowed_token_ids(state)
+
+Generate a list of allowed tokens for the next step.
+The initialization of the FSM builds an index which maps FSM states to a +map from authorized tokens to the state in which the FSM needs to move +if said token is generated. Therefore the authorized tokens at the +current state are the keys of the map returned by the value of the index +for current state.
+If the current state is not contained in the end this means that we are +in a final state of the FSM. We only authorize EOS tokens in the final +state.
+state + The current state of the FSM.
+A list that contains the tokens to mask.
+ +outlines/fsm/fsm.pycopy()
+
+next_state(state, token_id)
+
+Update the state of the FSM.
+We use the index to determine to which state the FSM should transition +given the token that was just generated.
+state + The current state of the FSM. +token_id + The id of the token that was just generated.
+The new state of the FSM.
+ +outlines/fsm/fsm.pyStopAtEosFSM
+
+
+
+ Bases: FSM
FSM to generate text until EOS has been generated.
+ +outlines/fsm/fsm.pyallowed_token_ids(state)
+
+Generate a list of allowed tokens for the next step.
+When in the initial state we allow every token to be generated.
+In the final state the only allowed token is stop_token_id.
state + The current state of the FSM.
+A list that contains the tokens to mask.
+ +outlines/fsm/fsm.pycopy()
+
+next_state(state, token_id)
+
+Update the state of the FSM.
+The FSM stays in the initial state 0 unless the specified stop token
+has been generated or the maximum number of tokens has been reached. In
+which case the FSM moves to the final state -1.
state + The current state of the FSM. +token_id + The id of the token that was just generated.
+The new state of the FSM.
+ +outlines/fsm/fsm.pybuild_regex_from_schema(schema, whitespace_pattern=None)
+
+Turn a JSON schema into a regex that matches any JSON object that follows + this schema.
+JSON Schema is a declarative language that allows to annotate JSON documents + with types and descriptions. These schemas can be generated from any Python + datastructure that has type annotation: namedtuples, dataclasses, Pydantic + models. And by ensuring that the generation respects the schema we ensure + that the output can be parsed into these objects. + This function parses the provided schema and builds a generation schedule which + mixes deterministic generation (fixed strings), and sampling with constraints.
+Parameters
+schema
+ A string that represents a JSON Schema.
+ whitespace_pattern
+ Pattern to use for JSON syntactic whitespace (doesn't impact string literals)
+ Example: allow only a single space or newline with whitespace_pattern=r"[
+]?"
Returns
+A generation schedule. A list of strings that represent the JSON + schema's structure and regular expression that define the structure of + the fields.
+References
+.. [0] JSON Schema. https://json-schema.org/
+ +outlines/fsm/json_schema.pyget_schema_from_signature(fn)
+
+Turn a function signature into a JSON schema.
+Every JSON object valid to the output JSON Schema can be passed
+to fn using the ** unpacking syntax.
outlines/fsm/json_schema.pyto_regex(resolver, instance, whitespace_pattern=None)
+
+Translate a JSON Schema instance into a regex that validates the schema.
+Note
+Many features of JSON schema are missing:
+ - Handle additionalProperties keyword
+ - Handle types defined as a list
+ - Handle constraints on numbers
+ - Handle special patterns: date, uri, etc.
This does not support recursive definitions.
+Parameters
+resolver
+ An object that resolves references to other instances within a schema
+ instance
+ The instance to translate
+ whitespace_pattern
+ Pattern to use for JSON syntactic whitespace (doesn't impact string literals)
+ Example: allow only a single space or newline with whitespace_pattern=r"[
+]?"
outlines/fsm/json_schema.py98 + 99 +100 +101 +102 +103 +104 +105 +106 +107 +108 +109 +110 +111 +112 +113 +114 +115 +116 +117 +118 +119 +120 +121 +122 +123 +124 +125 +126 +127 +128 +129 +130 +131 +132 +133 +134 +135 +136 +137 +138 +139 +140 +141 +142 +143 +144 +145 +146 +147 +148 +149 +150 +151 +152 +153 +154 +155 +156 +157 +158 +159 +160 +161 +162 +163 +164 +165 +166 +167 +168 +169 +170 +171 +172 +173 +174 +175 +176 +177 +178 +179 +180 +181 +182 +183 +184 +185 +186 +187 +188 +189 +190 +191 +192 +193 +194 +195 +196 +197 +198 +199 +200 +201 +202 +203 +204 +205 +206 +207 +208 +209 +210 +211 +212 +213 +214 +215 +216 +217 +218 +219 +220 +221 +222 +223 +224 +225 +226 +227 +228 +229 +230 +231 +232 +233 +234 +235 +236 +237 +238 +239 +240 +241 +242 +243 +244 +245 +246 +247 +248 +249 +250 +251 +252 +253 +254 +255 +256 +257 +258 +259 +260 +261 +262 +263 +264 +265 +266 +267 +268 +269 +270 +271 +272 +273 +274 +275 +276 +277 +278 +279 +280 +281 +282 +283 +284 +285 +286 +287 +288 +289 +290 +291 +292 +293 +294 +295 +296 +297 +298 +299 +300 +301 +302 +303 +304 +305 +306 +307 +308 +309 +310 +311 +312 +313 +314 +315 +316 +317 +318 +319 +320 +321 +322 +323 +324 +325 +326 +327 +328 +329 +330 +331 +332 +333 +334 +335 +336 +337 +338 +339 +340 +341 +342 +343 +344 +345 +346 +347 +348 +349 +350 +351 +352 +353 +354 +355 +356 +357 +358 +359 +360 | |
TransformerTokenizer
+
+
+
+ Bases: Tokenizer
Represents a tokenizer for models in the transformers library.
outlines/models/transformers.pyTransformers
+
+
+Represents a transformers model.
outlines/models/transformers.py118 +119 +120 +121 +122 +123 +124 +125 +126 +127 +128 +129 +130 +131 +132 +133 +134 +135 +136 +137 +138 +139 +140 +141 +142 +143 +144 +145 +146 +147 +148 +149 +150 +151 +152 +153 +154 +155 +156 +157 +158 +159 +160 +161 +162 +163 +164 +165 +166 +167 +168 +169 +170 +171 +172 +173 +174 +175 +176 +177 +178 +179 | |
forward(input_ids, attention_mask, past_key_values=None)
+
+Compute a forward pass through the transformer model.
+input_ids + The input token ids. Must be one or two dimensional. +attention_mask + The attention mask. Must be one or two dimensional. +past_key_values + A tuple of tuples containing the cached key and value tensors for each + attention head.
+The computed logits and the new cached key and value tensors.
+ +outlines/models/transformers.pyget_llama_tokenizer_types()
+
+Get all the Llama tokenizer types/classes that need work-arounds.
+When they can't be imported, a dummy class is created.
+ +outlines/models/transformers.pytransformers(model_name, device=None, model_kwargs={}, tokenizer_kwargs={})
+
+Instantiate a model from the transformers library and its tokenizer.
model_name
+ The name of the model as listed on Hugging Face's model page.
+device
+ The device(s) on which the model should be loaded. This overrides
+ the device_map entry in model_kwargs when provided.
+model_kwargs
+ A dictionary that contains the keyword arguments to pass to the
+ from_pretrained method when loading the model.
+tokenizer_kwargs
+ A dictionary that contains the keyword arguments to pass to the
+ from_pretrained method when loading the tokenizer.
A TransformersModel model instance.
outlines/models/transformers.pyIntegration with OpenAI's API.
+ + + +OpenAI
+
+
+An object that represents the OpenAI API.
+ +outlines/models/openai.py69 + 70 + 71 + 72 + 73 + 74 + 75 + 76 + 77 + 78 + 79 + 80 + 81 + 82 + 83 + 84 + 85 + 86 + 87 + 88 + 89 + 90 + 91 + 92 + 93 + 94 + 95 + 96 + 97 + 98 + 99 +100 +101 +102 +103 +104 +105 +106 +107 +108 +109 +110 +111 +112 +113 +114 +115 +116 +117 +118 +119 +120 +121 +122 +123 +124 +125 +126 +127 +128 +129 +130 +131 +132 +133 +134 +135 +136 +137 +138 +139 +140 +141 +142 +143 +144 +145 +146 +147 +148 +149 +150 +151 +152 +153 +154 +155 +156 +157 +158 +159 +160 +161 +162 +163 +164 +165 +166 +167 +168 +169 +170 +171 +172 +173 +174 +175 +176 +177 +178 +179 +180 +181 +182 +183 +184 +185 +186 +187 +188 +189 +190 +191 +192 +193 +194 +195 +196 +197 +198 +199 +200 +201 +202 +203 +204 +205 +206 +207 +208 +209 +210 +211 +212 +213 +214 +215 +216 +217 +218 +219 +220 +221 +222 +223 +224 +225 +226 +227 +228 +229 +230 +231 +232 +233 +234 +235 +236 +237 +238 +239 +240 +241 +242 +243 +244 +245 +246 +247 +248 +249 +250 +251 +252 +253 +254 +255 +256 +257 +258 +259 +260 | |
__call__(prompt, max_tokens=None, stop_at=None, *, system_prompt=None, temperature=None, samples=None)
+
+Call the OpenAI API to generate text.
+prompt + A string or list of strings that will be used to prompt the model +max_tokens + The maximum number of tokens to generate +stop_at + A string or array of strings which, such that the generation stops + when they are generated. +system_prompt + The content of the system message that precedes the user's prompt. +temperature + The value of the temperature used to sample tokens +samples + The number of completions to generate for each prompt +stop_at + Up to 4 words where the API will stop the completion.
+ +outlines/models/openai.py__init__(client, config, tokenizer=None, system_prompt=None)
+
+Create an OpenAI instance.
This class supports the standard OpenAI API, the Azure OpeanAI API as +well as compatible APIs that rely on the OpenAI client.
+client
+ An instance of the API's async client.
+config
+ An instance of OpenAIConfig. Can be useful to specify some
+ parameters that cannot be set by calling this class' methods.
+tokenizer
+ The tokenizer associated with the model the client connects to.
outlines/models/openai.pygenerate_choice(prompt, choices, max_tokens=None, system_prompt=None)
+
+Call the OpenAI API to generate one of several choices.
+prompt + A string or list of strings that will be used to prompt the model +choices + The list of strings between which we ask the model to choose +max_tokens + The maximum number of tokens to generate +system_prompt + The content of the system message that precedes the user's prompt.
+ +outlines/models/openai.py161 +162 +163 +164 +165 +166 +167 +168 +169 +170 +171 +172 +173 +174 +175 +176 +177 +178 +179 +180 +181 +182 +183 +184 +185 +186 +187 +188 +189 +190 +191 +192 +193 +194 +195 +196 +197 +198 +199 +200 +201 +202 +203 +204 +205 +206 +207 +208 +209 +210 +211 +212 +213 +214 +215 +216 +217 +218 +219 +220 +221 +222 +223 +224 +225 +226 +227 +228 +229 +230 +231 +232 +233 +234 +235 +236 +237 +238 +239 +240 +241 +242 +243 +244 +245 +246 +247 +248 +249 +250 | |
OpenAIConfig
+
+
+
+ dataclass
+
+
+Represents the parameters of the OpenAI API.
+The information was last fetched on 2023/11/20. We document below the +properties that are specific to the OpenAI API. Not all these properties are +supported by Outlines.
+model
+ The name of the model. Available models can be found on OpenAI's website.
+frequence_penalty
+ Number between 2.0 and -2.0. Positive values penalize new tokens based on
+ their existing frequency in the text,
+logit_bias
+ Modifies the likelihood of specified tokens to appear in the completion.
+ Number between -100 (forbid) and +100 (only allows).
+n
+ The number of completions to return for each prompt.
+presence_penalty
+ Similar to frequency penalty.
+response_format
+ Specifies the format the model must output. {"type": "json_object"}
+ enables JSON mode.
+seed
+ Two completions with the same seed value should return the same
+ completion. This is however not guaranteed.
+stop
+ Up to 4 words where the API will stop the completion.
+temperature
+ Number between 0 and 2. Higher values make the output more random, while
+ lower values make it more deterministic.
+top_p
+ Number between 0 and 1. Parameter for nucleus sampling.
+user
+ A unique identifier for the end-user.
outlines/models/openai.pybuild_optimistic_mask(transposed, max_mask_size=300)
+
+We build the largest mask possible.
+Tokens are added from left to right, so if the encoded choices are e.g.
+[[1,2], [3,4]], 1 and 3 will be added before 2 and 4.
transposed + A list of lists that contain the nth token of each choice.
+ +outlines/models/openai.pyerror_handler(api_call_fn)
+
+Handle OpenAI API errors and missing API key.
+ +outlines/models/openai.pyfind_longest_intersection(response, choice)
+
+Find the longest intersection between the response and the choice.
+ +outlines/models/openai.pyfind_response_choices_intersection(response, choices)
+
+Find the longest intersection between the response and the different +choices.
+Say the response is of the form [1, 2, 3, 4, 5] and we have the choices
+[[1, 2], [1, 2, 3], [6, 7, 8] then the function will return [1, 2, 3] as the
+intersection, and [[]] as the list of choices left.
response + The model's response +choices + The remaining possible choices
+A tuple that contains the longest intersection between the response and the +different choices, and the choices which start with this intersection, with the +intersection removed.
+ +outlines/models/openai.pygenerate_chat(prompt, system_prompt, client, config)
+
+
+ async
+
+
+Call OpenAI's Chat Completion API.
+prompt
+ The prompt we use to start the generation. Passed to the model
+ with the "user" role.
+system_prompt
+ The system prompt, passed to the model with the "system" role
+ before the prompt.
+client
+ The API client
+config
+ An OpenAIConfig instance.
A tuple that contains the model's response(s) and usage statistics.
+ +outlines/models/openai.pyPartialIndenter
+
+
+
+ Bases: Indenter
An Indenter that doesn't reset its state every time process is called.
outlines/fsm/parsing.pyPartialParserState
+
+
+
+ Bases: ParserState
outlines/fsm/parsing.py345 +346 +347 +348 +349 +350 +351 +352 +353 +354 +355 +356 +357 +358 +359 +360 +361 +362 +363 +364 +365 +366 +367 +368 +369 +370 +371 +372 +373 +374 +375 +376 +377 +378 +379 +380 +381 +382 +383 +384 +385 +386 +387 +388 +389 +390 +391 +392 +393 +394 +395 +396 +397 +398 +399 +400 +401 +402 +403 +404 +405 +406 +407 +408 +409 +410 +411 +412 +413 +414 +415 +416 +417 +418 +419 +420 +421 +422 +423 +424 +425 +426 +427 +428 +429 +430 +431 +432 +433 +434 +435 +436 +437 +438 +439 +440 +441 +442 +443 +444 +445 +446 +447 +448 +449 +450 +451 +452 +453 +454 +455 +456 +457 +458 +459 | |
feed_token_no_stack(token, is_end=False)
+
+This is a copy of ParserState.feed_token with all the value stack
+steps removed. Since we're not exactly parsing in order to obtain a
+CST or anything similar, we can avoid the growing expense of tracking
+the parse tree.
outlines/fsm/parsing.pyPartialParsingFrontend
+
+
+
+ Bases: ParsingFrontend
outlines/fsm/parsing.py161 +162 +163 +164 +165 +166 +167 +168 +169 +170 +171 +172 +173 +174 +175 +176 +177 +178 +179 +180 +181 +182 +183 +184 +185 +186 +187 +188 +189 +190 +191 +192 +193 +194 +195 +196 +197 +198 +199 +200 +201 +202 +203 +204 +205 +206 +207 +208 +209 +210 +211 +212 +213 +214 +215 +216 +217 +218 +219 +220 +221 +222 +223 +224 +225 +226 +227 +228 +229 +230 +231 +232 +233 +234 +235 +236 +237 +238 +239 +240 +241 +242 +243 +244 +245 +246 +247 +248 +249 +250 +251 +252 +253 +254 +255 +256 +257 +258 +259 +260 +261 +262 +263 +264 +265 +266 +267 +268 +269 +270 +271 +272 +273 +274 +275 | |
PartialScanner
+
+
+
+ Bases: Scanner
outlines/fsm/parsing.py511 +512 +513 +514 +515 +516 +517 +518 +519 +520 +521 +522 +523 +524 +525 +526 +527 +528 +529 +530 +531 +532 +533 +534 +535 +536 +537 +538 +539 +540 +541 +542 +543 +544 +545 +546 +547 +548 +549 +550 +551 +552 +553 +554 +555 +556 +557 +558 +559 +560 +561 +562 +563 +564 +565 +566 +567 +568 +569 +570 +571 +572 +573 +574 +575 +576 +577 +578 +579 +580 +581 +582 +583 +584 +585 +586 +587 | |
get_terminals_info(fsm_state_seq)
+
+Get the possible terminal symbols for an FSM state sequence.
+ +outlines/fsm/parsing.pymatch(text, pos, last_fsm_state_seq=None)
+
+Determine an FSM match over text starting at pos and continuing last_fsm_state_seq.
outlines/fsm/parsing.pyterminals_to_fsms(lp)
+
+Construct a dict mapping terminal symbol names to their finite state machines.
outlines/fsm/parsing.pyPrompt
+
+
+
+ dataclass
+
+
+Represents a prompt function.
+We return a Prompt class instead of a simple function so the
+template defined in prompt functions can be accessed.
outlines/prompts.py__call__(*args, **kwargs)
+
+Render and return the template.
+The rendered template as a Python str.
outlines/prompts.pyget_fn_description(fn)
+
+Returns the first line of a callable's docstring.
+ +outlines/prompts.pyget_fn_name(fn)
+
+Returns the name of a callable.
+ +outlines/prompts.pyget_fn_signature(fn)
+
+Return the signature of a callable.
+ +outlines/prompts.pyget_fn_source(fn)
+
+Return the source code of a callable.
+ +outlines/prompts.pyget_schema_dict(model)
+
+get_schema_pydantic(model)
+
+Return the schema of a Pydantic model.
+ +outlines/prompts.pyparse_pydantic_schema(raw_schema, definitions)
+
+Parse the output of Basemodel.[schema|model_json_schema]().
This recursively follows the references to other schemas in case +of nested models. Other schemas are stored under the "definitions" +key in the schema of the top-level model.
+ +outlines/prompts.pyprompt(fn)
+
+Decorate a function that contains a prompt template.
+This allows to define prompts in the docstring of a function and simplify their
+manipulation by providing some degree of encapsulation. It uses the render
+function internally to render templates.
++++++import outlines
+@outlines.prompt +def build_prompt(question): +... "I have a ${question}" +... +prompt = build_prompt("How are you?")
+
This API can also be helpful in an "agent" context where parts of the prompt +are set when the agent is initialized and never modified later. In this situation +we can partially apply the prompt function at initialization.
+++++++import outlines +import functools as ft +... +@outlines.prompt +... def solve_task(name: str, objective: str, task: str): +... '''Your name is {{name}}. +.. Your overall objective is to {{objective}}. +... Please solve the following task: {{task}} +... ''' +... +hal = ft.partial(solve_task, "HAL", "Travel to Jupiter")
+
A Prompt callable class which will render the template when called.
outlines/prompts.pyrender(template, **values)
+
+Parse a Jinaj2 template and translate it into an Outlines graph.
+This function removes extra whitespaces and linebreaks from templates to +allow users to enter prompts more naturally than if they used Python's +constructs directly. See the examples for a detailed explanation.
+Outlines follow Jinja2's syntax
+++++++import outlines +outline = outlines.render("I like {{food}} and {{sport}}", food="tomatoes", sport="tennis") +I like tomatoes and tennis
+
If the first line of the template is empty, render removes it
++++++from outlines import render
+tpl = ''' +... A new string''' +tpl +... '\nA new string' +render(tpl) +... 'a new string'
+
Similarly, render ignores linebreaks introduced by placing the closing quotes
+underneath the text:
++++++tpl = ''' +... A new string +... ''' +tpl +... '\nA new string\n' +render(tpl) +... 'A new string'
+
If you want to insert a linebreak at the end of the rendered template, you will +need to leave an empty line at the end of the template:
+++++++tpl = ''' +... A new string +... +... ''' +tpl +... '\nA new string\n\n' +render(tpl) +... 'A new string\n'
+
render removes the identation in docstrings. This is particularly important
+when using prompt functions
++++++tpl = ''' +... a string +... and another string''' +tpl +... '\n a string\n and another string' +render(tpl) +... 'a string\nand another string'
+
The indentation of the first line is assumed to be the same as the second line's
+++++++tpl = '''a string +... and another''' +tpl +... 'a string\n and another' +render(tpl) +... 'a string\nand another'
+
To get a different indentation for the first and the second line, we can start the +prompt on the string's second line:
+++++++tpl = ''' +... First line +... Second line''' +render(tpl) +... 'First Line\n Second Line'
+
template + A string that contains a template written with the Jinja2 syntax. +**values + Map from the variables in the template to their value.
+A string that contains the rendered template.
+ +outlines/prompts.py94 + 95 + 96 + 97 + 98 + 99 +100 +101 +102 +103 +104 +105 +106 +107 +108 +109 +110 +111 +112 +113 +114 +115 +116 +117 +118 +119 +120 +121 +122 +123 +124 +125 +126 +127 +128 +129 +130 +131 +132 +133 +134 +135 +136 +137 +138 +139 +140 +141 +142 +143 +144 +145 +146 +147 +148 +149 +150 +151 +152 +153 +154 +155 +156 +157 +158 +159 +160 +161 +162 +163 +164 +165 +166 +167 +168 +169 +170 +171 +172 +173 +174 +175 +176 +177 +178 +179 +180 +181 +182 +183 +184 +185 +186 +187 +188 +189 +190 +191 +192 +193 +194 +195 +196 +197 +198 +199 +200 +201 +202 +203 +204 +205 +206 +207 +208 +209 +210 +211 +212 +213 | |
regex(model, regex_str, sampler=multinomial())
+
+Generate structured text in the language of a regular expression.
+model:
+ An instance of Transformer that represents a model from the
+ transformers library.
+regex_str:
+ The regular expression that the output must follow.
+sampler:
+ The sampling algorithm to use to generate token ids from the logits
+ distribution.
A SequenceGenerator instance that generates text constrained by the
+regular expression.
outlines/generate/regex.pyBeamSearchSampler
+
+
+Beam Search sampling algorithm.
+samples + The number of samples taken for each input sequence.
+ +outlines/samplers.py235 +236 +237 +238 +239 +240 +241 +242 +243 +244 +245 +246 +247 +248 +249 +250 +251 +252 +253 +254 +255 +256 +257 +258 +259 +260 +261 +262 +263 +264 +265 +266 +267 +268 +269 +270 +271 +272 +273 +274 +275 +276 +277 +278 +279 +280 +281 +282 +283 +284 +285 +286 +287 +288 +289 +290 +291 +292 +293 +294 +295 +296 +297 +298 +299 +300 +301 +302 +303 +304 +305 +306 +307 | |
__call__(next_token_logits, sequence_weights, _)
+
+Call the beam search sampler.
+next_token_logits
+ A tensor of shape (n_seqs, vocab_size,) that represents the
+ probability distribution of the next token over the vocabulary.
+sequence_weights
+ A tensor of shape (n_seqs,) that represents the cumulative
+ weight of each sequence.
+rng
+ A random number generator.
A tuple with an array that contains the ids of the sampled tokens of
+shape (n_seqs, 1), an array that contains the ancestors of each
+sampled id of shape (n_seqs,) and an array that contains the updated
+cumulative weights of each sequence of shape (n_seqs,).
outlines/samplers.pyGreedySampler
+
+
+Greedy Sampling algorithm.
+Greedy sampling consists in choosing the token with the largest +likelihood at every step.
+We don't allow more than one sample. We could attribute this a meaning, for +instance the k-th sample represents the k-th most likely token. In which +case it would be equivalent to beam search without the sequence weights.
+samples + The number of samples taken for each input sequence.
+ +outlines/samplers.py__call__(next_token_logits, sequence_weights, _)
+
+Call the greedy sampler.
+next_token_logits
+ A tensor of shape (n_seqs, vocab_size,) that represents the
+ probability distribution of the next token over the vocabulary.
+sequence_weights
+ A tensor of shape (n_seqs,) that represents the cumulative
+ weight of each sequence.
+rng
+ A random number generator.
A tuple with an array that contains the ids of the sampled tokens of
+shape (n_seqs, 1), an array that contains the ancestors of each
+sampled id of shape (n_seqs,) and an array that contains the updated
+cumulative weights of each sequence of shape (n_seqs,).
outlines/samplers.pyMultinomialSampler
+
+
+Multinomial sampling algorithm.
+Multinomial sampling consists in randomly sampling the next token assuming +its distribution is a Categorical distribution parametrized by the +next-token logits.
+samples + The number of samples taken for each input sequence.
+ +outlines/samplers.py80 + 81 + 82 + 83 + 84 + 85 + 86 + 87 + 88 + 89 + 90 + 91 + 92 + 93 + 94 + 95 + 96 + 97 + 98 + 99 +100 +101 +102 +103 +104 +105 +106 +107 +108 +109 +110 +111 +112 +113 +114 +115 +116 +117 +118 +119 +120 +121 +122 +123 +124 +125 +126 +127 +128 +129 +130 +131 +132 +133 +134 +135 +136 +137 +138 +139 +140 +141 +142 +143 +144 +145 +146 +147 +148 +149 +150 +151 +152 +153 +154 | |
__call__(next_token_logits, sequence_weights, rng)
+
+Call the multinomial sampler.
+next_token_logits
+ A tensor of shape (n_seqs, vocab_size,) that represents the
+ probability distribution of the next token over the vocabulary.
+sequence_weights
+ A tensor of shape (n_seqs,) that represents the cumulative
+ weight of each sequence.
+rng
+ A random number generator.
A tuple with an array that contains the ids of the sampled tokens of
+shape (n_seqs, 1), an array that contains the ancestors of each
+sampled id of shape (n_seqs,) and an array that contains the updated
+cumulative weights of each sequence of shape (n_seqs,).
outlines/samplers.pykeep_top_k_logits(k)
+
+Build a function that masks logits values smaller than the top k ones.
k
+ The ranking below which logit values are replaced by -math.inf.
outlines/samplers.pykeep_top_p_logits(p)
+
+Build a function that masks the lowest probability tokens whose +cumulative probability is below a certain threshold.
+p
+ The value of the threshold. We keep the highest probability tokens whose
+ cumulative distribution is greater than or equal to p and mask the
+ others. Its value must be between 0 (excluded) and 1 (included).
outlines/samplers.pyrescale_logits(temperature)
+
+Build a function that rescales the token probabilities exponentially.
+temperature + The value by which we rescale the logits.
+ +outlines/samplers.pySCw?g1AZdtb{ZFl
z%_QRDDvV9d$zFv<9N43+R}q<=aJ<0cQRvq6Y>g57Nj0o)^pEM>={?FLD{Wf!R|_)5
ztd0|kul@rbYGAM>3Z1u}!9swXgv5kEC5*^a#?}+RI799q)hq(IB`%O7Pv^R?zQA$G
zpD&k0hs++Df%%M2pEM^E(-Ys)*>i-2VB?^tHt>@K5lWt+_fChi$l$&+TXpJ1hA#=%
zxagg4NtpL!(+*z77R?40Qehrg$dSzC`T!nNP!lKcZUltkGt(IYw8M{1l~0}pk05B9
zg{JfC!<=NMi9^JUhq(%mpz2x&uRA_&O6Q~skMR2k{*c3;em43u9kO8riQ9*?NY+IY
zDiI4fO%Ckqz2TDiCsltfo8{fd%$mVE>;pGCawuW3xPv!Gck?KeV6B|&oKj3eYOHXe
z2<-Qy8Co7!0JxFzRKx4D_G;-7qt<)V<4qn5i9cXUS2zh%T$;~i0g>;wM*-l+{l=EF
zY1oZ_#NLkhBW>>CkO#%(>$02{_D&%kFtb34cV` n)eKl*BiAN1Tp5@I|=#iLr)t
zVpgJ_)=S(njLO`vyp1<6s