Generating grammars automatically following lark grammar

[AdrienCorenflos]

Hi,

I was trying to be a bit lazy today and implement a MWE to ask a model to generate a usable grammar automatically for a problem at hand: in this instance, I wanted to generate a grammar that would be compatible with pgfplots in tikz/LaTex.

The script I wrote is the following (adapted from the README example):

import outlines

model = outlines.models.transformers("TinyLlama/TinyLlama_v1.1")

prompt = r"""Create a Lark grammar for a tikzpicture environment in LaTeX. 
The grammar should be able to parse plots implemented using pgfplots.
"""

with open("lark.lark", "r") as f:
    lark_grammar = f.read()

generator = outlines.generate.cfg(model, lark_grammar)
answer = generator(prompt)

print(answer)

The lark.lark file is posted below: it's a slightly modified version of https://github.com/lark-parser/lark/blob/master/lark/grammars/lark.lark which does not seem to parse due to the Terminal _STRING not being defined (I need to open an issue in their repo).

I got past the finite state machine compilation, but it does not seem to do anything at all after that:

Any idea what is going on? Is my laptop too slow (I wouldn't be surprised :D)

# Lark grammar of Lark's syntax
# Note: Lark is not bootstrapped, its parser is implemented in load_grammar.py

start: (_item? _NL)* _item?

_item: rule
     | token
     | statement

rule: RULE rule_params priority? ":" expansions
token: TOKEN token_params priority? ":" expansions

rule_params: ["{" RULE ("," RULE)* "}"]
token_params: ["{" TOKEN ("," TOKEN)* "}"]

priority: "." NUMBER

statement: "%ignore" expansions                    -> ignore
         | "%import" import_path ["->" name]       -> import
         | "%import" import_path name_list         -> multi_import
         | "%override" rule                        -> override_rule
         | "%declare" name+                        -> declare

!import_path: "."? name ("." name)*
name_list: "(" name ("," name)* ")"

?expansions: alias (_VBAR alias)*

?alias: expansion ["->" RULE]

?expansion: expr*

?expr: atom [OP | "~" NUMBER [".." NUMBER]]

?atom: "(" expansions ")"
     | "[" expansions "]" -> maybe
     | value

?value: STRING ".." STRING -> literal_range
      | name
      | (REGEXP | STRING) -> literal
      | name "{" value ("," value)* "}" -> template_usage

name: RULE
    | TOKEN

_VBAR: _NL? "|"
OP: /[+*]|[?](?![a-z])/
RULE: /!?[_?]?[a-z][_a-z0-9]*/
TOKEN: /_?[A-Z][_A-Z0-9]*/
STRING: _STRING "i"?
REGEXP: /\/(?!\/)(\\\/|\\\\|[^\/])*?\/[imslux]*/
_NL: /(\r?\n)+\s*/
_STRING: /"(\\"|[^"])*"/

%import common.ESCAPED_STRING -> _STRING
%import common.SIGNED_INT -> NUMBER
%import common.WS_INLINE

COMMENT: /\s*/ "//" /[^\n]/* | /\s*/ "#" /[^\n]/*

%ignore WS_INLINE
%ignore COMMENT

[lapp0]

Since the index compiled 100%, it's likely that the model is infinitely generating. Could you try setting a token limit and ensuring the generation process is running?

outlines.generate.cfg(model, lark_grammar, max_tokens=10)