OutputParserException: Failed to parse CodeCompletion from completion

[christopherblum]

I tried the following:

%%code 
# using matplotlib, create a two-dimensional scatter plot of 1000 points drawn from a two-dimensional standard normal distribution

but obtained the following error (I believe the parser does not escape the \n properly):

---------------------------------------------------------------------------
ValidationError                           Traceback (most recent call last)
File ~/py3.10/lib/python3.10/site-packages/langchain/output_parsers/pydantic.py:26, in PydanticOutputParser.parse(self, text)
     25     json_object = json.loads(json_str, strict=False)
---> 26     return self.pydantic_object.parse_obj(json_object)
     28 except (json.JSONDecodeError, ValidationError) as e:

File ~/py3.10/lib/python3.10/site-packages/pydantic/main.py:526, in pydantic.main.BaseModel.parse_obj()

File ~/py3.10/lib/python3.10/site-packages/pydantic/main.py:341, in pydantic.main.BaseModel.__init__()

ValidationError: 1 validation error for CodeCompletion
source
  field required (type=value_error.missing)

During handling of the above exception, another exception occurred:

OutputParserException                     Traceback (most recent call last)
Cell In[6], line 1
----> 1 get_ipython().run_cell_magic('code', '', '# using matplotlib, create a two-dimensional scatter plot of 1000 points drawn from a two-dimensional standard normal distribution\n')

File ~/py3.10/lib/python3.10/site-packages/IPython/core/interactiveshell.py:2478, in InteractiveShell.run_cell_magic(self, magic_name, line, cell)
   2476 with self.builtin_trap:
   2477     args = (magic_arg_s, cell)
-> 2478     result = fn(*args, **kwargs)
   2480 # The code below prevents the output from being displayed
   2481 # when using magics with decodator @output_can_be_silenced
   2482 # when the last Python token in the expression is a ';'.
   2483 if getattr(fn, magic.MAGIC_OUTPUT_CAN_BE_SILENCED, False):

File ~/py3.10/lib/python3.10/site-packages/notebook_copilot/notebook_copilot.py:39, in CopilotMagics.code(self, line, cell)
     37 run_history = get_ipython_run_history()
     38 cell = cell.replace(line, '')
---> 39 code_completion = get_cells_completion(self.build_llm_from_args(line), CompletionType.CODE, run_history,
     40                                        cell=cell)
     41 generate_notebook_cell_below(CellCompletion(cell_type=CellType.CODE, source=code_completion.source))

File ~/py3.10/lib/python3.10/site-packages/notebook_copilot/chains.py:40, in get_cells_completion(llm, completion_type, run_history, cell)
     38 chain = LLMChain(llm=llm, prompt=prompt)
     39 output = chain.predict(run_history=run_history, cell={cell})
---> 40 return output_parser.parse(output)

File ~/py3.10/lib/python3.10/site-packages/langchain/output_parsers/pydantic.py:31, in PydanticOutputParser.parse(self, text)
     29 name = self.pydantic_object.__name__
     30 msg = f"Failed to parse {name} from completion {text}. Got: {e}"
---> 31 raise OutputParserException(msg)

OutputParserException: Failed to parse CodeCompletion from completion {"properties": {"source": "# using matplotlib, create a two-dimensional scatter plot of 1000 points drawn from a two-dimensional standard normal distribution\nimport matplotlib.pyplot as plt\nimport numpy as np\n\n# Generate random data\nx = np.random.normal(0, 1, 1000)\ny = np.random.normal(0, 1, 1000)\n\n# Create scatter plot\nplt.scatter(x, y)\nplt.xlabel('X')\nplt.ylabel('Y')\nplt.title('Scatter Plot')\nplt.show()"}, "required": ["source"]}. Got: 1 validation error for CodeCompletion
source
  field required (type=value_error.missing)

I'm running Python 3.10 on Ubuntu 22.04 with the following package versions:

aiohttp==3.8.5
aiosignal==1.3.1
anyio==3.7.1
argon2-cffi==23.1.0
argon2-cffi-bindings==21.2.0
arrow==1.2.3
asttokens==2.2.1
async-lru==2.0.4
async-timeout==4.0.3
attrs==23.1.0
Babel==2.12.1
backcall==0.2.0
beautifulsoup4==4.12.2
bleach==6.0.0
certifi==2023.7.22
cffi==1.15.1
charset-normalizer==3.2.0
comm==0.1.4
contourpy==1.1.0
cycler==0.11.0
dataclasses-json==0.5.14
debugpy==1.6.7.post1
decorator==5.1.1
defusedxml==0.7.1
exceptiongroup==1.1.3
executing==1.2.0
faiss-cpu==1.7.4
fastjsonschema==2.18.0
fonttools==4.42.1
fqdn==1.5.1
frozenlist==1.4.0
greenlet==2.0.2
idna==3.4
ipykernel==6.25.1
ipython==8.13.2
ipywidgets==8.0.6
isoduration==20.11.0
jedi==0.19.0
Jinja2==3.1.2
json5==0.9.14
jsonpointer==2.4
jsonschema==4.19.0
jsonschema-specifications==2023.7.1
jupyter-events==0.7.0
jupyter-lsp==2.2.0
jupyter_client==8.3.0
jupyter_core==5.3.1
jupyter_server==2.7.2
jupyter_server_terminals==0.4.4
jupyterlab==4.0.5
jupyterlab-pygments==0.2.2
jupyterlab-widgets==3.0.8
jupyterlab_server==2.24.0
kiwisolver==1.4.5
langchain==0.0.187
langsmith==0.0.27
MarkupSafe==2.1.3
marshmallow==3.20.1
matplotlib==3.7.2
matplotlib-inline==0.1.6
mistune==3.0.1
multidict==6.0.4
mypy-extensions==1.0.0
nbclient==0.8.0
nbconvert==7.7.4
nbformat==5.9.2
nest-asyncio==1.5.7
notebook-copilot==0.2.0
notebook_shim==0.2.3
numexpr==2.8.5
numpy==1.25.2
openai==0.27.9
openapi-schema-pydantic==1.2.4
overrides==7.4.0
packaging==23.1
pandas==2.0.3
pandocfilters==1.5.0
parso==0.8.3
pexpect==4.8.0
pickleshare==0.7.5
Pillow==10.0.0
platformdirs==3.10.0
prometheus-client==0.17.1
prompt-toolkit==3.0.39
psutil==5.9.5
ptyprocess==0.7.0
pure-eval==0.2.2
pycparser==2.21
pydantic==1.10.12
Pygments==2.16.1
pyparsing==3.0.9
python-dateutil==2.8.2
python-json-logger==2.0.7
pytz==2023.3
PyYAML==6.0.1
pyzmq==25.1.1
referencing==0.30.2
regex==2023.8.8
requests==2.31.0
rfc3339-validator==0.1.4
rfc3986-validator==0.1.1
rpds-py==0.10.0
scipy==1.11.2
Send2Trash==1.8.2
six==1.16.0
sniffio==1.3.0
soupsieve==2.4.1
SQLAlchemy==2.0.20
stack-data==0.6.2
tenacity==8.2.3
terminado==0.17.1
tiktoken==0.4.0
tinycss2==1.2.1
tomli==2.0.1
tornado==6.3.3
tqdm==4.66.1
traitlets==5.9.0
typing-inspect==0.9.0
typing_extensions==4.7.1
tzdata==2023.3
uri-template==1.3.0
urllib3==2.0.4
wcwidth==0.2.6
webcolors==1.13
webencodings==0.5.1
websocket-client==1.6.2
widgetsnbextension==4.0.8
yarl==1.9.2