Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

troubleshooting dangling proxy objects #125

Open
memetb opened this issue Mar 25, 2025 · 1 comment
Open

troubleshooting dangling proxy objects #125

memetb opened this issue Mar 25, 2025 · 1 comment

Comments

@memetb
Copy link

memetb commented Mar 25, 2025

Hello, I was just troubleshooting this following strange situation:

async def reaper():
	reaper_simdjson_parser = simdjson.Parser()
	while True:
		await asyncio.sleep(1)

		text = await get_json_payload()
		doc = reaper_simdjson_parser.parse(text)
		*results, status = doc

		for x in results:
			# do something

		# del x # forgetting to explicitly delete x would cause intermittent problems
		del status
		del results
		del doc

the error is this:

    doc = reaper_simdjson_parser.parse(text) 
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "simdjson/csimdjson.pyx", line 435, in csimdjson.Parser.parse
RuntimeError: Tried to re-use a parser while simdjson.Object and/or simdjson.Array objects still exist referencing the old parser.

Perhaps specifying which reference is dangling may be helpful? At the very least putting it in the documentation as a potential gotcha might be helpful.

Thank you for pysimdjson and simdjson in general. Huge fan.

@TkTech
Copy link
Owner

TkTech commented Mar 25, 2025

Unfortunately we only keep a reference count, since tracking with more detail would be a significant performance and potentially significant memory hit.

Version 7.0.0 (#124) switches to using the new(ish) document object in simdjson, which means the lifecycle of the objects and the parser can be disconnected without issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants