Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Strange issues when using memoize/cache and refreshing #174

Open
ghedb opened this issue Mar 13, 2020 · 2 comments
Open

Strange issues when using memoize/cache and refreshing #174

ghedb opened this issue Mar 13, 2020 · 2 comments

Comments

@ghedb
Copy link

ghedb commented Mar 13, 2020

Hello there.

I am running into a strange error when using memoize type caching for db calls (to mariadb).
I am trying to keep db connections down, so I am trying to only open a connection when needed.

This is with using aiocache, but getting essentially the same thing when using py-memoize.

This is what I am doing (in starlette).

@cached(ttl=60 * 2, cache=SimpleMemoryCache)
async def get_db_data(reference_type):
    query = references.select().where(references.c.Type == reference_type)
    await database.connect()
    reference = await database.fetch_one(query)
    await database.disconnect()
.....

This is the error I am getting.

File "/usr/local/lib/python3.6/site-packages/databases/core.py", line 137, in fetch_one
return await connection.fetch_one(query, values)
File "/usr/local/lib/python3.6/site-packages/databases/core.py", line 212, in __aexit__
await self._connection.release()
File "/usr/local/lib/python3.6/site-packages/databases/backends/mysql.py", line 96, in release
await self._database._pool.release(self._connection)
File "/usr/local/lib/python3.6/site-packages/aiomysql/pool.py", line 204, in release
assert conn in self._used, (conn, self._used)
AssertionError: (<aiomysql.connection.Connection object at 0x7f636b74dc50>, {<aiomysql.connection.Connection object at 0x7f636b77e128>})

This is specific to aiocache, but in py-memoize I am getting

AssertionError: (<aiomysql.connection.Connection object at 0x7fadc54a09e8>, set())

Not sure if the issue is here or in the caching implementations, but it seems odd to get the same in two different implementations. I'm pretty clueless at the moment. Any help would be appreciated, and let me know if I can help in any way.

@ghedb
Copy link
Author

ghedb commented Mar 16, 2020

Actually starting to realize that its not having a significant impact doing the connect in the function rather than the on_startup/on_shutdown.

I'm reverting to use the events, so problem solved for me. But I will leave the ticket here since perhaps there might be some underlying issue here.
I wish I was more familiar with asyncio and could troubleshoot it more myself.

@FichteFoll
Copy link

FichteFoll commented Sep 2, 2024

I got the same AssertionError with supposedly normal usage, only with asyncio.gather involved, so I suspect this issue is related to #516 (I also got DatabaseBackend is not running errors on the other running tasks) and #456 (#230). The error does also not reproduce 100% of the time, but it is frequent enough to be quite unstable. Furthermore, after removing one asyncio.gather usage where a connection was shared, the problem still occured.

Additionally, I also seem to have a problem with the async context managers never returning sometimes because the last statements within the blocks (logging) are executed but the first one after the block is not.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants