Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Upgrade to SQLAlchemy 2.0 #126

Draft
wants to merge 9 commits into
base: master
Choose a base branch
from
Draft

Conversation

smsearcy
Copy link
Owner

@smsearcy smsearcy commented Jun 26, 2024

Upgrade to SQLAlchemy 2.0 and utilize the new API as much as possible.

  • Test collection process
  • Clean up Alembic migrations
  • Use 2.0 select() statements everywhere

(Optimizing db updates will get its own branch)

@smsearcy smsearcy linked an issue Jun 26, 2024 that may be closed by this pull request
@smsearcy smsearcy self-assigned this Jun 26, 2024
@smsearcy
Copy link
Owner Author

smsearcy commented Jul 4, 2024

Test failed, it looks like something is broken with the Node.links relationship. Not sure if that is causing the actual exception.

/home/smsearcy/Projects/mesh-info/meshinfo/collector.py:418: SAWarning: Multiple rows returned with uselist=False for lazily-loaded attribute 'Node.links'  (This warning originated from the Session 'autoflush' process, which was invoked automatically in response to a user-initiated operation.)
  .one_or_none()
Traceback (most recent call last):
  File "/home/smsearcy/Projects/mesh-info/.venv/bin/meshinfo", line 8, in <module>
    sys.exit(main())
             ^^^^^^
  File "/home/smsearcy/Projects/mesh-info/meshinfo/cli.py", line 82, in main
    collector.main(
  File "/home/smsearcy/Projects/mesh-info/meshinfo/collector.py", line 95, in main
    asyncio.run(service(collection, polling_period=config.period))
  File "/usr/lib64/python3.11/asyncio/runners.py", line 190, in run
    return runner.run(main)
           ^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.11/asyncio/runners.py", line 118, in run
    return self._loop.run_until_complete(task)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.11/asyncio/base_events.py", line 654, in run_until_complete
    return future.result()
           ^^^^^^^^^^^^^^^
  File "/home/smsearcy/Projects/mesh-info/meshinfo/collector.py", line 109, in service
    await collect()
  File "/home/smsearcy/Projects/mesh-info/meshinfo/collector.py", line 168, in collector
    link_models = save_links(links, dbsession, count=summary)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/smsearcy/Projects/mesh-info/meshinfo/collector.py", line 418, in save_links
    .one_or_none()
     ^^^^^^^^^^^^^
  File "/home/smsearcy/Projects/mesh-info/.venv/lib64/python3.11/site-packages/sqlalchemy/orm/query.py", line 2754, in one_or_none
    return self._iter().one_or_none()  # type: ignore
           ^^^^^^^^^^^^
  File "/home/smsearcy/Projects/mesh-info/.venv/lib64/python3.11/site-packages/sqlalchemy/orm/query.py", line 2827, in _iter
    result: Union[ScalarResult[_T], Result[_T]] = self.session.execute(
                                                  ^^^^^^^^^^^^^^^^^^^^^
  File "/home/smsearcy/Projects/mesh-info/.venv/lib64/python3.11/site-packages/sqlalchemy/orm/session.py", line 2351, in execute
    return self._execute_internal(
           ^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/smsearcy/Projects/mesh-info/.venv/lib64/python3.11/site-packages/sqlalchemy/orm/session.py", line 2215, in _execute_internal
    ) = compile_state_cls.orm_pre_session_exec(
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/smsearcy/Projects/mesh-info/.venv/lib64/python3.11/site-packages/sqlalchemy/orm/context.py", line 549, in orm_pre_session_exec
    session._autoflush()
  File "/home/smsearcy/Projects/mesh-info/.venv/lib64/python3.11/site-packages/sqlalchemy/orm/session.py", line 3039, in _autoflush
    self.flush()
  File "/home/smsearcy/Projects/mesh-info/.venv/lib64/python3.11/site-packages/sqlalchemy/orm/session.py", line 4341, in flush
    self._flush(objects)
  File "/home/smsearcy/Projects/mesh-info/.venv/lib64/python3.11/site-packages/sqlalchemy/orm/session.py", line 4476, in _flush
    with util.safe_reraise():
  File "/home/smsearcy/Projects/mesh-info/.venv/lib64/python3.11/site-packages/sqlalchemy/util/langhelpers.py", line 146, in __exit__
    raise exc_value.with_traceback(exc_tb)
  File "/home/smsearcy/Projects/mesh-info/.venv/lib64/python3.11/site-packages/sqlalchemy/orm/session.py", line 4437, in _flush
    flush_context.execute()
  File "/home/smsearcy/Projects/mesh-info/.venv/lib64/python3.11/site-packages/sqlalchemy/orm/unitofwork.py", line 466, in execute
    rec.execute(self)
  File "/home/smsearcy/Projects/mesh-info/.venv/lib64/python3.11/site-packages/sqlalchemy/orm/unitofwork.py", line 591, in execute
    self.dependency_processor.process_saves(uow, states)
  File "/home/smsearcy/Projects/mesh-info/.venv/lib64/python3.11/site-packages/sqlalchemy/orm/dependency.py", line 604, in process_saves
    self._synchronize(
  File "/home/smsearcy/Projects/mesh-info/.venv/lib64/python3.11/site-packages/sqlalchemy/orm/dependency.py", line 625, in _synchronize
    sync.clear(dest, self.mapper, self.prop.synchronize_pairs)
  File "/home/smsearcy/Projects/mesh-info/.venv/lib64/python3.11/site-packages/sqlalchemy/orm/sync.py", line 88, in clear
    raise AssertionError(
AssertionError: Dependency rule on column 'node.node_id' tried to blank-out primary key column 'link.source_id' on instance '<Link at 0x7f4d8d8704d0>'

@smsearcy smsearcy force-pushed the 95-migrate-to-sqlalchemy-20-api branch 5 times, most recently from f6362d1 to 4d11617 Compare July 17, 2024 20:31
@smsearcy smsearcy force-pushed the 95-migrate-to-sqlalchemy-20-api branch from 4d11617 to e9c01ee Compare July 25, 2024 19:31
@smsearcy smsearcy force-pushed the 95-migrate-to-sqlalchemy-20-api branch from e9c01ee to d7a07ae Compare July 25, 2024 19:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Migrate to SQLAlchemy 2.0 API
1 participant