[Feature Request] Create a cassandra session per migration script

[witzatom]

I have a few cql scripts and a python script to migrate. The python script is something like this:

from cassandra.cluster import Session
from cassandra.query import SimpleStatement, PreparedStatement, dict_factory
import logging

log = logging.getLogger(__name__)


def execute(session: Session):
    session.row_factory = dict_factory
    __migrate_table(session, "table", newtable_schema)


def __migrate_table(session: Session, original_table: str, new_schema: str):
    log.info(f"Migrating table {original_table} to a new schema {new_schema}")
    # do stuff, deleted since it doesnt matter

After I run the migrations the migration exits with status code 1 and the following exception:

INFO:Migrator:Creating keyspace ''
INFO:Migrator:Creating table 'database_migrations' in keyspace ''
INFO:Migrator:Pending migrations found. Current version: None, Latest version: 3
INFO:Migrator:Advancing to version 1
INFO:Migrator:Writing in-progress migration version 1: Migration("v00001_initial_schema.cql")
INFO:Migrator:Applying cql migration
INFO:Migrator:Executing migration with 9 CQL statements
INFO:Migrator:Finalizing migration version with state SUCCEEDED
INFO:Migrator:Advancing to version 2
INFO:Migrator:Writing in-progress migration version 2: Migration("v00002_.cql")
INFO:Migrator:Applying cql migration
INFO:Migrator:Executing migration with 1 CQL statements
INFO:Migrator:Finalizing migration version with state SUCCEEDED
INFO:Migrator:Advancing to version 3
INFO:Migrator:Writing in-progress migration version 3: Migration("v00003_.py")
INFO:Migrator:Applying python script
INFO:v00003_nvector_partitioning:Migrating table table to a new schema 
    CREATE TABLE IF NOT EXISTS table(
        id int,
        time bigint,
        PRIMARY KEY (id, time)
    );
    
INFO:v00003_nvector_partitioning:Copying table loginnvector to tmp_loginnvector started
INFO:v00003_nvector_partitioning:Copying table loginnvector to tmp_loginnvector finished
INFO:Migrator:Finalizing migration version with state SUCCEEDED
Traceback (most recent call last):
  File "/usr/local/bin/cassandra-migrate", line 18, in <module>
    main()
  File "/usr/local/lib/python3.6/site-packages/cassandra_migrate/cli.py", line 149, in main
    cmd_method(opts)
  File "/usr/local/lib/python3.6/site-packages/cassandra_migrate/migrator.py", line 100, in wrapper
    return func(self, opts, *args, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/cassandra_migrate/migrator.py", line 542, in migrate
    force=opts.force)
  File "/usr/local/lib/python3.6/site-packages/cassandra_migrate/migrator.py", line 509, in _advance
    self._apply_migration(version, migration, skip=skip)
  File "/usr/local/lib/python3.6/site-packages/cassandra_migrate/migrator.py", line 468, in _apply_migration
    if not result or not result[0].applied:
AttributeError: 'dict' object has no attribute 'applied'
+ final_exit_code=1

You can see the python script seems to have been applied successfully, only afterwards in the migrator tool internals it crashes.

Initially I was really quite confused as to why its crashing. The issue is in the row_factory being set by my script. Ofc this was not a hard fix for me, I just have to reset it to whatever was in it before my script, but its a bit error prone. I really think that the session the migration scripts use should be kept separate from each other, and from the session that the migrator internals use. I cant imagine what kind of mischief you could cause, by accident or on purpose.

[danielkza]

I had not hit any cases where sharing the session caused issues, but your example seems perfectly acceptable and it should not fail. I will look into a solution ASAP.

[witzatom]

cheers, thanks for your time