Merge pull request #29 from lsst-dm/tickets/DM-42076

DM-42076: Add script for one-shot collections migration

doc/lsst.daf.butler_migrate/migrations/collections.rst

@@ -5,3 +5,16 @@ Migrations for collections manager
 The ``collections`` tree has a number of migrations scripts created but there is no actual migration implemented in any of them.
 These scripts reflect the history of versions that existed in ``daf_butler`` code, but all production version of database should have the latest version of the schema.
 The ``collections`` tree has two branches, ``collections-SynthIntKeyCollectionManager`` and ``collections-NameKeyCollectionManager``, for two separate implementations.
+
+
+One-shot migration to integer collection IDs
+============================================
+
+This is a one-shot migration from ``NameKeyCollectionManager`` 2.0.0 to ``SynthIntKeyCollectionManager`` 2.0.0::
+
+    $ butler migrate show-history --one-shot collections/name_2.0.0_to_int_2.0.0
+    93341d68b814 -> 8c57494cabcc (collections) (head), Migration script for SynthIntKeyCollectionManager 2.0.0.
+    8d2e9de2f21f -> 93341d68b814 (collections), Migration script for NameKeyCollectionManager 2.0.0.
+    <base> -> 8d2e9de2f21f (collections), Initial pseudo-revision of the 'collections/name_2.0.0_to_int_2.0.0' tree.
+
+The migration script for this one-shot migration is in `8c57494cabcc.py <https://github.com/lsst-dm/daf_butler_migrate/blob/main/migrations/_oneshot/collections/name_2.0.0_to_int_2.0.0/8c57494cabcc.py>`_. JIRA ticket `DM-42076 <https://jira.lsstcorp.org/browse/DM-42076>`_ has extra info on performance of this migration on large repositories.

migrations/_alembic/script.py.mako

@@ -1,23 +1,40 @@
+<%!
+def fmt(item):
+    if item is None:
+        return "None"
+    elif isinstance(item, (list, tuple)):
+        strings = [fmt(i) for i in item]
+        return f'({", ".join(strings)},)'
+    else:
+        return f'"{item}"'
+%>\
 """${message}
 
 Revision ID: ${up_revision}
 Revises: ${down_revision | comma,n}
 Create Date: ${create_date}
 
 """
+import logging
+
 import sqlalchemy as sa
 from alembic import op
 ${imports if imports else ""}
 # revision identifiers, used by Alembic.
-revision = "${up_revision}"
-down_revision = "${down_revision}"
-branch_labels = ${repr(branch_labels)}
-depends_on = ${repr(depends_on)}
+revision = ${up_revision | n,fmt}
+down_revision = ${down_revision | n,fmt}
+branch_labels = ${branch_labels | n,fmt}
+depends_on = ${depends_on | n,fmt}
+
+# Logger name should start with lsst to work with butler logging option.
+_LOG = logging.getLogger(f"lsst.{__name__}")
 
 
 def upgrade() -> None:
+    """Perform schema upgrade."""
     ${upgrades if upgrades else "raise NotImplementedError()"}
 
 
 def downgrade() -> None:
+    """Perform schema downgrade."""
     ${downgrades if downgrades else "raise NotImplementedError()"}