Test migration from Fedora 4 to 6

[cjcolvar]

Testing will be done in #5357 with a new empty Avalon on Fedora 6. Just as important if not more so will be to see if Avalon can work with data migrated from Fedora 4.

Done Looks Like

• Fedora's tooling used to migrate a small collection like IJCCR or Archivo to Fedora 6
• Spot check and manually test Avalon with migrated Fedora 6
• Notes taken and issues written

[cjcolvar]

Tried walking through the migration instructions inside the avalon_archivo-stage container:

#Install java on archivo-staging container: 
apt-get install openjdk-8-jre
#Install newer java: 
curl -O -L https://download.java.net/java/GA/jdk21/fd2272bbf8e04c3dbaee13770090416c/35/GPL/openjdk-21_linux-x64_bin.tar.gz
tar xvzf openjdk-21_linux-x64_bin.tar.gz
#Downloaded fedora import/export tool (recommended 0.3.0 version):
curl -O -L https://github.com/fcrepo-exts/fcrepo-import-export/releases/download/fcrepo-import-export-0.3.0/fcrepo-import-export-0.3.0.jar
#Run export tool: 
jdk-21/bin/java -jar fcrepo-import-export-0.3.0.jar -b --dir archivo_fcrepo4.7.5_export --user fedoraAdmin:fedoraAdmin --mode export --resource http://localhost:8080/fedora/rest --binaries --versions
#INFO 15:21:34.345 (Exporter) Running exporter...
#...
#INFO 15:23:43.351 (Exporter) Exported http://localhost:8080/fedora/rest/09/3e/dc/e3/093edce3-a094-4c46-b066-5d91fd06807f to /home/app/tmp/export/archivo_fcrepo4.7.5_export/fedora/rest/09/3e/dc/e3/093edce3-a094-4c46-b066-5d91fd06807f.ttl
#Did this actually finish?
#Download migration tool: 
curl -v -O -L https://github.com/fcrepo-exts/fcrepo-upgrade-utils/releases/download/fcrepo-upgrade-utils-6.2.0/fcrepo-upgrade-utils-6.2.0.jar
#Run migration to fedora 5.x: 
jdk-21/bin/java -jar fcrepo-upgrade-utils-6.2.0.jar --input-dir archivo_fcrepo4.7.5_export --output-dir archivo_fcrepo5_export --source-version 4.7.5 --target-version 5+
#INFO 15:22:27.786 (F47ToF5UpgradeManager) Processing directory: /home/app/tmp/export/archivo_fcrepo4.7.5_export
#java.lang.RuntimeException: java.nio.file.NoSuchFileException: archivo_fcrepo4.7.5_export/fedora/rest/af/21/ab/71/af21ab71-c690-4c3b-a608-a76dd1529044
	at org.fcrepo.upgrade.utils.F47ToF5UpgradeManager.convertAcl(F47ToF5UpgradeManager.java:372)
#...
#Run migration from fedora 5.x to 6: 
jdk-21/bin/java --add-opens java.base/java.util.concurrent=ALL-UNNAMED -jar fcrepo-upgrade-utils-6.2.0.jar --input-dir archivo_fcrepo5_export --output-dir archivo_fcrepo6_export --source-version 5+ --target-version 6+ --base-uri http://localhost:8080/fedora/rest
#INFO 15:43:18.988 (F5ToF6UpgradeManager) Starting upgrade: config=Config{sourceVersion=V_5, targetVersion=V_6, inputDir=archivo_fcrepo5_export, outputDir=archivo_fcrepo6_export, srcRdfLang=Lang:Turtle, baseUri='http://localhost:8080/fedora/rest', threads=4, digestAlgorithm='null', fedoraUser='fedoraAdmin', fedoraUserAddress='info:fedora/fedoraAdmin', forceWindowsMode=false, writeToS3=false, s3Region='null', s3Endpoint='null', s3PathStyleAccess=false, s3Bucket=null, s3Prefix=null, resourceInfoFile=null}
#...
#...
#INFO 15:43:23.722 (ResourceMigrator) Failed to migration resource info:fedora/76/53/71/40/76537140z. Rolling back...	#ERROR 15:43:23.724 (MigrateResourceTask) Failed to process ResourceInfo{parentId='info:fedora', fullId='info:fedora/76/53/71/40/76537140z', nameEncoded='76537140z', outerDirectory=archivo_fcrepo5_export/fedora/rest/76/53/71/40, innerDirectory=archivo_fcrepo5_export/fedora/rest/76/53/71/40/76537140z, type=CONTAINER}
#java.lang.IllegalStateException: Failed to identify interaction model for resource info:fedora/76/53/71/40/76537140z
#...
#INFO 15:43:28.850 (F5ToF6UpgradeManager) Upgrade complete.

[cjcolvar]

I investigated the error further and found that the migration tool was looking for directories that didn't exist. It appears they weren't created for ACL nodes which didn't have any children. I reran the export and migration steps but after exporting ran a set of shell commands to create directories ACL nodes:

grep -r "ns002:accessControl" archivo_fcrepo4.7.5_export_backup/ | cut -d '<' -f 2 | cut -d '>' -f 1 | sort | uniq > 4.7.5_acl_ids
cat 4.7.5_acl_ids | cut -d '/' -f 6- > 4.7.5_acl_id_paths
for path in $(cat 4.7.5_acl_id_paths)
do
  mkdir "archivo_fcrepo4.7.5_export_backup/fedora/rest/${path}"
done

This allowed the migrations to run successfully!
I created an issue for this in the migration tool repo: fcrepo-exts/fcrepo-upgrade-utils#55