Updated API call for retrieving all bundle fqids of a project
Make AzulAgent handle 400 errors Switch to dcplib's IngestApiAgent and ingest_entities Fix file <-> validation relationship in entities
analyze-submission: update elasticsearch primary bundle search
analyze-submission:
- count extra bundles returned by a project UUID search in DSS
- sort bundle lists when printing
- save state after every phase
- save state on Ctrl-C
Fix Ingest authentication error
Dcpdig can show batch jobs and their logs: dcpdig @upload validation_id= --show batch_jobs,logs Dcpdig can show workflows: dcpdig @analysis bundle_uuid= --show bundles,projects Use Ingest's HTTPS API endpoints. Compatible with Upload database v4.1.0.
dcpdig: Allow project_uuid= --show submissions AND bundles
dcpdig: allow 'all' as an option for --show
dcpdig @upload [area|file_id]
analyze-submission: show accurate starting counts when we already have partial results Save all Ingest state in savefile, version the statefile schema.
Use bundle FQID instead of UUID everywhere we can
Now works in production environment too.
Fix load vs clean bug
analyze-submission:
- use --jobs to configure concurrency
- saves and reloads state automatically, use --clean to disable load
- verbosity: -v = bad news detail, -vv = good and bad news detail
scripts/dcpdig -d staging @ingest project_uuid= --show submissions scripts/analyze-submission -d <submission_id>
- checks bundle manifests are in AWS/GCP DSS
- searches DSS AWS & GCP for bundles by project_id
Refactor to make more extensible gitignore build and dist dirs fix error when no --show is supplied
UX cleanup, add search for submission uuid Add dcpdig usage notes to README
dcpdig @ingest bundle_uuid=x
dcpdig @ingest subm_uuid=x