WikibaseMigrator is a tool to migrate wikibase entities from one wikibase instance to another. During the migration process all used Item and Property IDs are mapped to the IDs from the target Wikibase instance.
pip install wikibasemigrator
Install WikibaseMigrator after cloning with
pip install .
for development with
pip install .[test]
- WikibaseMigrator offers a cli tool and a web UI.
- Migration is defined with a migration profile which needs to be defined. For details on the definition see Wikibase Migration Profile Configuration
docker run -p 8009:8080 -v ~/.config/WikibaseMigrator/profiles/WikibaseMigrationTest.yaml:/config.yaml tholzheim/wbmigrator
Exchange
~/.config/WikibaseMigrator/profiles/WikibaseMigrationTest.yaml
with the location of your migration profile configuration if needed
The webserver can be started over the cli with
wbmigrate webserver --config FactGrid.yaml --host localhost --port 9100
or with docker
After installing the project cli is available under the name ẁbmigrate
wbmigrate --help
Usage: wbmigrate [OPTIONS] COMMAND [ARGS]...
╭─ Options ──────────────────────────────────────────────────────────────────────────────────────────────────╮
│ --install-completion Install completion for the current shell. │
│ --show-completion Show completion for the current shell, to copy it or customize the │
│ installation. │
│ --help Show this message and exit. │
╰────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
╭─ Commands ─────────────────────────────────────────────────────────────────────────────────────────────────╮
│ app Run the WikibaseMigrator web server as local app Note: Experimental feature as some of the │
│ imported resources are not localized yet │
│ webserver Start the WikibaseMigrator web server │
│ migrate Migrate the provided entities │
╰────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
wbmigrate migrate --help
Usage: wbmigrate migrate [OPTIONS]
Migrate the provided entities
╭─ Options ──────────────────────────────────────────────────────────────────────────────────────────────────╮
│ * --config TEXT The configuration file defining the Wikibases │
│ [default: None] │
│ [required] │
│ * --summary TEXT Summary message to add to the wikibase edits │
│ [default: None] │
│ [required] │
│ --entity TEXT The items to migrate [default: None] │
│ --query TEXT The query querying the items to migrate. The items to │
│ migrate must have the binding ?items │
│ [default: None] │
│ --query-file TEXT The query file with a query querying the items to migrate. │
│ The items to migrate must have the binding ?item │
│ [default: None] │
│ --show-details --no-show-details Show detailed information during the migration process │
│ [default: no-show-details] │
│ --force --no-force If True migrate items directly to target wikibase │
│ [default: no-force] │
│ --help Show this message and exit. │
╰────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
wbmigrate webserver --help
Usage: wbmigrate webserver [OPTIONS]
Start the WikibaseMigrator web server
╭─ Options ──────────────────────────────────────────────────────────────────────────────────────────────────╮
│ * --config TEXT The configuration file defining the Wikibases [default: None] [required] │
│ --host TEXT host of the webserver [default: 0.0.0.0] │
│ --port INTEGER port of the webserver [default: 8080] │
│ --help Show this message and exit. │
╰────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
flowchart TD
source[(source Wikibase)]
target[(target Wikibase)]
config@{ shape: doc, label: "Mapping Profile" }
A@{ shape: circle, label: "Start" }
A-- provide --> B(Entity IDs)
B--start-->t
config-->t
q1<-.query.->target
q3<-.query.->target
subgraph Migrate
subgraph Translation
direction LR
t@{ shape: sm-circ, label: "Small start" }
t-->q1
q1[[query mappings]]
t-->q2[[query entity records]]
ser@{ shape: docs, label: "Source Entity Records"}
q2--result-->ser
map@{ shape: doc, label: "Mapping Table" }
q1--result-->map
translate[translate]
map-->translate
ser-->translate
translate-->TranslationResults@{ shape: docs, label: "Translated Entities"}
end
ex{exists<br>in target?}
TranslationResults --> ex
ex --noo-->TranslationResultsFinal
ex --yes-->m
TranslationResultsFinal@{ shape: docs, label: "Translated & Merged Entities"}
mergedEntities-->TranslationResultsFinal
subgraph merge
m@{ shape: sm-circ, label: "Small start" }
q3[[query entity records]]
m-->q3
targetEntities@{ shape: docs, label: "Target Entities"}
translatedEntities@{ shape: docs, label: "Translated Entities"}
q3--result-->targetEntities
m-->translatedEntities
merger[merge]
translatedEntities-->merger
targetEntities-->merger
mergedEntities@{ shape: docs, label: "Merged Entities"}
merger-->mergedEntities
end
migrator[migrate]
end
q2<-.query.->source
TranslationResultsFinal-->migrator
migrator --write--> target
stop@{ shape: framed-circle, label: "Stop" }
migrator-->stop
If an entity already exists in the target wikibase instance the source entity is first translated to the target and then merged with the target entity. The merging of entities differs from the default wikibase entity merging for cases where the statement does not have qualifiers. In this case if the statement value is equal the qualifiers of the source are merged into the target. Otherwise, the merge results in an additional statement to preserve both qualifier information (equal to the wikibase entity merge)
This project was funded by FactGrid
This repository is licensed under the Apache 2.0