Skip to content

Commit

Permalink
Merge pull request #442 from orichters/conv2005-17
Browse files Browse the repository at this point in the history
update tutorial
  • Loading branch information
orichters authored Feb 4, 2025
2 parents 55b5ebc + 4846710 commit f9b40f1
Show file tree
Hide file tree
Showing 3 changed files with 29 additions and 18 deletions.
1 change: 1 addition & 0 deletions .gitattributes
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
* text eol=lf
23 changes: 14 additions & 9 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ To improve the tutorial, edit [`tutorial.md`](./tutorial.md).
## Mappings

Mappings found in
[the `inst/mappings` folder](https://github.com/pik-piam/piamInterfaces/tree/master/inst/mappings)
the [`inst/mappings`](https://github.com/pik-piam/piamInterfaces/tree/master/inst/mappings) folder
serve to map variables from the PIAM framework to variables needed for the submission to databases.
The mappings are `;`-separated files, using `#` as comment character, with the following mandatory columns:

Expand Down Expand Up @@ -48,22 +48,27 @@ Additionally, some mappings use those columns:

### Editing a mapping

To edit a mapping in `R`, use:
You can use a text editor, `R` or a spreadsheet editor to adjust the mappings.

For `R`, use:
```
mappingdata <- getMapping("AR6")
...
... whatever data.frame operations you want ...
write.table(mappingdata, "inst/mappings/test_mapping.csv", na = "", dec = ".", sep = ";", row.names = FALSE, quote = FALSE)
```

Opening the csv files in Excel can be problematic, as it sometimes changes values and quotation marks.
You can edit the files in LibreOffice Calc using these settings in the Text Import dialog:
- Text Import with:
Using `Excel` can be problematic, as it sometimes changes values and quotation marks.

For `LibreOffice Calc`, use these settings:
```
Text Import:
- Character set: Unicode (UTF-8)
- Separated by: Semicolon.
- Save with:
Save -> Edit filter settings:
- Character set: Unicode (UTF-8)
- Field Delimiter: ;
- String Delimiter: (none)
```

The github diff on a large semicolon-separated file is often unreadable.
For a human-readable output, save the old version of the mapping and run:
Expand All @@ -74,15 +79,15 @@ On the PIK cluster, you can run `comparescenconf mapping_AR6.csv` in the `inst/m

After you adjusted the mapping, run `make test` on Linux (incl. the cluster) or `Rscript -e 'devtools::test(show_report = TRUE)'` on Windows, and various tests will be performed.

### Renaming a piam_variable
#### Renaming a piam_variable

If a variable used as `piam_variable` has to be renamed, please add it with its `old_name` to [`inst/renamed_piam_variables.csv`](./inst/renamed_piam_variables.csv).
Like this, if someone arrives with a dataset that contains the old name but not the new, [`renameOldVariables()`](./R/renameOldVariables.R) makes sure the data is automatically adjusted.
[`test-renameOldVariables.R`](./tests/testthat/test-renameOldVariables.R) enforces that the old variable name is not used anywhere in the mappings.
To adjust the mappings automatically, make sure you commit the current state to be able to reset its results, and then run `Rscript -e "devtools::load_all(); renameOldInMappings()"`.
Check the `diff` carefully, for example using `comparescenconf`, see above.

### piam_factor and unit checks
#### piam_factor and unit checks

While running the tests, an extensive check of the compatibility of `piam_unit`, `unit` and `piam_factor` is performed in each mapping.
It helps to find mismatches, for example mapping `Mt` to `Gt` with a factor of `1` or mapping `US$2005` to `US$2017` without accounting for inflation.
Expand Down
23 changes: 14 additions & 9 deletions tutorial.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ To improve the tutorial, edit [`tutorial.md`](./tutorial.md).
## Mappings

Mappings found in
[the `inst/mappings` folder](https://github.com/pik-piam/piamInterfaces/tree/master/inst/mappings)
the [`inst/mappings`](https://github.com/pik-piam/piamInterfaces/tree/master/inst/mappings) folder
serve to map variables from the PIAM framework to variables needed for the submission to databases.
The mappings are `;`-separated files, using `#` as comment character, with the following mandatory columns:

Expand Down Expand Up @@ -39,22 +39,27 @@ Additionally, some mappings use those columns:

### Editing a mapping

To edit a mapping in `R`, use:
You can use a text editor, `R` or a spreadsheet editor to adjust the mappings.

For `R`, use:
```
mappingdata <- getMapping("AR6")
...
... whatever data.frame operations you want ...
write.table(mappingdata, "inst/mappings/test_mapping.csv", na = "", dec = ".", sep = ";", row.names = FALSE, quote = FALSE)
```

Opening the csv files in Excel can be problematic, as it sometimes changes values and quotation marks.
You can edit the files in LibreOffice Calc using these settings in the Text Import dialog:
- Text Import with:
Using `Excel` can be problematic, as it sometimes changes values and quotation marks.

For `LibreOffice Calc`, use these settings:
```
Text Import:
- Character set: Unicode (UTF-8)
- Separated by: Semicolon.
- Save with:
Save -> Edit filter settings:
- Character set: Unicode (UTF-8)
- Field Delimiter: ;
- String Delimiter: (none)
```

The github diff on a large semicolon-separated file is often unreadable.
For a human-readable output, save the old version of the mapping and run:
Expand All @@ -65,15 +70,15 @@ On the PIK cluster, you can run `comparescenconf mapping_AR6.csv` in the `inst/m

After you adjusted the mapping, run `make test` on Linux (incl. the cluster) or `Rscript -e 'devtools::test(show_report = TRUE)'` on Windows, and various tests will be performed.

### Renaming a piam_variable
#### Renaming a piam_variable

If a variable used as `piam_variable` has to be renamed, please add it with its `old_name` to [`inst/renamed_piam_variables.csv`](./inst/renamed_piam_variables.csv).
Like this, if someone arrives with a dataset that contains the old name but not the new, [`renameOldVariables()`](./R/renameOldVariables.R) makes sure the data is automatically adjusted.
[`test-renameOldVariables.R`](./tests/testthat/test-renameOldVariables.R) enforces that the old variable name is not used anywhere in the mappings.
To adjust the mappings automatically, make sure you commit the current state to be able to reset its results, and then run `Rscript -e "devtools::load_all(); renameOldInMappings()"`.
Check the `diff` carefully, for example using `comparescenconf`, see above.

### piam_factor and unit checks
#### piam_factor and unit checks

While running the tests, an extensive check of the compatibility of `piam_unit`, `unit` and `piam_factor` is performed in each mapping.
It helps to find mismatches, for example mapping `Mt` to `Gt` with a factor of `1` or mapping `US$2005` to `US$2017` without accounting for inflation.
Expand Down

0 comments on commit f9b40f1

Please sign in to comment.