Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[AUTO] Update data #151

Closed

Conversation

damianooldoni-bot
Copy link
Contributor

Brief description

This is an automatically generated PR.
The following steps are all automatically performed:

  • Fetch raw data
  • Map raw data to DwC standard and save the output in ./data/processed
  • Get an overview of the changes
  • Run some tests, e.g. check the uniqueness of occurrenceID, check that all occurrences have a eventID and scientificName, ...

Note to the reviewer: the workflow automation is still in a development phase. Please, check the output thoroughly before merging to main. In case, improve the data fecthing fetch_data.Rmd, the mapping dwc_mapping.Rmd, both in ./src or the GitHub workflows fetch-data.yaml and mapping_and_testing.yaml in ./.github/workflows.

Files changed:
M	data/processed/occurrence.csv
@PietrH
Copy link
Member

PietrH commented Jan 22, 2024

This PR only has 2000 lines in it's occurrence.csv, something clearly has gone wrong

@PietrH
Copy link
Member

PietrH commented Jan 22, 2024

Records with multiple values for Waarneming are currently removed until #23 is fixed, however, this is detected on a separator including a space. This separator seems to be present in a lot of records even if there aren't multiple values.

Remove occurrences containing multiple type - value pairs information in column waarneming (patch until #23 is solved):

#remove records with multiple values for waarneming
input_data %<>%
  filter(is.na(.data$waarneming) | 
           !str_detect(.data$waarneming, pattern = "; "))

@PietrH PietrH closed this Jan 22, 2024
@PietrH PietrH deleted the automatic-update-2024-01-22T14-49-46Z branch January 22, 2024 15:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants