Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Mataro plans pipeline #79

Open
wants to merge 17 commits into
base: master
Choose a base branch
from
Open

Mataro plans pipeline #79

wants to merge 17 commits into from

Conversation

entantoencuanto
Copy link
Member

@entantoencuanto entantoencuanto commented Sep 25, 2023 •
edited
Loading

Closes PopulateTools/issues#1854

This PR adds 2 operations to import Mataró plans:

  • The transformation of the original JSON data to a JSON prepared to be processed by the gobierto plans API. The operation filters the items which has to be transformed and performs the transformations. The resulting JSON is prepared to update existing projects by their external id or create new ones if required
  • The upsert operation which takes the JSON created in the previous operation and sends it to the API
    The PR also adds a Jenkinsfile pipeline to complete the ETL

ferblape
ferblape approved these changes Sep 25, 2023
View reviewed changes
Copy link
Member

@ferblape ferblape left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just small details, the rest is ok

@entantoencuanto
Fix mataro id using environment variable
e495f7b
@entantoencuanto
Fix cache namespace
f09a88e
@entantoencuanto
Particularize transform operation depending on source data
33b56f0
@entantoencuanto
Add urban agenda plan pipeline
0a64fd7
@entantoencuanto
Fix Jenkinsfile syntax
3ebee0b
@entantoencuanto
Include a vocabulary custom field attribute in PAM transform operation
622687a
@entantoencuanto
Update import data of PAM
c507d8b
@entantoencuanto
Add custom field string type and improve getting value from source fo…
8c858cd 
…r vocabularies type
@entantoencuanto
Include categories and statuses vocabularies in plans API update requ…
231f1f7 
…ests
@entantoencuanto
Update plans configurations to include custom fields and allow vocabu…
89673ee 
…laries generation
@entantoencuanto
Add description, dates, and paragraph custom fields for comments and …
d481a25 
…evolution in PAM projects
@entantoencuanto
Move actual start and end dates to native project dates fields
44279a9
@entantoencuanto
Change date format in text
c90629d
@entantoencuanto
Add custom fields to urban_agenda_2030 plan configuration
8a331af
@ferblape ferblape force-pushed the mataro_plans_pipeline branch from 9fe3bee to 8a331af Compare February 11, 2024 06:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@ferblape ferblape ferblape approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@entantoencuanto @ferblape