Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added legacy formatting during export from main to exploitation db #303

Merged
merged 5 commits into from
Nov 27, 2023

Conversation

antoineludeau
Copy link
Member

@antoineludeau antoineludeau commented Oct 11, 2023

WARNING : need to install postgis in production before deploying this PR.

Context :

When BAL data is processed through ID-Fix and new BAN-ID APIs, data is stored in a postgreSQL DB.
For now, the data used by our users is stored in the legacy Mongo DB. In order to be able to use the data that has been processed by new APIs, we need to :

  • export the data from postgreSQL DB
  • format the data from our new data schema to the legacy data schema
  • import the data into MongoDB

Enhancements :

This PR aims to :

  • format the data from our new data schema (documentation) to the legacy data format
  • import the data into mongoDB temporary collections* : communes_legacy, voies_legacy, numeros_legacy

*For now, we use temporary collections because we are still duplicating the data. It is still a two ways architecture : the data is processed by id-fix and le legacy compose.
The last step would be to dissociate this two ways architecure into a one way architecture and finally import the formatted data into the legacy collections.

@antoineludeau antoineludeau marked this pull request as draft October 11, 2023 09:41
@antoineludeau antoineludeau self-assigned this Oct 11, 2023
@antoineludeau antoineludeau force-pushed the antoineludeau/export-postgres-to-legacy-format branch 4 times, most recently from 19df729 to 8d98792 Compare October 25, 2023 14:12
@antoineludeau antoineludeau marked this pull request as ready for review October 25, 2023 14:32
@antoineludeau antoineludeau force-pushed the antoineludeau/export-postgres-to-legacy-format branch 4 times, most recently from 392221f to 83fb7c2 Compare October 27, 2023 13:28
@antoineludeau antoineludeau force-pushed the antoineludeau/export-postgres-to-legacy-format branch from 83fb7c2 to e8a3efd Compare November 6, 2023 16:15
Copy link
Contributor

@mmortier mmortier left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Testé en détail le fonctionnement pour moi c'est bon. A noté un cas particulier. Voir dans un autre ticket car il va chercher le fantoir dans la commune ancienne puis dans la commune actuelle. Dans le code legacy on avait ensuite un filtrage... ici on n'a pas rajouté de filtrage on devrait limiter la recherche à l'ancienne commune si elle est déclarée.

idVoie | idVoieFantoir | nbNumeros | nomVoie
49244_C282 | 49244_C282 | 2 | L'Orée du Bois (St Florent-le-Vieil)
49244_C282 | 49244_C282 | 1 | L'Orée du Bois (La Pommeraye)
49244_C282 | 49244_C282 | 1 | Orée du Bois (St Laurent-de-la-Plaine)

idVoie idVoieFantoir nbNumeros nomVoie
49244_26zn3x null 1 Orée du Bois (St Laurent-de-la-Plaine)
49244_C282 49244_C282 1 L'Orée du Bois (La Pommeraye)
49244_zn02j8 null 2 L'Orée du Bois (St Florent-le-Vieil)

@antoineludeau antoineludeau force-pushed the antoineludeau/export-postgres-to-legacy-format branch 2 times, most recently from 07bc99c to 5292f6f Compare November 16, 2023 09:34
@antoineludeau antoineludeau force-pushed the antoineludeau/export-postgres-to-legacy-format branch from 5292f6f to 10c1476 Compare November 16, 2023 12:43
lib/api/helper.js Outdated Show resolved Hide resolved
@antoineludeau antoineludeau force-pushed the antoineludeau/export-postgres-to-legacy-format branch from f793d11 to 05db529 Compare November 21, 2023 13:55
@antoineludeau antoineludeau force-pushed the antoineludeau/export-postgres-to-legacy-format branch from 05ce658 to cc767c6 Compare November 24, 2023 16:03
@antoineludeau antoineludeau merged commit e5fd0bc into master Nov 27, 2023
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
3 participants