Fix/obs unit db ids #72
Open
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…ationUnit documents (still observationVariableDbId). fix GNP-6529
…ing a stack for handling data structures values. fix GNP-6529
…api-etl-faidare into fix/obsUnit-DbIds
removing extra deprecated modules and files. Pipfile is now the reference, no more Requirements.txt
Feat/upgrade to python3.12
Feat/upgrade to python3.12
…to expected output. GNP-6543
Add the make_hashable method to resolve unhashable type error
cpommier
reviewed
Nov 8, 2024
| if document_type != "observationVariable": | ||
| document[document_type + 'DbId'] = get_generated_uri_from_dict(source, document_type, document, True) | ||
| # transform other DbIds , skip observationVariable | ||
| # create a stack for the current document | ||
| stackDocument = [document] |
There was a problem hiding this comment.
stackDocument is slightly misleading. Rename for isntance to "stackDocumentFields" or "stackDocumentProperties"
| True) | ||
| while stackDocument: | ||
| # Retrieve the top item of the stack | ||
| currentItem = stackDocument.pop() |
There was a problem hiding this comment.
Same: currentDocumentItem or currentDocumentField
| currentItem = stackDocument.pop() | ||
|
|
||
| if isinstance(currentItem, dict): | ||
| for key, value in list(currentItem.items()): |
There was a problem hiding this comment.
Same: replace key with "fieldKey" or "fieldName" or something like that
replace value with "fieldValue" or something like that
| @@ -2,8 +2,16 @@ | |||
| from etl.transform.utils import get_generated_uri_from_str | |||
|
|
|||
|
|
|||
| def make_hashable(obj): | |||
There was a problem hiding this comment.
Add a comment explaining the origin of this method: synonyms were either list(string) or list(obj) which generate hash error. This methods solv the problem but the real problem was the data model that had changed.
| @@ -51,11 +48,11 @@ def handler(*_): | |||
| if 'transform_rdf' in options or 'etl_virtuoso' in options: | |||
| etl.transform.rdf.main(config) | |||
|
|
|||
| if 'load_elasticsearch' in options or 'etl_es' in options: | |||
| etl.load.elasticsearch.main(config) | |||
| #if 'load_elasticsearch' in options or 'etl_es' in options: | |||
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.