Skip to content

Commit

Permalink
docs(csv-enricher): Add warning for write_semantics (datahub-project#…
Browse files Browse the repository at this point in the history
  • Loading branch information
eboneil authored Oct 9, 2024
1 parent 732543f commit ef4805d
Show file tree
Hide file tree
Showing 2 changed files with 5 additions and 1 deletion.
Original file line number Diff line number Diff line change
Expand Up @@ -129,6 +129,10 @@ class CSVEnricherSource(Source):
If ownership_type_urn is set then ownership_type must be set to CUSTOM.
Note that you have the option in your recipe config to write as a PATCH or as an OVERRIDE. This choice will apply to
all metadata for the entity, not just a single aspect. So OVERRIDE will override all metadata, including performing
deletes if a metadata field is empty. The default is PATCH.
:::note
This source will not work on very large csv files that do not fit in memory.
:::
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ class CSVEnricherConfig(ConfigModel):
)
write_semantics: str = pydantic.Field(
default="PATCH",
description='Whether the new tags, terms and owners to be added will override the existing ones added only by this source or not. Value for this config can be "PATCH" or "OVERRIDE"',
description='Whether the new tags, terms and owners to be added will override the existing ones added only by this source or not. Value for this config can be "PATCH" or "OVERRIDE". NOTE: this will apply to all metadata for the entity, not just a single aspect.',
)
delimiter: str = pydantic.Field(
default=",", description="Delimiter to use when parsing CSV"
Expand Down

0 comments on commit ef4805d

Please sign in to comment.