Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

NR-347971 | OpenLineage event consumer writer #1887

Open
wants to merge 39 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 8 commits
Commits
Show all changes
39 commits
Select commit Hold shift + click to select a range
e5ee948
OLIN agent collection
devendra-nr Jan 21, 2025
234692f
Create DPO collections
devendra-nr Jan 21, 2025
03ad233
NR-347971 OLIN agent collection
devendra-nr Jan 21, 2025
8cf86d2
NR-347971 | Added identifiers and tags
devendra-nr Jan 30, 2025
9680c79
Merge branch 'main' into NR-347971-OLIN-event-consumer-writer
devendra-nr Jan 30, 2025
19db55c
Merge branch 'main' into NR-347971-OLIN-event-consumer-writer
devendra-nr Feb 2, 2025
b872e62
NR=347971 Update name as per OLIN spec
devendra-nr Feb 2, 2025
d654b44
NR-347971 Revert file change
devendra-nr Feb 2, 2025
744128d
NR-347971 | Use unique tag names
devendra-nr Feb 6, 2025
addf09f
Merge branch 'newrelic:main' into NR-347971-OLIN-event-consumer-writer
devendra-nr Feb 6, 2025
2ed1250
NR-347971 | Added conditions
devendra-nr Feb 7, 2025
a2677a6
NR-347971 | Added conditions
devendra-nr Feb 7, 2025
ca95fe5
Merge branch 'newrelic:main' into NR-347971-OLIN-event-consumer-writer
devendra-nr Feb 11, 2025
9dbe0dc
NR-347871 | Remove redundant line
devendra-nr Feb 12, 2025
16814aa
Merge pull request #2 from devendra-nr/NR-347971-add-condition
devendra-nr Feb 12, 2025
ca9750d
Merge branch 'main' into NR-347971-OLIN-event-consumer-writer
devendra-nr Feb 13, 2025
1197390
Merge branch 'main' into NR-347971-OLIN-event-consumer-writer
devendra-nr Feb 13, 2025
fc6f648
Merge branch 'main' into NR-347971-OLIN-event-consumer-writer
devendra-nr Feb 17, 2025
ed516e4
Merge branch 'main' into NR-347971-OLIN-event-consumer-writer
devendra-nr Feb 17, 2025
f69d9b6
Merge branch 'main' into NR-347971-OLIN-event-consumer-writer
devendra-nr Feb 18, 2025
5caa6c4
Added test case
devendra-nr Feb 18, 2025
a2624f2
Added more entity conditions
devendra-nr Feb 18, 2025
fba7ca4
Update test data
devendra-nr Feb 18, 2025
a3f0e5d
Merge branch 'main' into NR-347971-add-condition
devendra-nr Feb 18, 2025
1e6be59
Added entity type to test data
devendra-nr Feb 18, 2025
97b0286
Merge branch 'NR-347971-add-condition' of https://github.com/devendra…
devendra-nr Feb 18, 2025
e081da9
Merge branch 'main' into NR-347971-OLIN-event-consumer-writer
devendra-nr Feb 18, 2025
6f28171
Merge branch 'NR-347971-OLIN-event-consumer-writer' into NR-347971-ad…
devendra-nr Feb 18, 2025
56e12c8
Merge pull request #4 from devendra-nr/NR-347971-add-condition
devendra-nr Feb 18, 2025
e269c47
Update testcase
devendra-nr Feb 18, 2025
03640a6
Merge branch 'NR-347971-add-condition' of https://github.com/devendra…
devendra-nr Feb 18, 2025
0f2b23a
Merge pull request #5 from devendra-nr/NR-347971-add-condition
devendra-nr Feb 18, 2025
2604e18
Update test
devendra-nr Feb 18, 2025
910ded8
Merge pull request #6 from devendra-nr/NR-347971-add-condition
devendra-nr Feb 18, 2025
07cebf9
Update type
devendra-nr Feb 21, 2025
b83e0a2
Merge pull request #7 from devendra-nr/NR-347971-add-condition
devendra-nr Feb 21, 2025
644b20a
Update type
devendra-nr Feb 21, 2025
ad540bc
Merge pull request #8 from devendra-nr/NR-347971-add-condition
devendra-nr Feb 21, 2025
55a3573
Merge branch 'main' into NR-347971-OLIN-event-consumer-writer
devendra-nr Feb 21, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions entity-types/agent-openlineage/definition.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
domain: AGENT
type: OPENLINEAGE
configuration:
entityExpirationTime: MANUAL
alertable: false
21 changes: 21 additions & 0 deletions entity-types/dpo-dataset_versions/definition.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
domain: DPO
type: DATASET_VERSIONS
synthesis:
rules:
- compositeIdentifier:
separator: ":"
attributes:
- dataset.namespace
- dataset.name
- updatedAt
name: dataset.name
encodeIdentifierInGUID: true
conditions:
- attribute: dataset.name
tags:
dataset.namespace:
dataset.name:
dataset.updatedAt:
configuration:
entityExpirationTime: MANUAL
alertable: true
19 changes: 19 additions & 0 deletions entity-types/dpo-datasets/definition.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
domain: DPO
type: DATASETS
synthesis:
rules:
- compositeIdentifier:
separator: ":"
attributes:
- dataset.namespace
- dataset.name
name: dataset.name
encodeIdentifierInGUID: true
conditions:
- attribute: dataset.name
tags:
dataset.namespace:
dataset.name:
configuration:
entityExpirationTime: MANUAL
alertable: true
22 changes: 22 additions & 0 deletions entity-types/dpo-job_versions/definition.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
domain: DPO
type: JOB_VERSIONS
synthesis:
rules:
- compositeIdentifier:
separator: ":"
attributes:
- job.namespace
- job.name
- updatedAt
name: job.name
encodeIdentifierInGUID: true
conditions:
- attribute: job.name
tags:
job.namespace:
job.name:
job.updatedAt:
pipeline.name:
configuration:
entityExpirationTime: MANUAL
alertable: true
20 changes: 20 additions & 0 deletions entity-types/dpo-jobs/definition.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
domain: DPO
type: JOBS
synthesis:
rules:
- compositeIdentifier:
separator: ":"
attributes:
- job.namespace
- job.name
name: job.name
encodeIdentifierInGUID: true
conditions:
- attribute: job.name
tags:
job.namespace:
job.name:
pipeline.name:
configuration:
entityExpirationTime: MANUAL
alertable: true
19 changes: 19 additions & 0 deletions entity-types/dpo-pipelines/definition.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
domain: DPO
type: PIPELINES
synthesis:
rules:
- compositeIdentifier:
separator: ":"
attributes:
- pipeline.namespace
- pipeline.name
name: pipeline.name
encodeIdentifierInGUID: true
conditions:
- attribute: pipeline.name
tags:
pipeline.namespace:
pipeline.name:
configuration:
entityExpirationTime: MANUAL
alertable: true