Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Document current & desired ETL steps and data flow #4455

Closed
AetherUnbound opened this issue Jun 6, 2024 · 0 comments · Fixed by #4465
Closed

Document current & desired ETL steps and data flow #4455

AetherUnbound opened this issue Jun 6, 2024 · 0 comments · Fixed by #4465
Assignees
@AetherUnbound
Labels
📄 aspect: text Concerns the textual material in the repository 🌟 goal: addition Addition of new feature 🟨 priority: medium Not blocking but should be addressed soon 🧱 stack: documentation Related to Sphinx documentation

Comments

@AetherUnbound
Copy link
Collaborator

Description

Recent discussions about the projects #430, #431, and #3925 have brought up some interesting questions about where in our general data flow certain operations are occurring. We have many different steps in our data pipeline where various ETL steps are occurring, and it would be useful to enumerate the current state and operations. We should also describe a desired state that will be the result of the completion of the three aforementioned projects.

Additional context

https://make.wordpress.org/openverse/2024/05/20/openverse-monthly-priorities-meeting-2024-06-05/#comment-282
@AetherUnbound AetherUnbound added 🌟 goal: addition Addition of new feature 📄 aspect: text Concerns the textual material in the repository 🟨 priority: medium Not blocking but should be addressed soon 🧱 stack: documentation Related to Sphinx documentation labels Jun 6, 2024
@AetherUnbound AetherUnbound self-assigned this Jun 6, 2024
This was referenced Jun 7, 2024
Merged
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@AetherUnbound AetherUnbound

Labels
📄 aspect: text Concerns the textual material in the repository 🌟 goal: addition Addition of new feature 🟨 priority: medium Not blocking but should be addressed soon 🧱 stack: documentation Related to Sphinx documentation
Projects
Openverse Backlog
Archived in project
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Add data flow diagram for various ETL steps in pipelines WordPress/openverse
1 participant
@AetherUnbound