Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Abstract dataset #125

Closed
wants to merge 44 commits into from
Closed

Abstract dataset #125

wants to merge 44 commits into from

Conversation

bpmeek
Copy link
Contributor

@bpmeek bpmeek commented Mar 9, 2023

Description

To allow implementations of AbstractDataSet and AbstractVersionedDataSet to be shared with and used by non-Kedro users. Kedro issue #2409

Development notes

There are four commits:

  1. patches necessary files from Kedro to kedro-datasets necessary to remove Kedro as a dependency for installation while keeping history.
  2. changes imports from kedro.io.core to kedro_datasets.io.core in both kedro_datasets/ and tests/
  3. changes requirements.txt to require cachetools~=5.3 instead of Kedro
  4. Updates Release notes.

After updating imports to import from kedro-datasets instead of kedro pytest executed with no failures.

Checklist

  • Opened this PR as a 'Draft Pull Request' if it is work-in-progress
  • Updated the documentation to reflect the code changes
  • Added a description of this change in the relevant RELEASE.md file
  • Added Updated tests to cover my changes

idanov and others added 30 commits May 16, 2019 16:54
* Reverting license to standard Apache 2.0

* Removing make legal

* Delete license_and_headers.py

* Removed all occurrence of the copyright header from codebase

* Removed copyrighted header from build-docs.sh

* Removing copyright footer from all docs

* Changes based on review

* removed whitespace

* cleanup
Lorena Bălan and others added 14 commits March 31, 2022 14:18
Merge develop into main in preparation for 0.18.0 release
* Replace back-ticks with single quotes

Signed-off-by: Ahdra Merali <[email protected]>

* Replace more backticks

Signed-off-by: Ahdra Merali <[email protected]>

* Linting

Signed-off-by: Ahdra Merali <[email protected]>

* Update kedro/extras/datasets/pandas/gbq_dataset.py

Co-authored-by: Antony Milne <[email protected]>

* Update kedro/framework/cli/project.py

Co-authored-by: Antony Milne <[email protected]>

* Fix tests pt 1

Signed-off-by: Ahdra Merali <[email protected]>

* Fix tests pt 2

Signed-off-by: Ahdra Merali <[email protected]>

* Fix tests pt 3

Signed-off-by: Ahdra Merali <[email protected]>

* Change quotes to follow convention

Co-authored-by: Antony Milne <[email protected]>

* Change quotes to follow convention pt 2

Co-authored-by: Antony Milne <[email protected]>

* Fix tests

Signed-off-by: Ahdra Merali <[email protected]>

* Fix e2e tests

Signed-off-by: Ahdra Merali <[email protected]>

Co-authored-by: Antony Milne <[email protected]>
…related issues better (#1881)

* Update message for VersionNotFoundError

Signed-off-by: Ankita Katiyar <[email protected]>

* Add test for VersionNotFoundError for cloud protocols

* Update test_data_catalog.py

Update NoVersionFoundError test

* minor linting update

* update docs link + styling changes

* Revert "update docs link + styling changes"

This reverts commit 6088e00159a9ee844dfee312673654b6d248f931.

* Update test with styling changes

* Update RELEASE.md

Signed-off-by: ankatiyar <[email protected]>

Signed-off-by: Ankita Katiyar <[email protected]>
Signed-off-by: ankatiyar <[email protected]>
Co-authored-by: Ahdra Merali <[email protected]>
* remove a redundant function call

Signed-off-by: Nok Chan <[email protected]>

* Remove redundant resolove_load_version & fix test

Signed-off-by: Nok Chan <[email protected]>

* Fix HoloviewWriter tests with more specific error message pattern & Lint

Signed-off-by: Nok Chan <[email protected]>

* Rename tests

Signed-off-by: Nok Chan <[email protected]>

Signed-off-by: Nok Chan <[email protected]>
…ity than `kedro.extras.datasets` (#1734)

* Update release doc

Signed-off-by: Nok Chan <[email protected]>

* Make kedro.datasets with higher priority

Signed-off-by: Nok Chan <[email protected]>

* Update the library priorities

Signed-off-by: Nok Chan <[email protected]>

* Update release notes

Signed-off-by: Nok Chan <[email protected]>

* Add test

Signed-off-by: Nok Chan <[email protected]>

* Modify test to remove the dummy obj

Signed-off-by: Nok Chan <[email protected]>

* Fix mocker.spy with different API for Python 3.7

Signed-off-by: Nok Chan <[email protected]>

* Please the linter

Signed-off-by: Nok Chan <[email protected]>

* Black

Signed-off-by: Nok Chan <[email protected]>

Signed-off-by: Nok Chan <[email protected]>
* Add failing test

Signed-off-by: Jannic Holzer <[email protected]>

* Add patch specific for abfss

Signed-off-by: Jannic Holzer <[email protected]>

* Fix linting

Signed-off-by: Jannic Holzer <[email protected]>

* Add release note

Signed-off-by: Jannic Holzer <[email protected]>

* Add comment explaining adding username to abfss path

Signed-off-by: Jannic Holzer <[email protected]>

* Modify release note to 'fixed'

Signed-off-by: Jannic Holzer <[email protected]>

Signed-off-by: Jannic Holzer <[email protected]>
@bpmeek
Copy link
Contributor Author

bpmeek commented Mar 9, 2023

Not sure what to do about the DCO Action required, I can't sign off on those commits as I'm obviously not the Author of those commits.

@ankatiyar
Copy link
Contributor

Thank you for your efforts @bpmeek! Closing this PR following the discussion on kedro-org/kedro#2409 :)

@ankatiyar ankatiyar closed this Jul 13, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.