Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update on credentials.md #2787

Merged
merged 12 commits into from
Aug 26, 2023
Merged

Update on credentials.md #2787

merged 12 commits into from
Aug 26, 2023

Conversation

jmnunezd
Copy link
Contributor

@jmnunezd jmnunezd commented Jul 11, 2023

Updating example code in credentials.md to make it usable when copy/pasting. Since as of now it will generate a TypeError

NOTE: Kedro datasets are moving from kedro.extras.datasets to a separate kedro-datasets package in
kedro-plugins repository. Any changes to the dataset implementations
should be done by opening a pull request in that repository.

Description

I was reading the docs on how to load credentials directly in code, I tried using the copy button and paste it in my local to test it.

Screenshot 2023-07-11 at 2 40 36 PM

The problem is that even if you change project_path you will still get an TypeError since Python won't allow to use "/" between two strings.

For example:
conf_path = str(project_path / settings.CONF_SOURCE) changing project_path:
conf_path = str("my_project_path" / "conf") <----- Will raise TypeError

Specifically:
TypeError: unsupported operand type(s) for /: 'str' and 'str'

Development notes

I changed the credentials.md from:

conf_path = str(project_path / settings.CONF_SOURCE)

To:
conf_path = f"{project_path}/{settings.CONF_SOURCE}"

Checklist

  • Read the contributing guidelines
  • [N/A] Opened this PR as a 'Draft Pull Request' if it is work-in-progress
  • [N/A] Updated the documentation to reflect the code changes
  • [N/A] Added a description of this change in the RELEASE.md file
  • [N/A] Added tests to cover my changes

Updating example code in credentials.md to make it usable when copy/pasting. Since as of now it will generate a TypeError

Signed-off-by: Jose <[email protected]>
@ankatiyar ankatiyar added the Community Issue/PR opened by the open-source community label Jul 13, 2023
…at project_path is a pathlib.Path object

Signed-off-by: Jose <[email protected]>
@jmnunezd jmnunezd requested a review from astrojuanlu August 1, 2023 18:33
from kedro.config import ConfigLoader
from kedro.framework.project import settings

conf_path = str(project_path / settings.CONF_SOURCE)
conf_path = str(Path(project_path) / settings.CONF_SOURCE)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

project_path needs to be assigned

@jmnunezd jmnunezd requested a review from astrojuanlu August 3, 2023 17:31
Copy link
Member

@astrojuanlu astrojuanlu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The code snippet still fails because project_path is not defined, right?

@jmnunezd
Copy link
Contributor Author

jmnunezd commented Aug 7, 2023

The code snippet still fails because project_path is not defined, right?

That's right! but again, I think the user should be able to manually give the path they want to use and assign it to project_path, does that makes sense? what would be the alternative?

@noklam
Copy link
Contributor

noklam commented Aug 7, 2023

@jmnunezd Late to this conversation - Maybe we should just keep it consistent with <project_root> to indicate that this is something user need to define themselve?

@stichbury
Copy link
Contributor

It'd be great to get this in @jmnunezd as we are close to making a release and these changes have been on the cards a while. How can I help push it over the line? cc @astrojuanlu @noklam

@noklam
Copy link
Contributor

noklam commented Aug 23, 2023

@stichbury I think it's good to check our terminology, is it project_root or project_path? The code snippet here is not working because project_path does not exist. Would it be enough to change it to <project_root> to indicate that this is something user need to put in themselves since it is different for every case.

@stichbury
Copy link
Contributor

@stichbury I think it's good to check our terminology, is it project_root or project_path? The code snippet here is not working because project_path does not exist. Would it be enough to change it to <project_root> to indicate that this is something user need to put in themselves since it is different for every case.

Thanks @noklam yes I agree. We use this term reasonably consistently and define it here https://docs.kedro.org/en/stable/tutorial/spaceflights_tutorial.html#terminology so I think we should change it (and maybe put a comment in the code to explain?

@noklam noklam enabled auto-merge (squash) August 24, 2023 12:22
@astrojuanlu
Copy link
Member

Thanks @jmnunezd for your patience and for kicking this off!

@astrojuanlu
Copy link
Member

After merging this we should probably add a more practical example on how credentials are normally used, see also #2669 (and more broadly, #1646)

@noklam noklam merged commit 3f87895 into kedro-org:main Aug 26, 2023
ankatiyar added a commit that referenced this pull request Sep 12, 2023
* Remove non-user-facing release notes about pip pin (#2854)

Signed-off-by: Jason Hite <[email protected]>

* Allow for customization of pipeline templates

Controlled via `settings.PIPELINE_TEMPLATE_PATH`.

Signed-off-by: Jason Hite <[email protected]>

* Update lookup based on feedback

Instead of lookups on `settings.py` just statically look for a `templates`
directory in the project root. If this folder is present it will override
the default global template, otherwise fall back to the previous global
default.

Signed-off-by: Jason Hite <[email protected]>

* Add CLI flag for pipeline creation template -t/--template

Precedence for template_path is:
   command line > project templates/pipeline dir > global default

pipeline create now will also print the template folder used.

Signed-off-by: Jason Hite <[email protected]>

* First basic unit test

TODO is testing the CLI flag too, and maybe testing that the cleanup works
with custom templates.

Signed-off-by: Jason Hite <[email protected]>

* Add test to check overriding template path on the CLI

Signed-off-by: Jason Hite <[email protected]>

* Linter formatting fixes

Signed-off-by: Jason Hite <[email protected]>

* Release 0.18.12 (#2871)

* bump version and prepare for new release

Signed-off-by: SajidAlamQB <[email protected]>

* Update RELEASE.md

Added community contributions

* add DeltaTableDataSet to rst

Signed-off-by: SajidAlamQB <[email protected]>

* Update setup.py

Signed-off-by: SajidAlamQB <[email protected]>

* Update setup.py

Signed-off-by: SajidAlamQB <[email protected]>

* Update setup.py

Co-authored-by: Juan Luis Cano Rodríguez <[email protected]>

* temporarily use the fixed kedro-plugins branch

Signed-off-by: SajidAlamQB <[email protected]>

* Update setup.py

Signed-off-by: SajidAlamQB <[email protected]>

* test docsbuild on merels branch

Signed-off-by: SajidAlamQB <[email protected]>

* use kedro-datasets new release

Signed-off-by: SajidAlamQB <[email protected]>

---------

Signed-off-by: SajidAlamQB <[email protected]>
Co-authored-by: Jo Stichbury <[email protected]>
Co-authored-by: Juan Luis Cano Rodríguez <[email protected]>
Signed-off-by: Jason Hite <[email protected]>

* Fix broken link to Prefect website in deploy guide (#2885)

Signed-off-by: Jason Hite <[email protected]>

* Clarify the <micropkg_name> argument to kedro micropkg package (#2835)

Co-authored-by: Juan Luis Cano Rodríguez <[email protected]>
Co-authored-by: Merel Theisen <[email protected]>
Signed-off-by: Jason Hite <[email protected]>

* Allow registering of custom resolvers to `OmegaConfigLoader` (#2869)

* Allow registering of custom resolvers to OCL

Signed-off-by: Ankita Katiyar <[email protected]>

* Complete doc string

Signed-off-by: Ankita Katiyar <[email protected]>

* Add test for overwritten resolvers

Signed-off-by: Ankita Katiyar <[email protected]>

* Update test for overwritten resolvers

Signed-off-by: Ankita Katiyar <[email protected]>

* Remove replace=True by default

Signed-off-by: Ankita Katiyar <[email protected]>

* Update release notes

Signed-off-by: Ankita Katiyar <[email protected]>

* Update release notes

Signed-off-by: Ankita Katiyar <[email protected]>

* Add debug level log for registering new resolver

Signed-off-by: Ankita Katiyar <[email protected]>

---------

Signed-off-by: Ankita Katiyar <[email protected]>
Signed-off-by: Jason Hite <[email protected]>

* Document the use of custom resolvers with `OmegaConfigLoader` (#2896)

* Update docs for custom resolvers for OCL

Signed-off-by: Ankita Katiyar <[email protected]>

* Update FAQ with correct link

Signed-off-by: Ankita Katiyar <[email protected]>

* Fix typo

Co-authored-by: Merel Theisen <[email protected]>

* Add note for oc.env

Signed-off-by: Ankita Katiyar <[email protected]>

* Suggestions from code review

Signed-off-by: Ankita Katiyar <[email protected]>

---------

Signed-off-by: Ankita Katiyar <[email protected]>
Co-authored-by: Merel Theisen <[email protected]>
Signed-off-by: Jason Hite <[email protected]>

* Update kedro pipeline create to use new /conf file structure (#2856)

* add parameters_pipelineNane.yml template in the conf folder

Signed-off-by: Dmitry Sorokin <[email protected]>

* del yml template from parameters folder

Signed-off-by: Dmitry Sorokin <[email protected]>

* modify delete pipeline command to use correct path to yml files

Signed-off-by: Dmitry Sorokin <[email protected]>

* modify test_pipeline move parameters foldername to the yml filename

Signed-off-by: Dmitry Sorokin <[email protected]>

* collect configs to package not only from parameters folder, but from core conf folder also

Signed-off-by: Dmitry Sorokin <[email protected]>

* move parameters foldername to the yml filename, remove nesting

Signed-off-by: Dmitry Sorokin <[email protected]>

* move parameters foldername to the yml filename, remove nesting

Signed-off-by: Dmitry Sorokin <[email protected]>

* Add changes to RELEASE.md

Signed-off-by: Dmitry Sorokin <[email protected]>

* Update RELEASE.md

Co-authored-by: Merel Theisen <[email protected]>

* Update RELEASE.md, change to release 0.18.13

Signed-off-by: Dmitry Sorokin <[email protected]>

* save old structure for 'pipeline delete' command for backward compatibility

Signed-off-by: Dmitry Sorokin <[email protected]>

* modify test_pipeline for 'pipeline delete' backward compability

Signed-off-by: Dmitry Sorokin <[email protected]>

---------

Signed-off-by: Dmitry Sorokin <[email protected]>
Co-authored-by: Merel Theisen <[email protected]>
Signed-off-by: Jason Hite <[email protected]>

* Update kedro catalog create to use new /conf structure (#2884)

* modify test_catalog, remove nesting in the catalog folder

Signed-off-by: Dmitry Sorokin <[email protected]>

* move catalog foldername to the yml filename, remove nesting

Signed-off-by: Dmitry Sorokin <[email protected]>

* Update RELEASE.md

Signed-off-by: Dmitry Sorokin <[email protected]>

* Modify docs, fix broken link

Signed-off-by: Dmitry Sorokin <[email protected]>

---------

Signed-off-by: Dmitry Sorokin <[email protected]>
Signed-off-by: Jason Hite <[email protected]>

* Add migration steps for `ConfigLoader` to `OmegaConfigLoader` (#2887)

* Add migration steps for CL to OCL

Signed-off-by: Merel Theisen <[email protected]>
Signed-off-by: Jason Hite <[email protected]>

* Fix #2498 Adding logging issue 2498 (#2842)

* Fixing typo n on line 114 file /docs/source/nodes_and_pipelines/run_a_pipeline.md from issue #2607

Signed-off-by: laizaparizotto <[email protected]>

* Removed redundant packages at lines 66, 67 and 68 on file kedro/docs/source/nodes_and_pipelines/run_a_pipeline.md, packages:  from kedro.io import AbstractDataSet, DataCatalog, MemoryDataSet,  from kedro.pipeline import Pipeline and  from kedro.runner.runner import AbstractRunner

Signed-off-by: laizaparizotto <[email protected]>

* Add logger object for logging when computing micropackage version

Signed-off-by: laizaparizotto <[email protected]>

* Added new format to the logging

Signed-off-by: laizaparizotto <[email protected]>

---------

Signed-off-by: laizaparizotto <[email protected]>
Co-authored-by: Merel Theisen <[email protected]>
Signed-off-by: Jason Hite <[email protected]>

* Try only trigger docs build when release notes updated (#2907)

Signed-off-by: Merel Theisen <[email protected]>
Signed-off-by: Jason Hite <[email protected]>

* Add Python 3.11 support to Kedro (#2851)

* add 3.11

Signed-off-by: SajidAlamQB <[email protected]>

* Update setup.py

Signed-off-by: SajidAlamQB <[email protected]>

* limit number of python version tests

Signed-off-by: SajidAlamQB <[email protected]>

* fix setup.py

Signed-off-by: SajidAlamQB <[email protected]>

* delta-spark version

Signed-off-by: SajidAlamQB <[email protected]>

* try original setup

Signed-off-by: SajidAlamQB <[email protected]>

* remove 3.11 from circleci

Signed-off-by: SajidAlamQB <[email protected]>

* Update setup.py

Signed-off-by: SajidAlamQB <[email protected]>

* Update setup.py

Signed-off-by: SajidAlamQB <[email protected]>

* pin matplotlib to 3.7.2 for python 3.11

Signed-off-by: SajidAlamQB <[email protected]>

* Update test_import.py

Signed-off-by: SajidAlamQB <[email protected]>

* add noks test adjustments

Signed-off-by: SajidAlamQB <[email protected]>

* Update setup.py

Signed-off-by: SajidAlamQB <[email protected]>

* Update setup.py

Signed-off-by: SajidAlamQB <[email protected]>

* replace eq with is for multi_catalog

Signed-off-by: SajidAlamQB <[email protected]>

* minor fixes

Signed-off-by: SajidAlamQB <[email protected]>

* remove matplotlib 3.11 pin

Signed-off-by: SajidAlamQB <[email protected]>

* matplotlib pin for 3.11

Signed-off-by: SajidAlamQB <[email protected]>

* run original matplotlib pin with ubuntu only

Signed-off-by: SajidAlamQB <[email protected]>

* run circleci

Signed-off-by: SajidAlamQB <[email protected]>

* pin spark < 3.4 as its breaking

Signed-off-by: SajidAlamQB <[email protected]>

* spark datasets revert earlier changes back to original

Signed-off-by: SajidAlamQB <[email protected]>

* delta

Signed-off-by: SajidAlamQB <[email protected]>

* remove all windows tests from GHA

Signed-off-by: SajidAlamQB <[email protected]>

* skip coverage as not needed

Signed-off-by: SajidAlamQB <[email protected]>

* Checking only WIN tests (will revert)

Signed-off-by: SajidAlamQB <[email protected]>

* Update unit-tests.yml

Signed-off-by: SajidAlamQB <[email protected]>

* undo previous commit

Signed-off-by: SajidAlamQB <[email protected]>

* add msbuild

Signed-off-by: SajidAlamQB <[email protected]>

* add ubuntu tests back in and remove 3.11 tests from circleci add in msbuild for the other tests

Signed-off-by: SajidAlamQB <[email protected]>

* set hdf5 disable check and lint

Signed-off-by: SajidAlamQB <[email protected]>

* pin tables for 3.8

Signed-off-by: SajidAlamQB <[email protected]>

* pin 3.8 tables

Signed-off-by: SajidAlamQB <[email protected]>

* lint

Signed-off-by: SajidAlamQB <[email protected]>

* remove 3.11 completely from circleci and tidy up

Signed-off-by: SajidAlamQB <[email protected]>

* add 3.11 to pyarrow pin for docs

Signed-off-by: SajidAlamQB <[email protected]>

* undo previous commit

Signed-off-by: SajidAlamQB <[email protected]>

* update rtd to python 3.11

Signed-off-by: SajidAlamQB <[email protected]>

* try 3.10 on rtd

Signed-off-by: SajidAlamQB <[email protected]>

* fix outdated pylint to ruff

Signed-off-by: SajidAlamQB <[email protected]>

* changes based on review

Signed-off-by: SajidAlamQB <[email protected]>

* add mock for pyarrow and deltalake for docs

Signed-off-by: SajidAlamQB <[email protected]>

* undo previous commit

Signed-off-by: SajidAlamQB <[email protected]>

* Update setup.py

Signed-off-by: SajidAlamQB <[email protected]>

* Update .readthedocs.yml

Signed-off-by: SajidAlamQB <[email protected]>

* Update conf.py

Signed-off-by: SajidAlamQB <[email protected]>

* changes based on review

Signed-off-by: SajidAlamQB <[email protected]>

* fix comments

Signed-off-by: SajidAlamQB <[email protected]>

* test running 3.10 parallel

Signed-off-by: SajidAlamQB <[email protected]>

* Add to release notes and update badges on docs

Signed-off-by: SajidAlamQB <[email protected]>

* Add migration tips

Signed-off-by: SajidAlamQB <[email protected]>

---------

Signed-off-by: SajidAlamQB <[email protected]>
Signed-off-by: Jason Hite <[email protected]>

* Revise FAQs and README (#2909)

* Revise FAQs and README

Signed-off-by: Jo Stichbury <[email protected]>

* Add back the data layers FAQ as I've no idea where else it fits

Signed-off-by: Jo Stichbury <[email protected]>

* minor changes from review

Signed-off-by: Jo Stichbury <[email protected]>

---------

Signed-off-by: Jo Stichbury <[email protected]>
Signed-off-by: Jason Hite <[email protected]>

* Update Generator example (#2858)

* update release note

Signed-off-by: Nok <[email protected]>

* update to binary mode  when opening file

Signed-off-by: Nok Chan <[email protected]>

* Simplify the implementation of Dataset

Signed-off-by: Nok Chan <[email protected]>

* simplify the example - wip

Signed-off-by: Nok Chan <[email protected]>

* Update the nodes

Signed-off-by: Nok Chan <[email protected]>

* update example

Signed-off-by: Nok Chan <[email protected]>

* Update example

Signed-off-by: Nok <[email protected]>

* Apply suggestions from code review

Co-authored-by: Jo Stichbury <[email protected]>

* Apply suggestions from code review

Co-authored-by: Jo Stichbury <[email protected]>

* Apply suggestions from code review

Co-authored-by: Merel Theisen <[email protected]>
Co-authored-by: Jo Stichbury <[email protected]>

* Apply suggestions from code review

Co-authored-by: Jo Stichbury <[email protected]>

* add more instruction to guide user to create the custom dataset

Signed-off-by: Nok <[email protected]>

* update the reference from `pipeline.py` -> `nodes.py`

Signed-off-by: Nok <[email protected]>

* Collapse the nodes.py function into a expandable section

Signed-off-by: Nok <[email protected]>

* Update nodes.md

* Update nodes.md

---------

Signed-off-by: Nok <[email protected]>
Signed-off-by: Nok Chan <[email protected]>
Co-authored-by: Jo Stichbury <[email protected]>
Co-authored-by: Merel Theisen <[email protected]>
Signed-off-by: Jason Hite <[email protected]>

* update docs to reflect change to /conf file structure (#2913)

* update docs to reflect change to /conf file structure

Signed-off-by: Dmitry Sorokin <[email protected]>

* update docs, folder structure diagrams

Signed-off-by: Dmitry Sorokin <[email protected]>

---------

Signed-off-by: Dmitry Sorokin <[email protected]>
Signed-off-by: Jason Hite <[email protected]>

* Change CONTRIBUTING.md file based on PR #2894 (#2914)

Signed-off-by: lrcouto <[email protected]>
Signed-off-by: Jason Hite <[email protected]>

* Move contribution guidelines from CONTRIBUTING.md to the Wiki (#2894)

* Add drafts for new contributing.md and wiki page

Signed-off-by: lrcouto <[email protected]>

* Add contribution details to wiki page draft

Signed-off-by: lrcouto <[email protected]>

* Remove contributing-to-Kedro.md draft from the branch

Signed-off-by: lrcouto <[email protected]>

* Fix broken link to Prefect website in deploy guide (#2885)

Signed-off-by: lrcouto <[email protected]>

* Clarify the <micropkg_name> argument to kedro micropkg package (#2835)

Co-authored-by: Juan Luis Cano Rodríguez <[email protected]>
Co-authored-by: Merel Theisen <[email protected]>
Signed-off-by: lrcouto <[email protected]>

* Allow registering of custom resolvers to `OmegaConfigLoader` (#2869)

* Allow registering of custom resolvers to OCL

Signed-off-by: Ankita Katiyar <[email protected]>

* Complete doc string

Signed-off-by: Ankita Katiyar <[email protected]>

* Add test for overwritten resolvers

Signed-off-by: Ankita Katiyar <[email protected]>

* Update test for overwritten resolvers

Signed-off-by: Ankita Katiyar <[email protected]>

* Remove replace=True by default

Signed-off-by: Ankita Katiyar <[email protected]>

* Update release notes

Signed-off-by: Ankita Katiyar <[email protected]>

* Update release notes

Signed-off-by: Ankita Katiyar <[email protected]>

* Add debug level log for registering new resolver

Signed-off-by: Ankita Katiyar <[email protected]>

---------

Signed-off-by: Ankita Katiyar <[email protected]>
Signed-off-by: lrcouto <[email protected]>

* Some adjustments to ordering of text

Signed-off-by: Jo Stichbury <[email protected]>
Signed-off-by: lrcouto <[email protected]>

* Document the use of custom resolvers with `OmegaConfigLoader` (#2896)

* Update docs for custom resolvers for OCL

Signed-off-by: Ankita Katiyar <[email protected]>

* Update FAQ with correct link

Signed-off-by: Ankita Katiyar <[email protected]>

* Fix typo

Co-authored-by: Merel Theisen <[email protected]>

* Add note for oc.env

Signed-off-by: Ankita Katiyar <[email protected]>

* Suggestions from code review

Signed-off-by: Ankita Katiyar <[email protected]>

---------

Signed-off-by: Ankita Katiyar <[email protected]>
Co-authored-by: Merel Theisen <[email protected]>
Signed-off-by: lrcouto <[email protected]>

* Update kedro pipeline create to use new /conf file structure (#2856)

* add parameters_pipelineNane.yml template in the conf folder

Signed-off-by: Dmitry Sorokin <[email protected]>

* del yml template from parameters folder

Signed-off-by: Dmitry Sorokin <[email protected]>

* modify delete pipeline command to use correct path to yml files

Signed-off-by: Dmitry Sorokin <[email protected]>

* modify test_pipeline move parameters foldername to the yml filename

Signed-off-by: Dmitry Sorokin <[email protected]>

* collect configs to package not only from parameters folder, but from core conf folder also

Signed-off-by: Dmitry Sorokin <[email protected]>

* move parameters foldername to the yml filename, remove nesting

Signed-off-by: Dmitry Sorokin <[email protected]>

* move parameters foldername to the yml filename, remove nesting

Signed-off-by: Dmitry Sorokin <[email protected]>

* Add changes to RELEASE.md

Signed-off-by: Dmitry Sorokin <[email protected]>

* Update RELEASE.md

Co-authored-by: Merel Theisen <[email protected]>

* Update RELEASE.md, change to release 0.18.13

Signed-off-by: Dmitry Sorokin <[email protected]>

* save old structure for 'pipeline delete' command for backward compatibility

Signed-off-by: Dmitry Sorokin <[email protected]>

* modify test_pipeline for 'pipeline delete' backward compability

Signed-off-by: Dmitry Sorokin <[email protected]>

---------

Signed-off-by: Dmitry Sorokin <[email protected]>
Co-authored-by: Merel Theisen <[email protected]>
Signed-off-by: lrcouto <[email protected]>

* Update CONTRIBUTING.md

Signed-off-by: lrcouto <[email protected]>

* Fix broken link

Signed-off-by: Jo Stichbury <[email protected]>
Signed-off-by: lrcouto <[email protected]>

* Update kedro catalog create to use new /conf structure (#2884)

* modify test_catalog, remove nesting in the catalog folder

Signed-off-by: Dmitry Sorokin <[email protected]>

* move catalog foldername to the yml filename, remove nesting

Signed-off-by: Dmitry Sorokin <[email protected]>

* Update RELEASE.md

Signed-off-by: Dmitry Sorokin <[email protected]>

* Modify docs, fix broken link

Signed-off-by: Dmitry Sorokin <[email protected]>

---------

Signed-off-by: Dmitry Sorokin <[email protected]>
Signed-off-by: lrcouto <[email protected]>

* Add migration steps for `ConfigLoader` to `OmegaConfigLoader` (#2887)

* Add migration steps for CL to OCL

Signed-off-by: Merel Theisen <[email protected]>
Signed-off-by: lrcouto <[email protected]>

* Fix #2498 Adding logging issue 2498 (#2842)

* Fixing typo n on line 114 file /docs/source/nodes_and_pipelines/run_a_pipeline.md from issue #2607

Signed-off-by: laizaparizotto <[email protected]>

* Removed redundant packages at lines 66, 67 and 68 on file kedro/docs/source/nodes_and_pipelines/run_a_pipeline.md, packages:  from kedro.io import AbstractDataSet, DataCatalog, MemoryDataSet,  from kedro.pipeline import Pipeline and  from kedro.runner.runner import AbstractRunner

Signed-off-by: laizaparizotto <[email protected]>

* Add logger object for logging when computing micropackage version

Signed-off-by: laizaparizotto <[email protected]>

* Added new format to the logging

Signed-off-by: laizaparizotto <[email protected]>

---------

Signed-off-by: laizaparizotto <[email protected]>
Co-authored-by: Merel Theisen <[email protected]>
Signed-off-by: lrcouto <[email protected]>

* Try only trigger docs build when release notes updated (#2907)

Signed-off-by: Merel Theisen <[email protected]>
Signed-off-by: lrcouto <[email protected]>

* Add Python 3.11 support to Kedro (#2851)

* add 3.11

Signed-off-by: SajidAlamQB <[email protected]>

* Update setup.py

Signed-off-by: SajidAlamQB <[email protected]>

* limit number of python version tests

Signed-off-by: SajidAlamQB <[email protected]>

* fix setup.py

Signed-off-by: SajidAlamQB <[email protected]>

* delta-spark version

Signed-off-by: SajidAlamQB <[email protected]>

* try original setup

Signed-off-by: SajidAlamQB <[email protected]>

* remove 3.11 from circleci

Signed-off-by: SajidAlamQB <[email protected]>

* Update setup.py

Signed-off-by: SajidAlamQB <[email protected]>

* Update setup.py

Signed-off-by: SajidAlamQB <[email protected]>

* pin matplotlib to 3.7.2 for python 3.11

Signed-off-by: SajidAlamQB <[email protected]>

* Update test_import.py

Signed-off-by: SajidAlamQB <[email protected]>

* add noks test adjustments

Signed-off-by: SajidAlamQB <[email protected]>

* Update setup.py

Signed-off-by: SajidAlamQB <[email protected]>

* Update setup.py

Signed-off-by: SajidAlamQB <[email protected]>

* replace eq with is for multi_catalog

Signed-off-by: SajidAlamQB <[email protected]>

* minor fixes

Signed-off-by: SajidAlamQB <[email protected]>

* remove matplotlib 3.11 pin

Signed-off-by: SajidAlamQB <[email protected]>

* matplotlib pin for 3.11

Signed-off-by: SajidAlamQB <[email protected]>

* run original matplotlib pin with ubuntu only

Signed-off-by: SajidAlamQB <[email protected]>

* run circleci

Signed-off-by: SajidAlamQB <[email protected]>

* pin spark < 3.4 as its breaking

Signed-off-by: SajidAlamQB <[email protected]>

* spark datasets revert earlier changes back to original

Signed-off-by: SajidAlamQB <[email protected]>

* delta

Signed-off-by: SajidAlamQB <[email protected]>

* remove all windows tests from GHA

Signed-off-by: SajidAlamQB <[email protected]>

* skip coverage as not needed

Signed-off-by: SajidAlamQB <[email protected]>

* Checking only WIN tests (will revert)

Signed-off-by: SajidAlamQB <[email protected]>

* Update unit-tests.yml

Signed-off-by: SajidAlamQB <[email protected]>

* undo previous commit

Signed-off-by: SajidAlamQB <[email protected]>

* add msbuild

Signed-off-by: SajidAlamQB <[email protected]>

* add ubuntu tests back in and remove 3.11 tests from circleci add in msbuild for the other tests

Signed-off-by: SajidAlamQB <[email protected]>

* set hdf5 disable check and lint

Signed-off-by: SajidAlamQB <[email protected]>

* pin tables for 3.8

Signed-off-by: SajidAlamQB <[email protected]>

* pin 3.8 tables

Signed-off-by: SajidAlamQB <[email protected]>

* lint

Signed-off-by: SajidAlamQB <[email protected]>

* remove 3.11 completely from circleci and tidy up

Signed-off-by: SajidAlamQB <[email protected]>

* add 3.11 to pyarrow pin for docs

Signed-off-by: SajidAlamQB <[email protected]>

* undo previous commit

Signed-off-by: SajidAlamQB <[email protected]>

* update rtd to python 3.11

Signed-off-by: SajidAlamQB <[email protected]>

* try 3.10 on rtd

Signed-off-by: SajidAlamQB <[email protected]>

* fix outdated pylint to ruff

Signed-off-by: SajidAlamQB <[email protected]>

* changes based on review

Signed-off-by: SajidAlamQB <[email protected]>

* add mock for pyarrow and deltalake for docs

Signed-off-by: SajidAlamQB <[email protected]>

* undo previous commit

Signed-off-by: SajidAlamQB <[email protected]>

* Update setup.py

Signed-off-by: SajidAlamQB <[email protected]>

* Update .readthedocs.yml

Signed-off-by: SajidAlamQB <[email protected]>

* Update conf.py

Signed-off-by: SajidAlamQB <[email protected]>

* changes based on review

Signed-off-by: SajidAlamQB <[email protected]>

* fix comments

Signed-off-by: SajidAlamQB <[email protected]>

* test running 3.10 parallel

Signed-off-by: SajidAlamQB <[email protected]>

* Add to release notes and update badges on docs

Signed-off-by: SajidAlamQB <[email protected]>

* Add migration tips

Signed-off-by: SajidAlamQB <[email protected]>

---------

Signed-off-by: SajidAlamQB <[email protected]>
Signed-off-by: lrcouto <[email protected]>

* Revise FAQs and README (#2909)

* Revise FAQs and README

Signed-off-by: Jo Stichbury <[email protected]>

* Add back the data layers FAQ as I've no idea where else it fits

Signed-off-by: Jo Stichbury <[email protected]>

* minor changes from review

Signed-off-by: Jo Stichbury <[email protected]>

---------

Signed-off-by: Jo Stichbury <[email protected]>

* Update Generator example (#2858)

* update release note

Signed-off-by: Nok <[email protected]>

* update to binary mode  when opening file

Signed-off-by: Nok Chan <[email protected]>

* Simplify the implementation of Dataset

Signed-off-by: Nok Chan <[email protected]>

* simplify the example - wip

Signed-off-by: Nok Chan <[email protected]>

* Update the nodes

Signed-off-by: Nok Chan <[email protected]>

* update example

Signed-off-by: Nok Chan <[email protected]>

* Update example

Signed-off-by: Nok <[email protected]>

* Apply suggestions from code review

Co-authored-by: Jo Stichbury <[email protected]>

* Apply suggestions from code review

Co-authored-by: Jo Stichbury <[email protected]>

* Apply suggestions from code review

Co-authored-by: Merel Theisen <[email protected]>
Co-authored-by: Jo Stichbury <[email protected]>

* Apply suggestions from code review

Co-authored-by: Jo Stichbury <[email protected]>

* add more instruction to guide user to create the custom dataset

Signed-off-by: Nok <[email protected]>

* update the reference from `pipeline.py` -> `nodes.py`

Signed-off-by: Nok <[email protected]>

* Collapse the nodes.py function into a expandable section

Signed-off-by: Nok <[email protected]>

* Update nodes.md

* Update nodes.md

---------

Signed-off-by: Nok <[email protected]>
Signed-off-by: Nok Chan <[email protected]>
Co-authored-by: Jo Stichbury <[email protected]>
Co-authored-by: Merel Theisen <[email protected]>
Signed-off-by: lrcouto <[email protected]>

* Update CONTRIBUTING.md

---------

Signed-off-by: lrcouto <[email protected]>
Signed-off-by: Ankita Katiyar <[email protected]>
Signed-off-by: Jo Stichbury <[email protected]>
Signed-off-by: Dmitry Sorokin <[email protected]>
Signed-off-by: Merel Theisen <[email protected]>
Signed-off-by: laizaparizotto <[email protected]>
Signed-off-by: SajidAlamQB <[email protected]>
Signed-off-by: Nok <[email protected]>
Signed-off-by: Nok Chan <[email protected]>
Co-authored-by: Deepyaman Datta <[email protected]>
Co-authored-by: Jonathan Cohen <[email protected]>
Co-authored-by: Juan Luis Cano Rodríguez <[email protected]>
Co-authored-by: Merel Theisen <[email protected]>
Co-authored-by: Ankita Katiyar <[email protected]>
Co-authored-by: Jo Stichbury <[email protected]>
Co-authored-by: Dmitry Sorokin <[email protected]>
Co-authored-by: Laíza Milena Scheid Parizotto <[email protected]>
Co-authored-by: Sajid Alam <[email protected]>
Co-authored-by: Nok Lam Chan <[email protected]>
Signed-off-by: Jason Hite <[email protected]>

* Remove redundant pages and direct users to wiki (#2917)

* Remove redundant pages and direct users to wiki

Signed-off-by: lrcouto <[email protected]>

* Lint docs changes

Signed-off-by: Jo Stichbury <[email protected]>

* Few tweaks to index page

Signed-off-by: Jo Stichbury <[email protected]>

---------

Signed-off-by: lrcouto <[email protected]>
Signed-off-by: Jo Stichbury <[email protected]>
Co-authored-by: Jo Stichbury <[email protected]>
Signed-off-by: Jason Hite <[email protected]>

* Deprecate abstract "DataSet" in favor of "Dataset" (#2746)

* Deprecate abstract "DataSet" in favor of "Dataset"

Signed-off-by: Deepyaman Datta <[email protected]>

* Update docs references to abstract "DataSet" class

Signed-off-by: Deepyaman Datta <[email protected]>

* Fix `AbstractDataSet` reference to `AbstractDataset`

Signed-off-by: Deepyaman Datta <[email protected]>

* Update docs/source/conf.py

Signed-off-by: Deepyaman Datta <[email protected]>

* Change remaining `_DEPRECATED_ERROR_CLASSES` to `_DEPRECATED_CLASSES`, update type hints

Signed-off-by: Deepyaman Datta <[email protected]>

* Update RELEASE.md

* Reformat kedro/io/__init__.py with Black and isort

Signed-off-by: Deepyaman Datta <[email protected]>

* Remove duplicate imports

Signed-off-by: Deepyaman Datta <[email protected]>

* Move imports

Signed-off-by: Deepyaman Datta <[email protected]>

---------

Signed-off-by: Deepyaman Datta <[email protected]>
Signed-off-by: Jason Hite <[email protected]>

* Reorganise and improve the data catalog documentation (#2888)

* First drop of newly organised data catalog docs

Signed-off-by: Jo Stichbury <[email protected]>

* linter

Signed-off-by: Jo Stichbury <[email protected]>

* Added to-do notes

Signed-off-by: Jo Stichbury <[email protected]>

* Afternoon's work in rewriting/reorganising content

Signed-off-by: Jo Stichbury <[email protected]>

* More changes

Signed-off-by: Jo Stichbury <[email protected]>

* Further changes

Signed-off-by: Jo Stichbury <[email protected]>

* Another chunk of changes

Signed-off-by: Jo Stichbury <[email protected]>

* Final changes

Signed-off-by: Jo Stichbury <[email protected]>

* Revise ordering of pages

Signed-off-by: Jo Stichbury <[email protected]>

* Add new CLI commands to dataset factory docs (#2935)

* Add changes from #2930

Signed-off-by: Ahdra Merali <[email protected]>

* Lint

Signed-off-by: Ahdra Merali <[email protected]>

* Apply suggestions from code review

Co-authored-by: Jo Stichbury <[email protected]>

* Make code snippets collapsable

Signed-off-by: Ahdra Merali <[email protected]>

---------

Signed-off-by: Ahdra Merali <[email protected]>
Co-authored-by: Jo Stichbury <[email protected]>

* Bunch of changes from feedback

Signed-off-by: Jo Stichbury <[email protected]>

* A few more tweaks

Signed-off-by: Jo Stichbury <[email protected]>

* Update h1,h2,h3 font sizes

Signed-off-by: Tynan DeBold <[email protected]>

* Add code snippet for using DataCatalog with Kedro config

Signed-off-by: Ankita Katiyar <[email protected]>

* Few more tweaks

Signed-off-by: Jo Stichbury <[email protected]>

* Update docs/source/data/data_catalog.md

* Upgrade kedro-datasets for docs

Signed-off-by: Juan Luis Cano Rodríguez <[email protected]>

* Improve prose

Signed-off-by: Juan Luis Cano Rodríguez <[email protected]>

Co-authored-by: Jo Stichbury <[email protected]>

---------

Signed-off-by: Jo Stichbury <[email protected]>
Signed-off-by: Ahdra Merali <[email protected]>
Signed-off-by: Tynan DeBold <[email protected]>
Signed-off-by: Ankita Katiyar <[email protected]>
Signed-off-by: Juan Luis Cano Rodríguez <[email protected]>
Co-authored-by: Ahdra Merali <[email protected]>
Co-authored-by: Tynan DeBold <[email protected]>
Co-authored-by: Ankita Katiyar <[email protected]>
Co-authored-by: Juan Luis Cano Rodríguez <[email protected]>
Signed-off-by: Jason Hite <[email protected]>

* Add line about Viz to PR template (#2929)

Signed-off-by: Jason Hite <[email protected]>

* Add architecture graphic back to docs with revisions (#2916)

* Revise FAQs and README

Signed-off-by: Jo Stichbury <[email protected]>

* Add back the data layers FAQ as I've no idea where else it fits

Signed-off-by: Jo Stichbury <[email protected]>

* minor changes from review

Signed-off-by: Jo Stichbury <[email protected]>

* Re-add kedro arch diagram, with revised graphic

Signed-off-by: Jo Stichbury <[email protected]>

* fix broken anchor

Signed-off-by: Jo Stichbury <[email protected]>

* fix broken anchor

Signed-off-by: Jo Stichbury <[email protected]>

* Update docs/source/extend_kedro/architecture_overview.md

Co-authored-by: Nok Lam Chan <[email protected]>

* Update docs/source/extend_kedro/architecture_overview.md

Co-authored-by: Nok Lam Chan <[email protected]>

* Update docs/source/extend_kedro/architecture_overview.md

Co-authored-by: Nok Lam Chan <[email protected]>

* Update docs/source/extend_kedro/architecture_overview.md

Co-authored-by: Nok Lam Chan <[email protected]>

* Changes to architecture page following review

Signed-off-by: Jo Stichbury <[email protected]>

* Change diagram following reivew

* Add links to API docs

Signed-off-by: Jo Stichbury <[email protected]>

* Added in type of users

* Fix linting error

Signed-off-by: Jo Stichbury <[email protected]>

---------

Signed-off-by: Jo Stichbury <[email protected]>
Co-authored-by: Nok Lam Chan <[email protected]>
Co-authored-by: Yetunde Dada <[email protected]>
Signed-off-by: Jason Hite <[email protected]>

* Add kedro catalog resolve command (#2891)

* Implement working draft

Signed-off-by: Ahdra Merali <[email protected]>

* Remove comments

Signed-off-by: Ahdra Merali <[email protected]>

* Remove raw_config

Signed-off-by: Ahdra Merali <[email protected]>

* Use config loader to access catalog config

Signed-off-by: Ahdra Merali <[email protected]>

* Use dict comprehension

Signed-off-by: Ahdra Merali <[email protected]>

* Remove pipeline filtering

Signed-off-by: Ahdra Merali <[email protected]>

* Prevent overwrite or param inclusion

Signed-off-by: Ahdra Merali <[email protected]>

* Trim filepath

Signed-off-by: Ahdra Merali <[email protected]>

* Appease linter

Signed-off-by: Ahdra Merali <[email protected]>

* Add test for resolve

Signed-off-by: Ahdra Merali <[email protected]>

* Add test to exclude params

Signed-off-by: Ahdra Merali <[email protected]>

* Add test for overwrite (not working)

Signed-off-by: Ahdra Merali <[email protected]>

* Fix test

Signed-off-by: Ahdra Merali <[email protected]>

* Remove print

Signed-off-by: Ahdra Merali <[email protected]>

* Add changes to RELEASE.md

Signed-off-by: Ahdra Merali <[email protected]>

* Trim unreachable code

Signed-off-by: Ahdra Merali <[email protected]>

* Make helper function private

Signed-off-by: Ahdra Merali <[email protected]>

* Refactor

Signed-off-by: Ahdra Merali <[email protected]>

* Add changes to the docs

Signed-off-by: Ahdra Merali <[email protected]>

* Refactor 2

Signed-off-by: Ahdra Merali <[email protected]>

* Fix docs build

Signed-off-by: Ahdra Merali <[email protected]>

* Add suggestions from code review (docs)

Signed-off-by: Ahdra Merali <[email protected]>

* Add suggestions from code review (code)

Signed-off-by: Ahdra Merali <[email protected]>

* Rename catalog_config variable to explicit_datasets

Signed-off-by: Ahdra Merali <[email protected]>

* Apply suggestions from code review

Co-authored-by: Merel Theisen <[email protected]>

* Fix mocking in test

Signed-off-by: Ahdra Merali <[email protected]>

* Change test fixture name

Signed-off-by: Ahdra Merali <[email protected]>

* Lint

Signed-off-by: Ahdra Merali <[email protected]>

---------

Signed-off-by: Ahdra Merali <[email protected]>
Co-authored-by: Merel Theisen <[email protected]>
Signed-off-by: Jason Hite <[email protected]>

* Replace "DataSet" with "Dataset" in Markdown files (#2735)

* LambdaDataSet->LambdaDataset in .md files

Signed-off-by: Deepyaman Datta <[email protected]>

* MemoryDataSet->MemoryDataset in .md files

Signed-off-by: Deepyaman Datta <[email protected]>

* PartitionedDataSet->PartitionedDataset in .md files

Signed-off-by: Deepyaman Datta <[email protected]>

* IncrementalDataSet->IncrementalDataset in .md files

Signed-off-by: Deepyaman Datta <[email protected]>

* CachedDataSet->CachedDataset in .md files

Signed-off-by: Deepyaman Datta <[email protected]>

* DataSetError->DatasetError in .md files

Signed-off-by: Deepyaman Datta <[email protected]>

* DataSetNotFoundError->DatasetNotFoundError in .md files

Signed-off-by: Deepyaman Datta <[email protected]>

* Replace "DataSet" with "Dataset" in Markdown files

* Update RELEASE.md

* Fix remaining instance of "*DataSet*"->"*Dataset*"

Signed-off-by: Deepyaman Datta <[email protected]>

* `find . -name '*.md' -print0 | xargs -0 sed -i "" "s/\([^A-Za-z]\)DataSet/\1Dataset/g"`

Signed-off-by: Deepyaman Datta <[email protected]>

* Change non-class instances of Dataset to dataset

* Replace any remaining instances of DataSet in docs

* Fix a broken link to docs for `PartitionedDataset`

---------

Signed-off-by: Deepyaman Datta <[email protected]>
Co-authored-by: Jo Stichbury <[email protected]>
Co-authored-by: Juan Luis Cano Rodríguez <[email protected]>
Signed-off-by: Jason Hite <[email protected]>

* Update h1,h2,h3 font sizes in the docs pages (#2938)

* Update h1,h2,h3 font sizes

Signed-off-by: Tynan DeBold <[email protected]>

* Update h2 font size

Signed-off-by: Tynan DeBold <[email protected]>

---------

Signed-off-by: Tynan DeBold <[email protected]>
Co-authored-by: Jo Stichbury <[email protected]>
Co-authored-by: Juan Luis Cano Rodríguez <[email protected]>
Signed-off-by: Jason Hite <[email protected]>

* Automatically trigger `kedro-starters` release on the release of `kedro` (#2889)

* Add GA for triggering starters release

Signed-off-by: Ankita Katiyar <[email protected]>

* Update token name

Signed-off-by: Ankita Katiyar <[email protected]>

---------

Signed-off-by: Ankita Katiyar <[email protected]>
Signed-off-by: Jason Hite <[email protected]>

* Create issues_metrics.yml (#2814)

* Create issues_metrics.yml

* Update issues_metrics.yml

Signed-off-by: Jason Hite <[email protected]>

* Clean up setuptools and wheel requirements to align with PEP-518 (#2927)

Signed-off-by: Jason Hite <[email protected]>

* Clean up `kedro pipeline create` outdated docs (#2945)

* remove the README template generate by kedro pipeline create

Signed-off-by: Nok <[email protected]>

* fix docstring for kedro catalog create

Signed-off-by: Nok <[email protected]>

* Fix broken link

Signed-off-by: Nok <[email protected]>

* fix broken link

Signed-off-by: Nok <[email protected]>

---------

Signed-off-by: Nok <[email protected]>
Co-authored-by: Juan Luis Cano Rodríguez <[email protected]>
Signed-off-by: Jason Hite <[email protected]>

* Add globals feature for `OmegaConfigLoader` using a globals resolver (#2921)

* Refactor load_and_merge_dir()

Signed-off-by: Ankita Katiyar <[email protected]>

* Try adding globals resolver

Signed-off-by: Ankita Katiyar <[email protected]>

* Minor change

Signed-off-by: Ankita Katiyar <[email protected]>

* Add globals resolver

Signed-off-by: Ankita Katiyar <[email protected]>

* Revert refactoring

Signed-off-by: Ankita Katiyar <[email protected]>

* Add test + remove self.globals

Signed-off-by: Ankita Katiyar <[email protected]>

* Allow for nested variables in globals

Signed-off-by: Ankita Katiyar <[email protected]>

* Add documentation

Signed-off-by: Ankita Katiyar <[email protected]>

* Typo

Signed-off-by: Ankita Katiyar <[email protected]>

* Add error message + test

Signed-off-by: Ankita Katiyar <[email protected]>

* Apply suggestions from code review

Co-authored-by: Merel Theisen <[email protected]>

* Split test into multiple tests

Signed-off-by: Ankita Katiyar <[email protected]>

* Restrict the globals config_patterns

Signed-off-by: Ankita Katiyar <[email protected]>

* Release notes

Signed-off-by: Ankita Katiyar <[email protected]>

* Update docs/source/configuration/advanced_configuration.md

Co-authored-by: Merel Theisen <[email protected]>

* Add helpful error message for keys starting with _

Signed-off-by: Ankita Katiyar <[email protected]>

* Enable setting default value for globals resolver

Signed-off-by: Ankita Katiyar <[email protected]>

* Typo

Signed-off-by: Ankita Katiyar <[email protected]>

* Move test for keys starting with _ to the top

Signed-off-by: Ankita Katiyar <[email protected]>

* Fix cross ref link in docs

Signed-off-by: Ankita Katiyar <[email protected]>

---------

Signed-off-by: Ankita Katiyar <[email protected]>
Co-authored-by: Merel Theisen <[email protected]>
Co-authored-by: Juan Luis Cano Rodríguez <[email protected]>
Co-authored-by: Nok Lam Chan <[email protected]>
Signed-off-by: Jason Hite <[email protected]>

* Consolidate two `ruff-pre-commit` entries into one (#2881)

* Clean up Ruff configuration in base pyproject.toml

Signed-off-by: Deepyaman Datta <[email protected]>

* Consolidate two `ruff-pre-commit` entries into one

Signed-off-by: Deepyaman Datta <[email protected]>

* Move Ruff hook to the top, because it's aggressive

Signed-off-by: Deepyaman Datta <[email protected]>

* Add `kedro` as `known-first-party` for `isort` run

Signed-off-by: Deepyaman Datta <[email protected]>

* Add noqa in docs/source/conf.py

Signed-off-by: Deepyaman Datta <[email protected]>

* Fix ruff linter precommit config

Signed-off-by: Nok <[email protected]>

---------

Signed-off-by: Deepyaman Datta <[email protected]>
Signed-off-by: Nok <[email protected]>
Co-authored-by: Nok <[email protected]>
Signed-off-by: Jason Hite <[email protected]>

* Fix typos across the documentation (#2956)

* Fix typos across docs

Signed-off-by: Ankita Katiyar <[email protected]>

* Capitalisation stuff

Signed-off-by: Ankita Katiyar <[email protected]>

---------

Signed-off-by: Ankita Katiyar <[email protected]>
Signed-off-by: Jason Hite <[email protected]>

* Setup Vale linter as a GHA workflow (#2953)

* Add vale GHA

Signed-off-by: Ankita Katiyar <[email protected]>

* Update condition

Signed-off-by: Ankita Katiyar <[email protected]>

* Update .github/styles/Kedro/ignore.txt

Co-authored-by: Jo Stichbury <[email protected]>

* add some styles

Signed-off-by: Jo Stichbury <[email protected]>

* Remove styles for now

Signed-off-by: Ankita Katiyar <[email protected]>

---------

Signed-off-by: Ankita Katiyar <[email protected]>
Signed-off-by: Jo Stichbury <[email protected]>
Co-authored-by: Jo Stichbury <[email protected]>
Signed-off-by: Jason Hite <[email protected]>

* Fix README to show graphics on PyPI (#2961)

* Fix for PyPI

Signed-off-by: Jo Stichbury <[email protected]>

* Few more tweaks

Signed-off-by: Jo Stichbury <[email protected]>

* Few more tweaks

Signed-off-by: Jo Stichbury <[email protected]>

* Few more tweaks

Signed-off-by: Jo Stichbury <[email protected]>

* Few more tweaks

Signed-off-by: Jo Stichbury <[email protected]>

* Update README.md

* Update README.md

---------

Signed-off-by: Jo Stichbury <[email protected]>
Signed-off-by: Jason Hite <[email protected]>

* Add some Vale styles (#2963)

* add some styles

Signed-off-by: Jo Stichbury <[email protected]>

* Tweaks to styles

Signed-off-by: Jo Stichbury <[email protected]>

* more tweaks

Signed-off-by: Jo Stichbury <[email protected]>

* Revise some styles

Signed-off-by: Jo Stichbury <[email protected]>

* Revise severity of some styles

Signed-off-by: Jo Stichbury <[email protected]>

* Update .github/styles/Kedro/words.yml

Co-authored-by: Deepyaman Datta <[email protected]>

---------

Signed-off-by: Jo Stichbury <[email protected]>
Co-authored-by: Deepyaman Datta <[email protected]>
Signed-off-by: Jason Hite <[email protected]>

* Minor changes to test + release notes + docs

Signed-off-by: Ankita Katiyar <[email protected]>
Signed-off-by: Jason Hite <[email protected]>

* Update anyconfig requirement from ~=0.10.0 to >=0.10,<0.14 (#2876)

Updates the requirements on [anyconfig](https://github.com/ssato/python-anyconfig) to permit the latest version.
- [Changelog](https://github.com/ssato/python-anyconfig/blob/next/NEWS)
- [Commits](https://github.com/ssato/python-anyconfig/compare/RELEASE_0.10.0...RELEASE_0.13.0)

Signed-off-by: Jason Hite <[email protected]>
---
updated-dependencies:
- dependency-name: anyconfig
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Juan Luis Cano Rodríguez <[email protected]>
Signed-off-by: Jason Hite <[email protected]>

* Move default template to static `pyproject.toml`, take 2 (#2853)

* Remove leftover reference to micropkg setup.py

Signed-off-by: Juan Luis Cano Rodríguez <[email protected]>

* Replace outdated extras requirement with kedro-datasets

Signed-off-by: Juan Luis Cano Rodríguez <[email protected]>

* Remove outdated mention to environment.yml

Signed-off-by: Juan Luis Cano Rodríguez <[email protected]>

* Move flake8 configuration to dedicated file

Signed-off-by: Juan Luis Cano Rodríguez <[email protected]>

* Replace setup.py with pyproject.toml in default project template

Signed-off-by: Juan Luis Cano Rodríguez <[email protected]>

* Adapt link to moving file in docs

Signed-off-by: Juan Luis Cano Rodríguez <[email protected]>

* Revert "Replace outdated extras requirement with kedro-datasets"

This reverts commit dfa8a9e8377ca60dcc3026e93087c5538791a0b5.

Signed-off-by: Juan Luis Cano Rodríguez <[email protected]>

* Add release notes for pyproject.toml change

Signed-off-by: Juan Luis Cano Rodríguez <[email protected]>

---------

Signed-off-by: Juan Luis Cano Rodríguez <[email protected]>
Signed-off-by: Jason Hite <[email protected]>

* Add deprecation warnings to top-level use of layer in catalog definition (#2964)

* add deprecation

Signed-off-by: SajidAlamQB <[email protected]>

* lint

Signed-off-by: SajidAlamQB <[email protected]>

* Use FutureWarning

Signed-off-by: SajidAlamQB <[email protected]>

* changes based on review

Signed-off-by: SajidAlamQB <[email protected]>

* add link to docs

Signed-off-by: SajidAlamQB <[email protected]>

* Update RELEASE.md

Co-authored-by: Jo Stichbury <[email protected]>

---------

Signed-off-by: SajidAlamQB <[email protected]>
Co-authored-by: Jo Stichbury <[email protected]>
Signed-off-by: Jason Hite <[email protected]>

* Update on credentials.md (#2787)

* Update on credentials.md

Updating example code in credentials.md to make it usable when copy/pasting. Since as of now it will generate a TypeError

Signed-off-by: Jose <[email protected]>

* adding a more explicit code example, now we directly show the user that project_path is a pathlib.Path object

Signed-off-by: Jose <[email protected]>

* Update docs/source/configuration/credentials.md

Co-authored-by: Jo Stichbury <[email protected]>

---------

Signed-off-by: Jose <[email protected]>
Co-authored-by: Jo Stichbury <[email protected]>
Co-authored-by: Nok Lam Chan <[email protected]>
Signed-off-by: Jason Hite <[email protected]>

* Cap pluggy 1.3 release (#2981)

Signed-off-by: Dmitry Sorokin <[email protected]>
Signed-off-by: Jason Hite <[email protected]>

* Make vale linter only run when PR opened or reopened (#2982)

Signed-off-by: Ankita Katiyar <[email protected]>
Signed-off-by: Jason Hite <[email protected]>

* Update merge-gatekeeper.yml (#2960)

Signed-off-by: Jason Hite <[email protected]>

* Configure starters to use OmegaConfigLoader (#2974)

* Change starters to use OmegaConfigLoader

Signed-off-by: lrcouto <[email protected]>

* Fix linter error

Signed-off-by: lrcouto <[email protected]>

* Suppress import outside toplevel linting for starters template

Signed-off-by: L. R. Couto <[email protected]>

* Fix linter error

Signed-off-by: lrcouto <[email protected]>

* Add changes to release notes

Signed-off-by: lrcouto <[email protected]>

---------

Signed-off-by: lrcouto <[email protected]>
Signed-off-by: L. R. Couto <[email protected]>
Signed-off-by: Jason Hite <[email protected]>

* Stop OmegaConfigLoader from reading config from hidden directory like `ipynb_checkpoints` (#2977)

* Check plugins implement valid hooks

Signed-off-by: Nok Chan <[email protected]>

* Add release note

Signed-off-by: Nok Chan <[email protected]>

* Staging work - add custom functions to check hidden folder and files. Tests still failing

Signed-off-by: Nok Chan <[email protected]>

* Fix test - checkpoints should use the same environment

Signed-off-by: Nok Chan <[email protected]>

* Revert "Check plugins implement valid hooks"

This reverts commit f10bede0401b80e3385e93458995835a2db721c9.

* Update RELEASE.md

Co-authored-by: Ankita Katiyar <[email protected]>

* fix lint

Signed-off-by: Nok <[email protected]>

---------

Signed-off-by: Nok Chan <[email protected]>
Signed-off-by: Nok <[email protected]>
Co-authored-by: Ankita Katiyar <[email protected]>
Signed-off-by: Jason Hite <[email protected]>

* Add migration steps for `TemplatedConfigLoader` to `OmegaConfigLoader` (#2904)

Signed-off-by: Merel Theisen <[email protected]>
Signed-off-by: Jason Hite <[email protected]>

* Introduce a sentinel value _NO_VALUE to improve Global resolvers to support defaults `0` or `None` (#2976)

* Add None support to globals

Signed-off-by: Ankita Katiyar <[email protected]>

* Add warning when default value is used

Signed-off-by: Ankita Katiyar <[email protected]>

* Check keys

Signed-off-by: Ankita Katiyar <[email protected]>

* Nok's suggestions

Signed-off-by: Ankita Katiyar <[email protected]>

* Create the test to check the non-existing keys

Signed-off-by: Nok <[email protected]>

* add more tests to catch case when global key is not a dict

Signed-off-by: Nok <[email protected]>

* Fix the null test

Signed-off-by: Nok <[email protected]>

* Introduce sentinel value _NO_VALUE

Signed-off-by: Nok <[email protected]>

* rename test

Signed-off-by: Nok <[email protected]>

* Improve error mesasge and raise InterpolationResolutionError when key does not exist and no default

Signed-off-by: Nok <[email protected]>

* Fix non exist default test

Signed-off-by: Nok <[email protected]>

* Fix test

Signed-off-by: Nok <[email protected]>

* Use omegaconf to replace the custom resolving logic

Signed-off-by: Nok <[email protected]>

* uncommented some tests

Signed-off-by: Nok <[email protected]>

* Remove dead code

Signed-off-by: Ankita Katiyar <[email protected]>

* Update error message

Signed-off-by: Ankita Katiyar <[email protected]>

---------

Signed-off-by: Ankita Katiyar <[email protected]>
Signed-off-by: Nok <[email protected]>
Co-authored-by: Nok <[email protected]>
Signed-off-by: Jason Hite <[email protected]>

* Release 0.18.13 (#2988)

Signed-off-by: Jason Hite <[email protected]>

* Fix docstrings on kedro/extras/datasets (#2995)

* Fix docstrings on kedro/extras/datasets

Signed-off-by: lrcouto <[email protected]>

* Fix formatting error

Signed-off-by: lrcouto <[email protected]>

* Explicitly define code block

Signed-off-by: lrcouto <[email protected]>

* Fix empty line under code block

Signed-off-by: lrcouto <[email protected]>

* Fix formatting error

Signed-off-by: lrcouto <[email protected]>

* Fix broken link

Signed-off-by: lrcouto <[email protected]>

* Bump kedro-datasets version

Signed-off-by: Ankita Katiyar <[email protected]>

* Update links in partitioned and incremental datasets

Signed-off-by: Ankita Katiyar <[email protected]>

* Update links in partitioned and incremental datasets

Signed-off-by: Ankita Katiyar <[email protected]>

* Update links in partitioned dataset

Signed-off-by: Ankita Katiyar <[email protected]>

* Update links in partitioned dataset

Signed-off-by: Ankita Katiyar <[email protected]>

* Add polars.GenericDataSet to .rst

Signed-off-by: Ankita Katiyar <[email protected]>

---------

Signed-off-by: lrcouto <[email protected]>
Signed-off-by: Ankita Katiyar <[email protected]>
Co-authored-by: Ankita Katiyar <[email protected]>
Signed-off-by: Jason Hite <[email protected]>

* Minor docs changes on data section to create a PR and test Vale styles (#2966)

* Minor changes to create a PR and test Vale styles

Signed-off-by: Jo Stichbury <[email protected]>

* fix some vale warnings

Signed-off-by: Jo Stichbury <[email protected]>

---------

Signed-off-by: Jo Stichbury <[email protected]>
Co-authored-by: Juan Luis Cano Rodríguez <[email protected]>
Signed-off-by: Jason Hite <[email protected]>

* PiP/pyproject.toml and Conda/Meta.yaml Sync (#2922)

* Resynced

Signed-off-by: rxm7706 <[email protected]>

* restore lower bound for pluggy

Signed-off-by: rxm7706 <[email protected]>

---------

Signed-off-by: rxm7706 <[email protected]>
Co-authored-by: Juan Luis Cano Rodríguez <[email protected]>
Signed-off-by: Jason Hite <[email protected]>

* fix typo for build (#3001)

Signed-off-by: rxm7706 <[email protected]>
Signed-off-by: Jason Hite <[email protected]>

* Add hook example to access `metadata`  (#2998)

* Check plugins implement valid hooks

Signed-off-by: Nok Chan <[email protected]>

* add Metadtahook

Signed-off-by: Nok Chan <[email protected]>

* Fix docs according to comments

Signed-off-by: Nok <[email protected]>

---------

Signed-off-by: Nok Chan <[email protected]>
Signed-off-by: Nok <[email protected]>
Co-authored-by: Jo Stichbury <[email protected]>
Signed-off-by: Jason Hite <[email protected]>

* Bump release notes version

Signed-off-by: Jason Hite <[email protected]>

* Expand docs for customized pipeline templates.

Signed-off-by: Jason Hite <[email protected]>

* Update release notes and docs

Signed-off-by: Ankita Katiyar <[email protected]>
Signed-off-by: Jason Hite <[email protected]>

* Update language linter to also run when PR converted ready for review

Signed-off-by: Ankita Katiyar <[email protected]>
Signed-off-by: Jason Hite <[email protected]>

* Fix ci - Use `--resolver=backtracking` with `kedro build-reqs` in e2e tests (#3010)

* Bump build version

Signed-off-by: Ankita Katiyar <[email protected]>

* Remove telemetry from test default starter

Signed-off-by: Ankita Katiyar <[email protected]>

* Add package_name back

Signed-off-by: Ankita Katiyar <[email protected]>

* Pin build only for 3.7

Signed-off-by: Ankita Katiyar <[email protected]>

* Try upgrade pip

Signed-off-by: Ankita Katiyar <[email protected]>

* Add constraint

Signed-off-by: Ankita Katiyar <[email protected]>

* Update for windows

Signed-off-by: Ankita Katiyar <[email protected]>

* Run build-reqs with backtracking resolver

Signed-off-by: Ankita Katiyar <[email protected]>

* Run build-reqs with backtracking resolver

Signed-off-by: Ankita Katiyar <[email protected]>

---------

Signed-off-by: Ankita Katiyar <[email protected]>
Signed-off-by: Jason Hite <[email protected]>

* Update docs

Signed-off-by: Ankita Katiyar <[email protected]>
Signed-off-by: Jason Hite <[email protected]>

* Update style to catch some more US spellings

Signed-off-by: Jo Stichbury <[email protected]>
Signed-off-by: Jason Hite <[email protected]>

* Make dataset factory resolve nested dict properly (#2993)

Signed-off-by: Jason Hite <[email protected]>

* Apply suggestions from code review

Co-authored-by: Jo Stichbury <[email protected]>
Signed-off-by: Jason Hite <[email protected]>

* Resolve suggestions from code review

Signed-off-by: Jason Hite <[email protected]>

* Revert changes ukspelling.yml

Signed-off-by: Ankita Katiyar <[email protected]>

---------

Signed-off-by: Jason Hite <[email protected]>
Signed-off-by: SajidAlamQB <[email protected]>
Signed-off-by: Ankita Katiyar <[email protected]>
Signed-off-by: Dmitry Sorokin <[email protected]>
Signed-off-by: Merel Theisen <[email protected]>
Signed-off-by: laizaparizotto <[email protected]>
Signed-off-by: Jo Stichbury <[email protected]>
Signed-off-by: Nok <[email protected]>
Signed-off-by: Nok Chan <[email protected]>
Signed-off-by: lrcouto <[email protected]>
Signed-off-by: Deepyaman Datta <[email protected]>
Signed-off-by: Ahdra Merali <ahdra.merali…
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Community Issue/PR opened by the open-source community
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants