Merge branch 'master' of github.com:flyteorg/flyte into fix-databricks

.github/workflows/flyteidl-release.yml

@@ -33,6 +33,7 @@ jobs:
       run:
         working-directory: flyteidl
     steps:
+      - uses: actions/checkout@v2
       - uses: actions/setup-node@v1
         with:
           node-version: "12.x"

.github/workflows/tests.yml

@@ -27,20 +27,35 @@ jobs:
   docs:
     runs-on: ubuntu-latest
     steps:
-      - name: Fetch the code
+      - name: Fetch flyte code
+        uses: actions/checkout@v4
+        with:
+          path: "${{ github.workspace }}/flyte"
+      - name: Fetch flytekit code
         uses: actions/checkout@v4
+        with:
+          repository: flyteorg/flytekit
+          path: "${{ github.workspace }}/flytekit"
       - uses: conda-incubator/setup-miniconda@v3
         with:
           auto-update-conda: true
           python-version: 3.9
       - shell: bash -el {0}
+        working-directory: ${{ github.workspace }}/flyte
         run: |
           conda install -c conda-forge conda-lock
           conda-lock install -n monodocs-env monodocs-environment.lock.yaml
       - shell: bash -el {0}
+        working-directory: ${{ github.workspace }}/flyte
         run: |
           conda activate monodocs-env
+          export SETUPTOOLS_SCM_PRETEND_VERSION="2.0.0"
           pip install -e ./flyteidl
+      - shell: bash -el {0}
+        working-directory: ${{ github.workspace }}/flytekit
+        run: |
+          conda activate monodocs-env
+          pip install -e .
           conda info
           conda list
           conda config --show-sources
@@ -49,6 +64,7 @@ jobs:
       - name: Setup Graphviz
         uses: ts-graphviz/setup-graphviz@v1
       - name: Build the documentation
+        working-directory: ${{ github.workspace }}/flyte
         shell: bash -el {0}
         run: |
           conda activate monodocs-env

CHANGELOG/CHANGELOG-v0.11.0.md

@@ -4,7 +4,7 @@
 * New to flyte? https://start.flyte.org takes you through first run experience. (Thanks to @jeevb)
 * [Grafana templates](https://docs.flyte.org/en/latest/howto/monitoring/index.html) for monitoring Flyte System and User Workflows.
 * [Extend Flyte](https://docs.flyte.org/en/latest/plugins/index.html) docs.
-* [FlyteIdl Docs](https://docs.flyte.org/projects/flyteidl/en/latest/) are published! You can learn about the core language that makes it all work.
+* [FlyteIdl Docs](https://docs.flyte.org/en/latest/reference_flyteidl.html) are published! You can learn about the core language that makes it all work.
 * [Additional knob](https://github.com/flyteorg/flytepropeller/pull/219/files#diff-91657d6448dfbf87f4cecf126ad02bd668ea233edcf74e860ef4f54bdd4cb552R78) for fine tuning flyte propeller performance that speeds up executions drastically.
 * OidC support for Google Idp (And other OidC compliant Idps)
 * Various stabilization bugs.

CHANGELOG/CHANGELOG-v0.14.0.md

@@ -11,7 +11,7 @@
 - performance improvement for dynamic workflows
 
 ## Flyteconsole
- - Bug fixes 
+ - Bug fixes
  - More updates coming soon
 
 ## Flytekit
@@ -27,6 +27,6 @@
      - More use case driven examples in flytesnacks
 
 ## flytectl
- - flytectl is ready for BETA. check it out - https://docs.flyte.org/projects/flytectl/en/latest/
+ - flytectl is ready for BETA. check it out - https://docs.flyte.org/en/latest/flytectl/overview.html
 
 Please see the [flytekit release](https://github.com/flyteorg/flytekit/releases/tag/v0.18.0) for the full list and more details.

CHANGELOG/CHANGELOG-v0.17.0.md

@@ -1,16 +1,16 @@
 # Flyte v0.17.0
 
 ## Platform
-1. Recovery Mode: Executions that fail due to external system failures (e.g. external system being down) can now be rerun in recovery mode ([flytectl --recover docs](https://docs.flyte.org/projects/flytectl/en/latest/gen/flytectl_create_execution.html)). It's also available in the UI:
+1. Recovery Mode: Executions that fail due to external system failures (e.g. external system being down) can now be rerun in recovery mode ([flytectl --recover docs](https://docs.flyte.org/en/latest/flytectl/gen/flytectl_create_execution.html)). It's also available in the UI:
 
    <img src="https://i.imgur.com/hYYzkLK.png" alt="Recovery mode in the UI" width="800"/>
 
 
 ## Flytekit
-1. Great Expectations Integration ([docs](https://docs.flyte.org/projects/cookbook/en/latest/auto/integrations/flytekit_plugins/greatexpectations/index.html#great-expectations)).
+1. Great Expectations Integration ([docs](https://docs.flyte.org/en/latest/flytesnacks/examples/greatexpectations_plugin/index.html)).
 1. Access to durable blob stores (AWS/GCS/etc) are now pluggable.
 1. Local task execution has been updated to also trigger the type engine.
-1. Tasks that have `cache=True` should now be cached when running locally as well ([docs](https://docs.flyte.org/projects/cookbook/en/latest/auto/core/flyte_basics/task_cache.html#how-local-caching-works)).
+1. Tasks that have `cache=True` should now be cached when running locally as well ([docs](https://docs.flyte.org/en/latest/user_guide/development_lifecycle/caching.html#how-does-local-caching-work)).
 
 Please see the [flytekit release](https://github.com/flyteorg/flytekit/releases/tag/v0.22.0) for the full list and more details.
 
@@ -20,7 +20,7 @@ Please see the [flytekit release](https://github.com/flyteorg/flytekit/releases/
    <img src="https://i.imgur.com/HTfuios.png" alt="New Graph UX" width="800"/>
 1. JSON Validation for json-based types in the UI.
 
-   
+
 
 | Before | After |
 | -------- | -------- |
@@ -32,7 +32,7 @@ Please see the [flytekit release](https://github.com/flyteorg/flytekit/releases/
 
 
 ## FlyteCtl
-1. `flytectl upgrade` to automatically upgrade itself ([docs](https://docs.flyte.org/projects/flytectl/en/latest/gen/flytectl_upgrade.html)).
+1. `flytectl upgrade` to automatically upgrade itself ([docs](https://docs.flyte.org/en/latest/flytectl/gen/flytectl_upgrade.html)).
 1. `--dryRun` is available in most commands with server-side-effects to simulate the operations before committing any changes.
 
 And various stabilization [fixes](https://github.com/flyteorg/flyte/milestone/17?closed=1)!

CHANGELOG/CHANGELOG-v0.5.0.md

@@ -2,12 +2,12 @@
 
 ## Infrastructure
 - Moved CI/CD to Github Actions
-- Added end-to-end tests as part of the PR & master merges. 
+- Added end-to-end tests as part of the PR & master merges.
 - Enable CI system to run on forks.
 
 ## Core Platform
-- [Single Task Execution](https://flyte.readthedocs.io/en/latest/user/features/single_task_execution.html) to enable registering and launching tasks outside the scope of a workflow to enable faster iteration and a more intuitive development workflow.
-- [Run to completion](https://flyte.readthedocs.io/en/latest/user/features/on_failure_policy.html) to enable workflows to continue executing even if one or more branches fail.
+- [Single Task Execution](https://docs.flyte.org/en/latest/user_guide/development_lifecycle/running_tasks.html) to enable registering and launching tasks outside the scope of a workflow to enable faster iteration and a more intuitive development workflow.
+- [Run to completion](https://docs.flyte.org/en/latest/protos/docs/core/core.html#ref-flyteidl-core-workflowmetadata-onfailurepolicy) to enable workflows to continue executing even if one or more branches fail.
 - Fixed retries for dynamically yielded nodes.
 - PreAlpha Support for Raw container with FlyteCoPilot. (docs coming soon). [Sample Notebooks](https://github.com/lyft/flytekit/blob/master/sample-notebooks/raw-container-shell.ipynb). This makes it possible to run workflows with arbitrary containers
 

CHANGELOG/CHANGELOG-v0.9.0.md

@@ -1,7 +1,7 @@
 
 
 ### Flytekit
-* Fast register - code only changes no longer need a rebuild of the container to run ([read the docs!](https://flyte.readthedocs.io/en/latest/user/features/fast_registration.html))
+* [Fast register](https://docs.flyte.org/en/latest/flyte_fundamentals/registering_workflows.html) - code only changes no longer need a rebuild of the container to run.
 * Project archive/activate command in flyte-cli
 * Bug fixes and misc improvements
 

CHANGELOG/CHANGELOG-v1.1.0.md

@@ -4,7 +4,7 @@
 ### User Improvements
 Support for [Optional types](https://github.com/flyteorg/flyte/issues/2426). With the inclusion of Union types in flytekit, we can now support optional types.
 
-[Flyte Deck](https://github.com/flyteorg/flyte/issues/2175) is now available. Please take a look at the [documentation](https://docs.flyte.org/projects/cookbook/en/latest/auto/core/flyte_basics/deck.html#sphx-glr-auto-core-flyte-basics-deck-py) and also the [OSS presentation](https://www.youtube.com/watch?v=KqyBYIaAZ7c) that was done a few weeks back.
+[Flyte Deck](https://github.com/flyteorg/flyte/issues/2175) is now available. Please take a look at the [documentation](https://docs.flyte.org/en/latest/user_guide/development_lifecycle/decks.html) and also the [OSS presentation](https://www.youtube.com/watch?v=KqyBYIaAZ7c) that was done a few weeks back.
 
 
 ### Backend Improvements

CHANGELOG/CHANGELOG-v1.10.0.md

@@ -8,9 +8,9 @@ Programmatically consuming inputs and outputs using flyteremote became a lot eas
 
 ![Usage snippet](./images/v1.10.0-flyteconsole-programmatic-access.png)
 
-You'll now be able to use offloaded types in [eager workflows](https://docs.flyte.org/projects/cookbook/en/latest/auto_examples/advanced_composition/eager_workflows.html#id1).
+You'll now be able to use offloaded types in [eager workflows](https://docs.flyte.org/en/latest/user_guide/advanced_composition/eager_workflows.html).
 
-More ergonomic improvements to [pyflyte](https://docs.flyte.org/projects/flytekit/en/latest/pyflyte.html#pyflyte-cli), including the inclusion of a progress bar, the ability to activate launchplans, and the ability to interact with gate nodes in local executions.
+More ergonomic improvements to [pyflyte](https://docs.flyte.org/en/latest/api/flytekit/pyflyte.html), including the inclusion of a progress bar, the ability to activate launchplans, and the ability to interact with gate nodes in local executions.
 
 And much more. Here's the exhaustive list of changes: