diff --git a/rsts/deployment/plugins/index.rst b/rsts/deployment/plugins/index.rst index d83bd79c54f..88dd7516cdf 100644 --- a/rsts/deployment/plugins/index.rst +++ b/rsts/deployment/plugins/index.rst @@ -48,6 +48,15 @@ to use these plugins in your workflows written in ``flytekit``. ^^^^^^^^^^^^ Guide to setting up GCP-specific Plugins. + --- + + .. link-button:: deployment-plugin-setup-memverge + :type: ref + :text: MemVerge Plugin + :classes: btn-block stretched-link + ^^^^^^^^^^^^ + Guide to setting up MemVerge Plugin. + .. toctree:: :maxdepth: 1 :name: Plugin Setup @@ -57,3 +66,4 @@ to use these plugins in your workflows written in ``flytekit``. aws/index gcp/index webapi/index + memverge/index diff --git a/rsts/deployment/plugins/memverge/index.rst b/rsts/deployment/plugins/memverge/index.rst new file mode 100644 index 00000000000..090d1cc5cea --- /dev/null +++ b/rsts/deployment/plugins/memverge/index.rst @@ -0,0 +1,27 @@ +.. _deployment-plugin-setup-memverge: + +Configure MemVerge Plugin +========================= + +.. tags:: AWS, GCP, AliCloud, Integration, Advanced + +Discover the process of setting up MemVerge plugin for Flyte. + +.. panels:: + :header: text-center + :column: col-lg-12 p-2 + + .. link-button:: deployment-plugin-setup-memverge-mmcloud + :type: ref + :text: MMCloud Plugin + :classes: btn-block stretched-link + ^^^^^^^^^^^^ + Guide to setting up the MMCloud plugin. + + +.. toctree:: + :maxdepth: 1 + :name: MemVerge plugin setup + :hidden: + + mmcloud diff --git a/rsts/deployment/plugins/memverge/mmcloud.rst b/rsts/deployment/plugins/memverge/mmcloud.rst new file mode 100644 index 00000000000..8bb442fc2bd --- /dev/null +++ b/rsts/deployment/plugins/memverge/mmcloud.rst @@ -0,0 +1,119 @@ +.. _deployment-plugin-setup-memverge-mmcloud: + +MMCloud Plugin +================= + +MemVerge Memory Machine Cloud (MMCloud) empowers users to continuously optimize cloud resources during runtime, +safely execute stateful tasks on spot instances, +and monitor resource usage in real time. +These capabilities make it an excellent fit for long-running batch workloads. + +This guide provides an overview of how to set up MMCloud in your Flyte deployment. + +Set up MMCloud +-------------- + +To run a Flyte workflow with Memory Machine Cloud, you will need to deploy Memory Machine Cloud. +Check out the `MMCloud User Guide `_ to get started! + +By the end of this step, you should have deployed an MMCloud OpCenter. + +Spin up a cluster +----------------- + +.. tabs:: + + .. group-tab:: Flyte binary + + You can spin up a demo cluster using the following command: + + .. code-block:: bash + + flytectl demo start + + Or install Flyte using the :ref:`flyte-binary helm chart `. + + .. group-tab:: Flyte core + + If you've installed Flyte using the + `flyte-core helm chart `__, please ensure: + + * You have the correct kubeconfig and have selected the correct Kubernetes context. + * You have configured the correct flytectl settings in ``~/.flyte/config.yaml``. + +.. note:: + + Add the Flyte chart repo to Helm if you're installing via the Helm charts. + + .. code-block:: bash + + helm repo add flyteorg https://flyteorg.github.io/flyte + +Specify plugin configuration +---------------------------- + +Enable the MMCloud plugin by adding the following config to the relevant YAML file(s): + +.. code-block:: yaml + + tasks: + task-plugins: + enabled-plugins: + - agent-service + default-for-task-types: + - mmcloud_task: agent-service + +.. code-block:: yaml + + plugins: + agent-service: + agents: + mmcloud-agent: + endpoint: + insecure: true + supportedTaskTypes: + - mmcloud_task + agentForTaskTypes: + - mmcloud_task: mmcloud-agent + +Substitute ```` with the endpoint of your MMCloud agent. + +Upgrade the deployment +---------------------- + +.. tabs:: + + .. group-tab:: Flyte binary + + .. tabs:: + + .. group-tab:: Demo cluster + + .. code-block:: bash + + kubectl rollout restart deployment flyte-sandbox -n flyte + + .. group-tab:: Helm chart + + .. code-block:: bash + + helm upgrade flyteorg/flyte-binary -n --values + + Replace ```` with the name of your release (e.g., ``flyte-backend``), + ```` with the name of your namespace (e.g., ``flyte``), + and ```` with the name of your YAML file. + + .. group-tab:: Flyte core + + .. code-block:: + + helm upgrade flyte/flyte-core -n --values values-override.yaml + + Replace ```` with the name of your release (e.g., ``flyte``) + and ```` with the name of your namespace (e.g., ``flyte``). + +Wait for the upgrade to complete. You can check the status of the deployment pods by running the following command: + +.. code-block:: + + kubectl get pods -n flyte