Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

move memverge doc #4085

Merged
merged 2 commits into from
Sep 28, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 5 additions & 3 deletions rsts/conf.py
Original file line number Diff line number Diff line change
Expand Up @@ -94,7 +94,7 @@
#
# This is also used if you do content translation via gettext catalogs.
# Usually you set "language" from the command line for these cases.
language = None
language = 'en'

# List of patterns, relative to source directory, that match files and
# directories to ignore when looking for source files.
Expand Down Expand Up @@ -130,8 +130,10 @@
"github_username": "flyteorg",
"github_commit": "master",
"docs_path": "rsts", # path to documentation source
"sphinx_gallery_src_dir": "cookbook", # path to directory of sphinx gallery source files relative to repo root
"sphinx_gallery_dest_dir": "auto", # path to root directory containing auto-generated sphinx gallery examples
# path to directory of sphinx gallery source files relative to repo root
"sphinx_gallery_src_dir": "cookbook",
# path to root directory containing auto-generated sphinx gallery examples
"sphinx_gallery_dest_dir": "auto",
}

# Theme options are theme-specific and customize the look and feel of a theme
Expand Down
1 change: 1 addition & 0 deletions rsts/deployment/agents/bigquery.rst
Original file line number Diff line number Diff line change
Expand Up @@ -87,4 +87,5 @@ Upgrade the Flyte Helm release
helm upgrade <RELEASE_NAME> flyte/flyte-core -n <YOUR_NAMESPACE> --values values-override.yaml

Replace ``<RELEASE_NAME>`` with the name of your release (e.g., ``flyte``)

and ``<YOUR_NAMESPACE>`` with the name of your namespace (e.g., ``flyte``).
12 changes: 12 additions & 0 deletions rsts/deployment/agents/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -18,9 +18,21 @@ Discover the process of setting up Agents for Flyte.
^^^^^^^^^^^^
Guide to setting up the Bigquery agent.

---

.. link-button:: deployment-agent-setup-mmcloud
:type: ref
:text: MMCloud Agent
:classes: btn-block stretched-link
^^^^^^^^^^^^
Guide to setting up the MMCloud agent.


.. toctree::
:maxdepth: 1
:name: Agent setup
:hidden:

bigquery
mmcloud

119 changes: 119 additions & 0 deletions rsts/deployment/agents/mmcloud.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,119 @@
.. _deployment-agent-setup-mmcloud:

MMCloud Agent
=================

MemVerge Memory Machine Cloud (MMCloud) empowers users to continuously optimize cloud resources during runtime,
safely execute stateful tasks on spot instances,
and monitor resource usage in real time.
These capabilities make it an excellent fit for long-running batch workloads.

This guide provides an overview of how to set up MMCloud in your Flyte deployment.

Set up MMCloud
--------------

To run a Flyte workflow with Memory Machine Cloud, you will need to deploy Memory Machine Cloud.
Check out the `MMCloud User Guide <https://docs.memverge.com/mmce/current/userguide/olh/index.html>`_ to get started!

By the end of this step, you should have deployed an MMCloud OpCenter.

Spin up a cluster
-----------------

.. tabs::

.. group-tab:: Flyte binary

You can spin up a demo cluster using the following command:

.. code-block:: bash

flytectl demo start

Or install Flyte using the :ref:`flyte-binary helm chart <deployment-deployment-cloud-simple>`.

.. group-tab:: Flyte core

If you've installed Flyte using the
`flyte-core helm chart <https://github.com/flyteorg/flyte/tree/master/charts/flyte-core>`__, please ensure:

* You have the correct kubeconfig and have selected the correct Kubernetes context.
* You have configured the correct flytectl settings in ``~/.flyte/config.yaml``.

.. note::

Add the Flyte chart repo to Helm if you're installing via the Helm charts.

.. code-block:: bash

helm repo add flyteorg https://flyteorg.github.io/flyte

Specify agent configuration
----------------------------

Enable the MMCloud agent by adding the following config to the relevant YAML file(s):

.. code-block:: yaml

tasks:
task-agents:
enabled-agents:
- agent-service
default-for-task-types:
- mmcloud_task: agent-service

.. code-block:: yaml

agents:
agent-service:
agents:
mmcloud-agent:
endpoint: <AGENT_ENDPOINT>
insecure: true
supportedTaskTypes:
- mmcloud_task
agentForTaskTypes:
- mmcloud_task: mmcloud-agent

Substitute ``<AGENT_ENDPOINT>`` with the endpoint of your MMCloud agent.

Upgrade the deployment
----------------------

.. tabs::

.. group-tab:: Flyte binary

.. tabs::

.. group-tab:: Demo cluster

.. code-block:: bash

kubectl rollout restart deployment flyte-sandbox -n flyte

.. group-tab:: Helm chart

.. code-block:: bash

helm upgrade <RELEASE_NAME> flyteorg/flyte-binary -n <YOUR_NAMESPACE> --values <YOUR_YAML_FILE>

Replace ``<RELEASE_NAME>`` with the name of your release (e.g., ``flyte-backend``),
``<YOUR_NAMESPACE>`` with the name of your namespace (e.g., ``flyte``),
and ``<YOUR_YAML_FILE>`` with the name of your YAML file.

.. group-tab:: Flyte core

.. code-block::

helm upgrade <RELEASE_NAME> flyte/flyte-core -n <YOUR_NAMESPACE> --values values-override.yaml

Replace ``<RELEASE_NAME>`` with the name of your release (e.g., ``flyte``)
and ``<YOUR_NAMESPACE>`` with the name of your namespace (e.g., ``flyte``).

Wait for the upgrade to complete. You can check the status of the deployment pods by running the following command:

.. code-block::

kubectl get pods -n flyte
Loading