Skip to content

Commit

Permalink
move memverge doc (#4085)
Browse files Browse the repository at this point in the history
Signed-off-by: helenzhangyc <[email protected]>
Signed-off-by: Helen Zhang <[email protected]>
  • Loading branch information
helenzhangyc authored Sep 28, 2023
1 parent e57cac0 commit d35c29a
Show file tree
Hide file tree
Showing 4 changed files with 137 additions and 3 deletions.
8 changes: 5 additions & 3 deletions rsts/conf.py
Original file line number Diff line number Diff line change
Expand Up @@ -94,7 +94,7 @@
#
# This is also used if you do content translation via gettext catalogs.
# Usually you set "language" from the command line for these cases.
language = None
language = 'en'

# List of patterns, relative to source directory, that match files and
# directories to ignore when looking for source files.
Expand Down Expand Up @@ -130,8 +130,10 @@
"github_username": "flyteorg",
"github_commit": "master",
"docs_path": "rsts", # path to documentation source
"sphinx_gallery_src_dir": "cookbook", # path to directory of sphinx gallery source files relative to repo root
"sphinx_gallery_dest_dir": "auto", # path to root directory containing auto-generated sphinx gallery examples
# path to directory of sphinx gallery source files relative to repo root
"sphinx_gallery_src_dir": "cookbook",
# path to root directory containing auto-generated sphinx gallery examples
"sphinx_gallery_dest_dir": "auto",
}

# Theme options are theme-specific and customize the look and feel of a theme
Expand Down
1 change: 1 addition & 0 deletions rsts/deployment/agents/bigquery.rst
Original file line number Diff line number Diff line change
Expand Up @@ -87,4 +87,5 @@ Upgrade the Flyte Helm release
helm upgrade <RELEASE_NAME> flyte/flyte-core -n <YOUR_NAMESPACE> --values values-override.yaml
Replace ``<RELEASE_NAME>`` with the name of your release (e.g., ``flyte``)

and ``<YOUR_NAMESPACE>`` with the name of your namespace (e.g., ``flyte``).
12 changes: 12 additions & 0 deletions rsts/deployment/agents/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -18,9 +18,21 @@ Discover the process of setting up Agents for Flyte.
^^^^^^^^^^^^
Guide to setting up the Bigquery agent.

---

.. link-button:: deployment-agent-setup-mmcloud
:type: ref
:text: MMCloud Agent
:classes: btn-block stretched-link
^^^^^^^^^^^^
Guide to setting up the MMCloud agent.


.. toctree::
:maxdepth: 1
:name: Agent setup
:hidden:

bigquery
mmcloud

119 changes: 119 additions & 0 deletions rsts/deployment/agents/mmcloud.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,119 @@
.. _deployment-agent-setup-mmcloud:

MMCloud Agent
=================

MemVerge Memory Machine Cloud (MMCloud) empowers users to continuously optimize cloud resources during runtime,
safely execute stateful tasks on spot instances,
and monitor resource usage in real time.
These capabilities make it an excellent fit for long-running batch workloads.

This guide provides an overview of how to set up MMCloud in your Flyte deployment.

Set up MMCloud
--------------

To run a Flyte workflow with Memory Machine Cloud, you will need to deploy Memory Machine Cloud.
Check out the `MMCloud User Guide <https://docs.memverge.com/mmce/current/userguide/olh/index.html>`_ to get started!

By the end of this step, you should have deployed an MMCloud OpCenter.

Spin up a cluster
-----------------

.. tabs::

.. group-tab:: Flyte binary

You can spin up a demo cluster using the following command:

.. code-block:: bash
flytectl demo start
Or install Flyte using the :ref:`flyte-binary helm chart <deployment-deployment-cloud-simple>`.

.. group-tab:: Flyte core

If you've installed Flyte using the
`flyte-core helm chart <https://github.com/flyteorg/flyte/tree/master/charts/flyte-core>`__, please ensure:

* You have the correct kubeconfig and have selected the correct Kubernetes context.
* You have configured the correct flytectl settings in ``~/.flyte/config.yaml``.

.. note::

Add the Flyte chart repo to Helm if you're installing via the Helm charts.

.. code-block:: bash
helm repo add flyteorg https://flyteorg.github.io/flyte
Specify agent configuration
----------------------------

Enable the MMCloud agent by adding the following config to the relevant YAML file(s):

.. code-block:: yaml
tasks:
task-agents:
enabled-agents:
- agent-service
default-for-task-types:
- mmcloud_task: agent-service
.. code-block:: yaml
agents:
agent-service:
agents:
mmcloud-agent:
endpoint: <AGENT_ENDPOINT>
insecure: true
supportedTaskTypes:
- mmcloud_task
agentForTaskTypes:
- mmcloud_task: mmcloud-agent
Substitute ``<AGENT_ENDPOINT>`` with the endpoint of your MMCloud agent.

Upgrade the deployment
----------------------

.. tabs::

.. group-tab:: Flyte binary

.. tabs::

.. group-tab:: Demo cluster

.. code-block:: bash
kubectl rollout restart deployment flyte-sandbox -n flyte
.. group-tab:: Helm chart

.. code-block:: bash
helm upgrade <RELEASE_NAME> flyteorg/flyte-binary -n <YOUR_NAMESPACE> --values <YOUR_YAML_FILE>
Replace ``<RELEASE_NAME>`` with the name of your release (e.g., ``flyte-backend``),
``<YOUR_NAMESPACE>`` with the name of your namespace (e.g., ``flyte``),
and ``<YOUR_YAML_FILE>`` with the name of your YAML file.

.. group-tab:: Flyte core

.. code-block::
helm upgrade <RELEASE_NAME> flyte/flyte-core -n <YOUR_NAMESPACE> --values values-override.yaml
Replace ``<RELEASE_NAME>`` with the name of your release (e.g., ``flyte``)
and ``<YOUR_NAMESPACE>`` with the name of your namespace (e.g., ``flyte``).

Wait for the upgrade to complete. You can check the status of the deployment pods by running the following command:

.. code-block::
kubectl get pods -n flyte

0 comments on commit d35c29a

Please sign in to comment.