Skip to content

Commit

Permalink
Cherry pick #44008 and #44022 (#44095)
Browse files Browse the repository at this point in the history
Signed-off-by: Cheng Su <[email protected]>
  • Loading branch information
c21 authored Mar 18, 2024
1 parent e032d7c commit 27b68d1
Show file tree
Hide file tree
Showing 12 changed files with 52 additions and 345 deletions.
2 changes: 1 addition & 1 deletion doc/source/data/data-internals.rst
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ Ray Data Internals
This guide describes the implementation of Ray Data. The intended audience is advanced
users and Ray Data developers.

For a gentler introduction to Ray Data, see :ref:`Key concepts <data_key_concepts>`.
For a gentler introduction to Ray Data, see :ref:`Quickstart <data_quickstart>`.

.. _dataset_concept:

Expand Down
14 changes: 6 additions & 8 deletions doc/source/data/data.rst
Original file line number Diff line number Diff line change
Expand Up @@ -8,16 +8,14 @@ Ray Data: Scalable Datasets for ML
:hidden:

Overview <overview>
key-concepts
quickstart
user-guide
examples
api/api
data-internals

Ray Data is a scalable data processing library for ML workloads. It provides flexible and performant APIs for scaling :ref:`Offline batch inference <batch_inference_overview>` and :ref:`Data preprocessing and ingest for ML training <ml_ingest_overview>`. Ray Data uses `streaming execution <https://www.anyscale.com/blog/streaming-distributed-execution-across-cpus-and-gpus>`__ to efficiently process large datasets.

.. image:: images/dataset.svg

..
https://docs.google.com/drawings/d/16AwJeBNR46_TsrkOmMbGaBK7u-OPsf_V8fHjU-d2PPQ/edit
Expand Down Expand Up @@ -57,19 +55,19 @@ Learn more

.. grid-item-card::

**Key Concepts**
**Quickstart**
^^^

Understand the key concepts behind Ray Data. Learn what
:ref:`Datasets <dataset_concept>` are and how they're used.
Datasets are and how they're used.

+++
.. button-ref:: data_key_concepts
.. button-ref:: data_quickstart
:color: primary
:outline:
:expand:

Learn Key Concepts
Quickstart

.. grid-item-card::

Expand Down Expand Up @@ -118,7 +116,7 @@ Learn more

.. grid-item-card::

**Ray blogs**
**Ray Blogs**
^^^

Get the latest on engineering updates from the Ray team and how companies are using Ray Data.
Expand Down
249 changes: 0 additions & 249 deletions doc/source/data/examples/gptj_batch_prediction.ipynb

This file was deleted.

Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@
"\n",
"We will use Ray Data and a pretrained model from Hugging Face hub. Note that you can easily adapt this example to use other similar models.\n",
"\n",
"It is highly recommended to read [Ray Train Key Concepts](train-key-concepts) and [Ray Data Key Concepts](data_key_concepts) before starting this example.\n",
"It is highly recommended to read [Ray Train Key Concepts](train-key-concepts) and [Ray Data Quickstart](data_quickstart) before starting this example.\n",
"\n",
"```{note}\n",
"In order to run this example, make sure your Ray cluster has access to at least one GPU with 16 or more GBs of memory. The amount of memory needed will depend on the model.\n",
Expand Down
1 change: 0 additions & 1 deletion doc/source/data/images/dataset.svg

This file was deleted.

Loading

0 comments on commit 27b68d1

Please sign in to comment.