[Feature]: Allow listing available key resources such as `gpu`, `region`, and `backends`

[peterschmidt85]

Problem

Problem 1 (major): listing available GPUs per region and backend

As a user, sometimes, I want to know which GPUs are available within my project (and in which regions). Currently, I either have to specify a wide range in YAML and then try dstack apply and explore offers, or check it via specific backends' console UI (e.g. I do it often with RunPod - because they have a great and convenient UI for that).
It would be great if I could list available GPUs per region and backend and their availability (if this information is available) via the CLI.

Also, a minor yet related issue is when I don't know how certain GPUs are named with specific backends. This issue though should be better addressed by ensuring that dstack uses unified GPU names across all vendors and backends. But this may not be very easy to implement. That's why this issue is also related.

Note, the SSH backend is also relevant here as I may want to see what fleets are available (if I use dstack heavily on-pem machines).

Problem 2 (medium): listing available regions per backend

As a user, I may often want to see what regions are available so I can configure the backend or run accordingly, or to create volumes. Currently, I always have to either read the backend's documentation or use their console UI.

Overall, if I have to use backend's UI to work with backends, this isn't very productive nor convenient. We strive to make dstack the most simple way to work with multiple clouds. For that, we need to ensure all is possible without using backends' console UI.

Problem 3 (minor): listing available backends

As a user, sometimes I may want to know what backends are available. Currently, this information is only available via UI or the YAML files if you're a server admin. It would be a lot more convenient if I could access it via the CLI.

Solution

$ dstack list gpu [--backend BACKEND] [--region REGION] [--name NAME] [--all]

| <backend> | <GPU name/memory> | GPU vendor | <region> | <availability/quota information> |

$ dstack list region [--backend BACKEND] 

| <backend> | <region> |

$ dstack list backend

| <backend> |

Workaround

No response

Would you like to help us implement this feature by sending a PR?

Yes

[r4victor]

@peterschmidt85, should the command 1) list the GPUs/regions/backends available to the user (set up in the project) or 2) all resources supported by dstack. If it's 1), then you wouldn't be able "to see what regions are available so I can configure the backend" because you would only see the already configured regions.

[peterschmidt85]

@peterschmidt85, should the command 1) list the GPUs/regions/backends available to the user (set up in the project) or 2) all resources supported by dstack. If it's 1), then you wouldn't be able "to see what regions are available so I can configure the backend" because you would only see the already configured regions.

Yes, IMO, dstack list should show only what is available to the user (based on permissions and backend configurations); this way, it's easier for the user to discover what is available; (If backends' regions arent configured, the default is always all regions); also the user may re-configure to what they need.)

[Bihan]

Currently, I use the web consoles of cloud providers to browse through their available offers. This process is not that productive, as it requires logging into each provider's backend individually, and user interfaces vary between providers, requiring extra effort to figure out each time I log in. If dstack could provide a unified browsing experience through a dstack list command, it would simplify this process for users.

[peterschmidt85]

If dstack could provide a unified browsing experience through a dstack list command, it would simplify this process for users.

@Bihan Please check the suggested solution above and let me know if this would cover your use case, or of you have other comments/suggestions.

[Bihan]

@peterschmidt85 I think below command from the solutions would cover my use case. I think this command would list all the available gpu instances in all regions for a particular backend.

dstack list gpu [--backend BACKEND] [-all]

[github-actions]

This issue is stale because it has been open for 30 days with no activity.

[jvstme]

An alternative solution is to add a single command that would list all offers and allow filtering and aggregating by one or more arbitrary fields.

Aggregations will allow users to view concise lists of GPUs, regions, backends, as well as any other fields or combinations of fields available in the offers list. We can use the offers count and the price range as aggregate functions.

Examples

Base

$ dstack offers
 #  BACKEND  REGION     INSTANCE      RESOURCES                   SPOT  PRICE       
 1  gcp      us-west4   e2-highcpu-2  2xCPU, 2GB, 20.0GB (disk)   yes   $0.005966   
 2  aws      us-east-2  t2.small      1xCPU, 2GB, 100.0GB (disk)  yes   $0.0062     
 3  gcp      us-west4   e2-medium     2xCPU, 4GB, 20.0GB (disk)   yes   $0.006672
    ...

Filters

What on-demand instances are there in Vultr?

$ dstack offers --backend vultr --on-demand
 #  BACKEND  REGION  INSTANCE    RESOURCES                  SPOT  PRICE    
 1  vultr    ewr     vc2-1c-1gb  1xCPU, 1GB, 25.0GB (disk)  no    $0.007   
 2  vultr    ord     vc2-1c-1gb  1xCPU, 1GB, 25.0GB (disk)  no    $0.007   
 3  vultr    dfw     vc2-1c-1gb  1xCPU, 1GB, 25.0GB (disk)  no    $0.007
    ...

Aggregates

What backends are there?

$ dstack offers --aggregate backend
 #  BACKEND  OFFERS  PRICE
 1  aws      1000    from $0.0062 to $127.816
 2  gcp      3583    from $0.005966 to $126.643
 3  vultr    2883    from $0.007 to $23.92
    ...

Multiple aggregates

What backends provide which GPUs?

$ dstack offers --aggregate backend --aggregate resources.gpu.name
 #  BACKEND  GPU NAME  OFFERS  PRICE
 1  aws      A10G      128     from $0.2857 to $20.6757
 2  aws      L40S      64      from $0.5693 to $37.6761
 3  vultr    A100      19      from $0.123 to $9.589
    ...

Filters with aggregates

What regions in which backends provide 8 H100s?

$ dstack offers --gpu 8:H100 --aggregate backend --aggregate region
 #  BACKEND  REGION      OFFERS  PRICE
 1  aws      eu-north-1  2       from $24.45 to $105.202
 2  aws      us-east-2   2       from $24.7948 to $98.32
 3  vastai   is-iceland  1       $21.0045
    ...

Implementation details

• Aggregation should probably happen on the server to avoid transferring thousands of unaggregated offers to the client, and to allow reusing the implementation in the UI in the future.
• The format of server responses is an open question. The response structure can't be fixed because it depends on which aggregates the user requests, so the server may need to return raw tables. However, some typing and semantic details can be inferred from the field name, e.g. the CLI can add a dollar sign when displaying values in the "Price" field. We can also consider using strictly typed structures for unaggregated responses and raw tables for aggregated ones.
• There should probably be a row limit to avoid showing thousands of rows in the CLI. But we can allow overriding the limit without an upper bound using a CLI option.
• We'll need to allow aggregation by nested offer fields, such as GPU vendor. We can either provide a dot syntax, e.g. --aggregate resources.gpu.vendor, or provide aliases for all nested fields, e.g. --aggregate gpu_vendor. At the same time, we may want to provide a shorthand option to aggregate by a whole group of nested fields, e.g. --aggregate gpu could aggregate by GPU vendor+name+count+memory.
• Availability is an important field. It should be possible to filter by availability (e.g. --available) and aggregate by availability (e.g. --aggregate availability).
• Unlike in dstack apply, we may want to avoid any default filters (cpu: 2.., memory: 8GB.., etc) in dstack offers.

[github-actions]

This issue is stale because it has been open for 30 days with no activity.