Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Dockerfile for installing kit as a KServe ClusterStorageContainer #61

Merged
merged 1 commit into from
Mar 15, 2024

Conversation

amisevsk
Copy link
Contributor

@amisevsk amisevsk commented Mar 4, 2024

Description

Add a Dockerfile to enable using kit as a ClusterStorageContainer for KServe via the following CR:

apiVersion: "serving.kserve.io/v1alpha1"
kind: ClusterStorageContainer
metadata:
  name: kitops
spec:
  container:
    name: kit-storage-initializer
    image: <kit-kserve-image>
    imagePullPolicy: Always
    env:
      - name: KIT_EXPORT_FLAGS
        value: ""       # Optional: Add extra flags for the `kit export` command
    resources:
      requests:
        memory: 100Mi
        cpu: 100m
      limits:
        memory: 1Gi     # Note: Default value, not measured
    supportedUriFormats:
    - prefix: kit://

This will allow downloading modelkits by specifying storageUris with kit:// -- for example

apiVersion: "serving.kserve.io/v1beta1"
kind: "InferenceService"
metadata:
  name: "torchserve"
spec:
  predictor:
    model:
      modelFormat:
        name: pytorch
      storageUri: kit://<modelkit-reference>

Note that KServe inference services generally assume a certain file structure for the model/configuration; as a result building dedicated modelkits for KServe is still likely required.

@bmicklea bmicklea added enhancement New feature or request CLI Topics related to the CLI *Salmon not pink ModelKit Topics related to ModelKit and removed CLI Topics related to the CLI *Salmon not pink labels Mar 12, 2024
build/dockerfiles/KServe/README.md Show resolved Hide resolved
build/dockerfiles/KServe/README.md Show resolved Hide resolved
Add a Dockerfile that builds an image suitable to using `kit` as a
ClusterStorageContainer for KServe installs. Included is a short
README.md that gives a general overview of how to use this container.

To come: automating multi-platform builds of this image to allow it to
be more easily used.
@amisevsk amisevsk requested a review from gorkem March 15, 2024 19:06
@amisevsk amisevsk merged commit 8d73093 into main Mar 15, 2024
1 check passed
@amisevsk amisevsk deleted the kserve-storage branch March 15, 2024 19:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request ModelKit Topics related to ModelKit
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants