Embedder Service v0 with FlashBert

[magdyksaleh]

Initialize:

docker run --gpus all --shm-size 1g -p 8080:80 -v $volume:/data \
    ghcr.io/predibase/lorax:main --model-id WhereIsAI/UAE-Large-V1

Embed:

curl 127.0.0.1:8080/embed \
-X POST \
-d '{
"inputs": "[INST] Natalia sold clips to 48 of her friends in April, and then she sold half as many clips in May. How many clips did Natalia sell altogether in April and May? [/INST]"
}' \
-H 'Content-Type: application/json'

Python:

client = Client(...)
resp = client.embed("What is the capital of France?")
resp.embeddings

[magdyksaleh]

curl 127.0.0.1:8080/embed
-X POST
-d '{
"inputs": "[INST] Natalia sold clips to 48 of her friends in April, and then she sold half as many clips in May. How many clips did Natalia sell altogether in April and May? [/INST]"
}'
-H 'Content-Type: application/json'

;)

[tgaddair]

Looks great! Couple minor questions.

[tgaddair]

Is this optional for embedding responses, or required?

[tgaddair]

Conversely, what happens when someone tries to use the BERT model for text generation?