Api embeding as service #127

Kleczyk · 2024-04-04T01:37:14Z

embeding-api is working more info in README.md Closes #122

pgronkievitz

real quick, didn't took deeper look at it

pgronkievitz · 2024-04-04T05:25:36Z

.gitignore

@@ -5,5 +5,6 @@ models/**
 db/
 static/
 .idea/
+embeding_models/e5-large-v2/


issue: this shouldn't be like that, it's a submodule

I remove this and add new .gitignore

pgronkievitz · 2024-04-04T05:27:04Z

embedingModels/README.md

issue: tons of typos, please use some spellchecker and preferably Language Tool

it was late xd done

pgronkievitz · 2024-04-04T05:27:41Z

embedingModels/README.md

+#### IMPORTANT
+
+keep file tree like this !!!
+
+```sh
+embeding_models
+├── e5-large-v2
+│   ├── 1_Pooling
+│   │   └── config.json
+│   ├── config.json
+│   ├── handler.py
+│   ├── model.safetensors
+│   ├── modules.json
+│   ├── pytorch_model.bin
+│   ├── README.md
+│   ├── sentence_bert_config.json
+│   ├── special_tokens_map.json
+│   ├── tokenizer_config.json
+│   ├── tokenizer.json
+│   └── vocab.txt
+└── README.md
+```


suggestion: just don't mess with submodule, as it's another repo you don't have permissions for :v

I will fix to download only models in nextgod for now as in llm I have to download the repo for the container

pgronkievitz · 2024-04-04T05:28:06Z

embedingModels/README.md

+48 deploy:
+49      resources:
+50        reservations:
+51          devices:
+52          - driver: nvidia
+53            count: 1
+54            capabilities: [ gpu ]


issue: create separate profiles for gpu and cpu or separate compose for the gpu (you can use 2 compose files using -f flag and 2nd option will just override the first one)

in the future

pgronkievitz · 2024-04-04T05:30:34Z

embedingModels/pyproject.toml

+python = "^3.10"
+transformers = "^4.39.3"
+torch = "^2.2.2"


nitpick: use ~major.minor instead of ^major.minor.patch, it'll be "safer" as some tools can introduce breaking changes with minors (but they shouldn't)

I remove this

pgronkievitz · 2024-04-06T10:37:16Z

.gitignore

question: ???

README.md

embedding/.gitignore

pgronkievitz · 2024-04-06T10:39:29Z

embedding/README.md

@@ -0,0 +1,61 @@
+# embedding-api
+
+## How run servis


issue: SPELL. CHECK.

pgronkievitz · 2024-04-06T10:41:01Z

embedding/README.md

+48 deploy:
+49      resources:
+50        reservations:
+51          devices:
+52          - driver: nvidia
+53            count: 1
+54            capabilities: [ gpu ]


issue: i think i already told you about line numbers in here?

pgronkievitz · 2024-04-06T10:42:16Z

embedding/README.md

+```sh
+cd embedding_models
+git clone [email protected]:intfloat/e5-large-v2
+```


issue: DO NOT do it like this. either use submodules or subtrees. cloning on your own into folder inside repo is just dumb.

Ok. I'll do it as submodules

embedding/README.md

Co-authored-by: Mateusz <[email protected]>

Co-authored-by: Patryk Gronkiewicz <[email protected]>

pgronkievitz · 2024-04-18T10:45:00Z

.gitignore

pgronkievitz · 2024-04-18T10:45:36Z

README.md

 ```sh
-cd models
-git clone [email protected]:intfloat/e5-large-v2
+cd embedding/models
+git submodule add [email protected]:intfloat/e5-large-v2


issue: nope, that's wrong, read more about submodules and how to use 'em

pgronkievitz · 2024-04-18T10:46:02Z

embedding/.gitignore

issue: nope

Kleczyk · 2024-04-28T11:48:49Z

To explain why I removed most of the files. In lama.cpp all llm models have a "built-in embedding model" I knew this but I thought that if we use a newer one then llm will work better and this was also said at the meetings but nothing could be further from the truth since a particular model benefited from a specific embedding model during training will work better with the one it was trained with. Using the lama.cpp server I have access directly to the embedding model itself as a separate EP. For this reason, I discarded the idea of creating a separate service for embedding.

TheJimmyNowak · 2024-04-28T17:35:25Z

docker-compose.yml

@@ -28,11 +28,27 @@ services:
    depends_on:
      - db

-  llm:
-    profiles: [ "dev", "prod" ]
+  llm-embedding:


Add -cpu to this line.

because I deleted the files to which there were objections and went another way

Kleczyk added 2 commits April 4, 2024 01:57

add embeding-api all is working

47fa3bd

update and add README and cleaning

e7158e6

github-actions bot requested a review from Sygnator April 4, 2024 01:37

Kleczyk added 8 commits April 4, 2024 03:38

cleaning

70083c7

fix readme

9567025

fix readme

bb31bcd

fix readme

9497e30

fix readme

3c6fe8b

fix readme

b3d949e

fix readme

e6d7bfa

fix readme

2d616b6

pgronkievitz changed the title ~~Api embeding as servis~~ Api embeding as service Apr 4, 2024

pgronkievitz requested changes Apr 4, 2024

View reviewed changes

Kleczyk added 3 commits April 4, 2024 11:46

cleaning up and adding gitignre to the embedding model

3b58d68

cleaning up and adding gitignre to the embedding model

5f386b1

spell check and improvement

bfa74b3

Kleczyk requested a review from pgronkievitz April 4, 2024 10:11

pgronkievitz assigned Kleczyk Apr 4, 2024

pgronkievitz requested changes Apr 6, 2024

View reviewed changes

Sygnator reviewed Apr 15, 2024

View reviewed changes

embedding/README.md Outdated Show resolved Hide resolved

Kleczyk and others added 5 commits April 16, 2024 00:00

Update embedding/README.md

0eef691

Co-authored-by: Mateusz <[email protected]>

Update README.md

d97e97f

Co-authored-by: Patryk Gronkiewicz <[email protected]>

Update embedding/.gitignore

c2489cb

Co-authored-by: Patryk Gronkiewicz <[email protected]>

Merge branch 'main' of github.com:knmlprz/ChatKNML into api-embeding

60af287

change to submodules and fix read me

8cfb785

pgronkievitz previously requested changes Apr 18, 2024

View reviewed changes

Kleczyk added 3 commits April 22, 2024 16:35

Merge branch 'main' of github.com:knmlprz/ChatKNML into api-embeding

02e8ab7

remove files and services with external embedding

85a6cb5

add note for llm-embedding api for gpu and cpu services

ed4074d

Kleczyk requested a review from pgronkievitz April 28, 2024 11:49

TheJimmyNowak requested changes Apr 28, 2024

View reviewed changes

rename service from llm-embedding to llm-embedding-cpu

fe1077f

Kleczyk requested review from TheJimmyNowak and Sygnator April 28, 2024 18:20

Update .gitignore

7057f3d

TheJimmyNowak approved these changes May 8, 2024

View reviewed changes

Merge branch 'main' into api-embeding

0b23454

Kleczyk merged commit 9f7c2bc into main May 12, 2024
3 checks passed

Kleczyk deleted the api-embeding branch May 12, 2024 16:48

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Api embeding as service #127

Api embeding as service #127

Kleczyk commented Apr 4, 2024

pgronkievitz left a comment

pgronkievitz Apr 4, 2024

Kleczyk Apr 4, 2024

pgronkievitz Apr 4, 2024

Kleczyk Apr 4, 2024

pgronkievitz Apr 4, 2024

Kleczyk Apr 4, 2024 •

edited

Loading

pgronkievitz Apr 4, 2024

Kleczyk Apr 4, 2024

pgronkievitz Apr 4, 2024

Kleczyk Apr 4, 2024

pgronkievitz Apr 6, 2024

pgronkievitz Apr 18, 2024

pgronkievitz Apr 6, 2024

Kleczyk Apr 17, 2024

pgronkievitz Apr 6, 2024

Kleczyk Apr 17, 2024

pgronkievitz Apr 6, 2024

Kleczyk Apr 17, 2024 •

edited

Loading

pgronkievitz Apr 18, 2024

pgronkievitz Apr 18, 2024

pgronkievitz Apr 18, 2024

Kleczyk commented Apr 28, 2024

TheJimmyNowak Apr 28, 2024

Api embeding as service #127

Api embeding as service #127

Conversation

Kleczyk commented Apr 4, 2024

pgronkievitz left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Kleczyk Apr 4, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Kleczyk Apr 17, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Kleczyk commented Apr 28, 2024

Choose a reason for hiding this comment

Kleczyk Apr 4, 2024 •

edited

Loading

Kleczyk Apr 17, 2024 •

edited

Loading