forked from alexiusstrauss/AudioTopic
-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Add Readme and tests files
- Loading branch information
Showing
26 changed files
with
724 additions
and
800 deletions.
There are no files selected for viewing
Binary file not shown.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,32 @@ | ||
SHELL := /bin/bash | ||
FILES=$(shell docker ps -a -q --filter "name=audiotopic*") | ||
|
||
# Verifica versao do docker compose. | ||
COMPOSE_COMMAND=$(shell command -v docker-compose >/dev/null 2>&1 && echo "docker-compose" || echo "docker compose") | ||
|
||
build: | ||
docker build -t audiotopic-api:latest ./backend | ||
docker build -t audiotopic-app:latest ./frontend | ||
|
||
up: build | ||
docker compose up $(dettach) audiotopic-app | ||
|
||
down: | ||
docker compose down -v | ||
|
||
bash-api: | ||
docker exec -ti audiotopic-api bash | ||
|
||
bash-app: | ||
docker exec -ti audiotopic-app bash | ||
|
||
test-api: | ||
docker exec -ti audiotopic-api pipenv run pytest | ||
|
||
logs-api: | ||
docker logs -f audiotopic-api | ||
|
||
logs-app: | ||
docker logs -f audiotopic-app | ||
|
||
.PHONY: all clean install test |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1 +1,52 @@ | ||
## AudioTopic-app | ||
# AudioTopic | ||
|
||
|
||
**AudioTopic** é um projeto com base em um teste para processar arquivos de áudio nos formatos: .mp3 ou .wav | ||
converter em texto usando a tecnologia STT, fazer um resumo deste conteúdo utilizando LLM e criar um áudio do resumo, disponibilizando para ouvir ou fazer download do resumo. | ||
Segue o link para leitura do desfio técnico e suas especificações. | ||
[Arquivo de especificações](AudioTipic-test.pdf) | ||
|
||
|
||
|
||
### Backend | ||
No backend utilizei a linguagem **python na versão 3.10** e o framework **FastAPI** na construção da api. | ||
Utilizei o padrão de projeto "**Strategy**" para implementar através de uma interface a engine que será usada para resumir o texto usando LLM. Deixei disponível duas engines: **LangChain** e **Tensorflow** e Seguindo o **SOLID** e **Clean Code**, deixando código mais flexível e fácil de testar ou dar manutenção. | ||
Para rodar os testes automatizados via Docker basta rodar o comando: `make test-api` caso o container esteja rodando, ou `docker exec -ti audiotopic-api pipenv run pytest`. | ||
|
||
Deverá configurar um .env na pasta backend/src/ com as credenciais da OPENAI para utilização do serviço com LangChain. | ||
|
||
OPEN_AI_TOKEN=sk-change-me | ||
TENSORFLOW_MODEL_NAME="t5-small" | ||
|
||
No arquivo **api.py** poderá configurar qual engine usar para o processo de resumo escolhendo LangChain ou TensorFlow: | ||
|
||
|
||
service = DeepDive(llm_engine=LLM_ENGINES.get("LangChain")) | ||
service = DeepDive(llm_engine=LLM_ENGINES.get("TensorFlow")) | ||
|
||
**Tecnologias utilizadas:** | ||
- Python 3.10 | ||
- Pipenv | ||
- FastAPI | ||
- Pytest e Pytest-cov | ||
- Docker | ||
- Make | ||
|
||
|
||
### Frontend | ||
Utilizei o framework Nuxt3 e criei um componente responsável consumir a API. | ||
**Tecnologias utilizadas:** | ||
- Nuxt3 | ||
- Javascript | ||
- Docker | ||
- Make | ||
|
||
|
||
Ao rodar o comando: `make run` na raiz do projeto, será criado as imagens docker do audiotopic-api e audiotopic-app | ||
e iniciado os serviços via docker-compose up. | ||
o serviço: audiotopic-app vai aguardar que o serviço: audiotopic-api fique online e disponível para ficar online. | ||
se tudo estiver ok você poderá acessar o swagger da api no endereço: http://127.0.0.1:8000/docs | ||
e poderá acessar o app na página: http://127.0.0.1:3000. | ||
|
||
### Audios de exemplo: | ||
na pasta 'audios' contem dois arquivos para testar a aplicação. |
Binary file not shown.
Binary file not shown.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,26 @@ | ||
[run] | ||
omit = | ||
*/migrations/*, | ||
*/settings/*, | ||
*migrations*, | ||
*tests*, | ||
*test_*, | ||
*run.py*, | ||
*apps.py*, | ||
*admin.py*, | ||
*settings*, | ||
*manage.py*, | ||
*conftest.py*, | ||
*asgi.py*, | ||
*wsgi*, | ||
*urls*, | ||
*__init__.py*, | ||
|
||
source = src/ | ||
data_file = .coverage | ||
|
||
[report] | ||
fail_under = 60 | ||
show_missing = False | ||
skip_covered = False | ||
sort = Cover |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,31 @@ | ||
FROM python:3.10.6-slim-buster | ||
|
||
ENV APP_NAME=audiotopic-api | ||
ARG DEV_MODE=true | ||
ENV LANG C.UTF-8 | ||
ENV LC_ALL C.UTF-8 | ||
ENV TZ America/Sao_Paulo | ||
ENV PYTHONDONTWRITEBYTECODE=1 | ||
ENV PYTHONFAULTHANDLER=1 | ||
ENV PYTHONUNBUFFERED=1 | ||
|
||
WORKDIR /app | ||
|
||
RUN apt-get update -yy \ | ||
&& apt-get upgrade -yy \ | ||
&& apt-get install -yy libpq-dev git ffmpeg gcc \ | ||
&& pip install --upgrade pip \ | ||
&& pip install --no-cache-dir pipenv | ||
|
||
COPY Pipfile* . | ||
|
||
RUN bash -c "if [ $DEV_MODE == 'true' ] ; \ | ||
then pipenv install --dev ; else pipenv install; fi" | ||
|
||
COPY . . | ||
|
||
ENV PYTHONPATH $PROJECT_DIR/src:$PYTHONPATH | ||
|
||
EXPOSE 8000 | ||
|
||
CMD ["pipenv", "run", "uvicorn", "src.api:app", "--host", "0.0.0.0", "--port", "8000", "--reload"] |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,54 @@ | ||
SHELL := /bin/bash | ||
FILES=$(shell docker ps -a -q --filter "name=audiotopic-api*") | ||
|
||
# Verifica versao do docker compose. | ||
COMPOSE_COMMAND=$(shell command -v docker-compose >/dev/null 2>&1 && echo "docker-compose" || echo "docker compose") | ||
|
||
clean: | ||
@find . -name '*.pyc' -exec rm -rf {} \; | ||
@find . -name '__pycache__' -exec rm -rf {} \; | ||
@find . -name 'Thumbs.db' -exec rm -rf {} \; | ||
@find . -name '*~' -exec rm -rf {} \; | ||
rm -rf .cache | ||
rm -rf build | ||
rm -rf dist | ||
rm -rf *.egg-info | ||
rm -rf htmlcov | ||
rm -rf .coverage | ||
rm -rf .pytest_cache | ||
rm -rf dev.sqlite3 | ||
rm -rf src/.pytest_cache | ||
rm -rf src/dev.sqlite3 | ||
|
||
create-requirements: | ||
pipenv requirements > contrib/requirements.txt | ||
|
||
build: | ||
docker build -t audiotopic-api:latest . | ||
|
||
up: build | ||
docker compose up $(dettach) -d audiotopic-api | ||
|
||
down: | ||
docker compose down | ||
|
||
run-dev: | ||
uvicorn src.api:app --reload | ||
|
||
bash: | ||
docker exec -ti audiotopic-api bash | ||
|
||
logs-app: | ||
docker logs -f audiotopic-api | ||
|
||
test: | ||
pytest -vv | ||
|
||
isort: | ||
@isort -m 3 --trailing-comma --use-parentheses --honor-noqa src/. --verbose --diff | ||
|
||
style: ## Run isort and black auto formatting code style in the project | ||
@isort -m 3 --trailing-comma --use-parentheses --honor-noqa src/. | ||
@black -S -t py37 -l 120 src/. --exclude '/(\.git|\.venv|env|venv|build|dist)/' | ||
|
||
.PHONY: all clean install test |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.