Skip to content

Commit

Permalink
Add Readme and tests files
Browse files Browse the repository at this point in the history
Add Readme and tests files
  • Loading branch information
alexiusstrauss authored Nov 28, 2023
2 parents 6d67b93 + 0a1d7f4 commit b58d6de
Show file tree
Hide file tree
Showing 26 changed files with 724 additions and 800 deletions.
Binary file added AudioTipic-test.pdf
Binary file not shown.
674 changes: 0 additions & 674 deletions LICENSE

This file was deleted.

32 changes: 32 additions & 0 deletions Makefile
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
SHELL := /bin/bash
FILES=$(shell docker ps -a -q --filter "name=audiotopic*")

# Verifica versao do docker compose.
COMPOSE_COMMAND=$(shell command -v docker-compose >/dev/null 2>&1 && echo "docker-compose" || echo "docker compose")

build:
docker build -t audiotopic-api:latest ./backend
docker build -t audiotopic-app:latest ./frontend

up: build
docker compose up $(dettach) audiotopic-app

down:
docker compose down -v

bash-api:
docker exec -ti audiotopic-api bash

bash-app:
docker exec -ti audiotopic-app bash

test-api:
docker exec -ti audiotopic-api pipenv run pytest

logs-api:
docker logs -f audiotopic-api

logs-app:
docker logs -f audiotopic-app

.PHONY: all clean install test
53 changes: 52 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
@@ -1 +1,52 @@
## AudioTopic-app
# AudioTopic


**AudioTopic** é um projeto com base em um teste para processar arquivos de áudio nos formatos: .mp3 ou .wav
converter em texto usando a tecnologia STT, fazer um resumo deste conteúdo utilizando LLM e criar um áudio do resumo, disponibilizando para ouvir ou fazer download do resumo.
Segue o link para leitura do desfio técnico e suas especificações.
[Arquivo de especificações](AudioTipic-test.pdf)



### Backend
No backend utilizei a linguagem **python na versão 3.10** e o framework **FastAPI** na construção da api.
Utilizei o padrão de projeto "**Strategy**" para implementar através de uma interface a engine que será usada para resumir o texto usando LLM. Deixei disponível duas engines: **LangChain** e **Tensorflow** e Seguindo o **SOLID** e **Clean Code**, deixando código mais flexível e fácil de testar ou dar manutenção.
Para rodar os testes automatizados via Docker basta rodar o comando: `make test-api` caso o container esteja rodando, ou `docker exec -ti audiotopic-api pipenv run pytest`.

Deverá configurar um .env na pasta backend/src/ com as credenciais da OPENAI para utilização do serviço com LangChain.

OPEN_AI_TOKEN=sk-change-me
TENSORFLOW_MODEL_NAME="t5-small"

No arquivo **api.py** poderá configurar qual engine usar para o processo de resumo escolhendo LangChain ou TensorFlow:


service = DeepDive(llm_engine=LLM_ENGINES.get("LangChain"))
service = DeepDive(llm_engine=LLM_ENGINES.get("TensorFlow"))

**Tecnologias utilizadas:**
- Python 3.10
- Pipenv
- FastAPI
- Pytest e Pytest-cov
- Docker
- Make


### Frontend
Utilizei o framework Nuxt3 e criei um componente responsável consumir a API.
**Tecnologias utilizadas:**
- Nuxt3
- Javascript
- Docker
- Make


Ao rodar o comando: `make run` na raiz do projeto, será criado as imagens docker do audiotopic-api e audiotopic-app
e iniciado os serviços via docker-compose up.
o serviço: audiotopic-app vai aguardar que o serviço: audiotopic-api fique online e disponível para ficar online.
se tudo estiver ok você poderá acessar o swagger da api no endereço: http://127.0.0.1:8000/docs
e poderá acessar o app na página: http://127.0.0.1:3000.

### Audios de exemplo:
na pasta 'audios' contem dois arquivos para testar a aplicação.
Binary file added audios/O-homen-mais-rico-da-babilonia.wav
Binary file not shown.
Binary file added audios/diario-oficial.mp3
Binary file not shown.
26 changes: 26 additions & 0 deletions backend/.coveragerc
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
[run]
omit =
*/migrations/*,
*/settings/*,
*migrations*,
*tests*,
*test_*,
*run.py*,
*apps.py*,
*admin.py*,
*settings*,
*manage.py*,
*conftest.py*,
*asgi.py*,
*wsgi*,
*urls*,
*__init__.py*,

source = src/
data_file = .coverage

[report]
fail_under = 60
show_missing = False
skip_covered = False
sort = Cover
31 changes: 31 additions & 0 deletions backend/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
FROM python:3.10.6-slim-buster

ENV APP_NAME=audiotopic-api
ARG DEV_MODE=true
ENV LANG C.UTF-8
ENV LC_ALL C.UTF-8
ENV TZ America/Sao_Paulo
ENV PYTHONDONTWRITEBYTECODE=1
ENV PYTHONFAULTHANDLER=1
ENV PYTHONUNBUFFERED=1

WORKDIR /app

RUN apt-get update -yy \
&& apt-get upgrade -yy \
&& apt-get install -yy libpq-dev git ffmpeg gcc \
&& pip install --upgrade pip \
&& pip install --no-cache-dir pipenv

COPY Pipfile* .

RUN bash -c "if [ $DEV_MODE == 'true' ] ; \
then pipenv install --dev ; else pipenv install; fi"

COPY . .

ENV PYTHONPATH $PROJECT_DIR/src:$PYTHONPATH

EXPOSE 8000

CMD ["pipenv", "run", "uvicorn", "src.api:app", "--host", "0.0.0.0", "--port", "8000", "--reload"]
54 changes: 54 additions & 0 deletions backend/Makefile
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
SHELL := /bin/bash
FILES=$(shell docker ps -a -q --filter "name=audiotopic-api*")

# Verifica versao do docker compose.
COMPOSE_COMMAND=$(shell command -v docker-compose >/dev/null 2>&1 && echo "docker-compose" || echo "docker compose")

clean:
@find . -name '*.pyc' -exec rm -rf {} \;
@find . -name '__pycache__' -exec rm -rf {} \;
@find . -name 'Thumbs.db' -exec rm -rf {} \;
@find . -name '*~' -exec rm -rf {} \;
rm -rf .cache
rm -rf build
rm -rf dist
rm -rf *.egg-info
rm -rf htmlcov
rm -rf .coverage
rm -rf .pytest_cache
rm -rf dev.sqlite3
rm -rf src/.pytest_cache
rm -rf src/dev.sqlite3

create-requirements:
pipenv requirements > contrib/requirements.txt

build:
docker build -t audiotopic-api:latest .

up: build
docker compose up $(dettach) -d audiotopic-api

down:
docker compose down

run-dev:
uvicorn src.api:app --reload

bash:
docker exec -ti audiotopic-api bash

logs-app:
docker logs -f audiotopic-api

test:
pytest -vv

isort:
@isort -m 3 --trailing-comma --use-parentheses --honor-noqa src/. --verbose --diff

style: ## Run isort and black auto formatting code style in the project
@isort -m 3 --trailing-comma --use-parentheses --honor-noqa src/.
@black -S -t py37 -l 120 src/. --exclude '/(\.git|\.venv|env|venv|build|dist)/'

.PHONY: all clean install test
3 changes: 3 additions & 0 deletions backend/Pipfile
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,9 @@ speechrecognition = "*"
pydub = "*"
pylint = "*"
bs4 = "*"
pytest = "*"
pytest-cov = "*"
isort = "*"

[dev-packages]

Expand Down
Loading

0 comments on commit b58d6de

Please sign in to comment.