Update vocodehq-public (#609)

* [DOW-118] set up code linting and tests (#589) * adds github workflow * run black * run isort * adds precommit * adds vscode settings * adds pre-commit guidelines (#590) * creates docker image, updates telephony app deps (#601) * [DOW-105] refactor interruptions into the output device (#586) * [DOW-105] refactor interruptions into the output device (#562) * initial refactor works * remove notion of UtteranceAudioChunk and put all of the state in the callback * move per_chunk_allowance_seconds into output device * onboard onto vonage * rename to abstract output device and onboard other output devices * initial work to onboard twilio output device * twilio conversation works * some cleanup with better comments * unset poetry.lock * move abstract play method into ratelimitoutputdevice + dispatch to thread in fileoutputdevice * rename back to AsyncWorker * comments * work through a bit of mypy * asyncio.gather is g2g: * create interrupt lock * remove todo * remove last todo * remove log for interrupts * fmt * fix mypy * fix mypy * isort * creates first test and adds scaffolding * adds two other send_speech_to_output tests * make send_speech_to_output more efficient * adds tests for rate limit interruptions output device * makes some variables private and also makes the chunk id coming back from the mark match the incoming audio chunk * adds twilio output device tests * make typing better for output devices * fix mypy * resolve PR comments * resolve PR comments * [DOW-101] LiveKit integration (#591) * checkpoint * livekit v0 * in progress changes * integrate with worker * fix import * update deps and remove unneeded files * integrate it properly into app * fix interrupts * make transcript publish work * a confounding fix * isort * constants, some cleanup --------- Co-authored-by: Kian <[email protected]> * upgrade to latest cartesia 1.0.3 (#587) * upgrade to latest cartesia 1.0.3 * fixed linting conflict * finish streaming * make cartesia optional --------- Co-authored-by: Ajay Raj <[email protected]> * poetry version prerelease (#602) * feat: Add ability to configure OpenAI base URL in ChatGPTAgentConfig (#577) * feat: Add ability to configure OpenAI base URL in ChatGPTAgentConfig - Added `base_url` parameter to `ChatGPTAgentConfig` to allow customization of the OpenAI API base URL. - Updated `instantiate_openai_client` function to use the `base_url` parameter from the configuration. - Modified `ChatGPTAgent` to utilize the updated `instantiate_openai_client` function. - Added tests to verify the new `base_url` functionality in `tests/streaming/agent/test_base_agent.py`. This enhancement allows users to specify a custom OpenAI API base URL, providing greater flexibility in agent configuration. * adding capability to use the openai compatible endpoint with token estimation for llama * lint fix * changing openai base_url parameter for overall less code changes * missed logging update * Update vocode/streaming/agent/chat_gpt_agent.py * Update tests/streaming/agent/test_base_agent.py * fix test --------- Co-authored-by: Ajay Raj <[email protected]> * Support passthrough of AsyncHTTPTransport (#603) Support passthrough of AsyncHTTPTransport object * add script used to make PR * adds test target for vocodehq-public * Remove catch-all exception logger for asyncio tasks (#605) * remove error log from exception for asyncio tasks * remove log error on chatgpt query --------- Co-authored-by: Kian <[email protected]> Co-authored-by: rjheeta <[email protected]> Co-authored-by: Clay Elmore <[email protected]> Co-authored-by: vocode-petern <[email protected]> Co-authored-by: Adnaan Sachidanandan <[email protected]>
vocodedev · Jul 5, 2024 · 87308ba · 87308ba
1 parent 28ead9f
commit 87308ba
Show file tree

Hide file tree

Showing 84 changed files with 5,402 additions and 1,214 deletions.
diff --git a/.github/workflows/test.yml b/.github/workflows/test.yml
@@ -4,53 +4,89 @@ on:
   push:
     branches: [main]
   pull_request:
-    branches: [main, vocode-core-0.1.0]
+    branches: [main, vocode-core-0.1.0, vocodehq-public]
+
+env:
+  poetry-version: "1.8.3"
 
 jobs:
-  build:
+  lint:
+    strategy:
+      fail-fast: false
+      matrix:
+        python-version:
+        - "3.10"
+        - "3.11"
+
+    runs-on: ubuntu-latest
+
+    steps:
+    - uses: actions/checkout@v2
+
+    - name: Install ffmpeg
+      run: |
+        sudo apt-get update
+        sudo apt-get install -y ffmpeg
+
+    - name: Set up Python ${{ matrix.python-version }}
+      uses: actions/setup-python@v2
+      with:
+        python-version: ${{ matrix.python-version }}
+
+    - name: Set up Poetry
+      uses: abatilo/[email protected]
+      with:
+        poetry-version: ${{ env.poetry-version }}
+
+    - name: Install dependencies
+      run: poetry install -E all
+
+    - name: Run mypy
+      run: |
+        poetry run mypy -p vocode
+        poetry run mypy -p quickstarts
+        poetry run mypy -p playground
+
+    - name: Run black
+      if: success() || failure()
+      run: poetry run black --check .
+
+    - name: Run isort
+      shell: bash
+      if: success() || failure()
+      run: poetry run isort --check .
+
+  pytest:
     strategy:
       fail-fast: false
       matrix:
         python-version:
-          - "3.10"
-          - "3.11"
-        poetry-version:
-          - "1.7.1"
+        - "3.10"
+        - "3.11"
 
     runs-on: ubuntu-latest
 
     steps:
-      - uses: actions/checkout@v2
-
-      - name: Set up Python ${{ matrix.python-version }}
-        uses: actions/setup-python@v2
-        with:
-          python-version: ${{ matrix.python-version }}
-
-      - name: Install ffmpeg
-        run: |
-          sudo apt-get update
-          sudo apt-get install -y ffmpeg
-
-      - name: Install poetry ${{ matrix.poetry-version }}
-        run: |
-          python -m ensurepip
-          python -m pip install --upgrade pip
-          python -m pip install poetry==${{ matrix.poetry-version }}
-
-      - name: View poetry --help
-        run: poetry --help
-
-      - name: Install dependencies
-        shell: bash
-        run: python -m poetry install -E all
-
-      - name: Lint with mypy
-        run: |
-          python -m poetry run mypy -p vocode
-          python -m poetry run mypy -p quickstarts
-          python -m poetry run mypy -p playground
-
-      - name: Test with pytest
-        run: |
-          python -m poetry run python -m pytest -v tests
+    - uses: actions/checkout@v2
+
+    - name: Install ffmpeg
+      run: |
+        sudo apt-get update
+        sudo apt-get install -y ffmpeg
+
+    - name: Set up Python ${{ matrix.python-version }}
+      uses: actions/setup-python@v2
+      with:
+        python-version: ${{ matrix.python-version }}
+
+    - name: Set up Poetry
+      uses: abatilo/[email protected]
+      with:
+        poetry-version: ${{ env.poetry-version }}
+
+    - name: Install dependencies
+      run: poetry install -E all
+
+    - name: Test with pytest
+      run: |
+        poetry run pytest -v tests
diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml
@@ -1,13 +1,41 @@
+default_language_version:
+  python: python3.10
+
 repos:
-    - repo: https://github.com/psf/black
-      rev: 24.4.2
-      hooks:
-          - id: black
-    - repo: https://github.com/pre-commit/mirrors-mypy
-      rev: v1.8.0
-      hooks:
-          - id: mypy
-            args: [--ignore-missing-imports, ./]
-            language: system
-            pass_filenames: false
-            stages: [pre-push]
+- repo: https://github.com/pre-commit/pre-commit-hooks
+  rev: v4.6.0
+  hooks:
+  - id: check-ast
+  - id: trailing-whitespace
+  - id: check-toml
+  - id: end-of-file-fixer
+  - id: check-case-conflict
+  - id: check-merge-conflict
+  - id: check-symlinks
+  - id: detect-private-key
+  - id: mixed-line-ending
+    args:
+    - --fix=lf
+
+- repo: https://github.com/macisamuele/language-formatters-pre-commit-hooks
+  rev: v2.13.0
+  hooks:
+  - id: pretty-format-yaml
+    args:
+    - --autofix
+    - --preserve-quotes
+    - --indent=2
+
+- repo: https://github.com/pycqa/isort
+  rev: 5.13.2
+  hooks:
+  - id: isort
+    args:
+    - --settings-path=./pyproject.toml
+
+- repo: https://github.com/psf/black-pre-commit-mirror
+  rev: 24.4.2
+  hooks:
+  - id: black
+    args:
+    - --config=./pyproject.toml
diff --git a/.vscode/extensions.json b/.vscode/extensions.json
@@ -0,0 +1,8 @@
+{
+  "recommendations": [
+    "ms-python.black-formatter",
+    "ms-python.isort",
+    "ms-python.python",
+    "ms-python.vscode-pylance"
+  ]
+}
diff --git a/.vscode/settings.json b/.vscode/settings.json
@@ -0,0 +1,16 @@
+{
+  "[python]": {
+    "editor.defaultFormatter": "ms-python.black-formatter",
+    "editor.codeActionsOnSave": {
+      "source.fixAll": "explicit",
+      "source.organizeImports": "explicit"
+    }
+  },
+  "black-formatter.args": ["--config", "./pyproject.toml"],
+  "isort.args": ["--settings-path", "./pyproject.toml"],
+  "rewrap.autoWrap.enabled": true,
+  "rewrap.wrappingColumn": 100,
+  "python.testing.unittestEnabled": false,
+  "python.testing.pytestEnabled": true,
+  "isort.check": true
+}
diff --git a/apps/langchain_agent/call_transcript_utils.py b/apps/langchain_agent/call_transcript_utils.py
@@ -5,27 +5,21 @@
 
 
 def add_transcript(conversation_id: str, transcript: str) -> None:
-    transcript_path = os.path.join(
-        CALL_TRANSCRIPTS_DIR, "{}.txt".format(conversation_id)
-    )
+    transcript_path = os.path.join(CALL_TRANSCRIPTS_DIR, "{}.txt".format(conversation_id))
     with open(transcript_path, "a") as f:
         f.write(transcript)
 
 
 def get_transcript(conversation_id: str) -> Optional[str]:
-    transcript_path = os.path.join(
-        CALL_TRANSCRIPTS_DIR, "{}.txt".format(conversation_id)
-    )
+    transcript_path = os.path.join(CALL_TRANSCRIPTS_DIR, "{}.txt".format(conversation_id))
     if os.path.exists(transcript_path):
         with open(transcript_path, "r") as f:
             return f.read()
     return None
 
 
 def delete_transcript(conversation_id: str) -> bool:
-    transcript_path = os.path.join(
-        CALL_TRANSCRIPTS_DIR, "{}.txt".format(conversation_id)
-    )
+    transcript_path = os.path.join(CALL_TRANSCRIPTS_DIR, "{}.txt".format(conversation_id))
     if os.path.exists(transcript_path):
         os.remove(transcript_path)
         return True

diff --git a/apps/langchain_agent/main.py b/apps/langchain_agent/main.py
@@ -1,32 +1,27 @@
 import os
 import sys
 import typing
-from dotenv import load_dotenv
 
+from dotenv import load_dotenv
+from langchain.memory import ConversationBufferMemory
+from stdout_filterer import RedactPhoneNumbers
 from tools.contacts import get_all_contacts
 from tools.vocode import call_phone_number
 from tools.word_of_the_day import word_of_the_day
+
 from vocode.turn_based.synthesizer.azure_synthesizer import AzureSynthesizer
 from vocode.turn_based.synthesizer.gtts_synthesizer import GTTSSynthesizer
-from langchain.memory import ConversationBufferMemory
-
-
-from stdout_filterer import RedactPhoneNumbers
 
 load_dotenv()
 
+from langchain.agents import AgentType, initialize_agent
 from langchain.chat_models import ChatOpenAI
-from langchain.agents import initialize_agent
-from langchain.agents import AgentType
 
 if __name__ == "__main__":
     # Redirect stdout to our custom class
     sys.stdout = typing.cast(typing.TextIO, RedactPhoneNumbers(sys.stdout))
 
-    OBJECTIVE = (
-        input("Objective: ")
-        or "Find a random person in my contacts and tell them a joke"
-    )
+    OBJECTIVE = input("Objective: ") or "Find a random person in my contacts and tell them a joke"
     llm = ChatOpenAI(temperature=0, model_name="gpt-4")  # type: ignore
     memory = ConversationBufferMemory(memory_key="chat_history", return_messages=True)
     # Logging of LLMChains

diff --git a/apps/langchain_agent/telephony_app.py b/apps/langchain_agent/telephony_app.py
@@ -1,19 +1,16 @@
 import logging
 import os
 import typing
+
+from call_transcript_utils import add_transcript
+from dotenv import load_dotenv
 from fastapi import FastAPI
+
 from vocode.streaming.models.events import Event, EventType
 from vocode.streaming.models.transcript import TranscriptCompleteEvent
-from vocode.streaming.utils import events_manager
-
-from vocode.streaming.telephony.config_manager.redis_config_manager import (
-    RedisConfigManager,
-)
+from vocode.streaming.telephony.config_manager.redis_config_manager import RedisConfigManager
 from vocode.streaming.telephony.server.base import TelephonyServer
-
-from call_transcript_utils import add_transcript
-
-from dotenv import load_dotenv
+from vocode.streaming.utils import events_manager
 
 load_dotenv()
 

diff --git a/apps/langchain_agent/tools/contacts.py b/apps/langchain_agent/tools/contacts.py
@@ -1,4 +1,5 @@
 from typing import List
+
 from langchain.agents import tool
 
 CONTACTS = [{"name": "Ajay", "phone": "+15555555555"}]

diff --git a/apps/langchain_agent/tools/vocode.py b/apps/langchain_agent/tools/vocode.py
@@ -1,24 +1,25 @@
-import logging
 import asyncio
+import logging
 import os
-from langchain.agents import tool
+
+from call_transcript_utils import delete_transcript, get_transcript
 from dotenv import load_dotenv
+from langchain.agents import tool
 
 from vocode.streaming.models.message import BaseMessage
-from call_transcript_utils import delete_transcript, get_transcript
 
 load_dotenv()
 
-from vocode.streaming.telephony.conversation.outbound_call import OutboundCall
-from vocode.streaming.telephony.config_manager.redis_config_manager import (
-    RedisConfigManager,
-)
-from vocode.streaming.models.agent import ChatGPTAgentConfig
 import time
 
+from vocode.streaming.models.agent import ChatGPTAgentConfig
+from vocode.streaming.telephony.config_manager.redis_config_manager import RedisConfigManager
+from vocode.streaming.telephony.conversation.outbound_call import OutboundCall
+
 LOOP = asyncio.new_event_loop()
 asyncio.set_event_loop(LOOP)
 
+
 @tool("call phone number")
 def call_phone_number(input: str) -> str:
     """calls a phone number as a bot and returns a transcript of the conversation.

diff --git a/apps/langchain_agent/tools/word_of_the_day.py b/apps/langchain_agent/tools/word_of_the_day.py
@@ -1,6 +1,7 @@
 from typing import List
-from langchain.agents import tool
+
 import requests
+from langchain.agents import tool
 
 WORDNIK_URL_WITH_KEY = "https://api.wordnik.com/v4/words.json/wordOfTheDay?api_key=d52b63b6880f17811310d0fbd3b0d3a8ef163a248f58dc831"
 

diff --git a/apps/livekit/README.md b/apps/livekit/README.md
@@ -0,0 +1,3 @@
+```
+poetry run python app.py dev
+```