Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add llama.cpp agent #254

Merged
merged 24 commits into from
Aug 11, 2023
Merged

Add llama.cpp agent #254

merged 24 commits into from
Aug 11, 2023

Conversation

HHousen
Copy link
Contributor

@HHousen HHousen commented Jun 29, 2023

Usage example:

agent=LlamacppAgent(
    LlamacppAgentConfig(
        prompt_preamble="You are an AI assistant",
        llamacpp_kwargs={"model_path": "./nous-hermes-13b.ggmlv3.q4_0.bin", "n_gpu_layers": 37, "verbose": True},
        generate_responses=True,
        prompt_template="alpaca",
        initial_message=BaseMessage(text="Hello! How are you?"),
    )
),

Model tested is from https://huggingface.co/TheBloke/Nous-Hermes-13B-GGML.

@linear
Copy link

linear bot commented Jun 29, 2023

EPD-308 Llama.cpp Integration

HHousen and others added 18 commits June 28, 2023 18:23
@HHousen
Add some docs
562e714
@HHousen
Add LlamacppAgent to AgentFactory
6f9711a
@HHousen
Remove comment
16853fe
@HHousen
Refactor duplicated sentence parsing logic
db5f745
@zaptrem
Merge ActionAgent into Base and RespondAgents and make sentence split…
68da842 
…ting vendor agnostic
@HHousen
Fix some mypy
e732101
@zaptrem
factor out OpenAI token logic and complete ActionAgent merge
0078d24
@zaptrem
factor out function call logic
8e69043
@zaptrem
docs updates and cleanup/revert prompt change
87e8da4
@zaptrem
pass mypy
c1812f6
@zaptrem
pr changes and test
bf035f2
@zaptrem
improve type hinting
31e669c
@zaptrem
add function test
433ac1a
@zaptrem
add test case where the request is only func call
86d325a
@HHousen
Merge branch 'zaptrem/streaming-action-agent' into hayden/epd-308-lla…
7fcaaf7 
…macpp-integration
@zaptrem
Merge remote-tracking branch 'origin/main' into zaptrem/streaming-act…
3862f72 
…ion-agent
@HHousen
Use new collate_response_async function
adf1721
@HHousen
Fix mypy
33b3124
@HHousen
Copy link
Contributor Author

HHousen commented Jul 3, 2023 •
edited
Loading

Must merge #260 before merging this PR. #260 Merged.

@HHousen HHousen marked this pull request as ready for review July 3, 2023 21:20
@ajar98 ajar98 merged commit 6c726e7 into main Aug 11, 2023
4 checks passed
m5a0r7 pushed a commit to m5a0r7/vocode-python that referenced this pull request Oct 19, 2023
@HHousen @zaptrem @ajar98
Add llama.cpp agent (vocodedev#254)
6f5be09 
* Add llama.cpp agent

* Add some docs

* Add LlamacppAgent to AgentFactory

* Remove comment

* Refactor duplicated sentence parsing logic

* Merge ActionAgent into Base and RespondAgents and make sentence splitting vendor agnostic

* Fix some mypy

* factor out OpenAI token logic and complete ActionAgent merge

* factor out function call logic

* docs updates and cleanup/revert prompt change

* pass mypy

* pr changes and test

* improve type hinting

* add function test

* add test case where the request is only func call

* Use new collate_response_async function

* Fix mypy

* merge tests

---------

Co-authored-by: zaptrem <[email protected]>
Co-authored-by: Ajay Raj <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@ajar98 ajar98 ajar98 approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@HHousen @ajar98 @zaptrem