Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adds support for the Solar architecture #719

Merged
merged 8 commits into from
Dec 16, 2024
Merged

Adds support for the Solar architecture #719

merged 8 commits into from
Dec 16, 2024

Conversation

arnavgarg1
Copy link
Contributor

@arnavgarg1 arnavgarg1 commented Dec 16, 2024

Model: https://huggingface.co/upstage/solar-pro-preview-instruct

curl 127.0.0.1:8080/generate \
    -X POST \
    -d '{
        "inputs": "<|im_start|>user\n Natalia sold clips to 48 of her friends in April, and then she sold half as many clips in May. How many clips did Natalia sell altogether in April and May?<|im_end|>\n<|im_start|>assistant\n",
        "parameters": {
            "max_new_tokens": 256
        }
    }' \
    -H 'Content-Type: application/json'

Output (this model is known to be pretty repetitive):

To find out how many clips Natalia sold in May, we need to divide the number of clips sold in April (48) by 2.\n48 / 2 = 24\nSo, Natalia sold 24 clips in May.\nTo find out the total number of clips sold in both months, we need to add the number of clips sold in April (48) and the number of clips sold in May (24).\n48 + 24 = 64\nSo, Natalia sold 64 clips altogether in April and May.\nHowever,...\n48 + 24 = 64\n48 + 24 = 64 + 0 = 64.\n48 + 24 = 64.\nSo, Natals sold 64 clips altogether in April and May.\nNtsidn't know if I should continue or say \

For the same prompt, from the new solar-pro-241126 supported on Predibase:

In May, Natalia sold half of 48, which is 24. Adding the two amounts together, she sold 48 + 24 = 72 clips altogether in April and May. [Answer: 72] [EXPLAINED] In this problem, we used the concept of multiplication and division to solve it. We first used the concept of division to find out how many clips Natalia sold in May, by dividing the number of clips she sold in April by 2. Then, we used the concept of multiplication to find out how many clips she sold altogether in both months, by multiplying the number of clips she sold in one of the months by 2. Finally, we used the concept of addition to find out how many clips she sold altogether in both months, by multiplying the number of clieters she sold in one of the months by 2. [Answer: 72] [EXAMPLE] Let's say that in April, Natalia sold 100 clips to her friends. Then, in May, she sold half of that number, which is 50. So, the total number of clips she sold in both

@@ -186,13 +186,16 @@ def from_pb(
if batch_tokenized_inputs is None:
batch_inputs = []
max_truncation = 0
for r in pb.requests:
inputs = tokenizers.get_inputs(r, tokenizer)
for request in pb.requests:
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

updated from r to request since pdb treats r as return so you can't actually inspect this variable.

Copy link
Collaborator

@magdyksaleh magdyksaleh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good! Great stuff @arnavgarg1

@arnavgarg1 arnavgarg1 merged commit ddd1c2d into main Dec 16, 2024
2 checks passed
@arnavgarg1 arnavgarg1 deleted the solar branch December 16, 2024 18:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants