Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

High Number of Tokens for openai-completions Models #1936

Open
selinaxiao opened this issue Jun 7, 2024 · 0 comments
Open

High Number of Tokens for openai-completions Models #1936

selinaxiao opened this issue Jun 7, 2024 · 0 comments

Comments

@selinaxiao
Copy link

selinaxiao commented Jun 7, 2024

I created the config file with hard-coded prompts to try out gpt-3.5-turbo on 5 examples, but the number of tokens is exceedingly high for this config file. What should I do to reduce the number of tokens?
The following is my config file:

task: newyorker_caption_contest_zeroshot
dataset_path: csv
dataset_path: selinax10010/matching 
dataset_name: null 
dataset_kwargs: null 
output_type: multiple_choice
training_split: train 
validation_split: validation 
test_split: test 
process_docs: !function utils.process_docs
description: "role: system\nYou are CaptionContestGPT, an expert language model at understanding \
the famous New Yorker caption contest. You follow the contest each week, and \
understand what makes for a humorous caption for each cartoon. You are aware of\
the various theories of humor, and read/anaylze the caption contest entries and \
winners each week.\n\nSome things to remember:\n\n- You're well versed in the \
history of the New Yorker Caption contest, and the types of captions that are \
selected as finalists/winners vs. those that are not.\n- Provide the answer in \
the requested format. \n~~~\nrole: user\nI will describe a New Yorker cartoon \
to you. Then, I will give you 5 choices (labelled A-E) for captions. One of the \
captions was the winning caption for that cartoon, the other captions do not \
correspond to this cartoon. Your job is to find the correct match and respond \
with \"Answer: X\" where X is either A, B, C, D, or E.\n\n\n"
doc_to_text: "{{query}}\nChoices:\nA: {{choices_text[0]}}\nB: {{choices_text[1]}}\nC: \
{{choices_text[2]}}\nD: {{choices_text[3]}}\nE: {{choices_text[4]}}\n\nWhich of the 5 options \
(A, B, C, D, or E) is the caption that truly corresponds to the cartoon?\n~~~"
doc_to_target: "{{label}}"
doc_to_choice: "{{choices}}"
num_fewshot: 0
metric_list:
  - metric: acc
    aggregation: mean
    higher_is_better: true
  - metric: acc_norm
    aggregation: mean
    higher_is_better: true
  - metric: mcc
    aggregation: matthews_corrcoef
    higher_is_better: true
metadata:
  version: 1.0
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant