Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Gemini Enhancements #428

Merged
merged 16 commits into from
Oct 21, 2024
Merged

Gemini Enhancements #428

merged 16 commits into from
Oct 21, 2024

Conversation

risin42
Copy link
Contributor

@risin42 risin42 commented Oct 15, 2024

This PR enhances Gemini support with several improvements:

Gemini Translator Improvements:

  • Default Model: Switched to gemini-1.5-flash.
  • New Options: Added support for use_context, temperature, prompt, custom model lists (--model_list), and key/model rotation, matching other translators.
  • Output Limit: Increased max_output_tokens from 2048 to 8192.
  • Safety Settings: Relaxed thresholds(BLOCK_NONE) to reduce BlockedPromptException occurrences (though not entirely eliminated).
  • Exponential Backoff: Implemented for retries.

Retry Strategy: The retry mechanism prioritizes key rotation on the first retry, switching to model rotation only on subsequent retries to minimize potential inconsistencies in translation quality.

New Option:

  • Interval Option: Added a --interval [float] (seconds) option to control request frequency. Exponential backoff is now implemented but the original fixed sleep time is retained. This addresses Gemini's lower RPM limits (1000-2000 vs. OpenAI's 10,000) and free tier restrictions. Suggested values: 0.03s for Gemini Flash (2000 RPM limit) and 4s for the free tier (15 RPM).

Testing:

  1. Uncomment line 90 in gemini_translator.py (# print(model)) to inspect the model object and verify parameter changes.
  2. Default (Gemini Flash): python3 make_book.py --book_name test_books/animal_farm.epub --model gemini --test
  3. Gemini Pro: python3 make_book.py --book_name test_books/animal_farm.epub --model geminipro --test
  4. All Options(Free Tier API with Retries): python3 make_book.py --book_name test_books/animal_farm.epub --model gemini --model_list gemini-1.5-pro-002,gemini-1.5-flash-002 --test --test_num 30 --use_context --temperature 0.5 --prompt prompt_template_sample.json --interval 0.1

@yihong0618
Copy link
Owner

Thanks a lot will take a look these days

@yihong0618
Copy link
Owner

Nice! very clean code, Thanks

@yihong0618 yihong0618 merged commit 9261d92 into yihong0618:main Oct 21, 2024
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants