Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[TLM] Update max_tokens for TLM to match provider limits #324

Merged
merged 5 commits into from
Oct 1, 2024

Conversation

ulya-tkch
Copy link
Collaborator

No description provided.

@ulya-tkch
Copy link
Collaborator Author

ulya-tkch commented Sep 24, 2024

/test-tlm
Starting TLM tests...
If you want to run all the TLM tests again (because TLM code is ready for review), comment '/test-tlm' on this PR.
If you want to re-run only the failed tests (you are still developing), comment '/rerun-failed-test-tlm' on this PR.
View full GitHub Actions run log
Tests completed!
TLM Tests Results: ✅✅✅✅✅
TLM Property Tests Results: ❌❌❌❌❌
Click the Github Actions run log for more information.

@ulya-tkch
Copy link
Collaborator Author

/rerun-test-tlm

@ulya-tkch
Copy link
Collaborator Author

ulya-tkch commented Sep 24, 2024

/rerun-failed-test-tlm
Starting Rerun of Failed TLM tests...
If you want to run all the TLM tests (because TLM code is ready for review), comment '/test-tlm' on this PR.
If you want to re-run only the failed tests again (you are still developing), comment '/rerun-failed-test-tlm' on this PR.
View full GitHub Actions run log
Failed Property Test rerun completed!
TLM Property Previously Failed Tests Results: ❌❌❌❌❌
Note: These results are only for the tests that failed on the previous run and not for all tests.
Click the Github Actions run log for more information.

@jwmueller jwmueller removed the request for review from huiwengoh September 25, 2024 17:49
@ulya-tkch
Copy link
Collaborator Author

ulya-tkch commented Sep 26, 2024

/test-tlm
Starting TLM tests...
If you want to run all the TLM tests again (because TLM code is ready for review), comment '/test-tlm' on this PR.
If you want to re-run only the failed tests (you are still developing), comment '/rerun-failed-test-tlm' on this PR.
View full GitHub Actions run log
Tests completed!
TLM Tests Results: ✅✅✅✅✅
TLM Property Tests Results: ✅✅✅✅✅
Click the Github Actions run log for more information.

@jas2600
Copy link
Contributor

jas2600 commented Sep 26, 2024

we should also change the unit test fixture here

options["max_tokens"] = int(np.random.randint(64, 512))

if 4096 is too large for claude models for now, we can keep that 512 if the model is claude and change to 4096 if the model is gpt

Copy link

Ensure final changes made to the TLM code are tested before merging. To run the TLM tests, comment /test-tlm on this PR. To re-run failed property tests, comment /rerun-failed-test-tlm instead.

@ulya-tkch
Copy link
Collaborator Author

ulya-tkch commented Sep 26, 2024

/test-tlm
Starting TLM tests...
If you want to run all the TLM tests again (because TLM code is ready for review), comment '/test-tlm' on this PR.
If you want to re-run only the failed tests (you are still developing), comment '/rerun-failed-test-tlm' on this PR.
View full GitHub Actions run log
Tests completed!
TLM Tests Results: ✅✅✅✅✅
TLM Property Tests Results: ❌❌❌❌❌
Click the Github Actions run log for more information.
Tests completed!
TLM Tests Results: ✅✅✅✅✅
TLM Property Tests Results: ❌❌❌❌❌
Click the Github Actions run log for more information.

Copy link

Ensure final changes made to the TLM code are tested before merging. To run the TLM tests, comment /test-tlm on this PR. To re-run failed property tests, comment /rerun-failed-test-tlm instead.

Copy link
Contributor

@jas2600 jas2600 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

changes LGTM. let's verify tests can pass after the backend changes deployed before merge.

@ulya-tkch
Copy link
Collaborator Author

ulya-tkch commented Sep 27, 2024

/test-tlm
Starting TLM tests...
If you want to run all the TLM tests again (because TLM code is ready for review), comment '/test-tlm' on this PR.
If you want to re-run only the failed tests (you are still developing), comment '/rerun-failed-test-tlm' on this PR.
View full GitHub Actions run log
Tests completed!
TLM Tests Results: ✅✅✅✅✅
TLM Property Tests Results: ❌❌❌❌❌
Click the Github Actions run log for more information.
Tests completed!
TLM Tests Results: ✅✅✅✅✅
TLM Property Tests Results: ✅✅✅✅✅
Click the Github Actions run log for more information.

@jas2600
Copy link
Contributor

jas2600 commented Sep 30, 2024

/test-tlm
Starting TLM tests...
If you want to run all the TLM tests again (because TLM code is ready for review), comment '/test-tlm' on this PR.
If you want to re-run only the failed tests (you are still developing), comment '/rerun-failed-test-tlm' on this PR.
View full GitHub Actions run log
Tests completed!
TLM Tests Results: ✅✅✅✅✅
TLM Property Tests Results: ✅✅✅✅✅
Click the Github Actions run log for more information.

@jas2600 jas2600 merged commit 4eac0e7 into main Oct 1, 2024
7 checks passed
@jas2600 jas2600 deleted the tlm-update-max-tokens branch October 1, 2024 16:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants