ci: update Transformers to v4.51.3 #1282
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
dvrogozh
force-pushed
the
transformers
branch
2 times, most recently
from
62ed9a5 to
fef2644
Compare
|
Transformers v4.48.0 has a regression in tests: I don't see this issue in main at c23a1c193. It might be reasonable to wait for v4.49.0 or bring this issue to HF if it will persist. |
dvrogozh
force-pushed
the
transformers
branch
from
fef2644 to
ce2bbd3
Compare
Same issue observed on v4.49.0. I reported it to Transformers project and will exclude the test manually for now: |
dvrogozh
force-pushed
the
transformers
branch
from
ce2bbd3 to
e0a5875
Compare
dvrogozh
changed the title
dvrogozh
force-pushed
the
transformers
branch
from
e0a5875 to
cf9381b
Compare
dvrogozh
changed the title
dvrogozh
force-pushed
the
transformers
branch
from
cf9381b to
8eea727
Compare
dvrogozh
changed the title
dvrogozh
force-pushed
the
transformers
branch
4 times, most recently
from
d0551c7 to
93a1366
Compare
|
@dvrogozh do we still need this PR? There are some code conflict |
We need it. That's my attempt to update tested Transformers version. This unfortunately stepped into an issue of increased test time due to which we started to catch infrastructure errors (like fetching some content from HF) without a way to efficiently rerun some tests. I have an idea how to handle that, but this needs some time which slips away. I will work on that in the coming weeks. |
dvrogozh
force-pushed
the
transformers
branch
11 times, most recently
from
3dd39dc to
6284d3f
Compare
dvrogozh
force-pushed
the
transformers
branch
3 times, most recently
from
a1e33b7 to
3597fd5
Compare
dvrogozh
force-pushed
the
transformers
branch
12 times, most recently
from
fbc5958 to
ce5fbb6
Compare
Changes: * Benchmarking scripts are pruned from Transformers by v4.49.0 due to deprecation. So we don't need to test them anymore. * Some cuda specific tests were generalized to cover non-cuda devices which uncovered some issues. * Some new tests were added which fail for both cuda and xpu. * Few regressions due to changes on Transformers side Fixed tests: * huggingface/transformers@b912f5e * `tests/models/git/test_modeling_git.py::GitModelTest::test_inputs_embeds_matches_input_ids` * huggingface/transformers@b5aaf87 * `tests/pipelines/test_pipelines_video_classification.py::VideoClassificationPipelineTests::test_small_model_pt` * `tests/test_pipeline_mixin.py::VideoClassificationPipelineTests::test_small_model_pt` * huggingface/transformers@42c8ccf * `tests/generation/test_utils.py::GenerationIntegrationTests::test_generated_length_assisted_generation` * huggingface/transformers@9fd123a * `test_model_parallelization` * `test_model_parallel_equal_results` Commits which added new tests (or enabled previously skipped tests) which fail: * huggingface/transformers@23d782e * `tests/pipelines/test_pipelines_text_generation.py::TextGenerationPipelineTests::test_return_dict_in_generate` * `tests/test_pipeline_mixin.py::TextGenerationPipelineTests::test_return_dict_in_generate` * huggingface/transformers@2fa876d * `test_cpu_offload` (some of) * `test_disk_offload_bin` (some of) * `test_disk_offload_safetensors` (some of) * `tests/pipelines/test_pipelines_text_generation.py::TextGenerationPipelineTests::test_small_model_pt_bloom_accelerate` * huggingface/transformers@be2ac09 * `tests/models/paligemma/test_modeling_paligemma.py::PaliGemmaForConditionalGenerationModelTest::test_generate_compilation_all_outputs` * `tests/models/paligemma2/test_modeling_paligemma2.py::PaliGemma2ForConditionalGenerationModelTest::test_generate_compilation_all_outputs` * huggingface/transformers#36340 * `tests/pipelines/test_pipelines_audio_classification.py::AudioClassificationPipelineTests::test_small_model_pt_fp16` * huggingface/transformers@1fae54c * `tests/trainer/test_trainer.py::TrainerIntegrationPrerunTest::test_gradient_accumulation_loss_alignment_with_model_loss` * huggingface/transformers@15ec971 * `tests/models/qwen2_5_vl/test_processor_qwen2_5_vl.py::Qwen2_5_VLProcessorTest::test_chat_template_video_custom_sampling` * `tests/models/qwen2_5_vl/test_processor_qwen2_5_vl.py::Qwen2_5_VLProcessorTest::test_chat_template_video_special_processing` Regressions: * huggingface/transformers@365fecb * `tests/generation/test_utils.py::GenerationIntegrationTests::test_encoder_decoder_generate_attention_mask` * huggingface/transformers@da334bc * `tests/generation/test_utils.py::GenerationIntegrationTests::test_generate_input_features_as_encoder_kwarg` * huggingface/transformers@bcfc9d7 * `tests/models/llava/test_modeling_llava.py::LlavaForConditionalGenerationModelTest::test_config` * huggingface/transformers#36267 * `tests/utils/test_import_utils.py` * huggingface/transformers#36267 * `tests/models/marian/test_modeling_marian.py` Signed-off-by: Dmitry Rogozhkin <[email protected]>
Signed-off-by: Dmitry Rogozhkin <[email protected]>
Signed-off-by: Dmitry Rogozhkin <[email protected]>
dvrogozh
changed the title
Fixed tests: * huggingface/transformers@24e311f * `tests/generation/test_logits_process.py::LogitsProcessorTest::test_watermarking_processor` * huggingface/transformers@77aa9fc * `tests/generation/test_utils.py::GenerationIntegrationTests::test_encoder_decoder_generate_attention_mask` * huggingface/accelerate@8ab01d3 * `tests/models/blip/test_modeling_blip.py::BlipTextImageModelTest::test_cpu_offload` * `tests/models/blip/test_modeling_blip.py::BlipTextImageModelTest::test_disk_offload_bin` * `tests/models/blip/test_modeling_blip.py::BlipTextImageModelTest::test_disk_offload_safetensors` * etc. Signed-off-by: Dmitry Rogozhkin <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Changes:
Signed-off-by: Dmitry Rogozhkin [email protected]