Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature: Add data_dir optional argument to Huggingface DataLoader #2817

Merged

Conversation

Pablo1785
Copy link
Contributor

@Pablo1785 Pablo1785 commented Feb 16, 2025

Add data_dir optional argument to Huggingface DataLoader to enable some manual download use cases

Pull Request Template

Checklist

  • [ x] Confirmed that run-checks all script has been executed.
  • [x ] Made sure the book is up to date with changes in this PR.

Related Issues/PRs

#2811

Changes

Some datasets (e.x. facebook/covost2) require this argument to include the subset of the dataset downloaded manually.

Testing

./run_checks.sh all

Copy link

codecov bot commented Feb 18, 2025

Codecov Report

Attention: Patch coverage is 0% with 11 lines in your changes missing coverage. Please review.

Project coverage is 81.69%. Comparing base (2c4c039) to head (a823fe8).
Report is 7 commits behind head on main.

Files with missing lines Patch % Lines
.../burn-dataset/src/source/huggingface/downloader.rs 0.00% 11 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #2817      +/-   ##
==========================================
- Coverage   81.69%   81.69%   -0.01%     
==========================================
  Files         851      851              
  Lines      113982   113993      +11     
==========================================
  Hits        93122    93122              
- Misses      20860    20871      +11     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Copy link
Member

@laggui laggui left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! Thanks for addressing the issue yourself 🙏

@laggui laggui merged commit 610943d into tracel-ai:main Feb 18, 2025
11 checks passed
zlogic pushed a commit to zlogic/burn that referenced this pull request Feb 19, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants