WIP-DEBUG-PROFILE torch.compile

[IvanKobzarev]

Stack from ghstack (oldest at bottom):

• -> WIP-DEBUG-PROFILE torch.compile #2644

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/torchtune/2644

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 4 New Failures, 3 Cancelled Jobs

As of commit c3f6a7d with merge base 0991f97 ():

NEW FAILURES - The following jobs have failed:

• GPU tests / gpu_test (3.10, stable) (gh)
  tests/torchtune/modules/moe/test_moe.py::TestMoeLayer::test_forward
• GPU tests / gpu_test (3.11, stable) (gh)
  tests/torchtune/modules/moe/test_moe.py::TestMoeLayer::test_forward
• Lint / lint (3.10) (gh)
  torchtune/training/_profiler.py:88:5: F811 redefinition of unused 'time' from line 10
• Unit Test / unit_tests (3.10) (gh)
  tests/torchtune/modules/moe/test_experts.py::TestLoRAGroupedExperts::test_forward_disabled

CANCELLED JOBS - The following jobs were cancelled. Please retry:

• GPU tests / gpu_test (3.9, stable) (gh)
• Unit Test / unit_tests (3.11) (gh)
  ##[error]The operation was canceled.
• Unit Test / unit_tests (3.9) (gh)
  ##[error]The operation was canceled.

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[codecov-commenter]

Codecov Report

Attention: Patch coverage is 15.23810% with 89 lines in your changes missing coverage. Please review.

Please upload report for BASE (gh/IvanKobzarev/3/base@43b72a4). Learn more about missing BASE report.

Files with missing lines	Patch %	Lines
torchtune/modules/moe/indices.py	0.00%	68 Missing ⚠️
torchtune/modules/moe/experts.py	56.52%	10 Missing ⚠️
torchtune/modules/moe/moe.py	27.27%	8 Missing ⚠️
torchtune/training/_profiler.py	0.00%	3 Missing ⚠️

Additional details and impacted files

@@                    Coverage Diff                    @@
##             gh/IvanKobzarev/3/base    #2644   +/-   ##
=========================================================
  Coverage                          ?   66.39%           
=========================================================
  Files                             ?      400           
  Lines                             ?    24374           
  Branches                          ?        0           
=========================================================
  Hits                              ?    16182           
  Misses                            ?     8192           
  Partials                          ?        0           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.