Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix the legacy loop indexing traversal #3373

Open
wants to merge 2 commits into
base: main
Choose a base branch
from

Conversation

naoyam
Copy link
Collaborator

@naoyam naoyam commented Nov 7, 2024

This is a temporary WAR for #3374. It's temporary since the repro has no problem with the IdModel-based indexer. This is for unblocking @IvanYashchuk until we can make the new indexer enabled by default.

The root cause of the issue is when we attempt to find a correct indexing path from the loop domain to the allocation domain of the indexed tensor, the algorithm fails to find a path visiting a backward merge when the indexed tensor has only one of the inputs. That happens when the tensor is broadcast and gets inlined with broadcast forwarding. In the current code, in that case, it just picks the first traversal option, which I think happens to be working fine, but that's not necessarily the right chose, particularly because we are looking at all candidate next traversal targets that are permissively mapped.

The WAR is simply picking a candidate as long as it has at least one mapped ID. I think this would be good enough as a temporary WAR.

Fixes #3374

@naoyam
Copy link
Collaborator Author

naoyam commented Nov 8, 2024

!test --diff

@naoyam naoyam changed the title [WIP] Fix the legacy loop indexing traversal Fix the legacy loop indexing traversal Nov 8, 2024
@naoyam naoyam marked this pull request as ready for review November 8, 2024 02:55
@naoyam naoyam requested a review from zasdfgbnm November 8, 2024 03:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Indexing error with HF's Qwen 2 model
1 participant