Avoid duplicating arrays #515
Open
+1,671
−762
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
majosm
force-pushed
the
avoid-array-duplication
branch
from
e813eb2 to
1d49631
Compare
majosm
force-pushed
the
avoid-array-duplication
branch
2 times, most recently
from
a5dbc5a to
c55dee2
Compare
majosm
force-pushed
the
avoid-array-duplication
branch
from
c55dee2 to
f6a3014
Compare
majosm
force-pushed
the
avoid-array-duplication
branch
3 times, most recently
from
dd6cd37 to
8ac0b8d
Compare
…uper().rec usage in CachedMapAndCopyMapper Here is a sketch of what happens with super().rec vs Mapper.rec for the previous implementation of deduplicate_data_wrappers. Suppose we have 2 data wrappers a and b with the same data pointer. With super().rec: 1) map_fn maps a to itself, then mapper copies a to a'; mapper caches a -> a' (twice, once in super().rec and then again in rec), 2) map_fn maps b to a, then mapper maps (via cache in super().rec call) a to a'; mapper caches b -> a'. => Only a' in output DAG. With Mapper.rec: 1) map_fn maps a to itself, then mapper copies a to a'; caches a -> a', 2) map_fn maps b to a, then mapper copies a to a''; caches b -> a''. => Both a' and a'' in output DAG.
…nition_cache_key for extra args case ambiguous due to the fact that any arg can be specified with/without keyword
…sformMapperWithExtraArgs
apparently TypeVar(..., <type>) doesn't include subclasses of <type>
majosm
force-pushed
the
avoid-array-duplication
branch
2 times, most recently
from
98ce162 to
ae9c939
Compare
majosm
force-pushed
the
avoid-array-duplication
branch
from
ae9c939 to
2ca7a65
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Not quite ready yet, still draft for now.
Depends on
#503(merged),#512(merged),#531(merged),#549(merged), #585, #583, and #550.