Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[REVIEW] Implement Feature Request from #1077 on Left Padding #1126

Closed
wants to merge 36 commits into from
Closed
Changes from 1 commit
Commits
Show all changes
36 commits
Select commit Hold shift + click to select a range
a52a90c
Update docstrings for issue #1077
Sep 14, 2021
ed61663
Merge branch 'main' into 1077-implement
Sep 16, 2021
ff1e396
Implementation of left padding for issue #1077
Sep 16, 2021
d9e457d
Update #1077 implementation
Sep 16, 2021
e25c6e8
Implement #1077 update with docstring and type hinting.
Sep 16, 2021
295d4e2
Merge branch 'main' into 1077-implement
lesnikow Sep 16, 2021
5166d57
Merge branch 'main' into 1077-implement
lesnikow Sep 17, 2021
e55336c
Merge branch 'main' into 1077-implement
lesnikow Sep 20, 2021
af8aa57
Merge branch 'main' of github.com:NVIDIA/NVTabular into 1077-implement
Sep 23, 2021
299d356
Update tensorflow module docstring for docs syntax
Sep 23, 2021
1285783
Expose pad_left to user
Sep 24, 2021
364bcf1
Skip test_distributed_multigpu()
Sep 24, 2021
071b8bf
Add unit test for torch dataloader and padding argument
Sep 24, 2021
3cce162
Update torch test for padding argument
Sep 24, 2021
cebb715
Update unit test for padding argument
Sep 25, 2021
5acd76a
Update dataloader torch to pass new tests
Sep 25, 2021
1684289
Clean up loader/torch module
Sep 25, 2021
a319501
Clean up test_torch_dataloader module
Sep 25, 2021
0be389e
Update tests
Sep 27, 2021
d93f9c5
Add tests for the TensorFlow runtime dataloader
Sep 28, 2021
0c0ce69
Implement pad_left in _build_sparse_tensor TF
Sep 28, 2021
941d2f3
Update torch loader documentation
Sep 28, 2021
7944b2a
Merge branch 'main' of 1077-implement
Sep 28, 2021
76c0024
Cleanup _build_sparese_tensor for TF dataloader
Sep 28, 2021
46847cb
Add docstring to _build_sparse_tensor() for tf
Sep 28, 2021
c7ae873
Update docstring
Sep 28, 2021
d86cec3
Refactor torch dataloader pad_left and _build_spar
Sep 28, 2021
d90e1df
Update pytest decorator
Sep 28, 2021
b21c57d
Cleanup torch loader
Sep 28, 2021
2150ede
Implement pad_left with TF ops
Sep 29, 2021
a51aa44
Implement pad_left with TF ops cleanup
Sep 29, 2021
01749f9
Merge branch 'main' into 1077-implement
lesnikow Sep 29, 2021
b305afa
Update tensorflow dataloader implementation
Sep 30, 2021
2febf1a
Merge branch '1077-implement' of https://github.com/NVIDIA/NVTabular …
Sep 30, 2021
587ef0c
Update pad_left TF unit tests
Sep 30, 2021
dd9927e
Update pad_left code for TF sparse tensors
Sep 30, 2021
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Update pad_left TF unit tests
Update pad_left TF unit tests to make name consistent with other sparse tensor
test and to collect print statements.
  • Loading branch information
Adam Lesnikowski committed Sep 30, 2021

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
commit 587ef0c48d21b059449a6e2a857de571f8305f1c
9 changes: 3 additions & 6 deletions tests/unit/loader/test_tf_dataloader.py
Original file line number Diff line number Diff line change
@@ -519,7 +519,7 @@ def test_sparse_tensors(tmpdir, sparse_dense):


@pytest.mark.parametrize("pad_left", [False, True])
def test_sparse_tensor_left_padding(pad_left):
def test_sparse_tensors_left_padding(pad_left):
"""Tests the pad_left functionality of our TensorFlow dataloader
to pad data on the left for sparse tensors."""
df = cudf.DataFrame({"A": [[3, 1, 5, 1], [9, 2], [6]], "B": [[3, 1, 5, 1, 9], [2], [6, 5, 3]]})
@@ -543,14 +543,12 @@ def test_sparse_tensor_left_padding(pad_left):
features, labels = batch
for categorical_column in categorical_columns:
feature_tensor = features[categorical_column]
print("feature_tensor is:\n{}".format(feature_tensor))
print("categorical_column is:\n{}".format(categorical_column))
if pad_left:
if categorical_column == "A":
expected_tensor = tf.constant(
[[0, 3, 1, 5, 1], [0, 0, 0, 9, 2], [0, 0, 0, 0, 6]], dtype=tf.int64
)
print("expected_tensor is:\n{}".format(expected_tensor))
if categorical_column == "B":
expected_tensor = tf.constant(
[
@@ -560,13 +558,11 @@ def test_sparse_tensor_left_padding(pad_left):
],
dtype=tf.int64,
)
print("expected_tensor is:\n{}".format(expected_tensor))
elif not pad_left:
if categorical_column == "A":
expected_tensor = tf.constant(
[[3, 1, 5, 1, 0], [9, 2, 0, 0, 0], [6, 0, 0, 0, 0]], dtype=tf.int64
)
print("expected_tensor is:\n{}".format(expected_tensor))
if categorical_column == "B":
expected_tensor = tf.constant(
[
@@ -576,7 +572,8 @@ def test_sparse_tensor_left_padding(pad_left):
],
dtype=tf.int64,
)
print("expected_tensor is:\n{}".format(expected_tensor))
print("expected_tensor is:\n{}".format(expected_tensor))
print("feature_tensor is:\n{}".format(feature_tensor))
assert tf.experimental.numpy.allclose(feature_tensor, expected_tensor)