Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Keras 3 Featurespace running into error when using PyTorch backed #19317

Closed
sibyjackgrove opened this issue Mar 15, 2024 · 10 comments
Closed

Keras 3 Featurespace running into error when using PyTorch backed #19317

sibyjackgrove opened this issue Mar 15, 2024 · 10 comments
Assignees
@sachinprasadhs
Labels
keras-team-review-pending Pending review by a Keras team member. type:Bug

Comments

@sibyjackgrove
Copy link

I am trying to use Keras 3 FeatureSpace on a tf.data dataset. It works fine with the tensorflow backend.
However, I am encountering the below error when I try to run the below example using a Pytorch backed : https://keras.io/examples/structured_data/structured_data_classification_with_feature_space/

The only change I am doing is changing the backend to Pytorch using: os.environ["KERAS_BACKEND"] = "torch"
The error is occurring at the line where the feature space is mapped to the dataset.

  File "/home/dercybersecurity/datapreprocessing/structured_data_classification_with_feature_space.py", line 306, in <module>
    preprocessed_train_ds = train_ds.map(
                            ^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/tensorflow/python/data/ops/dataset_ops.py", line 2280, in map
    return map_op._map_v2(
           ^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/tensorflow/python/data/ops/map_op.py", line 40, in _map_v2
    return _ParallelMapDataset(
           ^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/tensorflow/python/data/ops/map_op.py", line 148, in __init__
    self._map_func = structured_function.StructuredFunctionWrapper(
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/tensorflow/python/data/ops/structured_function.py", line 265, in __init__
    self._function = fn_factory()
                     ^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/tensorflow/python/eager/polymorphic_function/polymorphic_function.py", line 1227, in get_concrete_function
    concrete = self._get_concrete_function_garbage_collected(*args, **kwargs)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/tensorflow/python/eager/polymorphic_function/polymorphic_function.py", line 1197, in _get_concrete_function_garbage_collected
    self._initialize(args, kwargs, add_initializers_to=initializers)
  File "/usr/local/lib/python3.11/site-packages/tensorflow/python/eager/polymorphic_function/polymorphic_function.py", line 695, in _initialize
    self._concrete_variable_creation_fn = tracing_compilation.trace_function(
                                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/tensorflow/python/eager/polymorphic_function/tracing_compilation.py", line 178, in trace_function
    concrete_function = _maybe_define_function(
                        ^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/tensorflow/python/eager/polymorphic_function/tracing_compilation.py", line 283, in _maybe_define_function
    concrete_function = _create_concrete_function(
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/tensorflow/python/eager/polymorphic_function/tracing_compilation.py", line 310, in _create_concrete_function
    traced_func_graph = func_graph_module.func_graph_from_py_func(
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/tensorflow/python/framework/func_graph.py", line 1059, in func_graph_from_py_func
    func_outputs = python_func(*func_args, **func_kwargs)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/tensorflow/python/eager/polymorphic_function/polymorphic_function.py", line 598, in wrapped_fn
    out = weak_wrapped_fn().__wrapped__(*args, **kwds)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/tensorflow/python/data/ops/structured_function.py", line 231, in wrapped_fn
    ret = wrapper_helper(*args)
          ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/tensorflow/python/data/ops/structured_function.py", line 161, in wrapper_helper
    ret = autograph.tf_convert(self._func, ag_ctx)(*nested_args)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/tensorflow/python/autograph/impl/api.py", line 693, in wrapper
    raise e.ag_error_metadata.to_exception(e)
  File "/usr/local/lib/python3.11/site-packages/tensorflow/python/autograph/impl/api.py", line 690, in wrapper
    return converted_call(f, args, kwargs, options=options)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/tensorflow/python/autograph/impl/api.py", line 439, in converted_call
    result = converted_f(*effective_args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/tmp/__autograph_generated_filej1ue84y6.py", line 5, in <lambda>
    tf__lam = lambda x, y: ag__.with_function_scope(lambda lscope: (ag__.converted_call(feature_space, (x,), None, lscope), y), 'lscope', ag__.STD)
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/tensorflow/python/autograph/core/function_wrappers.py", line 113, in with_function_scope
    return thunk(scope)
           ^^^^^^^^^^^^
  File "/tmp/__autograph_generated_filej1ue84y6.py", line 5, in <lambda>
    tf__lam = lambda x, y: ag__.with_function_scope(lambda lscope: (ag__.converted_call(feature_space, (x,), None, lscope), y), 'lscope', ag__.STD)
                                                                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/tensorflow/python/autograph/impl/api.py", line 377, in converted_call
    return _call_unconverted(f, args, kwargs, options)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/tensorflow/python/autograph/impl/api.py", line 460, in _call_unconverted
    return f(*args)
           ^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/keras/src/layers/preprocessing/feature_space.py", line 723, in __call__
    preprocessed_data = self._preprocess_features(data)
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/keras/src/layers/preprocessing/feature_space.py", line 555, in _preprocess_features
    return {
           ^
  File "/usr/local/lib/python3.11/site-packages/keras/src/layers/preprocessing/feature_space.py", line 556, in <dictcomp>
    name: self.preprocessors[name](features[name])
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/keras/src/utils/traceback_utils.py", line 123, in error_handler
    raise e.with_traceback(filtered_tb) from None
  File "/usr/local/lib/python3.11/site-packages/tree/__init__.py", line 435, in map_structure
    [func(*args) for args in zip(*map(flatten, structures))])
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/tree/__init__.py", line 435, in <listcomp>
    [func(*args) for args in zip(*map(flatten, structures))])
     ^^^^^^^^^^^
NotImplementedError: in user code:

    File "/home/dercybersecurity/datapreprocessing/structured_data_classification_with_feature_space.py", line 307, in None  *
        lambda x, y: (feature_space(x), y)
    File "/usr/local/lib/python3.11/site-packages/keras/src/layers/preprocessing/feature_space.py", line 723, in __call__  **
        preprocessed_data = self._preprocess_features(data)
    File "/usr/local/lib/python3.11/site-packages/keras/src/layers/preprocessing/feature_space.py", line 555, in _preprocess_features
        return {
    File "/usr/local/lib/python3.11/site-packages/keras/src/layers/preprocessing/feature_space.py", line 556, in <dictcomp>
        name: self.preprocessors[name](features[name])
    File "/usr/local/lib/python3.11/site-packages/keras/src/utils/traceback_utils.py", line 123, in error_handler
        raise e.with_traceback(filtered_tb) from None
    File "/usr/local/lib/python3.11/site-packages/tree/__init__.py", line 435, in map_structure
        [func(*args) for args in zip(*map(flatten, structures))])
    File "/usr/local/lib/python3.11/site-packages/tree/__init__.py", line 435, in <listcomp>
        [func(*args) for args in zip(*map(flatten, structures))])

    NotImplementedError: Cannot convert a symbolic tf.Tensor (ExpandDims_3:0) to a numpy array. This error may indicate that you're trying to pass a Tensor to a NumPy call, which is not supported.
@sachinprasadhs
Copy link
Collaborator

Hi,

Basically this error is happening due to the map function, where you're trying to pass symbolic tf.Tensor to feature_space of pytorch backend, which is not allowed when tf.Tensor tries to evaluate with the torch backend during map function.

You can modify the logiic by converting tf.data to numpy datatype using as_numpy_iterator() and then apply the feature_space.

@sibyjackgrove
Copy link
Author

@sachinprasadhs Do you mean converting tf.data to numpy iterator using train_ds = train_ds.as_numpy_iterator() and then applying map on that?
However, I am unable to apply the map function to the numpy iterator since I get the following error. Could you suggest how to apply the featurespace on the numpy iterator?

preprocessed_train_ds = train_ds.map(lambda x, y: (feature_space(x.as_numpy_iterator()), y), num_parallel_calls=tf.data.AUTOTUNE
                            ^^^^^^^^^^^^
AttributeError: 'NumpyIterator' object has no attribute 'map'

@sachinprasadhs
Copy link
Collaborator

After modifying everything, I was able to run many majority of the code and now it is stuck at prediction.

Here is the modified Gist for reference

@sibyjackgrove
Copy link
Author

Thank you for the new code. I am including it below for reference.

x = []
y =[]
for i, j in train_ds.as_numpy_iterator():
    x.append(feature_space(i))
    y.append(j)
x = keras.ops.convert_to_tensor(x)
y = keras.ops.convert_to_tensor(y)
preprocessed_train_ds = tf.data.Dataset.from_tensors((x,y)).prefetch(tf.data.AUTOTUNE)

The solution seems to require iterating through the entire tf.data dataset and load it into memory. This may be impractical if the dataset is very large. In my actual application, I am streaming from disk using tf.data.make_csv_dataset
Could you please suggest an alternate solution?

@sachinprasadhs
Copy link
Collaborator

AFAIK, symbolic tf.Tensor and pytorch backend is not allowed.

@sachinprasadhs sachinprasadhs added the keras-team-review-pending Pending review by a Keras team member. label Mar 16, 2024
@nkovela1
Copy link
Contributor

Hi @sibyjackgrove, you cannot use a non-TF featurespace in tf.data. We recommend either switching to the TF backend or not using tf.data. Thanks!

To keep using the PyTorch backend, you can use a TorchDataLoader or a Keras PyDataset.

@google-ml-butler Google-ML-Butler
Copy link

Are you satisfied with the resolution of your issue?
Yes
No

@sibyjackgrove
Copy link
Author

It is unfortunate that such a useful feature like Keras FeatureSpace doesn't have multi-backed support. I couldn't find anything in the documentation to suggest that Keras FeatureSpace doesn't work with PyTorch backend. I think this should be made explicit so that users don't utilize FeatureSpace thinking that it is mult-backend.

@sachinprasadhs
Copy link
Collaborator

As suggested in the above comment, you can use it with like below using torch backend.

To keep using the PyTorch backend, you can use a TorchDataLoader or a Keras PyDataset.

or you can use .as_numpy_iterator():

@sibyjackgrove
Copy link
Author

@sachinprasadhs Could you please share an example for using Keras PyDataset with FeatureSpace? I could only find limited documentation on PyDataset. I used tf.data dataset since it has the most documentation and is very flexible.
Also, if I use Jax backend will it work with tf.data and FeatureSpace?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@sachinprasadhs sachinprasadhs

Labels
keras-team-review-pending Pending review by a Keras team member. type:Bug
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@sibyjackgrove @nkovela1 @sachinprasadhs