Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ModuleNotFoundError: No module named 'tensorflow.compat.v1.keras' #17

Open
johnlockejrr opened this issue Oct 18, 2024 · 132 comments
Open

Comments

@johnlockejrr
Copy link

Hi!

I try to finetune a model and I get this error: ModuleNotFoundError: No module named 'tensorflow.compat.v1.keras'
Under my WSL 2 I couldn't install tensorflow==2.12.1 because with CUDA Version: 12.7 doesn't have GPU support, so I installed tf_nightly 2.19.0.dev20241017 that works.

(sbb_pixelwise_segmentation-py3.11) incognito@DESKTOP-H1BS9PO:~/eynollah/sbb_pixelwise_segmentation$ python train.py with config_params.json
2024-10-18 17:07:59.114698: I tensorflow/core/util/port.cc:153] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
2024-10-18 17:07:59.274724: E external/local_xla/xla/stream_executor/cuda/cuda_fft.cc:467] Unable to register cuFFT factory: Attempting to register factory for plugin cuFFT when one has already been registered
WARNING: All log messages before absl::InitializeLog() is called are written to STDERR
E0000 00:00:1729260479.374835   60937 cuda_dnn.cc:8498] Unable to register cuDNN factory: Attempting to register factory for plugin cuDNN when one has already been registered
E0000 00:00:1729260479.391601   60937 cuda_blas.cc:1410] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered
2024-10-18 17:07:59.541551: I tensorflow/core/platform/cpu_feature_guard.cc:210] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 AVX512F AVX512_VNNI FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
Traceback (most recent call last):
  File "/home/incognito/eynollah/sbb_pixelwise_segmentation/train.py", line 4, in <module>
    from tensorflow.compat.v1.keras.backend import set_session
ModuleNotFoundError: No module named 'tensorflow.compat.v1.keras'

I know that tensorflow.compat.v1.keras is deprecated in latest tensorflow, what can I do? Is that session really needed?

Thank you!

@vahidrezanezhad
Copy link
Member

Dear @johnlockejrr
I have provided the requirements and several improvements in the branch unifying-training-models. Could you please test training with that branch? If it doesn't work, I'll be able to assist you further. You can configure your settings based on the instructions provided in this : wiki.
Thank you

@johnlockejrr
Copy link
Author

I just chaged from tensorflow.compat.v1.keras.backend import set_session in the train.py to from tensorflow.python.keras.backend import set_session and trainer started until I got this error:

(sbb_pixelwise_segmentation-py3.11) incognito@DESKTOP-H1BS9PO:~/eynollah/sbb_pixelwise_segmentation$ python train.py with config_params.json
2024-10-18 17:35:24.283063: I tensorflow/core/util/port.cc:153] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
2024-10-18 17:35:24.292083: E external/local_xla/xla/stream_executor/cuda/cuda_fft.cc:467] Unable to register cuFFT factory: Attempting to register factory for plugin cuFFT when one has already been registered
WARNING: All log messages before absl::InitializeLog() is called are written to STDERR
E0000 00:00:1729262124.302741   61768 cuda_dnn.cc:8498] Unable to register cuDNN factory: Attempting to register factory for plugin cuDNN when one has already been registered
E0000 00:00:1729262124.305857   61768 cuda_blas.cc:1410] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered
2024-10-18 17:35:24.316812: I tensorflow/core/platform/cpu_feature_guard.cc:210] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 AVX512F AVX512_VNNI FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
WARNING - root - Added new config entry: "rotation"
WARNING - root - Added new config entry: "rotation_not_90"
WARNING - train - No observers have been added to this run
INFO - train - Running command 'run'
INFO - train - Started
I0000 00:00:1729262126.854722   61768 gpu_device.cc:2020] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 9711 MB memory:  -> device: 0, name: NVIDIA GeForce RTX 3060, pci bus id: 0000:04:00.0, compute capability: 8.6
33it [00:09,  3.46it/s]
5it [00:01,  3.76it/s]
I0000 00:00:1729262137.737353   61768 gpu_device.cc:2020] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 9711 MB memory:  -> device: 0, name: NVIDIA GeForce RTX 3060, pci bus id: 0000:04:00.0, compute capability: 8.6
AttributeError: module 'ml_dtypes' has no attribute 'float8_e3m4'
ERROR - train - Failed after 0:00:13!
Traceback (most recent calls WITHOUT Sacred internals):
  File "/home/incognito/eynollah/sbb_pixelwise_segmentation/train.py", line 187, in run
    optimizer=Adam(lr=learning_rate), metrics=['accuracy'])
              ^^^^^^^^^^^^^^^^^^^^^^
  File "/home/incognito/eynollah/sbb_pixelwise_segmentation/sbb_pixelwise_segmentation-py3.11/lib/python3.11/site-packages/keras/src/optimizers/adam.py", line 62, in __init__
    super().__init__(
  File "/home/incognito/eynollah/sbb_pixelwise_segmentation/sbb_pixelwise_segmentation-py3.11/lib/python3.11/site-packages/keras/src/backend/tensorflow/optimizer.py", line 23, in __init__
    super().__init__(*args, **kwargs)
  File "/home/incognito/eynollah/sbb_pixelwise_segmentation/sbb_pixelwise_segmentation-py3.11/lib/python3.11/site-packages/keras/src/optimizers/base_optimizer.py", line 90, in __init__
    raise ValueError(f"Argument(s) not recognized: {kwargs}")
ValueError: Argument(s) not recognized: {'lr': 0.0001}

@johnlockejrr
Copy link
Author

Dear @johnlockejrr I have provided the requirements and several improvements in the branch unifying-training-models. Could you please test training with that branch? If it doesn't work, I'll be able to assist you further. You can configure your settings based on the instructions provided in this : wiki. Thank you

I'll try right now.

@johnlockejrr
Copy link
Author

On the branch you told me, ran for about two minutes:

(sbb_pixelwise_segmentation-py3.11) incognito@DESKTOP-H1BS9PO:~/eynollah/sbb_pixelwise_segmentation$ python train.py with config_params.json
WARNING: All log messages before absl::InitializeLog() is called are written to STDERR
E0000 00:00:1729263995.153621   62806 cuda_dnn.cc:8498] Unable to register cuDNN factory: Attempting to register factory for plugin cuDNN when one has already been registered
E0000 00:00:1729263995.156839   62806 cuda_blas.cc:1410] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered
WARNING - train - No observers have been added to this run
INFO - train - Running command 'run'
INFO - train - Started
I0000 00:00:1729263998.450910   62806 gpu_device.cc:2020] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 9711 MB memory:  -> device: 0, name: NVIDIA GeForce RTX 3060, pci bus id: 0000:04:00.0, compute capability: 8.6
33it [01:07,  2.04s/it]
5it [00:01,  3.84it/s]
I0000 00:00:1729264067.002153   62806 gpu_device.cc:2020] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 9711 MB memory:  -> device: 0, name: NVIDIA GeForce RTX 3060, pci bus id: 0000:04:00.0, compute capability: 8.6
AttributeError: module 'ml_dtypes' has no attribute 'float8_e3m4'
Model: "functional_1"
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ Layer (type)                  ┃ Output Shape              ┃         Param # ┃ Connected to               ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ input_layer (InputLayer)      │ (None, 448, 224, 3)       │               0 │ -                          │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ zero_padding2d                │ (None, 454, 230, 3)       │               0 │ input_layer[0][0]          │
│ (ZeroPadding2D)               │                           │                 │                            │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ conv1 (Conv2D)                │ (None, 224, 112, 64)      │           9,472 │ zero_padding2d[0][0]       │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ bn_conv1 (BatchNormalization) │ (None, 224, 112, 64)      │             256 │ conv1[0][0]                │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ activation (Activation)       │ (None, 224, 112, 64)      │               0 │ bn_conv1[0][0]             │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ max_pooling2d (MaxPooling2D)  │ (None, 111, 55, 64)       │               0 │ activation[0][0]           │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ res2a_branch2a (Conv2D)       │ (None, 111, 55, 64)       │           4,160 │ max_pooling2d[0][0]        │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ bn2a_branch2a                 │ (None, 111, 55, 64)       │             256 │ res2a_branch2a[0][0]       │
│ (BatchNormalization)          │                           │                 │                            │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ activation_1 (Activation)     │ (None, 111, 55, 64)       │               0 │ bn2a_branch2a[0][0]        │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ res2a_branch2b (Conv2D)       │ (None, 111, 55, 64)       │          36,928 │ activation_1[0][0]         │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ bn2a_branch2b                 │ (None, 111, 55, 64)       │             256 │ res2a_branch2b[0][0]       │
│ (BatchNormalization)          │                           │                 │                            │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ activation_2 (Activation)     │ (None, 111, 55, 64)       │               0 │ bn2a_branch2b[0][0]        │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ res2a_branch2c (Conv2D)       │ (None, 111, 55, 256)      │          16,640 │ activation_2[0][0]         │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ res2a_branch1 (Conv2D)        │ (None, 111, 55, 256)      │          16,640 │ max_pooling2d[0][0]        │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ bn2a_branch2c                 │ (None, 111, 55, 256)      │           1,024 │ res2a_branch2c[0][0]       │
│ (BatchNormalization)          │                           │                 │                            │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ bn2a_branch1                  │ (None, 111, 55, 256)      │           1,024 │ res2a_branch1[0][0]        │
│ (BatchNormalization)          │                           │                 │                            │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ add (Add)                     │ (None, 111, 55, 256)      │               0 │ bn2a_branch2c[0][0],       │
│                               │                           │                 │ bn2a_branch1[0][0]         │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ activation_3 (Activation)     │ (None, 111, 55, 256)      │               0 │ add[0][0]                  │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ res2b_branch2a (Conv2D)       │ (None, 111, 55, 64)       │          16,448 │ activation_3[0][0]         │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ bn2b_branch2a                 │ (None, 111, 55, 64)       │             256 │ res2b_branch2a[0][0]       │
│ (BatchNormalization)          │                           │                 │                            │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ activation_4 (Activation)     │ (None, 111, 55, 64)       │               0 │ bn2b_branch2a[0][0]        │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ res2b_branch2b (Conv2D)       │ (None, 111, 55, 64)       │          36,928 │ activation_4[0][0]         │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ bn2b_branch2b                 │ (None, 111, 55, 64)       │             256 │ res2b_branch2b[0][0]       │
│ (BatchNormalization)          │                           │                 │                            │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ activation_5 (Activation)     │ (None, 111, 55, 64)       │               0 │ bn2b_branch2b[0][0]        │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ res2b_branch2c (Conv2D)       │ (None, 111, 55, 256)      │          16,640 │ activation_5[0][0]         │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ bn2b_branch2c                 │ (None, 111, 55, 256)      │           1,024 │ res2b_branch2c[0][0]       │
│ (BatchNormalization)          │                           │                 │                            │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ add_1 (Add)                   │ (None, 111, 55, 256)      │               0 │ bn2b_branch2c[0][0],       │
│                               │                           │                 │ activation_3[0][0]         │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ activation_6 (Activation)     │ (None, 111, 55, 256)      │               0 │ add_1[0][0]                │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ res2c_branch2a (Conv2D)       │ (None, 111, 55, 64)       │          16,448 │ activation_6[0][0]         │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ bn2c_branch2a                 │ (None, 111, 55, 64)       │             256 │ res2c_branch2a[0][0]       │
│ (BatchNormalization)          │                           │                 │                            │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ activation_7 (Activation)     │ (None, 111, 55, 64)       │               0 │ bn2c_branch2a[0][0]        │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ res2c_branch2b (Conv2D)       │ (None, 111, 55, 64)       │          36,928 │ activation_7[0][0]         │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ bn2c_branch2b                 │ (None, 111, 55, 64)       │             256 │ res2c_branch2b[0][0]       │
│ (BatchNormalization)          │                           │                 │                            │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ activation_8 (Activation)     │ (None, 111, 55, 64)       │               0 │ bn2c_branch2b[0][0]        │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ res2c_branch2c (Conv2D)       │ (None, 111, 55, 256)      │          16,640 │ activation_8[0][0]         │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ bn2c_branch2c                 │ (None, 111, 55, 256)      │           1,024 │ res2c_branch2c[0][0]       │
│ (BatchNormalization)          │                           │                 │                            │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ add_2 (Add)                   │ (None, 111, 55, 256)      │               0 │ bn2c_branch2c[0][0],       │
│                               │                           │                 │ activation_6[0][0]         │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ activation_9 (Activation)     │ (None, 111, 55, 256)      │               0 │ add_2[0][0]                │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ res3a_branch2a (Conv2D)       │ (None, 56, 28, 128)       │          32,896 │ activation_9[0][0]         │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ bn3a_branch2a                 │ (None, 56, 28, 128)       │             512 │ res3a_branch2a[0][0]       │
│ (BatchNormalization)          │                           │                 │                            │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ activation_10 (Activation)    │ (None, 56, 28, 128)       │               0 │ bn3a_branch2a[0][0]        │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ res3a_branch2b (Conv2D)       │ (None, 56, 28, 128)       │         147,584 │ activation_10[0][0]        │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ bn3a_branch2b                 │ (None, 56, 28, 128)       │             512 │ res3a_branch2b[0][0]       │
│ (BatchNormalization)          │                           │                 │                            │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ activation_11 (Activation)    │ (None, 56, 28, 128)       │               0 │ bn3a_branch2b[0][0]        │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ res3a_branch2c (Conv2D)       │ (None, 56, 28, 512)       │          66,048 │ activation_11[0][0]        │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ res3a_branch1 (Conv2D)        │ (None, 56, 28, 512)       │         131,584 │ activation_9[0][0]         │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ bn3a_branch2c                 │ (None, 56, 28, 512)       │           2,048 │ res3a_branch2c[0][0]       │
│ (BatchNormalization)          │                           │                 │                            │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ bn3a_branch1                  │ (None, 56, 28, 512)       │           2,048 │ res3a_branch1[0][0]        │
│ (BatchNormalization)          │                           │                 │                            │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ add_3 (Add)                   │ (None, 56, 28, 512)       │               0 │ bn3a_branch2c[0][0],       │
│                               │                           │                 │ bn3a_branch1[0][0]         │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ activation_12 (Activation)    │ (None, 56, 28, 512)       │               0 │ add_3[0][0]                │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ res3b_branch2a (Conv2D)       │ (None, 56, 28, 128)       │          65,664 │ activation_12[0][0]        │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ bn3b_branch2a                 │ (None, 56, 28, 128)       │             512 │ res3b_branch2a[0][0]       │
│ (BatchNormalization)          │                           │                 │                            │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ activation_13 (Activation)    │ (None, 56, 28, 128)       │               0 │ bn3b_branch2a[0][0]        │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ res3b_branch2b (Conv2D)       │ (None, 56, 28, 128)       │         147,584 │ activation_13[0][0]        │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ bn3b_branch2b                 │ (None, 56, 28, 128)       │             512 │ res3b_branch2b[0][0]       │
│ (BatchNormalization)          │                           │                 │                            │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ activation_14 (Activation)    │ (None, 56, 28, 128)       │               0 │ bn3b_branch2b[0][0]        │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ res3b_branch2c (Conv2D)       │ (None, 56, 28, 512)       │          66,048 │ activation_14[0][0]        │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ bn3b_branch2c                 │ (None, 56, 28, 512)       │           2,048 │ res3b_branch2c[0][0]       │
│ (BatchNormalization)          │                           │                 │                            │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ add_4 (Add)                   │ (None, 56, 28, 512)       │               0 │ bn3b_branch2c[0][0],       │
│                               │                           │                 │ activation_12[0][0]        │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ activation_15 (Activation)    │ (None, 56, 28, 512)       │               0 │ add_4[0][0]                │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ res3c_branch2a (Conv2D)       │ (None, 56, 28, 128)       │          65,664 │ activation_15[0][0]        │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ bn3c_branch2a                 │ (None, 56, 28, 128)       │             512 │ res3c_branch2a[0][0]       │
│ (BatchNormalization)          │                           │                 │                            │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ activation_16 (Activation)    │ (None, 56, 28, 128)       │               0 │ bn3c_branch2a[0][0]        │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ res3c_branch2b (Conv2D)       │ (None, 56, 28, 128)       │         147,584 │ activation_16[0][0]        │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ bn3c_branch2b                 │ (None, 56, 28, 128)       │             512 │ res3c_branch2b[0][0]       │
│ (BatchNormalization)          │                           │                 │                            │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ activation_17 (Activation)    │ (None, 56, 28, 128)       │               0 │ bn3c_branch2b[0][0]        │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ res3c_branch2c (Conv2D)       │ (None, 56, 28, 512)       │          66,048 │ activation_17[0][0]        │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ bn3c_branch2c                 │ (None, 56, 28, 512)       │           2,048 │ res3c_branch2c[0][0]       │
│ (BatchNormalization)          │                           │                 │                            │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ add_5 (Add)                   │ (None, 56, 28, 512)       │               0 │ bn3c_branch2c[0][0],       │
│                               │                           │                 │ activation_15[0][0]        │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ activation_18 (Activation)    │ (None, 56, 28, 512)       │               0 │ add_5[0][0]                │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ res3d_branch2a (Conv2D)       │ (None, 56, 28, 128)       │          65,664 │ activation_18[0][0]        │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ bn3d_branch2a                 │ (None, 56, 28, 128)       │             512 │ res3d_branch2a[0][0]       │
│ (BatchNormalization)          │                           │                 │                            │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ activation_19 (Activation)    │ (None, 56, 28, 128)       │               0 │ bn3d_branch2a[0][0]        │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ res3d_branch2b (Conv2D)       │ (None, 56, 28, 128)       │         147,584 │ activation_19[0][0]        │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ bn3d_branch2b                 │ (None, 56, 28, 128)       │             512 │ res3d_branch2b[0][0]       │
│ (BatchNormalization)          │                           │                 │                            │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ activation_20 (Activation)    │ (None, 56, 28, 128)       │               0 │ bn3d_branch2b[0][0]        │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ res3d_branch2c (Conv2D)       │ (None, 56, 28, 512)       │          66,048 │ activation_20[0][0]        │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ bn3d_branch2c                 │ (None, 56, 28, 512)       │           2,048 │ res3d_branch2c[0][0]       │
│ (BatchNormalization)          │                           │                 │                            │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ add_6 (Add)                   │ (None, 56, 28, 512)       │               0 │ bn3d_branch2c[0][0],       │
│                               │                           │                 │ activation_18[0][0]        │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ activation_21 (Activation)    │ (None, 56, 28, 512)       │               0 │ add_6[0][0]                │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ res4a_branch2a (Conv2D)       │ (None, 28, 14, 256)       │         131,328 │ activation_21[0][0]        │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ bn4a_branch2a                 │ (None, 28, 14, 256)       │           1,024 │ res4a_branch2a[0][0]       │
│ (BatchNormalization)          │                           │                 │                            │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ activation_22 (Activation)    │ (None, 28, 14, 256)       │               0 │ bn4a_branch2a[0][0]        │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ res4a_branch2b (Conv2D)       │ (None, 28, 14, 256)       │         590,080 │ activation_22[0][0]        │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ bn4a_branch2b                 │ (None, 28, 14, 256)       │           1,024 │ res4a_branch2b[0][0]       │
│ (BatchNormalization)          │                           │                 │                            │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ activation_23 (Activation)    │ (None, 28, 14, 256)       │               0 │ bn4a_branch2b[0][0]        │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ res4a_branch2c (Conv2D)       │ (None, 28, 14, 1024)      │         263,168 │ activation_23[0][0]        │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ res4a_branch1 (Conv2D)        │ (None, 28, 14, 1024)      │         525,312 │ activation_21[0][0]        │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ bn4a_branch2c                 │ (None, 28, 14, 1024)      │           4,096 │ res4a_branch2c[0][0]       │
│ (BatchNormalization)          │                           │                 │                            │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ bn4a_branch1                  │ (None, 28, 14, 1024)      │           4,096 │ res4a_branch1[0][0]        │
│ (BatchNormalization)          │                           │                 │                            │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ add_7 (Add)                   │ (None, 28, 14, 1024)      │               0 │ bn4a_branch2c[0][0],       │
│                               │                           │                 │ bn4a_branch1[0][0]         │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ activation_24 (Activation)    │ (None, 28, 14, 1024)      │               0 │ add_7[0][0]                │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ res4b_branch2a (Conv2D)       │ (None, 28, 14, 256)       │         262,400 │ activation_24[0][0]        │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ bn4b_branch2a                 │ (None, 28, 14, 256)       │           1,024 │ res4b_branch2a[0][0]       │
│ (BatchNormalization)          │                           │                 │                            │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ activation_25 (Activation)    │ (None, 28, 14, 256)       │               0 │ bn4b_branch2a[0][0]        │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ res4b_branch2b (Conv2D)       │ (None, 28, 14, 256)       │         590,080 │ activation_25[0][0]        │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ bn4b_branch2b                 │ (None, 28, 14, 256)       │           1,024 │ res4b_branch2b[0][0]       │
│ (BatchNormalization)          │                           │                 │                            │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ activation_26 (Activation)    │ (None, 28, 14, 256)       │               0 │ bn4b_branch2b[0][0]        │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ res4b_branch2c (Conv2D)       │ (None, 28, 14, 1024)      │         263,168 │ activation_26[0][0]        │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ bn4b_branch2c                 │ (None, 28, 14, 1024)      │           4,096 │ res4b_branch2c[0][0]       │
│ (BatchNormalization)          │                           │                 │                            │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ add_8 (Add)                   │ (None, 28, 14, 1024)      │               0 │ bn4b_branch2c[0][0],       │
│                               │                           │                 │ activation_24[0][0]        │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ activation_27 (Activation)    │ (None, 28, 14, 1024)      │               0 │ add_8[0][0]                │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ res4c_branch2a (Conv2D)       │ (None, 28, 14, 256)       │         262,400 │ activation_27[0][0]        │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ bn4c_branch2a                 │ (None, 28, 14, 256)       │           1,024 │ res4c_branch2a[0][0]       │
│ (BatchNormalization)          │                           │                 │                            │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ activation_28 (Activation)    │ (None, 28, 14, 256)       │               0 │ bn4c_branch2a[0][0]        │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ res4c_branch2b (Conv2D)       │ (None, 28, 14, 256)       │         590,080 │ activation_28[0][0]        │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ bn4c_branch2b                 │ (None, 28, 14, 256)       │           1,024 │ res4c_branch2b[0][0]       │
│ (BatchNormalization)          │                           │                 │                            │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ activation_29 (Activation)    │ (None, 28, 14, 256)       │               0 │ bn4c_branch2b[0][0]        │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ res4c_branch2c (Conv2D)       │ (None, 28, 14, 1024)      │         263,168 │ activation_29[0][0]        │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ bn4c_branch2c                 │ (None, 28, 14, 1024)      │           4,096 │ res4c_branch2c[0][0]       │
│ (BatchNormalization)          │                           │                 │                            │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ add_9 (Add)                   │ (None, 28, 14, 1024)      │               0 │ bn4c_branch2c[0][0],       │
│                               │                           │                 │ activation_27[0][0]        │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ activation_30 (Activation)    │ (None, 28, 14, 1024)      │               0 │ add_9[0][0]                │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ res4d_branch2a (Conv2D)       │ (None, 28, 14, 256)       │         262,400 │ activation_30[0][0]        │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ bn4d_branch2a                 │ (None, 28, 14, 256)       │           1,024 │ res4d_branch2a[0][0]       │
│ (BatchNormalization)          │                           │                 │                            │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ activation_31 (Activation)    │ (None, 28, 14, 256)       │               0 │ bn4d_branch2a[0][0]        │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ res4d_branch2b (Conv2D)       │ (None, 28, 14, 256)       │         590,080 │ activation_31[0][0]        │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ bn4d_branch2b                 │ (None, 28, 14, 256)       │           1,024 │ res4d_branch2b[0][0]       │
│ (BatchNormalization)          │                           │                 │                            │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ activation_32 (Activation)    │ (None, 28, 14, 256)       │               0 │ bn4d_branch2b[0][0]        │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ res4d_branch2c (Conv2D)       │ (None, 28, 14, 1024)      │         263,168 │ activation_32[0][0]        │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ bn4d_branch2c                 │ (None, 28, 14, 1024)      │           4,096 │ res4d_branch2c[0][0]       │
│ (BatchNormalization)          │                           │                 │                            │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ add_10 (Add)                  │ (None, 28, 14, 1024)      │               0 │ bn4d_branch2c[0][0],       │
│                               │                           │                 │ activation_30[0][0]        │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ activation_33 (Activation)    │ (None, 28, 14, 1024)      │               0 │ add_10[0][0]               │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ res4e_branch2a (Conv2D)       │ (None, 28, 14, 256)       │         262,400 │ activation_33[0][0]        │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ bn4e_branch2a                 │ (None, 28, 14, 256)       │           1,024 │ res4e_branch2a[0][0]       │
│ (BatchNormalization)          │                           │                 │                            │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ activation_34 (Activation)    │ (None, 28, 14, 256)       │               0 │ bn4e_branch2a[0][0]        │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ res4e_branch2b (Conv2D)       │ (None, 28, 14, 256)       │         590,080 │ activation_34[0][0]        │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ bn4e_branch2b                 │ (None, 28, 14, 256)       │           1,024 │ res4e_branch2b[0][0]       │
│ (BatchNormalization)          │                           │                 │                            │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ activation_35 (Activation)    │ (None, 28, 14, 256)       │               0 │ bn4e_branch2b[0][0]        │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ res4e_branch2c (Conv2D)       │ (None, 28, 14, 1024)      │         263,168 │ activation_35[0][0]        │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ bn4e_branch2c                 │ (None, 28, 14, 1024)      │           4,096 │ res4e_branch2c[0][0]       │
│ (BatchNormalization)          │                           │                 │                            │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ add_11 (Add)                  │ (None, 28, 14, 1024)      │               0 │ bn4e_branch2c[0][0],       │
│                               │                           │                 │ activation_33[0][0]        │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ activation_36 (Activation)    │ (None, 28, 14, 1024)      │               0 │ add_11[0][0]               │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ res4f_branch2a (Conv2D)       │ (None, 28, 14, 256)       │         262,400 │ activation_36[0][0]        │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ bn4f_branch2a                 │ (None, 28, 14, 256)       │           1,024 │ res4f_branch2a[0][0]       │
│ (BatchNormalization)          │                           │                 │                            │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ activation_37 (Activation)    │ (None, 28, 14, 256)       │               0 │ bn4f_branch2a[0][0]        │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ res4f_branch2b (Conv2D)       │ (None, 28, 14, 256)       │         590,080 │ activation_37[0][0]        │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ bn4f_branch2b                 │ (None, 28, 14, 256)       │           1,024 │ res4f_branch2b[0][0]       │
│ (BatchNormalization)          │                           │                 │                            │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ activation_38 (Activation)    │ (None, 28, 14, 256)       │               0 │ bn4f_branch2b[0][0]        │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ res4f_branch2c (Conv2D)       │ (None, 28, 14, 1024)      │         263,168 │ activation_38[0][0]        │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ bn4f_branch2c                 │ (None, 28, 14, 1024)      │           4,096 │ res4f_branch2c[0][0]       │
│ (BatchNormalization)          │                           │                 │                            │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ add_12 (Add)                  │ (None, 28, 14, 1024)      │               0 │ bn4f_branch2c[0][0],       │
│                               │                           │                 │ activation_36[0][0]        │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ activation_39 (Activation)    │ (None, 28, 14, 1024)      │               0 │ add_12[0][0]               │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ res5a_branch2a (Conv2D)       │ (None, 14, 7, 512)        │         524,800 │ activation_39[0][0]        │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ bn5a_branch2a                 │ (None, 14, 7, 512)        │           2,048 │ res5a_branch2a[0][0]       │
│ (BatchNormalization)          │                           │                 │                            │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ activation_40 (Activation)    │ (None, 14, 7, 512)        │               0 │ bn5a_branch2a[0][0]        │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ res5a_branch2b (Conv2D)       │ (None, 14, 7, 512)        │       2,359,808 │ activation_40[0][0]        │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ bn5a_branch2b                 │ (None, 14, 7, 512)        │           2,048 │ res5a_branch2b[0][0]       │
│ (BatchNormalization)          │                           │                 │                            │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ activation_41 (Activation)    │ (None, 14, 7, 512)        │               0 │ bn5a_branch2b[0][0]        │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ res5a_branch2c (Conv2D)       │ (None, 14, 7, 2048)       │       1,050,624 │ activation_41[0][0]        │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ res5a_branch1 (Conv2D)        │ (None, 14, 7, 2048)       │       2,099,200 │ activation_39[0][0]        │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ bn5a_branch2c                 │ (None, 14, 7, 2048)       │           8,192 │ res5a_branch2c[0][0]       │
│ (BatchNormalization)          │                           │                 │                            │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ bn5a_branch1                  │ (None, 14, 7, 2048)       │           8,192 │ res5a_branch1[0][0]        │
│ (BatchNormalization)          │                           │                 │                            │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ add_13 (Add)                  │ (None, 14, 7, 2048)       │               0 │ bn5a_branch2c[0][0],       │
│                               │                           │                 │ bn5a_branch1[0][0]         │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ activation_42 (Activation)    │ (None, 14, 7, 2048)       │               0 │ add_13[0][0]               │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ res5b_branch2a (Conv2D)       │ (None, 14, 7, 512)        │       1,049,088 │ activation_42[0][0]        │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ bn5b_branch2a                 │ (None, 14, 7, 512)        │           2,048 │ res5b_branch2a[0][0]       │
│ (BatchNormalization)          │                           │                 │                            │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ activation_43 (Activation)    │ (None, 14, 7, 512)        │               0 │ bn5b_branch2a[0][0]        │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ res5b_branch2b (Conv2D)       │ (None, 14, 7, 512)        │       2,359,808 │ activation_43[0][0]        │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ bn5b_branch2b                 │ (None, 14, 7, 512)        │           2,048 │ res5b_branch2b[0][0]       │
│ (BatchNormalization)          │                           │                 │                            │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ activation_44 (Activation)    │ (None, 14, 7, 512)        │               0 │ bn5b_branch2b[0][0]        │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ res5b_branch2c (Conv2D)       │ (None, 14, 7, 2048)       │       1,050,624 │ activation_44[0][0]        │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ bn5b_branch2c                 │ (None, 14, 7, 2048)       │           8,192 │ res5b_branch2c[0][0]       │
│ (BatchNormalization)          │                           │                 │                            │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ add_14 (Add)                  │ (None, 14, 7, 2048)       │               0 │ bn5b_branch2c[0][0],       │
│                               │                           │                 │ activation_42[0][0]        │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ activation_45 (Activation)    │ (None, 14, 7, 2048)       │               0 │ add_14[0][0]               │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ res5c_branch2a (Conv2D)       │ (None, 14, 7, 512)        │       1,049,088 │ activation_45[0][0]        │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ bn5c_branch2a                 │ (None, 14, 7, 512)        │           2,048 │ res5c_branch2a[0][0]       │
│ (BatchNormalization)          │                           │                 │                            │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ activation_46 (Activation)    │ (None, 14, 7, 512)        │               0 │ bn5c_branch2a[0][0]        │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ res5c_branch2b (Conv2D)       │ (None, 14, 7, 512)        │       2,359,808 │ activation_46[0][0]        │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ bn5c_branch2b                 │ (None, 14, 7, 512)        │           2,048 │ res5c_branch2b[0][0]       │
│ (BatchNormalization)          │                           │                 │                            │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ activation_47 (Activation)    │ (None, 14, 7, 512)        │               0 │ bn5c_branch2b[0][0]        │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ res5c_branch2c (Conv2D)       │ (None, 14, 7, 2048)       │       1,050,624 │ activation_47[0][0]        │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ bn5c_branch2c                 │ (None, 14, 7, 2048)       │           8,192 │ res5c_branch2c[0][0]       │
│ (BatchNormalization)          │                           │                 │                            │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ add_15 (Add)                  │ (None, 14, 7, 2048)       │               0 │ bn5c_branch2c[0][0],       │
│                               │                           │                 │ activation_45[0][0]        │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ activation_48 (Activation)    │ (None, 14, 7, 2048)       │               0 │ add_15[0][0]               │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ conv2d (Conv2D)               │ (None, 14, 7, 1024)       │       2,098,176 │ activation_48[0][0]        │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ batch_normalization           │ (None, 14, 7, 1024)       │           4,096 │ conv2d[0][0]               │
│ (BatchNormalization)          │                           │                 │                            │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ activation_49 (Activation)    │ (None, 14, 7, 1024)       │               0 │ batch_normalization[0][0]  │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ up_sampling2d (UpSampling2D)  │ (None, 28, 14, 1024)      │               0 │ activation_49[0][0]        │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ concatenate (Concatenate)     │ (None, 28, 14, 2048)      │               0 │ up_sampling2d[0][0],       │
│                               │                           │                 │ activation_39[0][0]        │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ zero_padding2d_2              │ (None, 30, 16, 2048)      │               0 │ concatenate[0][0]          │
│ (ZeroPadding2D)               │                           │                 │                            │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ conv2d_1 (Conv2D)             │ (None, 28, 14, 512)       │       9,437,696 │ zero_padding2d_2[0][0]     │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ batch_normalization_1         │ (None, 28, 14, 512)       │           2,048 │ conv2d_1[0][0]             │
│ (BatchNormalization)          │                           │                 │                            │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ activation_50 (Activation)    │ (None, 28, 14, 512)       │               0 │ batch_normalization_1[0][… │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ up_sampling2d_1               │ (None, 56, 28, 512)       │               0 │ activation_50[0][0]        │
│ (UpSampling2D)                │                           │                 │                            │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ concatenate_1 (Concatenate)   │ (None, 56, 28, 1024)      │               0 │ up_sampling2d_1[0][0],     │
│                               │                           │                 │ activation_21[0][0]        │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ zero_padding2d_3              │ (None, 58, 30, 1024)      │               0 │ concatenate_1[0][0]        │
│ (ZeroPadding2D)               │                           │                 │                            │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ conv2d_2 (Conv2D)             │ (None, 56, 28, 256)       │       2,359,552 │ zero_padding2d_3[0][0]     │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ batch_normalization_2         │ (None, 56, 28, 256)       │           1,024 │ conv2d_2[0][0]             │
│ (BatchNormalization)          │                           │                 │                            │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ activation_51 (Activation)    │ (None, 56, 28, 256)       │               0 │ batch_normalization_2[0][… │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ zero_padding2d_1              │ (None, 113, 57, 256)      │               0 │ activation_9[0][0]         │
│ (ZeroPadding2D)               │                           │                 │                            │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ up_sampling2d_2               │ (None, 112, 56, 256)      │               0 │ activation_51[0][0]        │
│ (UpSampling2D)                │                           │                 │                            │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ lambda (Lambda)               │ (None, 112, 56, 256)      │               0 │ zero_padding2d_1[0][0]     │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ concatenate_2 (Concatenate)   │ (None, 112, 56, 512)      │               0 │ up_sampling2d_2[0][0],     │
│                               │                           │                 │ lambda[0][0]               │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ zero_padding2d_4              │ (None, 114, 58, 512)      │               0 │ concatenate_2[0][0]        │
│ (ZeroPadding2D)               │                           │                 │                            │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ conv2d_3 (Conv2D)             │ (None, 112, 56, 128)      │         589,952 │ zero_padding2d_4[0][0]     │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ batch_normalization_3         │ (None, 112, 56, 128)      │             512 │ conv2d_3[0][0]             │
│ (BatchNormalization)          │                           │                 │                            │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ activation_52 (Activation)    │ (None, 112, 56, 128)      │               0 │ batch_normalization_3[0][… │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ up_sampling2d_3               │ (None, 224, 112, 128)     │               0 │ activation_52[0][0]        │
│ (UpSampling2D)                │                           │                 │                            │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ concatenate_3 (Concatenate)   │ (None, 224, 112, 192)     │               0 │ up_sampling2d_3[0][0],     │
│                               │                           │                 │ conv1[0][0]                │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ zero_padding2d_5              │ (None, 226, 114, 192)     │               0 │ concatenate_3[0][0]        │
│ (ZeroPadding2D)               │                           │                 │                            │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ conv2d_4 (Conv2D)             │ (None, 224, 112, 64)      │         110,656 │ zero_padding2d_5[0][0]     │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ batch_normalization_4         │ (None, 224, 112, 64)      │             256 │ conv2d_4[0][0]             │
│ (BatchNormalization)          │                           │                 │                            │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ activation_53 (Activation)    │ (None, 224, 112, 64)      │               0 │ batch_normalization_4[0][… │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ up_sampling2d_4               │ (None, 448, 224, 64)      │               0 │ activation_53[0][0]        │
│ (UpSampling2D)                │                           │                 │                            │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ concatenate_4 (Concatenate)   │ (None, 448, 224, 67)      │               0 │ up_sampling2d_4[0][0],     │
│                               │                           │                 │ input_layer[0][0]          │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ zero_padding2d_6              │ (None, 450, 226, 67)      │               0 │ concatenate_4[0][0]        │
│ (ZeroPadding2D)               │                           │                 │                            │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ conv2d_5 (Conv2D)             │ (None, 448, 224, 32)      │          19,328 │ zero_padding2d_6[0][0]     │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ batch_normalization_5         │ (None, 448, 224, 32)      │             128 │ conv2d_5[0][0]             │
│ (BatchNormalization)          │                           │                 │                            │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ activation_54 (Activation)    │ (None, 448, 224, 32)      │               0 │ batch_normalization_5[0][… │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ conv2d_6 (Conv2D)             │ (None, 448, 224, 2)       │              66 │ activation_54[0][0]        │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ batch_normalization_6         │ (None, 448, 224, 2)       │               8 │ conv2d_6[0][0]             │
│ (BatchNormalization)          │                           │                 │                            │
├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤
│ activation_55 (Activation)    │ (None, 448, 224, 2)       │               0 │ batch_normalization_6[0][… │
└───────────────────────────────┴───────────────────────────┴─────────────────┴────────────────────────────┘
 Total params: 38,211,210 (145.76 MB)
 Trainable params: 38,154,054 (145.55 MB)
 Non-trainable params: 57,156 (223.27 KB)
ERROR - train - Failed after 0:01:11!
Traceback (most recent calls WITHOUT Sacred internals):
  File "/home/incognito/eynollah/sbb_pixelwise_segmentation/train.py", line 281, in run
    optimizer=Adam(lr=learning_rate), metrics=['accuracy'])
              ^^^^^^^^^^^^^^^^^^^^^^
  File "/home/incognito/eynollah/sbb_pixelwise_segmentation/sbb_pixelwise_segmentation-py3.11/lib/python3.11/site-packages/keras/src/optimizers/adam.py", line 62, in __init__
    super().__init__(
  File "/home/incognito/eynollah/sbb_pixelwise_segmentation/sbb_pixelwise_segmentation-py3.11/lib/python3.11/site-packages/keras/src/backend/tensorflow/optimizer.py", line 23, in __init__
    super().__init__(*args, **kwargs)
  File "/home/incognito/eynollah/sbb_pixelwise_segmentation/sbb_pixelwise_segmentation-py3.11/lib/python3.11/site-packages/keras/src/optimizers/base_optimizer.py", line 90, in __init__
    raise ValueError(f"Argument(s) not recognized: {kwargs}")
ValueError: Argument(s) not recognized: {'lr': 0.0001}

@vahidrezanezhad
Copy link
Member

I haven't tested the script with Python 3.11, which it seems you're using. I can try running it with Python 3.11, or you can test it with Python 3.8 or 3.9, as those versions have already been verified. At the moment, I'm unsure why this error is occurring.

@johnlockejrr
Copy link
Author

johnlockejrr commented Oct 18, 2024

I just changed in tran.py lr to learning_rate, let's see if it goes.

Worked more, kind of, until:

 Total params: 38,211,210 (145.76 MB)
 Trainable params: 38,154,054 (145.55 MB)
 Non-trainable params: 57,156 (223.27 KB)
  0%|                                                                                                                                                                                                                                                             | 0/4 [00:00<?, ?it/s]
ERROR - train - Failed after 0:01:09!
Traceback (most recent calls WITHOUT Sacred internals):
  File "/home/incognito/eynollah/sbb_pixelwise_segmentation/train.py", line 307, in run
    model.fit_generator(
    ^^^^^^^^^^^^^^^^^^^
AttributeError: 'Functional' object has no attribute 'fit_generator'

I'll try on Python 3.8 or 3.9, but I think the problem is I installed latest tensorflow because the one in requirements won't work with GPU under my CUDA 12.7

UPDATE:
Some guy says:

the fit_generator attribute has been deprecated. all you have to do to fix your code is to literally remove "_generator" from the first line. the fit attribute itself can work with generators.

I'll try that also 💯

@johnlockejrr
Copy link
Author

It started to train now, fingers crossed :)

 Total params: 38,211,210 (145.76 MB)
 Trainable params: 38,154,054 (145.55 MB)
 Non-trainable params: 57,156 (223.27 KB)
  0%|                                                                                                                                                                                                                                                             | 0/4 [00:00<?, ?it/s]WARNING: All log messages before absl::InitializeLog() is called are written to STDERR
I0000 00:00:1729265498.347709   63495 service.cc:152] XLA service 0x7f46400040c0 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices:
I0000 00:00:1729265498.347753   63495 service.cc:160]   StreamExecutor device (0): NVIDIA GeForce RTX 3060, Compute Capability 8.6
I0000 00:00:1729265501.452696   63495 cuda_dnn.cc:529] Loaded cuDNN version 90500
I0000 00:00:1729265521.190219   63495 device_compiler.h:188] Compiled cluster using XLA!  This line is logged at most once for the lifetime of the process.
  123/16587 ━━━━━━━━━━━━━━━━━━━━ 20:56 76ms/step - accuracy: 0.7332 - loss: 0.5795

@vahidrezanezhad
Copy link
Member

Glad to hear that!

@johnlockejrr
Copy link
Author

Still running after 16 minutes.

@vahidrezanezhad
Copy link
Member

For reference, I'm using CUDA 11.8 with cuDNN 8.6. I’ve also tested this configuration with Python 3.11, and it works without requiring any modifications to the script.

@johnlockejrr
Copy link
Author

One more question if I'm not bothering you too much. The wiki is a little complex, I hope I understood. From my alto-xml I generated with the script the dataset using a json config (I attach it below), I want to train for textline segmentation. Can you assure my wht I did is right? I have in this dataset only 4 classes: paragraph, heading, marginalia and endnote.

train_sample.zip

@johnlockejrr
Copy link
Author

johnlockejrr commented Oct 18, 2024

UPDATE:
Just crashed when trying to save the model, should I add .h5 extension? Or as I found somewhere: How to fix it: use model.export(filepath) instead of model.save(filepath)

 Total params: 38,211,210 (145.76 MB)
 Trainable params: 38,154,054 (145.55 MB)
 Non-trainable params: 57,156 (223.27 KB)
  0%|                                                                                                                                                                                                                                                             | 0/4 [00:00<?, ?it/s]WARNING: All log messages before absl::InitializeLog() is called are written to STDERR
I0000 00:00:1729265498.347709   63495 service.cc:152] XLA service 0x7f46400040c0 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices:
I0000 00:00:1729265498.347753   63495 service.cc:160]   StreamExecutor device (0): NVIDIA GeForce RTX 3060, Compute Capability 8.6
I0000 00:00:1729265501.452696   63495 cuda_dnn.cc:529] Loaded cuDNN version 90500
I0000 00:00:1729265521.190219   63495 device_compiler.h:188] Compiled cluster using XLA!  This line is logged at most once for the lifetime of the process.
16587/16587 ━━━━━━━━━━━━━━━━━━━━ 1252s 73ms/step - accuracy: 0.9105 - loss: 0.3131 - val_accuracy: 0.9504 - val_loss: 0.1493
  0%|                                                                                                                                                                                                                                                             | 0/4 [20:52<?, ?it/s]
ERROR - train - Failed after 0:22:05!
Traceback (most recent calls WITHOUT Sacred internals):
  File "/home/incognito/eynollah/sbb_pixelwise_segmentation/train.py", line 314, in run
    model.save(os.path.join(dir_output,'model_'+str(i)))
  File "/home/incognito/eynollah/sbb_pixelwise_segmentation/sbb_pixelwise_segmentation-py3.11/lib/python3.11/site-packages/keras/src/utils/traceback_utils.py", line 122, in error_handler
    raise e.with_traceback(filtered_tb) from None
  File "/home/incognito/eynollah/sbb_pixelwise_segmentation/sbb_pixelwise_segmentation-py3.11/lib/python3.11/site-packages/keras/src/saving/saving_api.py", line 114, in save_model
    raise ValueError(
ValueError: Invalid filepath extension for saving. Please add either a `.keras` extension for the native Keras format (recommended) or a `.h5` extension. Use `model.export(filepath)` if you want to export a SavedModel for use with TFLite/TFServing/etc. Received: filepath=/home/incognito/eynollah/sbb_pixelwise_segmentation/output/model_0.

@vahidrezanezhad
Copy link
Member

No, you don't need to add the .h5 extension to the models. I save them in the SavedModel format instead. Please follow this recommendation " Use model.export(filepath) if you want to export a SavedModel for use with TFLite/TFServing/etc."

@johnlockejrr
Copy link
Author

johnlockejrr commented Oct 18, 2024

Yes, I thought so. You use keras v2 and my latest tensorflow has keras v3 that's the difference.
Did you had the chance or cared to look at my zip example if I'm on the right path? :)
Thank you so much for your support!

@vahidrezanezhad
Copy link
Member

So about the config files for training label preparation. The config_dataset.json should be as follows:
config_dataset_johnlockejrr.json

@vahidrezanezhad
Copy link
Member

So about the config files for training label preparation. The config_dataset.json should be as follows: config_dataset_johnlockejrr.json

and training config file will be like this:
config_params_train.json

@vahidrezanezhad
Copy link
Member

With artificial class you have 3 classes. Background with label 0, textline 1 and artifical class 2 as assigned in config file.

@johnlockejrr
Copy link
Author

Perfect! Thank you so much! Keep up the good work. Just yesterday I found your project because I want something good for segmentation so I can forward to my text detection PyLaia models, tried YOLOv8 and I was 98% satisfied and kept trying.

@vahidrezanezhad
Copy link
Member

The rgb colored label is like this :
newtestamentinsy0000unse_g7r6_0339

@vahidrezanezhad
Copy link
Member

and the label you need for training will be this one.
newtestamentinsy0000unse_g7r6_0339

@johnlockejrr
Copy link
Author

So with the json dataset config, the script generate_gt_for_training.py will generate for me the dataset from the alto-xml files as rgb colored labels and labels for training or I have to do something mode before running the training?

@vahidrezanezhad
Copy link
Member

So with the json dataset config, the script generate_gt_for_training.py will generate for me the dataset from the alto-xml files as rgb colored labels and labels for training or I have to do something mode before running the training?

For training you have to pass -to (type output) as 2d to generate labels for training.

@johnlockejrr
Copy link
Author

Yes I did it like this:

python generate_gt_for_training.py pagexml2label -dx xml_files -do out_labels -cfg dataset_conf.json -to 2d

@vahidrezanezhad
Copy link
Member

For your information, we've trained a new model for textline detection that performs well on your document images. We'll be publishing it soon. In the meantime, I'll upload the results here.
Screenshot from 2024-10-18 18-24-50

@vahidrezanezhad
Copy link
Member

Screenshot from 2024-10-18 18-25-34

@johnlockejrr
Copy link
Author

johnlockejrr commented Oct 18, 2024

To use your model in combination with PyLaia I would need polygon coordinates of the detected lines (bounding boxes also would do but in this special case of Syriac with upper and lower vowels won't do; I need to train also for Arabic and Persian with vowels that won't fit well in a bbox), the inferencewill outputpolygons`? Is there a way to sort them or have them in order like a book is read?

@johnlockejrr
Copy link
Author

For your information, we've trained a new model for textline detection that performs well on your document images. We'll be publishing it soon. In the meantime, I'll upload the results here. Screenshot from 2024-10-18 18-24-50

Wow! Perfectly well!

@johnlockejrr
Copy link
Author

What UI interface you use in the image above to interact with the model?

@johnlockejrr
Copy link
Author

johnlockejrr commented Oct 18, 2024

UPDATE:
Train finished, seems it works with my hacks.

INFO - train - Completed after 1:23:08

But while trying inference:

(sbb_pixelwise_segmentation-py3.11) incognito@DESKTOP-H1BS9PO:~/eynollah/sbb_pixelwise_segmentation$ python inference.py -m output/model_3/ -i newtestamentinsy0000unse_g7r6_0013.jpg -s out.jpg
2024-10-18 20:42:09.783662: I tensorflow/core/util/port.cc:153] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
2024-10-18 20:42:09.794621: E external/local_xla/xla/stream_executor/cuda/cuda_fft.cc:467] Unable to register cuFFT factory: Attempting to register factory for plugin cuFFT when one has already been registered
WARNING: All log messages before absl::InitializeLog() is called are written to STDERR
E0000 00:00:1729273329.807413   65984 cuda_dnn.cc:8498] Unable to register cuDNN factory: Attempting to register factory for plugin cuDNN when one has already been registered
E0000 00:00:1729273329.811519   65984 cuda_blas.cc:1410] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered
2024-10-18 20:42:09.822880: I tensorflow/core/platform/cpu_feature_guard.cc:210] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 AVX512F AVX512_VNNI FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
I0000 00:00:1729273331.457233   65984 gpu_device.cc:2020] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 9711 MB memory:  -> device: 0, name: NVIDIA GeForce RTX 3060, pci bus id: 0000:04:00.0, compute capability: 8.6
Traceback (most recent call last):
  File "/home/incognito/eynollah/sbb_pixelwise_segmentation/sbb_pixelwise_segmentation-py3.11/lib/python3.11/site-packages/keras/src/saving/saving_lib.py", line 389, in _load_model_from_dir
    raise ValueError(
ValueError: Expected a model.weights.h5 or model.weights.npz file.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/incognito/eynollah/sbb_pixelwise_segmentation/inference.py", line 639, in <module>
    main()
  File "/home/incognito/eynollah/sbb_pixelwise_segmentation/sbb_pixelwise_segmentation-py3.11/lib/python3.11/site-packages/click/core.py", line 1157, in __call__
    return self.main(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/incognito/eynollah/sbb_pixelwise_segmentation/sbb_pixelwise_segmentation-py3.11/lib/python3.11/site-packages/click/core.py", line 1078, in main
    rv = self.invoke(ctx)
         ^^^^^^^^^^^^^^^^
  File "/home/incognito/eynollah/sbb_pixelwise_segmentation/sbb_pixelwise_segmentation-py3.11/lib/python3.11/site-packages/click/core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/incognito/eynollah/sbb_pixelwise_segmentation/sbb_pixelwise_segmentation-py3.11/lib/python3.11/site-packages/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/incognito/eynollah/sbb_pixelwise_segmentation/inference.py", line 636, in main
    x.run()
  File "/home/incognito/eynollah/sbb_pixelwise_segmentation/inference.py", line 559, in run
    res=self.predict()
        ^^^^^^^^^^^^^^
  File "/home/incognito/eynollah/sbb_pixelwise_segmentation/inference.py", line 227, in predict
    self.start_new_session_and_model()
  File "/home/incognito/eynollah/sbb_pixelwise_segmentation/inference.py", line 163, in start_new_session_and_model
    self.model = load_model(self.model_dir , compile=False,custom_objects = {"PatchEncoder": PatchEncoder, "Patches": Patches})
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/incognito/eynollah/sbb_pixelwise_segmentation/sbb_pixelwise_segmentation-py3.11/lib/python3.11/site-packages/keras/src/saving/saving_api.py", line 189, in load_model
    return saving_lib.load_model(
           ^^^^^^^^^^^^^^^^^^^^^^
  File "/home/incognito/eynollah/sbb_pixelwise_segmentation/sbb_pixelwise_segmentation-py3.11/lib/python3.11/site-packages/keras/src/saving/saving_lib.py", line 357, in load_model
    return _load_model_from_dir(
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/incognito/eynollah/sbb_pixelwise_segmentation/sbb_pixelwise_segmentation-py3.11/lib/python3.11/site-packages/keras/src/saving/saving_lib.py", line 413, in _load_model_from_dir
    weights_store.close()
    ^^^^^^^^^^^^^
UnboundLocalError: cannot access local variable 'weights_store' where it is not associated with a value

@johnlockejrr
Copy link
Author

johnlockejrr commented Oct 18, 2024

Tried to interfer with gradio also, another error:

Traceback (most recent call last):
  File "/home/incognito/eynollah-demo/eynollah-py3.10/lib/python3.10/site-packages/gradio/queueing.py", line 622, in process_events
    response = await route_utils.call_process_api(
  File "/home/incognito/eynollah-demo/eynollah-py3.10/lib/python3.10/site-packages/gradio/route_utils.py", line 323, in call_process_api
    output = await app.get_blocks().process_api(
  File "/home/incognito/eynollah-demo/eynollah-py3.10/lib/python3.10/site-packages/gradio/blocks.py", line 2014, in process_api
    result = await self.call_function(
  File "/home/incognito/eynollah-demo/eynollah-py3.10/lib/python3.10/site-packages/gradio/blocks.py", line 1567, in call_function
    prediction = await anyio.to_thread.run_sync(  # type: ignore
  File "/home/incognito/eynollah-demo/eynollah-py3.10/lib/python3.10/site-packages/anyio/to_thread.py", line 56, in run_sync
    return await get_async_backend().run_sync_in_worker_thread(
  File "/home/incognito/eynollah-demo/eynollah-py3.10/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 2441, in run_sync_in_worker_thread
    return await future
  File "/home/incognito/eynollah-demo/eynollah-py3.10/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 943, in run
    result = context.run(func, *args)
  File "/home/incognito/eynollah-demo/eynollah-py3.10/lib/python3.10/site-packages/gradio/utils.py", line 846, in wrapper
    response = f(*args, **kwargs)
  File "/home/incognito/eynollah-demo/app.py", line 240, in do_prediction
    img_height_model=model.layers[len(model.layers)-1].output_shape[1]
AttributeError: '_UserObject' object has no attribute 'layers'

@vahidrezanezhad
Copy link
Member

I think I'll just wait for you to relsease that seg model above you mentioned, I'm out of ideas for now.

Even after training your model, if the text lines still aren’t well isolated, you can try predicting the artificial class with thresholding to make the model more aggressive in this aspect.

@johnlockejrr
Copy link
Author

johnlockejrr commented Oct 25, 2024

The inference result is pretty good but some lines (less now) are still stuck together (model_textline_ens_5_6_7_9_11_nopatch_john.zip)

contours

Even so, looks way better than same dataset trained with YOLOv8:

newtestamentinsy0000unse_g7r6_0242

@johnlockejrr
Copy link
Author

johnlockejrr commented Oct 25, 2024

I think I'll just wait for you to relsease that seg model above you mentioned, I'm out of ideas for now.

Even after training your model, if the text lines still aren’t well isolated, you can try predicting the artificial class with thresholding to make the model more aggressive in this aspect.

I will try to train the model for about 40 epochs and see if improves.

And yes, many lines in my data are overlapping, and this is nothing compared to the same script in manuscripts :)

image

@johnlockejrr
Copy link
Author

Only two lines stuck together in my model trained by you... very good, I think a little more trainig or/and more data would do the trick.

image

@vahidrezanezhad
Copy link
Member

Only two lines stuck together in my model trained by you... very good, I think a little more trainig or/and more data would do the trick.

image

Before training new model just use this thresholding for artificial class and see how will be the result.

seg_art = label_p_pred[0, :,:,2]

seg_art[:,:][seg_art[:,:]<0.2] = 0
seg_art[:,:][seg_art[:,:]>0] = 1

seg = np.argmax(label_p_pred, axis=3)[0]

seg[:,:][seg_art[:,:]==1] = 2

@johnlockejrr
Copy link
Author

In the inference.py script? Or in eynollah?

@vahidrezanezhad
Copy link
Member

In the inference.py script? Or in eynollah?

First check it with inference.py

@johnlockejrr
Copy link
Author

I think here:

                        if thresholding_for_artificial_class_in_light_version:
                            seg_art = label_p_pred[:,:,:,2]

                            seg_art[seg_art<0.2] = 0
                            seg_art[seg_art>0] =1

                            seg[seg_art==1]=2

@johnlockejrr
Copy link
Author

Still stuck with the above thresholding.

image

@johnlockejrr
Copy link
Author

johnlockejrr commented Oct 25, 2024

johnlockejrr

I took a closer look at your ground truth and noticed nearby text lines are overlapping. To address this, I generated labels with a thicker artificial class and applied early erosion on textlines, then trained a model using these adjustments. The results seem improved, and I've shared the trained model here. Also, I'll share our new text-line model with you so you can use it as well.
textlinemodel_with_your_own_data
our_textline_model

To correct my comment. I have only applied an early erosion on textlines. gt_gen_utils.py I have changed this

erosion_rate = 0#1
dilation_rate = 2
dilation_early = 0
erosion_early = 1

Like this?

            if "artificial_class_label" in keys:
                img_boundary = np.zeros((y_len, x_len))
                erosion_rate = 0#1
                dilation_rate = 2
                dilation_early = 0
                erosion_early = 1
                co_use_case, img_boundary = update_region_contours(co_use_case, img_boundary, erosion_rate, dilation_rate, y_len, x_len, dilation_early=dilation_early, erosion_early=erosion_early )

@vahidrezanezhad
Copy link
Member

Still stuck with the above thresholding.

image

Hmm. You can start using our model—it should be compatible with your data. Additionally, in my free time, I'll work on your data to help resolve issues that may be causing it to get stuck.

@vahidrezanezhad
Copy link
Member

johnlockejrr

I took a closer look at your ground truth and noticed nearby text lines are overlapping. To address this, I generated labels with a thicker artificial class and applied early erosion on textlines, then trained a model using these adjustments. The results seem improved, and I've shared the trained model here. Also, I'll share our new text-line model with you so you can use it as well.
textlinemodel_with_your_own_data
our_textline_model

To correct my comment. I have only applied an early erosion on textlines. gt_gen_utils.py I have changed this

erosion_rate = 0#1
dilation_rate = 2
dilation_early = 0
erosion_early = 1

Like this?

            if "artificial_class_label" in keys:
                img_boundary = np.zeros((y_len, x_len))
                erosion_rate = 0#1
                dilation_rate = 2
                dilation_early = 0
                erosion_early = 1
                co_use_case, img_boundary = update_region_contours(co_use_case, img_boundary, erosion_rate, dilation_rate, y_len, x_len, dilation_early=dilation_early, erosion_early=erosion_early )

Yes, you can apply an early erosion of 2. This means the text lines will first undergo erosion.

@johnlockejrr
Copy link
Author

johnlockejrr commented Oct 25, 2024

hmm. should I change the update_region_contours function too?

(sbb_pixelwise_segmentation-py3.11) incognito@DESKTOP-H1BS9PO:~/eynollah/sbb_pixelwise_segmentation$ python generate_gt_for_training.py pagexml2label -dx syr_xml -do syr_xml_dataset -cfg config_dataset_syriacnt.json -to 2d
  0%|                                                                                                                                                                                                                                                            | 0/43 [00:00<?, ?it/s]
Traceback (most recent call last):
  File "/home/incognito/eynollah/sbb_pixelwise_segmentation/generate_gt_for_training.py", line 221, in <module>
    main()
  File "/home/incognito/eynollah/sbb_pixelwise_segmentation/sbb_pixelwise_segmentation-py3.11/lib/python3.11/site-packages/click/core.py", line 1157, in __call__
    return self.main(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/incognito/eynollah/sbb_pixelwise_segmentation/sbb_pixelwise_segmentation-py3.11/lib/python3.11/site-packages/click/core.py", line 1078, in main
    rv = self.invoke(ctx)
         ^^^^^^^^^^^^^^^^
  File "/home/incognito/eynollah/sbb_pixelwise_segmentation/sbb_pixelwise_segmentation-py3.11/lib/python3.11/site-packages/click/core.py", line 1688, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/incognito/eynollah/sbb_pixelwise_segmentation/sbb_pixelwise_segmentation-py3.11/lib/python3.11/site-packages/click/core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/incognito/eynollah/sbb_pixelwise_segmentation/sbb_pixelwise_segmentation-py3.11/lib/python3.11/site-packages/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/incognito/eynollah/sbb_pixelwise_segmentation/generate_gt_for_training.py", line 63, in pagexml2label
    get_images_of_ground_truth(gt_list,dir_xml,dir_out,type_output, config, config_params, printspace, dir_images, dir_out_images)
  File "/home/incognito/eynollah/sbb_pixelwise_segmentation/gt_gen_utils.py", line 276, in get_images_of_ground_truth
    co_use_case, img_boundary = update_region_contours(co_use_case, img_boundary, erosion_rate, dilation_rate, y_len, x_len, dilation_early=dilation_early, erosion_early=erosion_early )
                                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: update_region_contours() got an unexpected keyword argument 'erosion_early'

@vahidrezanezhad
Copy link
Member

erosion_early

No, you shouldn't change it. How did this happen? You have generated the label with only different parameters before, and now you've just changed the early erosion?!

@johnlockejrr
Copy link
Author

johnlockejrr commented Oct 25, 2024

I wanted to change the gt_gen_utils.py as you said you did, to retrain the model for 40 epochs. Otherwise why shoud I set erosion_early = 1 in gt_gen_utils.py because is never used.

To correct my comment. I have only applied an early erosion on textlines. [gt_gen_utils.py](https://github.com/qurator-spk/sbb_pixelwise_segmentation/blob/unifying-training-models/gt_gen_utils.py) I have changed this

erosion_rate = 0#1
dilation_rate = 2
dilation_early = 0
erosion_early = 1

@johnlockejrr
Copy link
Author

johnlockejrr

I took a closer look at your ground truth and noticed nearby text lines are overlapping. To address this, I generated labels with a thicker artificial class and applied early erosion on textlines, then trained a model using these adjustments. The results seem improved, and I've shared the trained model here. Also, I'll share our new text-line model with you so you can use it as well.
textlinemodel_with_your_own_data
our_textline_model

To correct my comment. I have only applied an early erosion on textlines. gt_gen_utils.py I have changed this

erosion_rate = 0#1
dilation_rate = 2
dilation_early = 0
erosion_early = 1

Here :)

@vahidrezanezhad
Copy link
Member

johnlockejrr

I took a closer look at your ground truth and noticed nearby text lines are overlapping. To address this, I generated labels with a thicker artificial class and applied early erosion on textlines, then trained a model using these adjustments. The results seem improved, and I've shared the trained model here. Also, I'll share our new text-line model with you so you can use it as well.
textlinemodel_with_your_own_data
our_textline_model

To correct my comment. I have only applied an early erosion on textlines. gt_gen_utils.py I have changed this

erosion_rate = 0#1
dilation_rate = 2
dilation_early = 0
erosion_early = 1

Here :)

I pushed a commit check it out

@johnlockejrr
Copy link
Author

You did exactly what I did locally 👯

@johnlockejrr
Copy link
Author

You trained the model with my dataset with "patches" : false or "patches" : true?

@vahidrezanezhad
Copy link
Member

vahidrezanezhad commented Oct 25, 2024

You trained the model with my dataset with "patches" : false or "patches" : true?

patches set to false. Please test this one

@johnlockejrr
Copy link
Author

You trained the model with my dataset with "patches" : false or "patches" : true?

patches set to false. Please test this one

Exactly the same issue:

image

@vahidrezanezhad
Copy link
Member

You trained the model with my dataset with "patches" : false or "patches" : true?

patches set to false. Please test this one

Exactly the same issue:

image

Could you share this image along with a few additional test images?

@johnlockejrr
Copy link
Author

johnlockejrr commented Oct 25, 2024

Yes! One sec.

samples.zip

@johnlockejrr
Copy link
Author

Unrelated question: you think we could segment something like this or needs pre-processing? Dewarping etc... I had trouble with any segmentation method with this kind of data.

IMG_1927

@vahidrezanezhad
Copy link
Member

You trained the model with my dataset with "patches" : false or "patches" : true?

patches set to false. Please test this one

Exactly the same issue:

image

Check out eynollah machine based reading order branch. With the latest model, I was able to isolate all text lines in your sample data. The latest model GT preparation was with these parameters

                erosion_rate = 0#1
                dilation_rate = 5
                dilation_early = 0
                erosion_early = 1

@vahidrezanezhad
Copy link
Member

Unrelated question: you think we could segment something like this or needs pre-processing? Dewarping etc... I had trouble with any segmentation method with this kind of data.

IMG_1927

Of course it is possible

layout:
john_layout

textline:

john_textline_overlayed

john_textline_only_textline

@vahidrezanezhad
Copy link
Member

@johnlockejrr
Copy link
Author

Perfect!

@johnlockejrr
Copy link
Author

You trained the model with my dataset with "patches" : false or "patches" : true?

patches set to false. Please test this one

Exactly the same issue:
image

Check out eynollah machine based reading order branch. With the latest model, I was able to isolate all text lines in your sample data. The latest model GT preparation was with these parameters

                erosion_rate = 0#1
                dilation_rate = 5
                dilation_early = 0
                erosion_early = 1

This branch did the trick 🗡️

image

@johnlockejrr
Copy link
Author

johnlockejrr commented Oct 25, 2024

I've retrained this model for 30 epochs, I'll try right now with that branch of eynollah.

No lines stuck on my model also. But something new: some lines are now split in two.

image

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants