Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

All loss values become nan #78

Open
AniketGurav opened this issue Aug 4, 2022 · 2 comments
Open

All loss values become nan #78

AniketGurav opened this issue Aug 4, 2022 · 2 comments

Comments

@AniketGurav
Copy link

Respected Sir,

Thanks for your work.
I am working on images that have only horizontal text.

During training following loss values eventually become nan after few steps.
Those loss are bbox_loss,seg_loss,angle_loss and loss
These values are present in train.py between lines 487 to 497

Can you please let me know what can be the issue and is there any
change required in code when only horizontal text is present.

--- Aniket

@MichalBusta
Copy link
Owner

Hi,
The horizontal and vertical text are supported.

  • with no additional information, I think the issue will be in data feeding. (nan values can be result of len(gt) > len(features) - for ctc loss the gt must be able to fit length of the feature vector).

you can run training with a debug option to see, what are you feeding in ...

the data format description: https://github.com/MichalBusta/E2E-MLT/tree/master/sample_train_data/MLT

  • note, that for ICDAR format, order of the coordinates matter.

Hope it helps, Michal

@AniketGurav
Copy link
Author

Thank you

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants