Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Does lightgbm_predict4j suit for the model generated by lightgbm v2.0.5 or higher? #3

Open
dragonclouder opened this issue Jan 18, 2018 · 16 comments

Comments

@dragonclouder
Copy link

Noticed here exists a demo suit for v2, and i can run the UseageTest in v2 package successfully, while when i use a model generared by lightgbm v2.0.5, it produced a error: "[Tree] Tree model string format error, should contain default_value field". The default_value is contained in models generated by lightgbm's earlier version. The model file cause the error is attached below.
Thanks for any kind help in advance!

lgbm_test.txt

@dragonclouder
Copy link
Author

For there are some friends meeting the same problem as list above email me to ask whether i solved it, i share the trick i used which is possibly not right but a compromise method: "I initialized the private double[] defaultValue; in the org.lightgbm.predict4j.v2.Tree with 0s , or you can substitute it with other values reasonable". Welcome masters to share perfect manners.

@benman1
Copy link

benman1 commented Aug 8, 2018

Please see my PR (or fork) for support of v2.

@dragonclouder
Copy link
Author

#4
yeah, thanks a lot for your positive contribution, it seems like that the default values are set to zeros as what i tried above. Is that reasonable or explainable from you point?

@benman1
Copy link

benman1 commented Aug 10, 2018

What you set as default_value, i.e. what the value should take if it's missing, should be up to the modeller really - I haven't looked into the lightgbm implementation in any detail to see how missing values are handled. It seems that default_value is entirely deprecated in the newer lightgbm releases (and therefore it's also not in the model file when saved from lightgbm). lightgbm_predict4j is behind in that it still expects it.

The way, I've included it, is that you can still include default_value in the model if you want to set it. It's set to 0.0 if not set otherwise.

It might be worthwhile to make sure lightgbm_predict4j (particularly v2) reflects the latest version of the lightgbm format.

@mrchor
Copy link

mrchor commented Apr 23, 2019

@benman1 Hello! Why feed different features to the model and the predicted results are the same?

@benman1
Copy link

benman1 commented Apr 23, 2019

Please explain clearly what you mean, maybe give an example, show your code, etc; otherwise, I am afraid it's not possible to answer.

@mrchor
Copy link

mrchor commented Apr 25, 2019

@benman1 What does default_value mean?
17:46:12.531 [main] DEBUG c.j.m.lightgbm.predictor.V2.Tree - Assuming default values as 0.0
17:46:12.532 [main] DEBUG c.j.m.lightgbm.predictor.V2.Tree - Assuming default values as 0.0
17:46:12.533 [main] DEBUG c.j.m.lightgbm.predictor.V2.Tree - Assuming default values as 0.0
17:46:12.533 [main] DEBUG c.j.m.lightgbm.predictor.V2.Tree - Assuming default values as 0.0
17:46:12.534 [main] DEBUG c.j.m.lightgbm.predictor.V2.Tree - Assuming default values as 0.0
17:46:12.535 [main] DEBUG c.j.m.lightgbm.predictor.V2.Tree - Assuming default values as 0.0
17:46:12.535 [main] DEBUG c.j.m.lightgbm.predictor.V2.Tree - Assuming default values as 0.0
17:46:12.536 [main] DEBUG c.j.m.lightgbm.predictor.V2.Tree - Assuming default values as 0.0
17:46:12.536 [main] DEBUG c.j.m.lightgbm.predictor.V2.Tree - Assuming default values as 0.0
17:46:12.536 [main] DEBUG c.j.m.lightgbm.predictor.V2.Tree - Assuming default values as 0.0
17:46:12.537 [main] DEBUG c.j.m.lightgbm.predictor.V2.Tree - Assuming default values as 0.0
17:46:12.537 [main] DEBUG c.j.m.lightgbm.predictor.V2.Tree - Assuming default values as 0.0
17:46:12.537 [main] DEBUG c.j.m.lightgbm.predictor.V2.Tree - Assuming default values as 0.0
17:46:12.538 [main] DEBUG c.j.m.lightgbm.predictor.V2.Tree - Assuming default values as 0.0
17:46:12.539 [main] DEBUG c.j.m.lightgbm.predictor.V2.Tree - Assuming default values as 0.0
17:46:12.539 [main] DEBUG c.j.m.lightgbm.predictor.V2.Tree - Assuming default values as 0.0
17:46:12.540 [main] DEBUG c.j.m.lightgbm.predictor.V2.Tree - Assuming default values as 0.0
17:46:12.541 [main] DEBUG c.j.m.lightgbm.predictor.V2.Tree - Assuming default values as 0.0
I suspect this is the problem.

@mrchor
Copy link

mrchor commented Apr 26, 2019

@benman1 Hello! In addition, I found that this prediction code has scoring errors when making binary prediction, for example, the label-1 sample probability score will be less than that labele-0 sample.
image
Excuse me, is this problem related to default_value?
Looking forward to your reply~

@benman1
Copy link

benman1 commented Apr 26, 2019 via email

@mrchor
Copy link

mrchor commented Apr 28, 2019

@benman1 I'm really sorry for not explaining my problem clearly. At present, I have found the problem, from my code. Thank you very much for your recent help. Thank you very much.~

@benman1
Copy link

benman1 commented Apr 28, 2019 via email

@KoreHuang
Copy link

兄弟,这个问题有结论了吗?lightgbm_predict4j到底能否适用于lightgbm v2.0.5更高版本的模型?

@kohry
Copy link

kohry commented May 5, 2020

I was having trouble with lgbm model 2.3.2 (2020 May) and had defaultvalue null array exception.
changing all defaultValue[node] to 0 helped me.

anyways, thank you for the issue raising here!

( LGBM is so huge, and I don't understand why people don't have much interest on developing java version. All of the people are talking about AI stuff and writing a book, but never take themselves into deep level of a production. phew..)

@buaanie
Copy link

buaanie commented Aug 13, 2021

@benman1 I'm really sorry for not explaining my problem clearly. At present, I have found the problem, from my code. Thank you very much for your recent help. Thank you very much.~

hello, do you remember what the problem is in your code. I got the difference between lightgbm_predict4j and lightgbm_python, and i dont know what's wrong with my code.

@buaanie
Copy link

buaanie commented Aug 17, 2021

@benman1 I'm really sorry for not explaining my problem clearly. At present, I have found the problem, from my code. Thank you very much for your recent help. Thank you very much.~

hello, do you remember what the problem is in your code. I got the difference between lightgbm_predict4j and lightgbm_python, and i dont know what's wrong with my code.

i know what happened...this java implement does not support "category feature"..

@linseyicey
Copy link

@benman1 I'm really sorry for not explaining my problem clearly. At present, I have found the problem, from my code. Thank you very much for your recent help. Thank you very much.~

hello, do you remember what the problem is in your code. I got the difference between lightgbm_predict4j and lightgbm_python, and i dont know what's wrong with my code.

i know what happened...this java implement does not support "category feature"..

I got the same question as u meet when i used lightgbm 3.34 or higher. Are you fix it?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

7 participants