[fix] skip empty bins when calculating cnt_in_bin in BinMapper::FindBin (fix #4301)

[shiyu1994]

As described in #4301 (comment), this is to fix #4301 by skipping the empty bins when calculating cnt_in_bin in BinMapper::FindBin. So that we can correctly decides the most_freq_bin_.

[jameslamb]

@shiyu1994 sorry for the inconvenience, could you please merge master into this? That should get you the fix from #4326, which I think should avoid timeout issues from QEMU builds.

[jameslamb]

thank you very much!

I built this branch locally and ran a modified version of the reproducible example from #4301 (comment) 10 times tonight.

import zipfile
from io import BytesIO

import lightgbm
import pandas as pd
import numpy as np
import requests

from sklearn.metrics import mean_absolute_error

data_url = "https://github.com/microsoft/LightGBM/files/6508547/weird.zip"

zipdata = BytesIO()
zipdata.write(requests.get(data_url, headers={"Accept": "application/octet-stream"}).content)
zip_contents = zipfile.ZipFile(zipdata)
data_file = zip_contents.open("weird.pkl")

test = pd.read_pickle(data_file)

bst = lightgbm.LGBMRegressor(verbose=1, num_iterations=5000).fit(test.drop(columns=["y"]), test["y"])

mae = mean_absolute_error(bst.predict(test.drop(columns=["y"])), test["y"])

print(f"target mean: {np.mean(test['y'])}")
print(f"MAE: {mae}")

Training succeeded every time, and increasing num_iterations led to a better fit to the training data. Given that and the fact that all CI jobs are passing, I'm confident in this fix.

[github-actions]

This pull request has been automatically locked since there has not been any recent activity since it was closed. To start a new related discussion, open a new issue at https://github.com/microsoft/LightGBM/issues including a reference to this.