The predicted probabilities of some issues are 99.99%

[mcxwx123]

Currently, many open issues in some projects have a GFI probability of 99.99%, and some of these issues clearly should not be marked as GFI.

The performance metric of the model is also unusually high.

I examined the code and found two features that may be problematic. The first is 'created_at_timestamp', which is not one of the features and should not be included in X (def get_x_y() in gfibot/model/utils.py). The second one is 'rpt_gfi_ratio', when I try to drop this feature, the model performance metrics appear to drop significantly.

The problems can be solved by the following steps:

1. Add 'created_at_timestamp' to

   gfi-bot/gfibot/model/utils.py

   Line 135 in 7ed0761

   ["owner", "name", "number", "is_gfi", "created_at", "closed_at"]

2. The gfi_ratio and gfi_num features should be calculated with a new issue list, which only includes issues closed before the data collection time.

   gfi-bot/gfibot/model/dataloader.py

   Line 112 in 7ed0761

   def _get_newcomer_ratio(n_user_commits: List[int], newcomer_thres: int) -> float:

   gfi-bot/gfibot/model/dataloader.py

   Line 118 in 7ed0761

   def _get_newcomer_num(n_user_commits: List[int], newcomer_thres: int) -> int:

   
   Now we use the following list. A new issues list like issues = [i for i in user.issues if i.closed_at <= t] should be created for calculating gfi_ratio and gfi_num later.
   

   gfi-bot/gfibot/data/dataset.py

   Line 205 in 7ed0761

   issues = [i for i in user.issues if i.created_at <= t]

There may be a situation where most of the prediction probabilities are close to 0 after the above features are corrected because of the imbalance of positive and negative instances in the training data, which can be solved by balancing the training dataset using methods such as SMOTE and ADASYN. Then we can check whether the '99.99% probabilities' problem is solved.