Confident Predictions Selection

[alinapark]

💸 Confident Predictions Selection is a bounty challenge 💸

Your task

To design a model capable of selecting the top 10% of predictions that will exhibit the smallest mean distance.
You have an option to work with either or both of the text data and raw predictions. Additionally, you may perform any form of aggregation or transformation on the raw predictions as you see fit.

Validation Metrics

The primary metric for this challenge is the mean distance of the selected top 10% predictions. Your objective is to minimize this value.
As a secondary metric, we designed the Class Representation Index (CRI). In essence, CRI compares the class distribution before and after filtering, giving a higher weight to classes that were initially larger. The primary purpose of this metric is to detect cases where a class is significantly less represented after filtering compared to its original size.

Please, see the dedicated repository for instructions

[Annyvichi]

Do you consider having a better solution for this issue and do i have some time to work on it?

[Annyvichi]

Do you consider having a better solution for this issue and do i have some time to work on it?

I'm afraid you can ruin my work by closing the issue before i get a good score, would you please keep it open?

[Annyvichi]

@evgenydmitriev I need to ask you about this issue. Did anyone exceed the winner's score significantly or is it impossible?

I liked this task very much and I'd like to work on other tasks like this.
Very curious about this position: Junior NLP Developer - Data Scraping, can I take it?

[evgenydmitriev]

@Annyvichi, I closed the issue because I saw no activity from the challenge organizers for almost a year, but if you email us at [email protected], I can forward your resume/profile to a few NLP teams.

[Annyvichi]

@evgenydmitriev, I will take some time to improve my profile and then will apply at the email and ask about open positions in your team.