Commit Scoring Logic

[Shwetakale]

Currently while scoring commits bugspots is used and in bugspot each file is rated based on historical commit messages on that file and timestamp of commit message. File whose historical commit message has max number of "fix" or "close" word gets higher weightage.

Incase of aasm highest weightage is for README and also CHANGELOG has higher weightage.

    Hotspots:
        5.9860 - README.md
        2.5098 - lib/aasm/core/transition.rb
        2.5010 - lib/aasm/instance_base.rb
        1.9753 - spec/unit/transition_spec.rb
        1.9007 - lib/aasm/persistence/active_record_persistence.rb
        1.7792 - lib/aasm/base.rb
        1.6691 - spec/unit/inspection_spec.rb
        1.5949 - lib/generators/active_record/templates/migration_existing.rb
        1.5705 - CHANGELOG.md

Incase of mongoid highest weightage is to CHANGELOG

    Hotspots:
        2.9300 - CHANGELOG.md
        0.8669 - lib/mongoid/contextual/mongo.rb
        0.7303 - spec/mongoid/scopable_spec.rb
        0.7236 - lib/mongoid/scopable.rb
        0.7053 - lib/mongoid/persistable/settable.rb

• Here while calculating score bugspot_score of file is used so if person do typo fix in readme update gets higher score than actual code changes incase of aasm.
  • If person adds totally new file for some feature will get 0 marks as bugspots_scores[file.filename] will be 0
• Here total_score is divided by number_of_files in the commit so if user has done good change and if test case file has less weightage then that commit will be scored less. If person commits test cases seperately and code seperately he will get more marks than person commiting both things together. but I think this can't be handled.

Thoughts?

[gautamrege]

I agree we need some more innovation around the scoring algorithm. Ideas are welcome.

[gautamrege]

Update: we are working on some ideas here.

Interestingly, we have seen instances where people look at the scoring algorithm and try to "tweak" their code to manipulate the scoring. Though I am a proponent of keeping the scoring algorithm open, I want some ideas about how we "hide" the logic from prying eyes. Remember, we spend real money on rewarding people, so we have to be extra careful that this is not mis-used.

[prasadsurase]

We can use the same technique as secrets.yml. Move the scoring logic to a file which isn't committed. Load the file containing the scoring logic when the application starts.

[thecodersblock]

Hey, just wondering. What's the rationale behind dividing the total score with the number of files?