Skip to content

Redesign contest estimator entry points #1741

Open
@Domonion

Description

@Domonion

Currently there 3 differents ways to start Contest:

  1. UtBotTool2 - as separate process for SBFT, calls ContestKt.main
  2. ContestEstimatorKt.main - calls runEstimator
  3. StatisticsMonitoringKt.main - calls runEstimator

All of these tools set options for Contest Estimator differently, which causes divergence in Grafana monitoring, local testing and result in SBFT. it is hard to repro bug occured in nightly statistics on local machine as settings differ and Grafana shows incorrect results.

Entry points should be reduced to single one, which will be used for all scenarios with same arguments.

Proposed solution:

  1. Merge ContestEstimator and StatisticsMonitoring in a single class, which will be used as an entry point both at CI and CLI.
  2. New entry point should have clean and simple way to configurate execution. Default arguments should correspond to statistics monitoring scenario and should work both from CI and CLI(for local repro).
  • Nice to have, but not mandatory: New entry point should use UtBotTool2 for generation. Monitoring will work as it is used in SBFT.

This would allow run this utbot-junit-contest in a single way as from CLI and Gradle.

Additional:
It is important either not to break current monitoring scripts or completely rewrite them.

Metadata

Metadata

Assignees

Labels

comp-contest-estimatorIssue is related to contest estimatorctg-enhancementNew feature, improvement or change request

Type

No type

Projects

Status

Todo

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions