Skip to content

Benchmark 3.5.25 (pooled) vs 2023.03

Jump to bottom
Ihar Suvorau edited this page Jul 28, 2023 · 1 revision

Benchmarking results for v3.5.25 (pooled resourced) against the version in Spring 2023

Metric names

Short name Full name
AED Absolute Event Distance
CAR Case Arrival Rate
CED Circadian Event Distribution
RED Relative Event Distribution
CTD Cycle Time Distance
NGD N-Gram Distance (size 2)
NGD(3) N-Gram Distance (size 3)

Delta values for each metric

Negative value means improvement:

comparison

Measurements

metric distance name simod_version
AED 324.7792762277447 BPIC17 3.5.25
CAR 391.7314262472885 BPIC17 3.5.25
CED 2.226791429637599 BPIC17 3.5.25
CTD 46.08636117136659 BPIC17 3.5.25
NGD 0.3647994467496542 BPIC17 3.5.25
NGD(3) 0.5682080739362043 BPIC17 3.5.25
RED 28.554793481841717 BPIC17 3.5.25
AED 82.19826590943886 CALL 3.5.25
CAR 84.69302387979607 CALL 3.5.25
CED 18.6579540244275 CALL 3.5.25
CTD 4.77069262907739 CALL 3.5.25
NGD 0.12447353237337251 CALL 3.5.25
NGD(3) 0.1658007963198866 CALL 3.5.25
RED 3.5194685273051585 CALL 3.5.25
AED 202.33132987036868 AC_CRE 3.5.25
CAR 251.0854271356784 AC_CRE 3.5.25
CED 2.690246187622668 AC_CRE 3.5.25
CTD 84.42713567839193 AC_CRE 3.5.25
NGD 0.1635565312843029 AC_CRE 3.5.25
NGD(3) 0.21210988600261632 AC_CRE 3.5.25
RED 46.51700283302519 AC_CRE 3.5.25
AED 210.0632204971677 BPIC12 3.5.25
CAR 285.38978494623655 BPIC12 3.5.25
CED 2.682191993529388 BPIC12 3.5.25
CTD 20.232190860215056 BPIC12 3.5.25
NGD 0.6308699258063794 BPIC12 3.5.25
NGD(3) 0.7162464155047958 BPIC12 3.5.25
RED 38.1019683048362 BPIC12 3.5.25
AED 117.32 AC_CRE 2023.03
CAR 110.38 AC_CRE 2023.03
CED 3.11 AC_CRE 2023.03
RED 48.19 AC_CRE 2023.03
CTD 62.23 AC_CRE 2023.03
NGD 0.24 AC_CRE 2023.03
AED 313.3 BPIC12 2023.03
CAR 336.42 BPIC12 2023.03
CED 2.1 BPIC12 2023.03
RED 96.82 BPIC12 2023.03
CTD 93.45 BPIC12 2023.03
NGD 0.56 BPIC12 2023.03
AED 314.92 BPIC17 2023.03
CAR 390.04 BPIC17 2023.03
CED 1.65 BPIC17 2023.03
RED 132.31 BPIC17 2023.03
CTD 102.85 BPIC17 2023.03
NGD 0.37 BPIC17 2023.03
AED 61.76 CALL 2023.03
CAR 61.68 CALL 2023.03
CED 4.72 CALL 2023.03
RED 0.0 CALL 2023.03
CTD 8.18 CALL 2023.03
NGD 0.08 CALL 2023.03

Raw CSV files with measurements

Configuration

version: 4
common:
  train_log_path: <train_log_path>
  test_log_path: <test_log_path>
  num_final_evaluations: 10
  evaluation_metrics:
    - 3_gram_distance
    - 2_gram_distance
    - absolute_event_distribution
    - relative_event_distribution
    - circadian_event_distribution
    - arrival_event_distribution
    - cycle_time_distribution
  log_ids:
    case: case_id
    activity: activity
    resource: resource
    start_time: start_time
    end_time: end_time
    enabled_time: enabled_time
preprocessing:
  multitasking: false
  enable_time_concurrency_threshold: 0.5
  concurrency_df: 0.75
  concurrency_l2l: 0.9
  concurrency_l1l: 0.9
control_flow:
  optimization_metric: n_gram_distance
  num_iterations: 30
  num_evaluations_per_iteration: 5
  gateway_probabilities: discovery
  discovery_algorithm: sm3
  epsilon:
    - 0.1
    - 1.0
  eta:
    - 0.2
    - 0.6
  replace_or_joins:
    - true
    - false
  prioritize_parallelism:
    - true
    - false
resource_model:
  optimization_metric: circadian_event_distribution
  num_iterations: 40
  num_evaluations_per_iteration: 5
  resource_profiles:
    discovery_type: pool
    granularity:
      - 15
      - 60
    confidence:
      - 0.1
      - 1.0
    support:
      - 0.1
      - 1.0
    participation: 0.4
extraneous_activity_delays:
  optimization_metric: relative_event_distribution
  num_iterations: 1
Clone this wiki locally