Skip to content

Commit

Permalink
Made small changes to repo README and sr-eval/README, added proper pr…
Browse files Browse the repository at this point in the history
…eds directory.
  • Loading branch information
jyoune committed Jul 1, 2024
1 parent c1b0dea commit 253d4e3
Show file tree
Hide file tree
Showing 3 changed files with 62,831 additions and 2 deletions.
1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ Each subdirectory of the repository is an evaluation task within the project. Ea
* golds - The gold-truth standard, humanly-annotated files by which apps are evaluated for predictive ability.
* often are `.tsv` or `.csv` or `.txt`.
* preds/predictions - The app-predicted files with predicted-annotations of what phenomena are to be evaluated. (e.g. time durations for slate detection.)
* each preds directory represents a batch, with naming conventions as follows:`preds@<APP_NAME><APP_VER>@<BATCH_NAME>`
* are always `.mmif` files with app views.
#### Outputs to Evaluations
* results - This should be the result system output of the evaluation numbers from a finished evaluation.
Expand Down
4 changes: 2 additions & 2 deletions sr-eval/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,8 +7,8 @@ To run this evaluation script, you need the following:

* Set of predictions in MMIF format (either from the preds folder in this repo
or generated from the [SWT app](https://github.com/clamsproject/app-swt-detection) )
* Set of golds in csv format (either the golds directory in this repo or downloaded from the annotations repository
using goldretriever.py, or your own set that exactly matches the format of the included golds.)
* Set of golds in csv format (either downloaded from the annotations repository
using goldretriever.py, or your own set that exactly matches the format present in [aapb-annotations](https://github.com/clamsproject/aapb-annotations/tree/main/scene-recognition/golds))

There are three arguments when running the script: `-mmif-dir`, `-gold-dir`, and `count-subtypes`.
The first two are directories that contain the predictions and golds, respectively. The third is a boolean value that
Expand Down
Loading

0 comments on commit 253d4e3

Please sign in to comment.