|
| 1 | +Sinolpack documentation |
| 2 | +======================= |
| 3 | + |
| 4 | +Sinolpack is a package standard currently used in SIO2. |
| 5 | + |
| 6 | +Package structure |
| 7 | +----------------- |
| 8 | + |
| 9 | +For a package to be considered a Sinolpack, it must |
| 10 | +have at least the following structure: |
| 11 | + |
| 12 | +.. code-block:: none |
| 13 | +
|
| 14 | + abc.tar.gz |
| 15 | + └── abc/ |
| 16 | + ├── in/ |
| 17 | + └── out/ |
| 18 | +
|
| 19 | +The package must have a top-level directory, which is the |
| 20 | +short name (task id) of the package. This directory must |
| 21 | +contain two subdirectories: `in` and `out`. |
| 22 | + |
| 23 | +The package can be compressed in these formats: `tar.gz`, |
| 24 | +`tgz` or `zip`. |
| 25 | + |
| 26 | +Below are the descriptions of all possible files and |
| 27 | +directories in a Sinolpack. |
| 28 | + |
| 29 | +`in` directory |
| 30 | +~~~~~~~~~~~~~~ |
| 31 | + |
| 32 | +The `in` directory contains all input tests for the package. |
| 33 | +Each test has to have a unique name: `<short_name><test id>.in`. |
| 34 | +Test id is a group number and an optional string of letters. |
| 35 | +Group number 0 is considered the example group -- points are |
| 36 | +not awarded for these tests and results are usually shown |
| 37 | +to the user. |
| 38 | + |
| 39 | +For example, the following files are valid input tests for a |
| 40 | +package with short name `abc`: |
| 41 | + |
| 42 | +- `abc0.in` -- example test |
| 43 | +- `abc0a.in` -- also an example test |
| 44 | +- `abc1ab.in` -- test from group 1 |
| 45 | + |
| 46 | +`out` directory |
| 47 | +~~~~~~~~~~~~~~~ |
| 48 | + |
| 49 | +The `out` directory contains all output tests for the package. |
| 50 | +Their names are the same as the input tests, but with the |
| 51 | +extension `.out`. |
| 52 | + |
| 53 | +`config.yml` file |
| 54 | +~~~~~~~~~~~~~~~~~ |
| 55 | + |
| 56 | +The `config.yml` file contains the configuration of the package. |
| 57 | +It is located in the root of the package: |
| 58 | + |
| 59 | +.. code-block:: none |
| 60 | +
|
| 61 | + abc.tar.gz |
| 62 | + └── abc/ |
| 63 | + ├── config.yml |
| 64 | + ├── in/ |
| 65 | + └── out/ |
| 66 | +
|
| 67 | +The configuration file is a YAML file with the following possible |
| 68 | +fields: |
| 69 | + |
| 70 | +- `title` -- the title of the package |
| 71 | +- `title_<lang>` -- the title of the package in a specific language |
| 72 | +- `scores` -- specifies the number of points for each test group. |
| 73 | + The key is the group number and the value is the number of points. |
| 74 | + By default, points are distributed evenly among the groups (if |
| 75 | + number of groups doesn't divide 100, then the last groups will |
| 76 | + have the remaining points). Group 0 always has zero points. |
| 77 | + Example: |
| 78 | + |
| 79 | + .. code-block:: yaml |
| 80 | +
|
| 81 | + scores: |
| 82 | + 1: 20 |
| 83 | + 2: 30 |
| 84 | + 3: 100 |
| 85 | +
|
| 86 | + In this example, group 1 has 20 points, group 2 has 30 points and |
| 87 | + group 3 has 100 points. As shown, the points don't have to sum up |
| 88 | + to 100. |
| 89 | +- `time_limit` -- the time limit for all tests in milliseconds |
| 90 | +- `time_limits` -- allows specifying time limits per group or |
| 91 | + per test. The key is the group number or the test id and the |
| 92 | + value is the time limit in milliseconds. The most specific |
| 93 | + time limit is used. Example: |
| 94 | + |
| 95 | + .. code-block:: yaml |
| 96 | +
|
| 97 | + time_limit: 500 |
| 98 | + time_limits: |
| 99 | + 1: 1000 |
| 100 | + 2: 2000 |
| 101 | + 2b: 3000 |
| 102 | +
|
| 103 | + In this example, group 1 has a time limit of 1000 ms, group 2 |
| 104 | + has a time limit of 2000 ms, except for test 2b which has a |
| 105 | + time limit of 3000 ms. All other tests have a time limit of 500 ms. |
| 106 | + |
| 107 | +- `memory_limit` -- the memory limit for all tests in kB. |
| 108 | +- `memory_limits` -- same as `time_limits`, but for memory limits. |
| 109 | + Specified in kB. |
| 110 | +- `override_limits` -- allows overriding time or memory limits per |
| 111 | + programming language. The key is the language and the value is |
| 112 | + a dictionary with `time_limit`, `time_limits`, `memory_limit` and |
| 113 | + `memory_limits` fields. Not all fields have to be specified. Example: |
| 114 | + |
| 115 | + .. code-block:: yaml |
| 116 | +
|
| 117 | + override_limits: |
| 118 | + py: |
| 119 | + time_limit: 1000 |
| 120 | + memory_limit: 256000 |
| 121 | + cpp: |
| 122 | + time_limits: |
| 123 | + 1: 2000 |
| 124 | + 2: 3000 |
| 125 | + memory_limit: 512000 |
| 126 | +
|
| 127 | + In this example, Python programs have a time limit of 1000 ms and |
| 128 | + a memory limit of 256 MB. C++ programs have a time limit of 2000 ms |
| 129 | + for group 1 and 3000 ms for group 2, and a memory limit of 512 MB. |
| 130 | +- `extra_compilation_args` -- allows specifying extra compilation |
| 131 | + arguments per programming language. The key is the language and |
| 132 | + the value is a list of arguments or a single value. Example: |
| 133 | + |
| 134 | + .. code-block:: yaml |
| 135 | +
|
| 136 | + extra_compilation_args: |
| 137 | + cpp: |
| 138 | + - -std=c++11 |
| 139 | + - -O2 |
| 140 | + py: single_value |
| 141 | +
|
| 142 | + In this example, C++ programs are compiled with the `-std=c++11` |
| 143 | + and `-O2` arguments. Python programs are compiled with a single |
| 144 | + value argument. |
| 145 | + |
| 146 | +- `extra_compilation_files` -- an array of files that should be |
| 147 | + included in the compilation process. The files are relative to |
| 148 | + the `prog/` directory. Example: |
| 149 | + |
| 150 | + .. code-block:: yaml |
| 151 | +
|
| 152 | + extra_compilation_files: |
| 153 | + - prog/abclib.cpp |
| 154 | + - prog/abclib.py |
| 155 | +
|
| 156 | +- `extra_execution_files` -- an array of files that should be |
| 157 | + present when the program is executed. The files are relative |
| 158 | + to the `prog/` directory. |
| 159 | + |
| 160 | +The `config.yml` can also contain other fields used by various |
| 161 | +tools, like `sinol-make` which uses fields starting with |
| 162 | +`sinol_`. These fields are ignored by SIO2 and are described |
| 163 | +`here <https://github.com/sio2project/sinol-make/blob/main/example_package/config.yml>`_. |
| 164 | + |
| 165 | +`prog` directory |
| 166 | +~~~~~~~~~~~~~~~~ |
| 167 | + |
| 168 | +The `prog` directory contains all source files for the package. |
| 169 | +The source files can be in any language. Here is a list of |
| 170 | +common files in the `prog` directory: |
| 171 | + |
| 172 | +- model solutions -- optional files that are used to see |
| 173 | + if grading is correct. The regex expression for them |
| 174 | + is `<short name>[bs]?[0-9]*(_.*)?\.(<languages>)`, |
| 175 | + where languages is a string of language extensions |
| 176 | + seperated by `|`. If the file contains a `b` in the |
| 177 | + first group, it is considered a bad solution and |
| 178 | + if it contains `s`, it's a slow solution. This |
| 179 | + information is only used to sort the model solutions |
| 180 | + in the results. |
| 181 | + |
| 182 | + A model solution named `<short name>.<language>` is |
| 183 | + used for generating output tests, so it should be |
| 184 | + a correct solution. |
| 185 | + |
| 186 | +- `<short name>ingen.<language>` -- optional file that |
| 187 | + is used to generate input tests. It is run in the |
| 188 | + `in` directory and should generate input tests in |
| 189 | + the format described above. |
| 190 | + |
| 191 | +- `<short name>inwer.<language>` -- optional file that |
| 192 | + is used to verify the correctness of the input tests. |
| 193 | + It is run for all input tests with it's first argument |
| 194 | + being the filename of the input test and on it's standard |
| 195 | + input the input test. If the test is correct, it should |
| 196 | + exit with code 0, otherwise with a non-zero code. |
| 197 | + |
| 198 | +`doc` directory |
| 199 | +~~~~~~~~~~~~~~~ |
| 200 | + |
| 201 | +The `doc` directory contains all documents for the package. |
| 202 | +SIO2 uses the `<short name>zad.pdf` file as the task |
| 203 | +description. The files `<short name>zad<language>.pdf` are |
| 204 | +used as task descriptions in specific languages. |
| 205 | + |
| 206 | +Some SIO2 instances allow compiling the task description |
| 207 | +from a LaTeX file. In that case, the LaTeX file should be |
| 208 | +named `<short name>zad.tex`. The LaTeX file can also be |
| 209 | +in a specific language, in which case it should be named |
| 210 | +`<short name>zad<language>.tex`. |
| 211 | + |
| 212 | +The task description can also be in HTML format. In that |
| 213 | +case, the files are in an archive `<short name>zad.html.zip`. |
| 214 | +The HTML archive can also be for a specific language, |
| 215 | +in which case it should be named `<short name>zad<language>.html.zip`. |
| 216 | + |
| 217 | +`attachments` directory |
| 218 | +~~~~~~~~~~~~~~~~~~~~~~~ |
| 219 | + |
| 220 | +Files from the `attachments` directory are shown in the |
| 221 | +`Files` section on SIO2. |
| 222 | + |
| 223 | + |
| 224 | +Task types |
| 225 | +---------- |
| 226 | + |
| 227 | +Sinolpack isn't very flexible, but supports various task |
| 228 | +types: normal, interactive (via library or IO) and |
| 229 | +output-only. The task type is determined by the presence |
| 230 | +of specific files in the package. |
| 231 | + |
| 232 | +Normal task type |
| 233 | +~~~~~~~~~~~~~~~~ |
| 234 | + |
| 235 | +Normal tasks are the most common type of tasks. They have |
| 236 | +model solutions, input and output tests and can have a |
| 237 | +checker for grading submissions. |
| 238 | + |
| 239 | +This task type is used when no other task type is detected. |
| 240 | + |
| 241 | +Custom files: |
| 242 | + |
| 243 | +- `prog/<short name>chk.<language>` -- output checker for |
| 244 | + the task. It is run with the first path to the input |
| 245 | + file, the second path to the user out file and the third |
| 246 | + path to the correct out file. If the checker exits with |
| 247 | + code greater than 2, the submission gets a System Error. |
| 248 | + The output of the checker must have at least one file, |
| 249 | + which is the result of the grading. If it's `OK`, the |
| 250 | + submission is correct, otherwise it's incorrect. In the |
| 251 | + second line of the output, the checker can print an optional |
| 252 | + comment that will be shown to the user. In the third line, |
| 253 | + the checker can print the number of points the submission |
| 254 | + got. The third line is passed to Python's |
| 255 | + `fractions.Fraction <https://docs.python.org/3/library/fractions.html>`_ |
| 256 | + constructor, so it can be a fraction. |
| 257 | + |
| 258 | +Interactive (via library) task type |
| 259 | +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
| 260 | + |
| 261 | +This task type is actually handled as a normal task type, |
| 262 | +but it uses `extra_compilation_args` and `extra_compilation_files` |
| 263 | +to specify the library that should be linked with the program. |
| 264 | +The library reads from the standard input and writes the results |
| 265 | +to the standard output. |
| 266 | + |
| 267 | +Interactive (via IO) task type |
| 268 | +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
| 269 | + |
| 270 | +This task type is used when the package contains the |
| 271 | +`prog/<short name>soc.<language>` file. This file is |
| 272 | +used to communicate with the user's program and grade |
| 273 | +it. |
| 274 | + |
| 275 | +This task type can have a field `num_processes` in the |
| 276 | +`config.yml` file. This field specifies the number of |
| 277 | +processes that will be run in parallel. The default |
| 278 | +value is 1. |
| 279 | + |
| 280 | +The `soc` program will be run with the following arguments: |
| 281 | + |
| 282 | +- number of the processes |
| 283 | +- pairs of numbers of file descriptors for writing and reading |
| 284 | + to the user's program |
| 285 | + |
| 286 | +The `soc` program on the standard input will receive the |
| 287 | +input test and on the standard output should write the |
| 288 | +grading result. The output is the same as for the normal |
| 289 | +task type's checker. |
| 290 | + |
| 291 | +Output-only task type |
| 292 | +~~~~~~~~~~~~~~~~~~~~~ |
| 293 | + |
| 294 | +I don't remember:( |
| 295 | + |
| 296 | + |
| 297 | +How the package is processed |
| 298 | +---------------------------- |
| 299 | + |
| 300 | +When a package is uploaded to SIO2, it is processed in the |
| 301 | +following way: |
| 302 | + |
| 303 | +1. The package is extracted to a temporary directory. |
| 304 | +2. The `config.yml` file is read and the configuration is |
| 305 | + stored in the database. |
| 306 | +3. Title and translated titles are stored in the database. |
| 307 | + If the title is not specified in `config.yml`, the title |
| 308 | + is taken from the LaTeX task description (if present) from |
| 309 | + the `\title` command. Only main title is taken from LaTeX, |
| 310 | + translated titles can only be specified in `config.yml`. |
| 311 | +4. Extra compilation and execution files are saved in the |
| 312 | + file storage. If any of the files are missing, the package |
| 313 | + is considered invalid. |
| 314 | +5. All files from `prog/` are saved as an archive in the |
| 315 | + file storage. |
| 316 | +6. Statements are processed and saved. If LaTeX files are |
| 317 | + present and the SIO2 instance allows compiling LaTeX, |
| 318 | + the compiled PDF is saved in the file storage. |
| 319 | +7. Tests are generated, limits are saved and scores for |
| 320 | + groups are assigned. There is one legacy method for |
| 321 | + setting memory limits, which is to specify the limit |
| 322 | + in LaTeX in the `\RAM` command. The actual memory |
| 323 | + limit is then calculate as such: |
| 324 | + `(value + (value + 31) // 32) * 1000`. This method is |
| 325 | + deprecated and should not be used. |
| 326 | +8. Checker program is saved in the file storage (if present). |
| 327 | +9. If the task type is interactive via IO, the `soc` program |
| 328 | + is saved in the file storage. |
| 329 | +10. Model solutions are created and saved in the database. |
| 330 | +11. Attachments are saved in the file storage. |
| 331 | +12. The package is marked as processed. |
0 commit comments