Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support for dynamic test case outputs #12

Open
Yash-Singh1 opened this issue Oct 31, 2022 · 1 comment
Open

Support for dynamic test case outputs #12

Yash-Singh1 opened this issue Oct 31, 2022 · 1 comment

Comments

@Yash-Singh1
Copy link

Some test cases can have multiple outputs such as https://cses.fi/problemset/task/1640. Currently, I believe that the judge doesn't validate dynamic outputs looking at the source code:

return given.trim() === expected.trim();

Also, submitting to it on one of my USACO Guide Groups gives me partial WA while CSES gives me all AC.

@megargayu
Copy link

An idea to fix this could be replacing .out files for all problems with multiple correct answers and instead replacing it with an executable or other script file which would validate an output, and then automatically detecting a lack of a .out file in the actual grader code. Probably would have to start changing the code below:

const regexMatch = file.Key!.match(/\/(\d+)\.(in|out)/);
if (!regexMatch) {
throw new Error("Failed to parse file name " + file.Key);
}
const testCaseNumber = parseInt(regexMatch[1]);
const isInput = regexMatch[2] === "in";
if (isInput) {
testCases[testCaseNumber - 1].input = content;
} else {
testCases[testCaseNumber - 1].expectedOutput = content;
}
})

Changing the Error thrown to instead check for an executable/script file and then if found set the necessary parameters so it can be used during grading.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants