Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Rust Autotest (DB Entries) #6906

Closed

Conversation

1whatleytay
Copy link

@1whatleytay 1whatleytay commented Jan 13, 2024

This PR is paired with markus-autotesting PR. It adds an example autotest target to rake db:autotest.

Motivation and Context

Rust autotesting allows courses or universities that are experimenting with Rust an easy platform to auto test their students submissions.

Your Changes

I have added an auto test target for Rust on the markus-autotesting server. With an updated auto test schema (you may have to wipe the autotest_settings table for the rust target to show up on MarkUs after updating the server), MarkUs now supports the Rust language via cargo-test. For more details on implementation, see the autotesting PR.

Type of change (select all that apply):

  • New feature (non-breaking change which adds functionality)
  • Test update (change that modifies or updates tests only)

Including the changes to markus-autotesting. I believe this specific PR only modifies development tests.

Testing

I tested this PR in a couple ways. Using docker and a fresh DB, I ran:

  • Ran rake db:autotest and ran tests manually on the autotest-rust project by selecting a submission and clicking on Run Tests (via MarkUs GUI).
  • Manually creating an additional assignment, configuring a Rust test group, submitting a submission as a student and running tests as an instructor.
Screenshot 2024-01-13 at 12 27 50 PM

Questions and Comments (if applicable)

I wanted to avoid parsing "human-readable" output of cargo test. Machine readable output is nightly-only for cargo test (which means we would have to use an unstable Rust version). I had to use an alternative test runner cargo-nextest that had a more stable Machine Readable test output, and was compatible with cargo-test formatted tests.

I don't think it makes sense to have a test-script schema property (neither cargo-test nor cargo-nextest seems to keep information about the file names to my knowledge). I can add a "modules to run" option, which works better with the input for both runners.

Checklist

  • I have performed a self-review of my own code.
  • I have verified that the pre-commit.ci checks have passed.
  • I have verified that the CI tests have passed.
  • I have reviewed the test coverage changes reported on Coveralls.
  • I have added tests for my changes. Taylor: Counting the db:autotest_run as tests. Seems to match the coverage of the other testers.
  • I have updated the Changelog.md file.
  • I have made changes to the documentation and linked to that pull request below.

Pull request to make documentation changes (if applicable)

MarkUsProject/Wiki#204

# copy test scripts and specs files into the destination directory
FileUtils.cp @test_scripts, test_file_destination
FileUtils.copy_entry @script_dir, test_file_destination
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will copy folders too. Rust expects the default library file to be called src/lib.rs (within the src folder). This is configurable in Cargo.toml, but I thought I would provide an example that uses the default project structure for clarity 👍

@@ -127,6 +129,24 @@ class AutotestSetup
)
end

def create_submission_files(transaction, glob)
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same here, this recursive definition is designed to support the src folder in the rust submission.

@1whatleytay 1whatleytay marked this pull request as ready for review January 15, 2024 00:47
@coveralls
Copy link
Collaborator

Pull Request Test Coverage Report for Build 8309673167

Details

  • 0 of 0 changed or added relevant lines in 0 files are covered.
  • No unchanged relevant lines lost coverage.
  • Overall coverage remained the same at 91.784%

Totals Coverage Status
Change from base Build 8308241425: 0.0%
Covered Lines: 39178
Relevant Lines: 42063

💛 - Coveralls

@david-yz-liu
Copy link
Contributor

@1whatleytay I'm closing this for now, but please feel free to reopen if you'd like to continue working on this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants