-
Notifications
You must be signed in to change notification settings - Fork 41
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Added browser tests using wasmtime test artifacts #42
base: main
Are you sure you want to change the base?
Conversation
@bjorn3 I hope these changes are again in the right direction. Testing could go any one of a dozen ways, and this seemed like the best cost/value. |
Using wasmtime's wasi tests is a great idea. Thanks for working on this. FYI I probably won't have time to review this in the next couple of days. Part of #7 |
dbe77a6
to
b719a54
Compare
@bjorn3 np, thanks for being willing and responsive! |
/playwright-report/ | ||
/playwright/.cache/ | ||
|
||
wasmtime/ |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit: missing trailing newline
|
||
path_unlink_file(path: string): number { | ||
delete this.dir.contents[path]; | ||
return wasi.ERRNO_SUCCESS; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Could you add a FIXME to check if the path actually exists and to return an error otherwise. Also is there no test in wasmtime's test suite for that?
if (this.file.readonly) return { ret: wasi.ERRNO_BADF, nwritten }; | ||
if ( | ||
this.file.readonly || | ||
(Number(this.fs_rights_base) & wasi.RIGHTS_FD_WRITE) === 0 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Would making RIGHTS_FD_WRITE
a Bigint
work to avoid this cast?
this.file.readonly || | ||
(Number(this.fs_rights_base) & wasi.RIGHTS_FD_WRITE) === 0 | ||
) | ||
return { ret: wasi.ERRNO_BADF, nwritten }; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
For please use {}
for if
where the action is not on the same line. That helps prevent issues like the duplicate "goto fail;" that caused a huge security issue in macOS a couple of years back.
WASMTIME_DIR=$SCRIPT_DIR/../wasmtime | ||
|
||
if ! [[ -d wasmtime ]]; then | ||
git clone --recurse-submodules https://github.com/bytecodealliance/wasmtime.git --depth=1 $WASMTIME_DIR; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Are the submodules actually necessary? And if so are all of them necessary?
WASMTIME_DIR=$SCRIPT_DIR/../wasmtime | ||
|
||
if ! [[ -d wasmtime ]]; then | ||
git clone --recurse-submodules https://github.com/bytecodealliance/wasmtime.git --depth=1 $WASMTIME_DIR; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Could you pin a specific commit of wasmtime for reproducibility in the future?
|
||
cargo build --manifest-path $WASMTIME_DIR/crates/test-programs/artifacts/Cargo.toml | ||
|
||
TARGET_DIR=$(ls -dt $WASMTIME_DIR/target/debug/build/test-programs-artifacts-* | head -n 1) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
TARGET_DIR=$(ls -dt $WASMTIME_DIR/target/debug/build/test-programs-artifacts-* | head -n 1) | |
TARGET_DIR=$(ls -dt $WASMTIME_DIR/target/debug/build/test-programs-artifacts-*/out | head -n 1) |
There is no guarantee that the first one is the one with the OUT_DIR
.
I'm not entirely sure all |
@jsoverson are you still interested in working on this or would you prefer if I work on splitting it up into several commits and landing it myself? |
@bjorn3 sorry, this fell off my radar during a few hectic weeks. I'll jump on it this coming week. |
Status: I'm going to fork wasmtime to create releases consisting of strictly the test wasm files. Getting CI setup to clone wasmtime to build the test artifacts is excessive. |
You can use |
It's more the overhead of setting up a rust environment to build wasm files that really shouldn't change. It will dramatically extend the duration of CI tests, vs. downloading a tarball of wasm artifacts. |
I see. How about caching the wasm test files in a github actions cache after they are built. And then only setup a rust toolchain for building if the cache is missing. That should be easier to keep up to date with new wasmtime versions than building locally and checking into a repo. |
This PR adds the start of browser-based tests. It adds a playwright configuration to test across Firefox, Chromium, and Webkit and uses test artifacts from the wasmtime project to assert compatibility.
Only one test has been added so far (test-programs/preview1_path_open_read_write). It required some project changes that need review before moving forward.
To run tests:
Change Notes:
initialize()
function was updated to only call_initialize()
if it's exported. AFAICT_initialize()
is optional. This lets all reactor libraries use the same shim init logic without needing the user to inspect the exports.wasiImport
types into a proper interface to keep satisfying TypeScript. This had the side effect of requiring some changes to satisfy types that would have been the source of errors before.Any change where(reverted)obj[key] = undefined
becomesdelete obj[key]
is due to this.Fd
implementations so thatfd_fdstat_get
could report correct values.path_unlink_file
implementation toDir
to satisfy the test wasm. It deletes the instance from the dir's contents and doesn't do anything more destructive.