-
Notifications
You must be signed in to change notification settings - Fork 22
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat: introduce experimental packaging format #278
Merged
Merged
Commits on Aug 27, 2024
-
Configuration menu - View commit details
-
Copy full SHA for c5a9869 - Browse repository at this point
Copy the full SHA c5a9869View commit details -
Configuration menu - View commit details
-
Copy full SHA for 5395e9e - Browse repository at this point
Copy the full SHA 5395e9eView commit details -
fix: use less fragile method for rodata segment init
Previous to this commit, we were expecting the rodata segments to be encoded in a specific order, and placed on the advice stack before anything else. With this commit, we now simply expect to the advice map to contain each segment keyed by its commitment hash, which we then move to the advice stack on demand, and immediately pipe to memory. This means the order of the segments no longer matters, and the advice stack is not sensitive to codegen changes or other influences which might perturb the advice stack or otherwise disrupt our assumptions. It also sets the stage for us to be able to initialize rodata after a context switch, as at that point the advice stack will be in an unknown condition, and using the advice map gives us certainty that we can arrange to have exactly what we need on the advice stack, when we need it. Additionally, I've updated the `midenc debug` input config file, as well as the usage documentation to reflect this. The last related change to this, will be emitting the rodata segments to disk in a convenient form, so that when the compiler emits the program, it also emits the segments alongside it, making it convenient to run the debugger against that program (or via the VM directly).
Configuration menu - View commit details
-
Copy full SHA for 71626a9 - Browse repository at this point
Copy the full SHA 71626a9View commit details -
feat: implement packaging prototype
This commit implements a basic package format for Miden programs and libraries, with the metadata needed to simplify using them together. The format is experimental, and can be changed at any time, however it does use a header that lets us identify whether a package was produced with the same specification or not. This also modifies the compiler and debugger to emit and consume, respectively, the package format as their primary artifact. The debugger uses this, for example, to initialize the VM with the necessary libraries and rodata segments required by the program. The package format is described using the structures in `codegen/masm/src/packaging/package.rs`, and is encoded using a tiny, but efficient binary format from the `bitcode` crate, using `serde` to facilitate the lowering of our high-level types to that format. This made it relatively straightforward to implement the format without having to mess around with the low-level byte representation. The `bitcode` crate is designed with stability in mind, though because we are using it via `serde`, the stability is subject to being violated if `serde` changes something that causes `bitcode` to serialize a type differently. For now, this is a non-issue, longer term we might want to actually implement `bitcode::{Encode, Decode}` for our types if we want to gain more stability.
Configuration menu - View commit details
-
Copy full SHA for 3322c53 - Browse repository at this point
Copy the full SHA 3322c53View commit details -
Configuration menu - View commit details
-
Copy full SHA for 4debaf6 - Browse repository at this point
Copy the full SHA 4debaf6View commit details -
fix: regression in midenc compile
Some recent changes caused the `midenc compile` command to emit no outputs when `--emit` was not provided. Also fixed: * Moved some unnecessary debug logs to trace level * Added more debug logging to the compiler to aid in troubleshooting * Fixed serialization of packages involving more complex inputs which were hitting an issue with fields marked as skippable via serde; this is not supported via bitcode, so those attributes were removed
Configuration menu - View commit details
-
Copy full SHA for 511f0d3 - Browse repository at this point
Copy the full SHA 511f0d3View commit details -
Configuration menu - View commit details
-
Copy full SHA for 86cf309 - Browse repository at this point
Copy the full SHA 86cf309View commit details -
Configuration menu - View commit details
-
Copy full SHA for f2f4045 - Browse repository at this point
Copy the full SHA f2f4045View commit details -
Configuration menu - View commit details
-
Copy full SHA for a1d7585 - Browse repository at this point
Copy the full SHA a1d7585View commit details -
Configuration menu - View commit details
-
Copy full SHA for c2b2f3b - Browse repository at this point
Copy the full SHA c2b2f3bView commit details -
Configuration menu - View commit details
-
Copy full SHA for ae525ea - Browse repository at this point
Copy the full SHA ae525eaView commit details -
Configuration menu - View commit details
-
Copy full SHA for 6d4353c - Browse repository at this point
Copy the full SHA 6d4353cView commit details -
Configuration menu - View commit details
-
Copy full SHA for 7bd7c72 - Browse repository at this point
Copy the full SHA 7bd7c72View commit details -
Configuration menu - View commit details
-
Copy full SHA for 5215164 - Browse repository at this point
Copy the full SHA 5215164View commit details -
Configuration menu - View commit details
-
Copy full SHA for 23c4d2d - Browse repository at this point
Copy the full SHA 23c4d2dView commit details -
Configuration menu - View commit details
-
Copy full SHA for 50ff2c9 - Browse repository at this point
Copy the full SHA 50ff2c9View commit details -
fix(frontend-wasm): reserve memory allocated for use by rust
It turns out we were assuming that Rust was laying out its memory using the default LLVM layout, but rustc actually specifies a larger shadow stack. We were not properly handling the defined memory in Wasm modules parsed by the frontend, and so we ended up incorrectly placing global variables and other data in the middle of memory that rust was using for either the shadow stack or static data. Because of the size of the shadow stack region reserved by Rust, this was not causing any corruption in our tests, but it was causing other assumptions to be invalid, and miscompilations were the result. This commit treats the Wasm memory for a parsed module as reserved for use by the code in that module, which causes midenc to place any global variables it defines after that region, and computes the start of the heap as immediately following any such data. It does so by propagating the page size and reserved number of pages through the IR and to the backend, taking that information into account as needed.
Configuration menu - View commit details
-
Copy full SHA for 5586482 - Browse repository at this point
Copy the full SHA 5586482View commit details -
Configuration menu - View commit details
-
Copy full SHA for ce99ba4 - Browse repository at this point
Copy the full SHA ce99ba4View commit details
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.