Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ZON #20271

Open
wants to merge 32 commits into
base: master
Choose a base branch
from
Open

ZON #20271

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
32 commits
Select commit Hold shift + click to select a range
2009cd3
Squashes all zon work into one commit for easier rebase
MasonRemaley Nov 4, 2024
b4b3d12
Rebases and gets runtime zon import tests passing, comptime import ne…
MasonRemaley Nov 5, 2024
6fd7fba
Gets some import zon behavior tests working, skips failing tests for now
MasonRemaley Nov 5, 2024
f5fb430
Gets strings working again
MasonRemaley Nov 5, 2024
909b072
Starts moving parsing into known result type
MasonRemaley Nov 5, 2024
f8470d4
Starts moving optionals to known result type parsing
MasonRemaley Nov 5, 2024
ec2f582
Parses integers to known result types. Not yet tested as those tests …
MasonRemaley Nov 6, 2024
8f1a665
Pulls out separate types into their own functions to make easier to j…
MasonRemaley Nov 6, 2024
7709564
Parses arrays to known result type
MasonRemaley Nov 6, 2024
d5301b4
Parses enums to known result type
MasonRemaley Nov 6, 2024
12c3530
WIP tuples: crash on string literals right now
MasonRemaley Nov 6, 2024
3d1e01c
Parses tuples to known result types
MasonRemaley Nov 6, 2024
30cbe7b
Parses strings to known result type
MasonRemaley Nov 6, 2024
72d17c0
Parses unions to explicit return types
MasonRemaley Nov 12, 2024
41b1880
Fixes bug in explicit union tag parsing
MasonRemaley Nov 12, 2024
7ddfae1
Parses void union fields from enum literals
MasonRemaley Nov 13, 2024
cbc34c7
Parses structs to known result types
MasonRemaley Nov 13, 2024
5bd6477
Fixes parsing integers as floats
MasonRemaley Nov 13, 2024
a9b2a93
Removes old implementation, renames parse to lower
MasonRemaley Nov 26, 2024
e45b8de
Implements slice coercion
MasonRemaley Nov 27, 2024
840e960
Starts work on compile error tests
MasonRemaley Nov 27, 2024
1edcd45
Fixes more failing compile error tests
MasonRemaley Dec 3, 2024
9156de2
Fixes more zon compile error tests
MasonRemaley Dec 3, 2024
93754f4
Gets all ZON tests passing locally
MasonRemaley Dec 3, 2024
8fefd9b
Removes platform specific separator from test case
MasonRemaley Dec 3, 2024
80c4de8
Fixes mistake in rebase
MasonRemaley Dec 18, 2024
d1852ba
Updates runtime parser to use ZonGen
MasonRemaley Dec 18, 2024
eae9770
Parses to zoir on import, but does not yet use result or lower errors
MasonRemaley Dec 21, 2024
9cd6219
Reports zongen errors when importing zon, doesn't yet use it for parsing
MasonRemaley Dec 21, 2024
8dc3413
Simplifies error handling
MasonRemaley Dec 22, 2024
54269df
Ports import zon to use Zoir
MasonRemaley Dec 22, 2024
423147a
Cleans up diff with master, resovles some feedback
MasonRemaley Dec 22, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion build.zig
Original file line number Diff line number Diff line change
Expand Up @@ -404,7 +404,7 @@ pub fn build(b: *std.Build) !void {
const optimization_modes = chosen_opt_modes_buf[0..chosen_mode_index];

const fmt_include_paths = &.{ "lib", "src", "test", "tools", "build.zig", "build.zig.zon" };
const fmt_exclude_paths = &.{"test/cases"};
const fmt_exclude_paths = &.{ "test/cases", "test/behavior/zon" };
MasonRemaley marked this conversation as resolved.
Show resolved Hide resolved
const do_fmt = b.addFmt(.{
.paths = fmt_include_paths,
.exclude_paths = fmt_exclude_paths,
Expand Down
1 change: 1 addition & 0 deletions lib/std/fmt.zig
Original file line number Diff line number Diff line change
Expand Up @@ -1581,6 +1581,7 @@ test parseInt {
try std.testing.expectEqual(@as(i5, -16), try std.fmt.parseInt(i5, "-10", 16));
}

/// Like `parseIntWithGenericCharacter`, but with a sign argument.
fn parseIntWithSign(
comptime Result: type,
comptime Character: type,
Expand Down
1 change: 1 addition & 0 deletions lib/std/std.zig
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,7 @@ pub const Thread = @import("Thread.zig");
pub const Treap = @import("treap.zig").Treap;
pub const Tz = tz.Tz;
pub const Uri = @import("Uri.zig");
pub const zon = @import("zon.zig");

pub const array_hash_map = @import("array_hash_map.zig");
pub const atomic = @import("atomic.zig");
Expand Down
2 changes: 1 addition & 1 deletion lib/std/zig/Ast.zig
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ tokens: TokenList.Slice,
/// references to the root node, this means 0 is available to indicate null.
nodes: NodeList.Slice,
extra_data: []Node.Index,
mode: Mode = .zig,
mode: Mode,

errors: []const Error,

Expand Down
13 changes: 12 additions & 1 deletion lib/std/zig/AstGen.zig
Original file line number Diff line number Diff line change
Expand Up @@ -9304,7 +9304,18 @@ fn builtinCall(
} else if (str.len == 0) {
return astgen.failTok(str_lit_token, "import path cannot be empty", .{});
}
const result = try gz.addStrTok(.import, str.index, str_lit_token);
const res_ty = try ri.rl.resultType(gz, node) orelse .none;
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Imports now store result types.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A possible ZIR optimization here would now be to replace the rvalue call below with special handling in the ty case, since Sema will do the coercion for us. Not a merge blocker, just worth noting!

const payload_index = try addExtra(gz.astgen, Zir.Inst.Import{
.res_ty = res_ty,
.path = str.index,
});
const result = try gz.add(.{
.tag = .import,
.data = .{ .pl_tok = .{
.src_tok = gz.tokenIndexToRelative(str_lit_token),
.payload_index = payload_index,
} },
});
const gop = try astgen.imports.getOrPut(astgen.gpa, str.index);
if (!gop.found_existing) {
gop.value_ptr.* = str_lit_token;
Expand Down
9 changes: 8 additions & 1 deletion lib/std/zig/Zir.zig
Original file line number Diff line number Diff line change
Expand Up @@ -1671,7 +1671,7 @@ pub const Inst = struct {
.func = .pl_node,
.func_inferred = .pl_node,
.func_fancy = .pl_node,
.import = .str_tok,
.import = .pl_tok,
.int = .int,
.int_big = .str,
.float = .float,
Expand Down Expand Up @@ -3542,6 +3542,13 @@ pub const Inst = struct {
/// If `.none`, restore unconditionally.
operand: Ref,
};

pub const Import = struct {
/// The result type of the import, or `.none` if none was available.
res_ty: Ref,
/// The import path.
path: NullTerminatedString,
};
};

pub const SpecialProng = enum { none, @"else", under };
Expand Down
18 changes: 16 additions & 2 deletions lib/std/zig/ZonGen.zig
Original file line number Diff line number Diff line change
Expand Up @@ -250,7 +250,21 @@ fn expr(zg: *ZonGen, node: Ast.Node.Index, dest_node: Zoir.Node.Index) Allocator
.block_two_semicolon,
.block,
.block_semicolon,
=> try zg.addErrorNode(node, "blocks are not allowed in ZON", .{}),
=> {
const size = switch (node_tags[node]) {
.block_two, .block_two_semicolon => @intFromBool(node_datas[node].lhs != 0) + @intFromBool(node_datas[node].rhs != 0),
.block, .block_semicolon => node_datas[node].rhs - node_datas[node].lhs,
else => unreachable,
};
if (size == 0) {
try zg.addErrorNodeNotes(node, "void literals are not available in ZON", .{}, &.{
try zg.errNoteNode(node, "void union payloads can be represented by enum literals", .{}),
try zg.errNoteNode(node, "for example, `.{{ .foo = {{}} }}` becomes `.foo`", .{}),
});
} else {
try zg.addErrorNode(node, "blocks are not allowed in ZON", .{});
}
},

.array_init_one,
.array_init_one_comma,
Expand Down Expand Up @@ -540,7 +554,7 @@ fn numberLiteral(zg: *ZonGen, num_node: Ast.Node.Index, src_node: Ast.Node.Index
if (unsigned_num == 0 and sign == .negative) {
try zg.addErrorTokNotes(num_token, "integer literal '-0' is ambiguous", .{}, &.{
try zg.errNoteTok(num_token, "use '0' for an integer zero", .{}),
try zg.errNoteTok(num_token, "use '-0.0' for a flaoting-point signed zero", .{}),
try zg.errNoteTok(num_token, "use '-0.0' for a floating-point signed zero", .{}),
});
return;
}
Expand Down
5 changes: 5 additions & 0 deletions lib/std/zig/number_literal.zig
Original file line number Diff line number Diff line change
Expand Up @@ -60,6 +60,11 @@ pub const Error = union(enum) {
period_after_exponent: usize,
};

const FormatWithSource = struct {
bytes: []const u8,
err: Error,
};

/// Parse Zig number literal accepted by fmt.parseInt, fmt.parseFloat and big_int.setString.
/// Valid for any input.
pub fn parseNumberLiteral(bytes: []const u8) Result {
Expand Down
90 changes: 90 additions & 0 deletions lib/std/zon.zig
Original file line number Diff line number Diff line change
@@ -0,0 +1,90 @@
//! ZON serialization and deserialization.
//!
//! # ZON
//! ZON, or Zig Object Notation, is a subset* of Zig used for data storage. ZON contains no type
//! names.
//!
//! Supported Zig primitives:
//! * boolean literals
//! * number literals (including `nan` and `inf`)
//! * character literals
//! * enum literals
//! * `null` literals
//! * string literals
//! * multiline string literals
//!
//! Supported Zig container types:
//! * anonymous struct literals
//! * anonymous tuple literals
//!
//! Here is an example ZON object:
//! ```zon
//! .{
//! .a = 1.5,
//! .b = "hello, world!",
//! .c = .{ true, false },
//! .d = .{ 1, 2, 3 },
//! }
//! ```
//!
//! Individual primitives are also valid ZON, for example:
//! ```zon
//! "This string is a valid ZON object."
//! ```
//!
//! * ZON is not currently a true subset of Zig, because it supports `nan` and
//! `inf` literals, which Zig does not.
//!
//! # Deserialization
//!
//! The simplest way to deserialize ZON at runtime is `parseFromSlice`. (For parsing ZON at
//! comptime, you can use `@import`.)
//!
//! Parsing from individual Zoir nodes is also available:
//! * `parseFromZoir`
//! * `parseFromZoirNode`
//! * `parseFromZoirNode`
//! * `parseFromZoirNodeNoAlloc`
//!
//! If you need lower level control than provided by this module, you can operate directly on the
//! results of `std.zig.Zoir` directly. This module is a good example of how that can be done.
//!
//!
//! # Serialization
//!
//! The simplest way to serialize to ZON is to call `stringify`.
//!
//! If you need to serialize recursive types, the following functions are also provided:
//! * `stringifyMaxDepth`
//! * `stringifyArbitraryDepth`
//!
//! If you need more control over the serialization process, for example to control which fields are
//! serialized, configure fields individually, or to stringify ZON values that do not exist in
//! memory, you can use `Stringifier`.
//!
//! Note that serializing floats with more than 64 bits may result in a loss of precision
//! (see https://github.com/ziglang/zig/issues/1181).

pub const ParseOptions = @import("zon/parse.zig").Options;
pub const ParseStatus = @import("zon/parse.zig").Status;
pub const parseFromSlice = @import("zon/parse.zig").parseFromSlice;
pub const parseFromZoir = @import("zon/parse.zig").parseFromZoir;
pub const parseFromZoirNoAlloc = @import("zon/parse.zig").parseFromZoirNoAlloc;
pub const parseFromZoirNode = @import("zon/parse.zig").parseFromZoirNode;
pub const parseFromZoirNodeNoAlloc = @import("zon/parse.zig").parseFromZoirNodeNoAlloc;
pub const parseFree = @import("zon/parse.zig").parseFree;

pub const StringifierOptions = @import("zon/stringify.zig").StringifierOptions;
pub const StringifyValueOptions = @import("zon/stringify.zig").StringifyValueOptions;
pub const StringifyOptions = @import("zon/stringify.zig").StringifyOptions;
pub const StringifyContainerOptions = @import("zon/stringify.zig").StringifyContainerOptions;
pub const Stringifier = @import("zon/stringify.zig").Stringifier;
pub const stringify = @import("zon/stringify.zig").stringify;
pub const stringifyMaxDepth = @import("zon/stringify.zig").stringifyMaxDepth;
pub const stringifyArbitraryDepth = @import("zon/stringify.zig").stringifyArbitraryDepth;
pub const stringifier = @import("zon/stringify.zig").stringifier;

test {
_ = @import("zon/parse.zig");
_ = @import("zon/stringify.zig");
}
Loading
Loading