Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support generation of Zig bindings #732

Open
wants to merge 23 commits into
base: master
Choose a base branch
from

Conversation

kassane
Copy link

@kassane kassane commented Jan 23, 2022

Zig is a general-purpose programming language and toolchain for maintaining robust, optimal, and reusable software.

Status: [ WIP ]

  • Add primitive types and C primitive types (c_void renamed to anyopaque - see) support
  • Add struct, enum, union with C compatibility
  • Functions
  • Casts & Functors
  • Pointer & Array types
  • Include compatibility
    - [ ] try comptime support
  • review all stages of tests

Test Project

Clone and build: https://github.com/kassane/zFFI

@kassane kassane marked this pull request as draft January 23, 2022 16:19
@kassane
Copy link
Author

kassane commented Jan 23, 2022

I ask for a little patience on this adjustment as I am still learning about the internal parsing mechanism of this api.
So any help is welcome!

@emilio, there any limit or deadline to still have PR open/draft?

@emilio
Copy link
Collaborator

emilio commented Jan 25, 2022

Definitely not! I'll try to get to it and provide some feedback, but I don't close PRs unless they're superseded by other work, is definitely-unwanted functionality, or have gone completely stale for other reasons. So no rush :)

@emilio
Copy link
Collaborator

emilio commented Jan 25, 2022

That said, a high-level question from taking a look at Zig and so on: Is there anything cbindgen would be able to do that @cImport-ing a cbindgen-generated header wouldn't? Just trying to understand what the improvements over that would be.

@kassane
Copy link
Author

kassane commented Jan 25, 2022

That said, a high-level question from taking a look at Zig and so on: Is there anything cbindgen would be able to do that @cImport-ing a cbindgen-generated header wouldn't? Just trying to understand what the improvements over that would be.

First of all, thanks for the feedback.

The goal in this feature would not only be the issue of compatibility with cImport because zig allows you to reuse C implementations (ffi) and can also translate C99 code into zig with zig translate-c.

But I think it would be interesting to evaluate a possible direct compatibility between rust and zig without relying on C code to make the transition for better security, even though for the rust side any foreign language would still be unsafe.
If you look at the test file you will notice that I added zig-build ReleaseSafe mode which could be explained here.

@kassane kassane marked this pull request as ready for review January 31, 2022 13:01
@kassane
Copy link
Author

kassane commented Jan 31, 2022

Pointers & Arrays

https://github.com/eqrion/cbindgen/blob/b94318a8b700d0d94e0da0efe9f2c1bcc27c855f/tests/rust/ptrs_as_arrays.rs#L1-L19
Generate code:

const std = @import("std")

pub fn ptr_as_array( n: u32,  arg: [3]u32,  _v: ?*u64) anyopaque;

pub fn ptr_as_array1( n: u32,  arg: [3]u32,  v: [4]u64) anyopaque;

pub fn ptr_as_array2( n: u32,  arg: []u32,  v: []u64) anyopaque;

pub fn ptr_as_array_wrong_syntax( _arg: ?*u32,  _v: ?*u32, _: ?*u32) anyopaque;

pub fn ptr_as_array_unnamed(_: ?*u32, _: ?*u32) anyopaque;

@kassane
Copy link
Author

kassane commented Feb 1, 2022

@eqrion @emilio
Who wants to test the syntax binding, just use zig fmt (lint) or build|run. There is also the possibility to use zig compiler-explorer

Since yesterday I focused on fixing the ones that are already marked. Now, I will need to look for other compatibilities between the languages.

Copy link

@camconn camconn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good except for the empty test stubs and commented portions.

src/bindgen/ir/ty.rs Outdated Show resolved Hide resolved
src/bindgen/ir/documentation.rs Outdated Show resolved Hide resolved
@camconn
Copy link

camconn commented Mar 27, 2022

Looks great so far, awesome job! I made some quick observations, but please take them with a grain of salt.

The comptime code may be able to merged into the existing C++ constexpr code, as I believe comptime is acceptable in a superset of cases of constexpr.

It would be interesting to see if there is a way to use custom global allocators once that RFC goes through.

@kassane
Copy link
Author

kassane commented Mar 28, 2022

Looks great so far, awesome job! I made some quick observations, but please take them with a grain of salt.

The comptime code may be able to merged into the existing C++ constexpr code, as I believe comptime is acceptable in a superset of cases of constexpr.

It would be interesting to see if there is a way to use custom global allocators once that RFC goes through.

No problem, thanks for the review!
I was waiting for some feedback before adding any other changes.

Also, I would like to share an important detail that discovered.
The zig does not depend on C ABI to interoperate with Rust or C++, because it can read the mangled function.


Example

Rust lib:

// #[no_mangle]
pub fn sum_add(x: i32, y: i32) -> i32 { x + y }

Cargo config (toml) => lib:
staticlib => ❌
cdylib, rlib, lib or dylib => 🆗

Run

$> objdump -x rust_mangle.so | grep sum_add
...   F.text   .... _ZN11rust_mangle7sum_add17hf6faed35db54da66E
$> echo "_ZN11rust_mangle7sum_add17hf6faed35db54da66E" | c++filt
rust_mangle::sum_add::hf6faed35db54da66

Zig app:

const std = @import("std");

extern "C++" fn @"_ZN11rust_mangle7sum_add17hf6faed35db54da66E" (x: i32, y: i32) i32;
//Why not rust? Missing librust

pub fn main() void {
     const x: i32 = 43;
     const y: i32 = 32;
     const add = @"_ZN11rust_mangle7sum_add17hf6faed35db54da66E";
     std.log.info("Value add {} + {} = {}\n",.{x, y, add(x, y)});
}

@kassane kassane force-pushed the zig-bindgen branch 2 times, most recently from 1dabc3a to 5fcb6f4 Compare April 7, 2022 18:50
@kassane
Copy link
Author

kassane commented Apr 8, 2022

New Progress

Source

https://github.com/eqrion/cbindgen/blob/a151cbfb720fa331cc24c5e439eb2c47cfcfb037/tests/rust/fns.rs#L1-L16

C

https://github.com/eqrion/cbindgen/blob/a151cbfb720fa331cc24c5e439eb2c47cfcfb037/tests/expectations/fns.c#L1-L16

Expected

// zig - translate-c generator

pub const Fns = extern struct {
    noArgs: ?fn () callconv(.C) void,
    anonymousArg: ?fn (i32) callconv(.C) void,
    returnsNumber: ?fn () callconv(.C) i32,
    namedArgs: ?fn (i32, i16) callconv(.C) i8,
    namedArgsWildcards: ?fn (i32, i16, i64) callconv(.C) i8,
};
pub extern fn root(_fns: Fns) void;
pub extern fn no_return() void;

Generated (cbindgen)

https://github.com/kassane/cbindgen/blob/a151cbfb720fa331cc24c5e439eb2c47cfcfb037/tests/expectations/fns.zig#L1-L13

const std = @import("std");

pub const Fns = extern struct {
   _noArgs: ?fn() anyopaque,
   _anonymousArg: ?fn() anyopaque,
   _returnsNumber: ?fn() i32,
   _namedArgs: ?fn(first: i32, snd: i16) i8,
   _namedArgsWildcards: ?fn(_: i32, named: i16, _1: i64) i8,
};

extern fn root(_fns: Fns) anyopaque;

extern fn no_return() anyopaque;

@kassane kassane force-pushed the zig-bindgen branch 2 times, most recently from f6f4270 to 044a0a9 Compare May 14, 2022 16:22
@kassane kassane mentioned this pull request May 16, 2022
3 tasks
@kassane
Copy link
Author

kassane commented Jun 26, 2022

If it is possible to review it, thank you. @emilio
I haven't solved the comptime issue yet, due to the fact that I need to refactor ty.rs in the arguments part regarding the generic types.
Other than that, I think the basics already work.

- remove anonymous struct;
- fix union namespace;
- remove comptime global variables
@kassane kassane closed this Sep 29, 2024
@BratishkaErik
Copy link

Why did you close it?

@kassane
Copy link
Author

kassane commented Sep 29, 2024

Why did you close it?

Hi @BratishkaErik ,

This PR needs some refactoring due to the recent changes. I've closed it, although will not erase work already done.
I haven't had time to keep up with all the recent changes.

Edit: Any help will be appreciated!

@BratishkaErik
Copy link

This PR needs some refactoring due to the recent changes. I've closed it, although will not erase work already done.
I haven't had time to keep up with all the recent changes.

Edit: Any help will be appreciated!

Hi, thank you for explanation! I was asking because I thought you met some difficult obstacle in design of language itself and was worrying if somehow it's impossible to generate using cbindgen (silly thought yes, sorry, accent on cbindgen). I'm not very proficient in Rust so if someone want to continue this work, I'll be glad to help by testing.

* fn funtions
* extern struct
* none_argname
@kassane
Copy link
Author

kassane commented Sep 29, 2024

@BratishkaErik ,
new commit: kassane@d156169

My sugestion: run cargo run -- -l zig tests/rust/(filename).rs | zig fmt --check --stdin

@kassane
Copy link
Author

kassane commented Sep 29, 2024

I was asking because I thought you met some difficult obstacle in design of language

Obstacles always occur. Some points are difficult to see right away (such as the struct/enum/union correlation). Since the project is focused on c-like syntax, the zig syntax requires a bit of rework and this should not compromise other languages.

@BratishkaErik
Copy link

BratishkaErik commented Sep 29, 2024

@BratishkaErik , new commit: kassane@d156169

My sugestion: run cargo run -- -l zig tests/rust/(filename).rs | zig fmt --check --stdin

Running tests/rust/union.rs, zig fmt does not complain but compilation fails:

$ cargo run -- -l zig tests/rust/union.rs | zig fmt --check --stdin
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.02s
     Running `target/debug/cbindgen -l zig tests/rust/union.rs`

$ cargo run -- -l zig tests/rust/union.rs > union.zig
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.02s
     Running `target/debug/cbindgen -l zig tests/rust/union.rs`

$ zig fmt union.zig
union.zig

$ zig run union.zig
union.zig:15:1: error: non-extern function has no body
fn root(a: ?*Opaque, b: Normal, c: NormalWithZST) callconv(.C) void;
^~

union.zig:

const std = @import("std");

const Opaque = ?*anyopaque;

const Normal = union {
    x: i32,
    y: f32,
};

const NormalWithZST = union {
    x: i32,
    y: f32,
};

fn root(a: ?*Opaque, b: Normal, c: NormalWithZST) callconv(.C) void;

constant.rs -> constant.zig:

const std = @import("std");


pub const 10

pub const ':'

pub const '{'

pub const '\''

pub const '\t'

pub const '\n'

pub const U'\U00002764'

pub const U'\U00010083'

pub const 3.14

 ///
/// A single-line doc comment.
 ///
pub const 1

 ///
/// A
/// multi-line
/// doc
/// comment.
 ///
pub const -1

pub const 3

pub const 1

pub const ((0 << SHIFT) | XBOOL)

pub const (1 << (SHIFT | XBOOL))

pub const ()'A'

pub const ()()1

const Foo = extern struct {
  x: [FOO]i32,
};

fn root(x: Foo) callconv(.C) void;

$ zig fmt constant.zig           
constant.zig:4:11: error: expected 'an identifier', found 'a number literal'
pub const 10
          ^~

@kassane
Copy link
Author

kassane commented Sep 29, 2024

@BratishkaErik , make pull and try build again (union sample).

@BratishkaErik
Copy link

BratishkaErik commented Sep 29, 2024

@BratishkaErik , make pull and try build again (union sample).

Yep, it works now:

const std = @import("std");

const Opaque = ?*anyopaque;

const Normal = union {
    x: i32,
    y: f32,
};

const NormalWithZST = union {
    x: i32,
    y: f32,
};

extern fn root(a: ?*Opaque, b: Normal, c: NormalWithZST) callconv(.C) void;
$ zig run union.zig

/usr/lib64/zig/9999/lib/std/start.zig:607:46: error: root struct of file 'union' has no member named 'main'
    const ReturnType = @typeInfo(@TypeOf(root.main)).@"fn".return_type.?;
                                         ~~~~^~~~~
union.zig:1:1: note: struct declared here
const std = @import("std");
^~~~~
/usr/lib64/zig/9999/lib/std/start.zig:580:20: note: called from here
    return callMain();
           ~~~~~~~~^~
/usr/lib64/zig/9999/lib/std/start.zig:535:36: note: called from here
    std.posix.exit(callMainWithArgs(argc, argv, envp));
                   ~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~
referenced by:
    _start: /usr/lib64/zig/9999/lib/std/start.zig:426:40
    comptime: /usr/lib64/zig/9999/lib/std/start.zig:92:63
    2 reference(s) hidden; use '-freference-trace=4' to see all references

UPD: Was Opaque supposed to be = opaque {}? Kinda like how zig translate-c tries to do today, it seems like root function now get ?* ?*anyopaque argument...

zig translate-c does this:

#include <stdarg.h>
#include <stdbool.h>
#include <stdint.h>
#include <stdlib.h>

typedef struct Opaque Opaque;

typedef union Normal {
  int32_t x;
  float y;
} Normal;

typedef union NormalWithZST {
  int32_t x;
  float y;
} NormalWithZST;

void root(struct Opaque *a, union Normal b, union NormalWithZST c);
pub const struct_Opaque = opaque {};
pub const Opaque = struct_Opaque;
pub const union_Normal = extern union {
    x: i32,
    y: f32,
};
pub const Normal = union_Normal;
pub const union_NormalWithZST = extern union {
    x: i32,
    y: f32,
};
pub const NormalWithZST = union_NormalWithZST;
pub extern fn root(a: ?*struct_Opaque, b: union_Normal, c: union_NormalWithZST) void;

Normalized:

pub const Opaque = opaque {};

pub const Normal = extern union {
    x: i32,
    y: f32,
};

pub const NormalWithZST = extern union {
    x: i32,
    y: f32,
};

pub extern fn root(a: ?*Opaque, b: Normal, c: NormalWithZST) void;

@kassane
Copy link
Author

kassane commented Sep 29, 2024

@BratishkaErik , fixed constant semicolon. But need fix casts. try git pull again.

Based from clike.rs

Literal::Cast { ref ty, ref value } => {
out.write("(");
self.write_type(out, ty);
out.write(")");
self.write_literal(out, value);
}

@BratishkaErik
Copy link

BratishkaErik commented Sep 29, 2024

Same errors (union.zig did not change):

const std = @import("std");


pub const 10;

pub const ':';

pub const '{';

pub const '\'';

pub const '\t';

pub const '\n';

pub const U'\U00002764';

pub const U'\U00010083';

pub const 3.14;

 ///
/// A single-line doc comment.
 ///
pub const 1;

 ///
/// A
/// multi-line
/// doc
/// comment.
 ///
pub const -1;

pub const 3;

pub const 1;

pub const ((0 << SHIFT) | XBOOL);

pub const (1 << (SHIFT | XBOOL));

pub const ()'A';

pub const ()()1;

const Foo = extern struct {
  x: [FOO]i32,
};

extern fn root(x: Foo) callconv(.C) void;
constant.zig:4:11: error: expected 'an identifier', found 'a number literal'
pub const 10;
          ^~

@kassane
Copy link
Author

kassane commented Sep 29, 2024

@BratishkaErik, rebased (semicolon fix) and try again union & constants samples


C macros need bypass on zig.

@BratishkaErik
Copy link

@BratishkaErik, rebased (semicolon fix) and try again union & constants samples

There's missing equal sign between identifier and value:

constant.zig:4:15: error: expected ';' after declaration
pub const FOO 10;
              ^~

...after fixing I get:

constant.zig:16:20: error: expected ';' after declaration
pub const HEART = U'\U00002764';
                   ^~~~~~~~~~~~
const std = @import("std");


pub const FOO = 10;

pub const DELIMITER = ':';

pub const LEFTCURLY = '{';

pub const QUOTE = '\'';

pub const TAB = '\t';

pub const NEWLINE = '\n';

pub const HEART = U'\U00002764';

pub const EQUID = U'\U00010083';

pub const ZOM = 3.14;

 ///
/// A single-line doc comment.
 ///
pub const POS_ONE = 1;

 ///
/// A
/// multi-line
/// doc
/// comment.
 ///
pub const NEG_ONE = -1;

pub const SHIFT = 3;

pub const XBOOL = 1;

pub const XFALSE = ((0 << SHIFT) | XBOOL);

pub const XTRUE = (1 << (SHIFT | XBOOL));

pub const CAST = ()'A';

pub const DOUBLE_CAST = ()()1;

const Foo = extern struct {
  x: [FOO]i32,
};

extern fn root(x: Foo) callconv(.C) void;

@kassane
Copy link
Author

kassane commented Sep 29, 2024

@BratishkaErik , fixed now (missing '='). But "U'/U" need bypass.

zig translate-c (v0.14.0-dev) test:

pub const DELIMITER = ':';
pub const LEFTCURLY = '{';
pub const QUOTE = '\'';
pub const TAB = '\t';
pub const NEWLINE = '\n';
pub const HEART = @compileError("macro tokenizing failed: TODO unicode escape sequences");
// constants.c:18:9
pub const EQUID = @compileError("macro tokenizing failed: TODO unicode escape sequences");

syn::Lit::Char(ref value) => Ok(Literal::Expr(match value.value() as u32 {
0..=255 => format!("'{}'", value.value().escape_default()),
other_code => format!(r"U'\U{:08X}'", other_code),
})),

@BratishkaErik
Copy link

BratishkaErik commented Sep 29, 2024

For u/U I think logic should be similar to this, but IIUC this is wrong place to set this since it breaks other languages:

diff --git a/src/bindgen/ir/constant.rs b/src/bindgen/ir/constant.rs
index 4d11402..68a4aa4 100644
--- a/src/bindgen/ir/constant.rs
+++ b/src/bindgen/ir/constant.rs
@@ -344,7 +344,7 @@ impl Literal {
                     syn::Lit::Byte(ref value) => Ok(Literal::Expr(format!("{}", value.value()))),
                     syn::Lit::Char(ref value) => Ok(Literal::Expr(match value.value() as u32 {
                         0..=255 => format!("'{}'", value.value().escape_default()),
-                        other_code => format!(r"U'\U{:08X}'", other_code),
+                        other_code => format!(r"'\u{{{:08X}}}'", other_code),
                     })),
                     syn::Lit::Int(ref value) => {
                         let suffix = match value.suffix() {
pub const FOO = 10;

pub const DELIMITER = ':';

pub const LEFTCURLY = '{';

pub const QUOTE = '\'';

pub const TAB = '\t';

pub const NEWLINE = '\n';

pub const HEART = '\u{00002764}';

pub const EQUID = '\u{00010083}';

pub const ZOM = 3.14;

// ...

comptime {
    @import("std").testing.refAllDecls(@This());
    @compileError(std.fmt.comptimePrint("{u}, {u}", .{ HEART, EQUID }));
}
constant.zig:54:5: error: ❤, 𐂃
    @compileError(std.fmt.comptimePrint("{u}, {u}", .{ HEART, EQUID }));
    ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/usr/lib64/zig/9999/lib/std/start.zig:607:46: error: root struct of file 'constant' has no member named 'main'
    const ReturnType = @typeInfo(@TypeOf(root.main)).@"fn".return_type.?;
                                         ~~~~^~~~~
...

I have also tried to fix casts myself, but get no results

BTW this syntax is also supported, but maybe I got ranges wrong:

diff --git a/src/bindgen/ir/constant.rs b/src/bindgen/ir/constant.rs
index 4d11402..2accc0e 100644
--- a/src/bindgen/ir/constant.rs
+++ b/src/bindgen/ir/constant.rs
@@ -342,9 +342,9 @@ impl Literal {
             syn::Expr::Lit(syn::ExprLit { ref lit, .. }) => {
                 match lit {
                     syn::Lit::Byte(ref value) => Ok(Literal::Expr(format!("{}", value.value()))),
-                    syn::Lit::Char(ref value) => Ok(Literal::Expr(match value.value() as u32 {
-                        0..=255 => format!("'{}'", value.value().escape_default()),
-                        other_code => format!(r"U'\U{:08X}'", other_code),
+                    syn::Lit::Char(ref value) => Ok(Literal::Expr(match value.value() {
+                        '\0' | '\n' | '\r' | '\t' | '\\' | '\'' | '\"' => format!("'{}'", value.value().escape_default()),
+                        other_code => format!(r"'{}'", value.value()),
                     })),
                     syn::Lit::Int(ref value) => {
                         let suffix = match value.suffix() {
const std = @import("std");


pub const FOO = 10;

pub const DELIMITER = ':';

pub const LEFTCURLY = '{';

pub const QUOTE = '\'';

pub const TAB = '\t';

pub const NEWLINE = '\n';

pub const HEART = '❤';

pub const EQUID = '𐂃';

// ...

@kassane kassane reopened this Sep 29, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants