Publish GirTool as a standalone NuGet tool

[adamreeve]

The project README says that one of the features is "Allows 3rd party developers to write bindings for other GObject-based libraries." One thing that would make this project more usable for 3rd parties is publishing the GirTool application as a NuGet tool so it can be used without needing to check out the source code for this repository.

In addition (and maybe this should be a separate issue), it would be great if 3rd party libraries could depend on the standard GObject libraries without having to include copies of their .gir files (eg. at least GLib, GObject, Gio and GModule, but maybe anything that gir.core provides bindings for?).

I'm imagining that the GirTool project could include these .gir files as embedded resources and search in the resources if a .gir file isn't found in the search path, or maybe there should be an extra argument to explicitly set which libraries should use the internal .gir files. Then the 3rd party project would need to depend on the corresponding GirCore NuGet packages.

[badcel]

Thank you for the feedback. This was actually the goal in the beginning.

As no requests in this direction came up I think it is not very important currently. I think if there are requests for public widely used libraries it would probably make sense to include them in this project to have some kind of "GirCore" eco system to avoid fragmentation.

So this would only be needed for custom, probably non open source code.

Perhaps it would be enough to update the readme and remove the passage for now?

[adamreeve]

To add a bit more context, I'd like to use gir.core to create bindings for the Apache Arrow library. This is an open source project and includes implementations of the Arrow standard in multiple languages, including an existing C# implementation and a GObject based C library that live in the same repository. The GObject library includes more functionality than the C# library so I'd like to create .NET bindings to the GObject library to provide this additional functionality to .NET users.

I don't think it would make sense for Arrow bindings to live in this gir.core repository given that the Arrow repository already contains the .NET library and GObject library as well as other implementations, and I'd also like to provide a higher level layer on top of the GObject based binding that would live in the Arrow repository too.

I'll admit that this scenario might be fairly unique but it would be a shame if the GirCore tooling can only be used for projects within the GirCore project as it does seem to be designed to be usable outside of GTK etc, unlike the GtkSharp project which is much more awkward to use for 3rd parties. I can understand this would add additional work and maintenance burden but it might also reduce future maintenance work by reducing the number of libraries that need to have bindings in GirCore itself. I can probably also help contribute to this if you're open to the idea.

[badcel]

Ah this is an interesting use case.

Out of curiosity: Can you provide a link to the GObject code which is part of the Arrow project? I did not find it at a first glance.

You are right, I think your use case is not suited to be included in the regular "GirCore ecosystem".

In general I'm open to contributions to create the generator as a dotnet tool and distribute it via Nuget so it helps your use case.

The generator itself is a bit dated and could probably be made a bit more versatile to better suit your needs in addition to the current supported feature set.

Before you start I would ask you to let me know what you want to change before you do it, so i can get a feeling if your changes would align with my expectations to avoid wasted work.

Perhaps you could even provide a written down description of your preferred commandline interface of the tool?

In general the PRs should be as minimal as possible so it develops incrementally into your desired direction.

To reach me we could comminicate via github discussions or matrix. Whatever suits your needs better.

[adamreeve]

Out of curiosity: Can you provide a link to the GObject code which is part of the Arrow project? I did not find it at a first glance.

Sure, it's in https://github.com/apache/arrow/tree/main/c_glib, there's a core "arrow-glib" library there as well as additional separate libraries that build on this. These are all implemented as wrappers around the Arrow C++ library. If you're interested I also have a very rough draft PR open with my attempt so far at building C# bindings: apache/arrow#41886

Before you start I would ask you to let me know what you want to change before you do it, so i can get a feeling if your changes would align with my expectations to avoid wasted work.

Yes absolutely, I will make an issue to describe anything I'd like to implement before doing any work. For full transparency I should say I'm not yet 100% sure whether I will continue with this project given it has turned out to be a lot more work than I initially anticipated, but I will keep you updated.

Perhaps you could even provide a written down description of your preferred commandline interface of the tool?

👍 I haven't thought too much about this yet, the existing interface seems pretty usable but I can imagine I might want a way to specify which libraries to use as dependencies from gir.core rather than providing gir files, and it might be more flexible to be able to generate bindings for one library at a time (eg. generate arrow-dataset-glib bindings without also generating arrow-glib bindings at the same time). I will make issues for any proposed command line interface change.

In general the PRs should be as minimal as possible so it develops incrementally into your desired direction.

👍

To reach me we could comminicate via github discussions or matrix. Whatever suits your needs better.

Thanks, I've joined the matrix channel although haven't used matrix much so will likely stick with GitHub.

[adamreeve]

Hi @badcel. I've been busy with other work but am now keen to get back to making gir.core usable for generating Apache Arrow bindings. I think for now I'm quite happy with the command line interface of GirTool.

For a first step, I'd like to make GirTool more suitable for this use case by not generating output code for repositories that aren't included in the list of input repository files (eg. don't generate code for GObject/GLib etc as they will be used via package references). I need to check whether this will break any existing uses though, maybe this should be configurable using a command line argument. Then I'll look into being able to load the repository files for these from internal resources rather than needing to include them in the search path directories.

I'll keep this issue open for eventually publishing GirTool as a NuGet tool and will make new issues for this other work. Does that sound like a reasonable plan?

[badcel]

Hi, I'm not 100% certain but I think you can't just omit one GirFile (e.g. GObject) from the input.

For example if a method from a library returns a class / record / alias from a library you don't want to generate the generator can't look up which kind of type it actually is and can't render the appropriate code. Each of those types need to be handled differently.

But you can verify this if you want to.

Perhaps another idea would be to have a list of GirFiles to generate and another list which is just there to help lookup information.

[adamreeve]

Yeah I'm thinking I would need to still load these all as inputs, I'd just want to skip the final code generation step that happens at

gir.core/src/Generation/GirTool/GenerateCommand.cs

Line 35 in 7545f0f

Parallel.ForEach(namespaces, x => PlatformGenerator.Generate(x, output));

. Given that happens in parallel I'm fairly confident that there isn't a dependency on doing the generation between different namespaces. I'll make a new issue to continue the discussion.

[badcel]

Another thought: can't you just create a Gir file from your custom GObject/GLib and generate the whole C# stack without omitting any Gir files?

Because if you don't have the Gir.core GObject.Object base class you will probably run into trouble, too.

Edit:
Another important class is the GObject.Value class. The generator assumes certain methods to be available which probably is not the case for a custom base project.

There are more assumptions build in, in regard to type registration which probably won't fit too.

[adamreeve]

I don't want to omit any files from the inputs or the namespaces that get loaded, I need them to be able to know how to generate code to interact with the other libraries as you say. The code generator definitely needs to know about GObject.Object etc. I just don't want to do the final step of writing out the C# code for all the libraries because I'll be referencing some of them from NuGet packages rather than using references to local projects.

It's not a big blocker, I can just ignore the generated code or delete it after code generation, but it seems like nicer behaviour and less surprising to not generate extra code that wasn't asked for.

[badcel]

I would strongly suggest that you try this manual process before implementing. If you reference libraries which are not generated from the GirCore generator all the internal APIs are probably missing.

I would even think that it is easier and more maintainable to just let GirCore generate all your C# code. Otherwise you need to always follow up on any internal API change.

The generated code is written in a way that it should be extensible on the public api level. But the internal structure is pretty coupled.

If your custom code is annotated in a way g-ir-scanner understands than it is probably pretty easy to create bindings for it.

[adamreeve]

If you reference libraries which are not generated from the GirCore generator all the internal APIs are probably missing.

I'm not really following what you're saying here, I don't think this is what I'm proposing to do, and I don't think I've managed to clearly explain what I want to do. I will still be referencing the libraries generated by GirCore, I just want to reference the published NuGet packages for things like GirCore.GObject rather than create local projects to reference. If I use a version of GirTool from the same commit as the published NuGet package then the generated code for my library should be compatible with these packages.

At the moment I need to do something like this to generate bindings for the Arrow library:

• Create separate linux/macos/windows directories to hold gir files
• Copy Arrow-1.0.gir files into these directories
• Copy the platform specific gir files for GObject-2.0, GLib-2.0, Gio-2.0 and GModule-2.0 from the gir-files repository into these directories
• Run GirTool with something like: GirTool generate Arrow-1.0.gir --output $OUTPUT_DIR --search-path-linux=$LINUX_DIR ...
• Delete all the generated code for GObject-2.0, GLib-2.0, Gio-2.0, GModule-2.0 from my output directory. I don't need this code as I've created .csproj files that reference these packges from NuGet, and I didn't list these as inputs to the generate command

All I'm proposing is removing the need for that last step of deleting the generated code files that weren't asked for. GirTool will still need to load the gir files for GObject-2.0 etc and generate an internal model of these libraries to know how to generate code to use them.

It might also be nice if GirTool would know about these standard libraries and store their repository files internally so I don't need my own git submodule of gir-files and can skip that third step, but that's a separate thing.

[badcel]

Ah sorry I was somehow focusing on the thought that you have a forked version of GObject and want to actually use it as a new base library.

Yes it would be very nice if the generator would store the resources internally if used as a nuget tool.