🧩 Introducing CuraEngine Plug-ins ~ Input is welcome

[MariMakes]

Cura Engine Plug-ins

Since the launch of the Cura Marketplace we've received around 80 different plug-ins for Cura. These have all been modifications limited to Cura's front-end and post-processing scripts. We are planning to expand the possible plug-in options.

We want to allow for plugins to extend/replace certain operations within CuraEngine. 🎉

We've been working on detangling the CuraEngine to lower the threshold for anyone to start working plug-ins that influence what actually happens when Cura is slicing. These plugins can influence specific steps in the slicing process like generating the Platform Adhesion, the Skin Generation, or recognizing the Support+Overhang areas.
In our first tests, we proved that these plugins can be written in a different languages (Python, Rust, Java etc.).

We predict that it's possible for the following plug-ins to be developed and loaded into Cura if when we unlock this possibility.

• Paint on support. This plugin also requires some front-end work where the model can be painted on. This plugin then generates overhang areas based on those areas that are marked to require support in the front end.
• Version number in print Modifies the skin area on layer-0 to have the version number or any custom text impressed on the bottom of the model.
• Texture Modifies the input model to have a texture embossed on the surface.
• Custom print instruction flavors Based on our abstract data type, it generates gcode for specific gcode-flavors or even other formats.

Two alternatives

For the last few weeks, we've been brainstorming on different ways to implement the option to write plug-ins for the CuraEngine. We found two potential alternatives on how to enable plug-ins to extend/replace certain operations within CuraEngine. We cannot promise that we are actually going to implement it like this, but unless we get different insights, these seem like the best options.

The main difference between the alternatives is that for alternative A we allow for more flexibility on the plugin side whereas alternative B is a bit more restrictive. This is due to the fixed input data for alternative B. With the experience of building an engine, we know what input data is expected for each step. For alternative B we only provide that input data. For alternative A we don’t provide any input data with a step, but instead broadcast more generic information at various points during slicing, and a plugin can choose to receive data from multiple origins and use that to generate data for a specific step. The downside of this is that it requires a bit more bookkeeping on the plugin side.

Alternative A:

Slicing involves certain sequential steps (think of slicing the model into outlines, generating the various infill-regions etc). Using the plugin system each of these steps can be overwritten, we refer to these steps as slots.

For each slot there are 3 operations, these are:

1. Generation: Generate the data for a specific slot, there can only be one plugin fulfilling this slot. The generate function does not have any input arguments. If the plugin needs input data for the generation step it needs to listen to broadcast slots from previous steps.
2. Modification: Modify the generated data, there can be multiple plugins fulfilling this slot. The modified plugin receives the data from the previous modify step (if the plugin is the first modify step it receives the data from the generation step). Then modification for that plugin is applied and returned. This is repeated until all modify-plugins for the slot are executed. As the plugins modify the data the order in which the plugins are executed matter and will be determined beforehand.
3. Broadcast: Send out the, possibly modified, data generated from this slot. The data received from a broadcast is guaranteed to be final data. There can be multiple plugins listening to the same broadcast slot. As all plugins listening to broadcast receive the same data the order in which the broadcasts are executed.
   Note: not all operations are implemented for all slots.

Alternative B:

Slicing involves specific sequential steps (think of slicing the model into outlines, generating the various infill-regions etc). Using the plugin system each of these steps can be overwritten, we refer to these steps as slots. For each, there is some input data that is relevant to produce the data required for that slot.

Examples of slots

Below a few examples of plugin slots are listed. If you are interested in other steps please let us know, we can try to provide more examples

Alternative A:

Model: Only the Modify and Broadcast operations exist for the model slot. The Modify slot could be used to for instance alter the model to have embossed text on the surface. Model is a triangle soup representation of the slice-model.

ModelService {
  Modify(Model) -> Model
  Broadcast(Model) -> ()
}

Generate areas: Generate the various infill areas for each layer. These areas include: skin, infill areas for the various infill densities, the inner wall areas, and the outer wall areas.

AreasService {
  Generate() -> Areas
  Modify(Map<ModelId, Areas>) -> Areas
  Broadcast(Areas) -> ()
}
struct Areas {
  outer_walls: Polygons,
  inner_walls: Polygons,
  infill_densities: [Polygons],
}

Skin: For each layer generate the skin-toolpaths

SkinService {
  Generate() -> Toolpaths
  Modify(Toolpaths) -> ModelId
  Broadcast(Toolpaths) -> ()
}

In order to generate data for this slot the plugin should probably listen to at least the settings and slice broadcast-slots.

Alternative B:

Support: Input is the and the overhang and the disallowed areas areas. Result will be the support areas for each layer

SupportService(settings: Settings, support_input_per_layer: [SupportInput]) -> [Polygons]

struct SupportInput {
  disallowed_areas: Polygons
  overhang_areas: Polygons
}

Infill areas: Input for this slot are the model outlines for each layer, then the various infill areas are generated

InfillAreasService(settings: Settings, outlines_per_layer: [Polygons]) -> [InfillAreas]

message InfillAreas {
  skin: Polygons
  infill: [Polygons] // infill areas for the various infill densities
  inner_wall: Polygons
  outer_wall: Polygons
}

Skin: Based on the skin-area generate toolpaths that fill the skin
SkinService(settings: Settings, skin_area_per_layer: [Polygons]) -> [Toolpaths]

How can you help?

We cannot make any promises on the actual implementation. But we do want to work together with our amazing plug-in developers and community members to create something that enables you to make all the amazing things you want.

You can help us by sharing your feedback on the proposal. Especially by providing answers to the following two questions:

1. Which of these alternatives has your preference, and why?

2. When reading through this proposal, what kind of potential for plugins jump to mind?

[MariMakes]

Some exciting changes are coming to the Cura and the Marketplace. 😊
What do you think?
@fieldOfView, @5axes, @Ghostkeeper, @nallath, @awiegel, @kartchnb, @jjgraphix, @Jeredian, @rgomezjnr, @Spanni26

Or @BagelOrb @ThomasRahm, @wawanbreton, @Piezoid, @ckvsoft @Cycov @richfelker, @smartavionics

I'm not trying to play favorites, but you jumped into my mind when I saw the proposal.
I'm sorry if I forgot anyone, if you see someone who might have some feedback, please tag them 😊

[5axes]

@kartchnb, @ChristopherHoffman

Concerning potential plugin

Possibility to define the print order of model by changing the print order sequence

[Jeredian]

Basically, both ideas are great, and can bring us a lot of new and useful.
In the end, it will all come down to where we want to store the logic of interaction between plugins more.

In option A - there will be more of the basic logic of working with the received data on the side of the final plugin developer.
This option will most likely be convenient to those who have been working with Cura for a long time and know the internal logic.
But on the other side we will have more tasks based on the interaction of plugins with each other, because the more flexibility we introduce into the data stream, the greater the chance of losing consistency.

I see only a few disadvantages here. It is a large amount of documentation to give a proper understanding of which parts of the data can be available and needed to the user. And the potential performance issues for this approach

In option B - the area of responsibility for the data consistency will be more with the Cura development team.
This option will be easier for those who are new to plugins, and needs less documentation. In this case, more frequent requests will be to expand Input data with a new fields, which are missing for the plugin in a particular step.

I would wonder how often it would be necessary to listen to data from any previous step, not just the last step.

Will we be able to choose the order of plugins execution within a step?

[nallath]

In case of the Modification plugins, yes; An order will need to be selected. If that wouldn't be possible, I think they would be pretty darn useless.
The modification plugins can be best compared to how the post-processing plugin works in the front-end right now.

[wawanbreton]

The basic idea of having the slicer splitted in plugins is really amazing... Now some thoughts about the options :

• Option A
  • [+] It seems to me that it provides more flexibility. You can have a plugin that does just a simple task, or multiple tasks altogether.
  • [+] Opens an option for multithreading executions of each step : as soon as the data for a step is ready, it can start processing, even if the "previous" one is not complete. Your linear workflow becomes a graph.
  • [-] With great power comes great responsibility : it will probably be harder for beginners to get in and understand how the whole thing works. But a good documentation should help with that.
• Option B
  • [+] Very easy to understand and write a hello world plugin
  • [+] Linear execution : makes debugging easier
  • [-] Possible plugins "types" have to be defined beforehand and can't be changed

Overall, I would prefer option A because it seems to open some possibilities that option B closes. If you start integrating this architecture, it will be a choice to be assumed for a few years, so you have to make sure that what you will do later is not limited. If I'm correct, option B is actually a subset of option A, so you can still have option B working by implementing option A 😃
However, it will require more effort in the beginning, if you want to make things easy enough :

• Documentation +++
• Maybe a graphical tool to display the actual processing flow (something like ROS graph or this) and be sure that all the required data is processed by someone.
• A very kind "data manager" object to register which data your plugin needs and send you a notification when it is ready for processing. If this is properly done, writing the plugin may be quite as easy as option B
• A way to record the data transferred from a step to an other one, so that you can replay a test set very quickly and develop your plugin by try-and-error. But maybe is already exists somewhere.

[wawanbreton]

By the way, one feature I would really love, is actually something you mentioned : I often add text to existing models, either with an outset or inset. Or sometimes with a second color. So I would love an integrated tool to add a text on a surface (flat or not !). Sometimes I also add icons, so a texture would also be fine.

An other very cool feature would be an auto-cut plugin : sometimes I print parts that are too big to be printed in one piece, so I have to split them and make proper joints. But maybe that would be more of a pre-processing step, not during actual slicing. Or maybe this is too project-dependant to make it automatic.

[MariMakes]

Autoparting would be so cool. I know more than one person worked on something like this but didn't follow through. I love this example of a school that changed from a physical to a digital 3D printing assignment during Covid. One of the deliverables was this cool plug-in.
You can see their Auto Parter Plug-in here.

[wawanbreton]

Waw, this plugin is really cool... Would have saved me a lot of time 😄
Why isn't it in the marketplace ? Is it up to the developer ?

[5axes]

This plugin was never submitted by the developers to the marketplace. They had (according to source, the idea of continuing the project, but it remained as is. I had made a small update to be compatible with versions 5.0 (https://github.com/5axes/AutoParter) But without continuing the project. In the meantime, Prusa has released a version of their Own Part function that makes this plugin obsolete for anyone looking for this kind of function.

[nallath]

Well, it doesn't quite make it obsolete as Cura doesn't have that feature yet. But yeah, it was a student project and they probably moved on to different classes / courses, so it never got any updates after that.

[fieldOfView]

I think the plugin system needs to be very flexible in what data the plugin has access to. So instead of having slots that come with a limited, fixed set of setting values, provide a way for the plugin to access the full settings relevant to the current mesh/meshgroup.

[Piezoid]

A lazy and generic version of SliceDataStorage could serve as an orchestration hub, where every transformation/view of the slice is computed and cached on demand. I'm thinking of an architecture reminiscent of query-based compilers. I think this could be achieved with a fixed marshalling protocol using few protobuf types like polygons, face lists, extrusion toolpaths; alongside a flexible query protocol that enables a well-rounded orchestration of the different stages.

I guess this is similar to alternative A, but with an immutable and pull-based management of the data flow.

[nallath]

That is the idea that we had with option A. So if you want all the settings in order to modify something, you would first subscribe to the setting broadcast and also subscribe to the step you want to modify.

[KimmoHop]

Could there be a wrapper of type A, which offers type B interface for type B plug-ins? Easy implementation for simple tasks and new developers, without sacrificing complexity generality when needed :)
Not sure, how the wrapper or type B developer would know, which broadcasts it would have to subscribe to :/

At least IMO (engine) development should become less "impossible" and while
a) having just type B might not have enough flexibility,
b) having just type A - if it is not done very carefully and correctly - might not be simple enough to attract developers.

How would these plugins alter / react with UI?

[nallath]

We are going to make example plugins regardless of the option that we will pick. We've really noticed that documentation, although valuable, tends to work a lot less well than examples (and obviously; the combination of both is best).

We don't have all the answers regarding UI and these plugins. The backend plugins will likely have to work together with a matching frontend plugin to add any required UI /UX elements

[wawanbreton]

We don't have all the answers regarding UI and these plugins. The backend plugins will likely have to work together with a matching frontend plugin to add any required UI /UX elements

I was wondering how the plugins are going to work from the end-user point of view. Usually, installing plugins is just "download and install" and then you have new buttons to make something cool. In this case, if you have multiple plugins, the end-user will have to chose how to mix them, right ? In case of alternative A, this could become very difficult...

[nallath]

We have identified the following problems so far:

1. Plugins that add a new option to an existing feature (for example: Add a new type of infill)
2. Modification plugins, as the order of these plugins can be important
3. Modification plugins, since you might want to run a single modification multiple times (eg; postprocessing)
4. Plugins that add new settings (also; Colisions in setting keys / upgrades/ etc)

Problem 1 will also lead to issues if you have a project file that you then open in a version of Cura that does not have that plugin.

But yeah, the idea is indeed that a backend plugin is always accompanied with a frontend plugin (also to ensure that the already existing plugin system in the frontend recognises them correctly and can deal with them). In some cases (problem 2 & 3) we can come up with a generalized interface. In other cases (1 & 4) we have to put a bit more thought into that.

I expect that for 1 we will need to add a required list of plugins to a project file so that we can notify that a dependency is needed. We already have a system that does that to a certain extent for material files downloaded from the marketplace. Problem 4 will mean that we need to add a system by which the front-end part of the plugin can register a new setting. That system would then need to ensure that it's a unique setting name (probably by prefixing the setting name with the plugin name to create a sort of namespace to prevent collisions).

[fieldOfView]

How are you going to handle security/validation of plugins for the Marketplace if they contain executable backend plugins (for multiple platforms)?

[fieldOfView]

Will the gcode postprocessing slot be called before or after print time estimation? It could be interesting to see the results of postprocessed gcode in the print time estimation. Come to think of it, it could also be interesting to replace the print time estimation altogether with something better.

[jellespijker]

The postprocessing slot as it is now, v0.0.x-alpha.0 will simply send over the gcode per layer. I'm looking into sending over the gcode as a single call.

This is all very naively, simple and quick. Mostly done to check the rest of the plugin plumbing.

Alternatively there will also come a future toolpath-postprocessing slot, which will send over toolpaths and commands before they are actual gcode words.

I don't think that we will send the print time estimation with either of the above mentioned slots. But we might create a broadcast slot for broadcasting the estimations such that plugins hooked into such a postprocessing slot can get that information anyway.

not sure how to approach this yet, but I do see value in this. Will discuss this tomorrow at the Cura&Cookies meeting.

[jellespijker]

The way we now estimate the print time is witch each individual GCode command.

See this example:
https://github.com/Ultimaker/CuraEngine/blob/ef46801bffb289cdcc3ec58b6dfff78b2889644d/src/gcodeExport.cpp#L884

[fieldOfView]

Looking at Ultimaker/CuraEngine_plugin_postprocess#1 (wow, that's a lot of boilerplate for a "simple" gcode postprocessing plugin), I can't help but wonder how this will scale to a situation where a user has multiple plugins installed. Will all of those be executables that launch with Cura, or are plugin authors supposed to launch the plugin executable before a slice and then close it immediately after?

How will port numbers be handled? Should a plugin just keep trying open a port until it finds a port that does not conflict (similar to how the Curaengine backend does it now in case of a conflict)?

[jellespijker]

The boiler plate is less in Python.
We are also, looking into make a C++ dependency which will abstract this away, such that a user can just provide a simple lambda or other function.

for now this is mostly here, in this state, such that QA can test how it behaves in the engine

[jellespijker]

for plugins in the frontend we have created a new type backend_plugin ports are assigned automatically. Probably will need some checking on availability in the future.

You can see the changes in the front end in this branch https://github.com/Ultimaker/Cura/tree/CURA-10475_engineplugin

and I'm currently working on a very basic front end plugin for postprocessing here:
https://github.com/Ultimaker/CuraEngine_plugin_postprocess/tree/CURA-10806_postprocess_gcode_cura_plugin/CuraEnginePostprocess

[teunko]

For a project I'm looking to gain more granular control over filaments with temperature dependent foaming capabilities. These filaments contain particles which expand to varying degrees based on the temperature they are exposed to (in this case a range of 170-230C). There are many interesting use cases in research where local differing degrees of foaming intensity within a geometry is desired; lightweight structural optimizations, local fluid/gas permeability, compliant mechanisms, local hardness control in TPU, promote cell growth in specific regions, density control for imaging phantoms etc. etc. For this to be fully leveraged there would need to be a full spatial control of extrusion temperature.

Unfortunately, there are no suitable workflows for realizing these use cases. As someone who is specializing in material science I'm not too familiar with writing software. But I just want to throw this idea out here to maybe get the ball rolling!

For now, just assuming a fully solid print, I think there'd be two possibilities of tackling this. Either changing the hotend temperature in real time during extrusion of the toolpath, or segmenting each layer into 'temperature regions' where every temperature step is printed separately until the layer is fully printed. The former might not be feasible in practice, since I doubt the hotend is able to change its temperature so quickly during extrusion.

Then there's the fronted part on configuring which regions of the geometry should have which temperature. In a simple form you might be able to segment a geometry into sections in CAD before you generate the STL file, and have this be split into different regions assigned with different temperature in the slicer? (I think SuperSlicer can do something like this for multi material printing). On the more extreme part of the spectrum it would be cool to be able to import files for very granular temperature changes; think of converting a bicycle seat or foot sole measurement pressure map into corresponding temperature regions in the geometry for controlling the hardness. Or, currently I'm playing with structural optimization where the geometry isn't optimized based on taking away volume (like topology optimization), but rather into assigning temperature regions (which because of the foaming corresponds to different mechanical properties and density) for creating an optimal lightweight structure based on load conditions. A simulation likes this generates a tetrahedral mesh with a a specific temperature for each element of the mesh. I wonder if something like this could be imported into a slicer and generate a .gcode that takes these temperature regions into account?

Anyway, a lot of spit balling and a lack of knowledge on how to implement this. But seeing the news of CuraEngine plug-ins there might be some possibilities here? I'd love to hear feedback or talk further, or a push in the right direction on how to implement this would be really appreciated! Thanks!

[teunko]

Thanks for the tip! Eager to play around with this and see if it can be useful. However, I can't seem to find it in the settings; I'm on 5.5.0 and have enabled all settings but it does not show in in the experimental tab.

[5axes]

Oup's sorry, my version of Cura is the version 4.20 of @smartavionics (https://github.com/smartavionics/Cura) and I sometimes forget that this version's options don't exist in the official version (which is always a shame).

Note that the @smartavionics version can also integrate print speed variations to give the nozzle time to change temperature.

[teunko]

No worries! I'm trying it now. So if I understand correctly this parameter is supposed to be used on the 'Per Model Settings' tab? I'd need to have separate mesh entities to change temperatures locally; it will become quite tedious if there are a lot of temperature steps and regions, it's a start though I guess!

[5axes]

Yes you can use with the 'Per Model Settings' if you have different meshes, or via a Modifier 'Modify settings for overlaps' . Of course, this parameter is limited, especially for restricted use and for testing machine/material behavior in your case. But the advantage and possibility of using the parameters : mesh_temperature_dawdle_time and mesh_temperature_dawdle_speed which does not exist when using a multi material printing as you suggested

[teunko]

I noticed those parameters and you're right they seem really handy and are somewhat on the right track for allowing local temperature changes. Thanks a lot! I will see if I can test around with these settings.

[h3rm]

i had problems slicing the bottom part of https://angrygriffin.net/product/3d-printable-stl-token-dice-and-card-holder-for-warcry-heart-of-ghur with as less material as possible but ensure that the walls are not "too weak". It took me a long time working with the infill settings, support blockers and per model settings - the result was that i needed (afair) around 240g filament after many hours of playing around with all settings instead of ~350g with normal ~12% zigzag infill (3 layer raft included) - that was a frustrating experience ^^

so it would be cool do improve a few things for the infill and support settings. My ideas:

1. change infill density in combination with settings like gradual infill at defined metric height points easier with an extra interface. I know it already works with the per model settings or maybe - depending on the modell - gradual infill but it's a sometimes a lot of work to do it manually. Maybe with some input fields for the infill settings, a + symbol to add a new row/change and a copy function so that only a few parameters have to be change for the next row/change of the settings
2. a new function withing the support and infill section like "use support instead of infill" (within the object)
3. infill that is similar to the existing VERY good tree supports (i use it with 0% density in most of my prints - the supports are great) because the function "infill support" doesn't work well when you have eg. an object that look like the base/bottom of the design mentioned above
4. (nearly similar to 3. but not only a new infill pattern) a "conical infill" function that automatically detects where the infill is "really essentially needed" to support only the top layers of an object which has more than one top layer in different heights. Maybe, if it's possible: it would be cool to also combine this function with the global infill settings. Eg. like this:
   -> 6% (global) gyroid/zigzag/etc. infill that makes the object genereally a bit strong -> in combination with "conical [tree] infill" that only support the tops in different heights
   4.1. together with "conical infill" it would be nice to add a function to the walls like "infill support next to the walls". A little bit similar to "alternate wall" but for each layer and with all infill options for a defined number or percentage of "vertical inner wall infill support") to make sure that the - in my use case - nearly unsupported walls are supported horizontally by infill... this would be really great to reduce material costs

just my thoughts and sorry for my bad english...! if you have questions feel free to ask (maybe i will use a translator for the answer... :) ) thx!