Prioritize system libraries on linux SDL backend (fix build-related issues on linux) #160
Conversation
This is not a good idea. Anyone using Arc on Linux - most notably, me - will now be forced to sit through a (mostly irrelevant) native compilation phase every time they clone or clean the project. I don't know how good Gradle is at detecting whether the natives task is up to date, either; it's highly unlikely to happen only once, based on my experience. What's more, if someone wants to make a PR for Arc - and very few involve touching the SDL backend - they'll now need to install various packages needed for compiling these native libraries every time they wish to do so.
This is an even worse idea. Many systems don't have SDL2 installed, which is why it's bundled. It is not reasonable to expect people to install packages to play a game. |
I assume that by "mostly irrelevant", you mean that compiling the natives is an irrelevant and thus not worth it. Regardless, i would strongly argue that compiling at build time and per-system is not irrelevant at all. In fact, personally, mindustry wasn't running at all for me until this PR, neither on X11/Xwayland nor native wayland, regardless of what version of mindustry I used; I would get some variant of a The benefits you gain from using system libraries and compiling the library per-system (e.g. using native compilation flags) definitely outweighs any extra compilation time (which is most likely less than 30 seconds on any modern system anyway). Not to mention, it's only compiled if the library file doesn't exist in the first place, so you only have to do it per clone or clean like you said.
Like I just mentioned, it seems gradle will only compile Arc if the
This was actually an issue I greatly considered when writing this PR. This is why specifically the Regardless, the only packages a developer would theoretically need is
I would think this isn't the case; Linux's library ecosystem isn't like that of Windows in where all third part library files need to be bundled separately and with the application itself. SDL and glew are a common dependency of many packages regardless of the package manager, so I feel it is quite likely that the average linux user would already have SDL and glew installed. It's important to note that this only concerns users who directly download binary versions of linux mindustry (e.g. from github releases or itch.io) The versions of mindustry from a package manager (e.g. AUR or flathub) could just have SDL and glew listed as a dependency (and it would be very easy to do so) so that anyone installing mindustry from a package manager will automatically have SDL and glew installed. Steam on linux (or more specifically, the steam runtime) comes with SDL, so anyone running mindustry through steam couldn't have this issue either. Thus, this issue can only even happen in the first place to users who use pre-compiled binary versions of mindustry. I personally think that this isn't really much of an issue, and I believe it's fine to just always use the system libraries on linux. However, I can understand if you're still concerned about users not having the required libraries on their system to run the game, and although I think it's probably unnecessary and causes more hassle, there are a few solutions for said users who download binary distributions of mindustry. The easiest solution would be to leave package manager versions of mindustry (i.e. when mindustry is compiled from source) to be unchanged after this PR, but in binary distributions of mindustry (i.e. distributions that come with Mindustry.jar pre-compiled; e.g. the github releases or itch.io versions) add a My point is that there are many solutions to this concern, but in my opinion are not worth the extra hassle. It may be better to just always use the system libraries on linux, since I think it's unlikely that a linux user will even meet the conditions to run into this issue specifically. Overall, this isn't really a matter of debate; Using the system libraries is objectively better overall than using bundled library files and a pre-compiled version of glew, especially in terms of mindustry. I believe the benefits of using system libraries greatly outweighs any drawbacks such as having to deal with linux users maybe not having sdl or glew on their systems or any extra compilation. |
You can't simply call something "objectively" better when it has pros and cons. Saying "it's not a matter of debate" is also a bit presumptuous when there is, in fact, quite a bit to debate here.
Strangely enough, I don't have glew installed on my current PC, despite using it for many months for game development. There simply has been no reason for me to install it, since the only application that would need it is Mindustry, and the native builds run through CI.
Okay, just for the sake of argument, let's say we go your route, and skip the "unnecessary" step of bundling libraries. Let's suppose a typical user (like me!) without GLEW or SDL installed downloads the itch version of the game, and tries to run it. What happens? Probably nothing. The game crashes, because it can't find the libraries, and the crash dialog probably won't even show up, because that's part of SDL. What do you expect the user to do in this (common) situation? Check the program output by running it through the terminal? Try to figure out what the error means? File a Github issue?
But how is this unlikely? I have no reason to install SDL2 (and especially) GLEW unless I am specifically building the Mindustry natives. Is it really reasonable to expect every Mindustry player on Linux to have these? Let me lay out the pros and cons, as I see them. Pros:
Cons:
Factoring in the time, effort, and potential issues it could cause, this doesn't seem like the "objectively better" solution to me at all. To be clear, I'm not necessarily against loading system libraries as the first-tried option when the game runs. But I am against removing the current libraries from the repo, and making them build implicitly in any way. |
I apologize if I sounded rude. I probably should've worded that better. What I mean is that in principal, it is better to use system libraries on Linux overall - remember that the native library ecosystem on Linux is quite different from that on Windows. It's just a matter of principal and practice, not just my own opinion - there are numerous benefits to using system libraries in general - hence why I said that in isolation, this doesn't have much of a debate as it's outside of the scope of this issue. What I think should be discussed is how to go about implementing this within mindustry/Arc as they stand, as I believe that is the main issue we are facing. That is why I gave a couple of solutions to how to implement this, but didn't really explain why system libraries should be used in the first place. Hopefully that all makes sense, but to help clarify even further, let me try to highlight off the top of my head some of the isolated theoretical benefits to using system libraries: The main and critical concept behind using system libraries is that you give control of which library the program uses to the user/system. This has a cascade of benefits, including but not limited to (I'll explain these further if I need to, but I feel like I'll have explained enough for now):
In other words, by linking to shared libraries that are included within the same directory (or in this case the mindustry jar), I think you are partially defeating the purpose of using shared libraries in the first place; You are essentially inheriting the downsides of linking to static libraries without the actual benefits of doing so, since you're linking to shared ones but in the same directory. You may even also lose some of the benefits of linking to shared libraries (e.g. larger application size, no performance improvements, no reduction in memory footprint, etc.) since they're not truly shared anymore. Now, keeping that in mind moving forward:
I think the problem with why you don't see using system libraries as a positive solution is mainly because you didn't factor in the general benefits (see above) of using system libraries. I think I can also clarify some of the pros/cons you listed:
Like I said, an advantage of using system libraries is that you eliminate compatibility and configuration issues between the library and the system. My misconfigured GLEW was just a specific but perfect example to highlight this problem. Regardless of if the title is incorrect, the fact that this issue hasn't been reported even more highlights the problem even more in my opinion: My specific issue was unique to my rare setup (specifically, the exceedingly rare environment of running linux mindustry under a wayland DE (GNOME) and with NVIDIA proprietary drivers), which highlights that it's unlikely you can accommodate every different system configuration with how Arc is currently setup.
That's not entirely correct. This PR inherently adds wayland support/capabilities on all Linux platforms, 100%. Since you're linking to the system SDL, anyone with a properly configured SDL can just run any version of mindustry on wayland regardless, without any actual change to Mindustry's or Arc's source code.
As I mentioned earlier, the only time you need to have these packages install is supposed to be when you specifically need to generate Arc's library file (i.e. the However, it's critical to note that you already require SDL2 to be installed anyway for this. You use The easiest solution for developers needing to install GLEW is to just have build instructions and a list of dependencies listed in Arc's README (which is basically empty right now). But even regardless of that, a user needing SDL and GLEW to run mindustry is quite different from a developer needing them to build mindustry/Arc. Hence, I think this is it's own separate issue from the main issue at hand here; Not to mention, the best course of action for this development issue depends on how we implement linking to system libraries. Thus, I think we should figure this out after we come up with a solution for the current issue. Here are some further comments on this, which may not be relevant (meaning they can be ignored), and don't factor in what I just said about this issue so farI will quickly say that conceptually, I personally don't find it unreasonable for a developer to have to install libraries on their system; although you may not have seen it often in java projects, but I have seen it myself with java projects that leverage jni, and it is extremely common in many other C family languages (e.g. C#, C, C++, rust) to have library dependencies (especially when building) and thus require developers to install said dependencies on their own (via their package manager, vcpkg, conan, cargo, nuget, or whatever). That may not seem relevant, but it's important to note that you're using native C/C++ libraries (which has a significantly different ecosystem than java libraries), so I believe it applies here. Thus, I think the best solution is to just list SDL and GLEW as a build dependency in gradle if need be (so in order words, *developers* trying to build the arc library file will be notified that they need SDL and GLEW on their system to do so). The opposite alternative is to go through the process of trying to find and link to system libraries in the gradle build *first* and then use a bundled or pre-compiled version of GLEW and SDL if it can't find them in the system. This might make the CI simpler or unchanged, but this obviously requires a lot more work in the gradle build; I personally don't think it's worth the hassle just so that developers can avoid installing GLEW and SDL on their system. But of course, what solution to go with is your decision in the end.
With the previously proposed solution of bundling GLEW and SDL on binary distributions, this isn't true. The library loading code does not need to be touched/changed at all, as you can see in the commit. By simply changing the gradle build and removing the bundled GLEW and SDL, it will load the system SDL and GLEW. Hence:
Is true for this specific solution. However, this is only applicable on solutions in which you bundle SDL and GLEW on specific distributions and prioritize loading the bundled versions over the system ones. If that is not the case, then this issue is not applicable, and the previous one is instead. In other words, these two previous issues are mutually exclusive (at least with the feasible solutions I can think of) depending on which solution you choose. I'll go into more detail about this below.
I actually think the opposite is true. As I mentioned at the top, using system libraries can actually lower the chance of inter-library issues. There is much less of a risk of version incompatibilities when using system libraries, and subsequently a much higher risk when using bundled libraries (but notably, regardless of what I think, there is some risk either way). This is more theoretical, but think about what happens if GLEW gets updated or changed (which you manually update in Arc) and, since you haven't updated SDL for everyone, it breaks compatibility with the specific version of SDL you bundled (maybe even not necessarily for everyone, maybe just on certain systems/distros which you personally can't account for). Now suddenly, mindustry is broken for the targeted group of people; You would have to go in and either downgrade GLEW or update SDL and then make sure they're compatible. Hopefully this highlights how much of a hassle it is for you to personally have to manage SDL and GLEW and similarly manually have to check compatibility issues and whatnot. By giving control to the user, you take this load off yourself. Instead of having to worry about SDL and GLEW yourself in every way manually, you can give control to the user and hence they can chose what version/configuration they want to use. If a certain system library isn't working, they can just switch versions or downgrade. Speaking of downgrade, think about the difference between you wanting to downgrade or change out the SDL mindustry uses versus the average user wanting to the same thing. If you want to downgrade, you can either just change the library file in the git clone and build Arc (i.e. the same way any other developer would), or push a new version of the library file to upstream and change it for everyone. But what is the average user going to do? Let's say they are having similar compatibility issues due to the SDL version. I find it very unreasonable for them to understand that they need to clone both Arc and Mindustry, figure out how to change out the SDL library file within that clone, and then know gradle well enough to know how to build Arc's library file and then build mindustry, The only solution that leaves is for them to replace or delete the SDL library file in Mindustry's desktop jar, and they have to (most likely manually) do this every time they update Mindustry. Hence, while it may be less likely that they need to downgrade their SDL when using a bundled version of it, they don't have much control to downgrade it in the first place; You're essentially the one in control of what SDL they use. In contrast, with system libraries, there may be a higher risk of specifically using a version of SDL that's incompatibility to the one Arc links to, but in that case they are in control of the SDL and thus have a much easier time changing versions. Also, critically, when using bundled library there is an (arguably greater) risk of issues from linking to an that's misconfigured or incompatible for that system - except now you can't easily change the version of the SDL because it's bundled within mindustry, as stated previously.
This definitely is a valid concern, but I think it's only applicable to graphical-related issues specifically on Linux. So, hopefully the SDL version won't be needed very often, since this probably only applies to a small subset of issues. With all that clarification, I'll lay out the pros and cons of using system libraries with Arc. Pros:
Cons:
Now for the issue at hand:
I should've worded my thoughts about this better. What I meant is that the subset of users specifically playing mindustry on Linux, without SDL and GLEW installed, and downloading specifically a binary distribution may not be very significant. Therefore (and this is what I was trying to say), I think it is worth considering/weighing whether it is worth the hassle and extra complexity to add CI code that injects pre-compiled libraries into the resulting binary, and thus choose one solution or the other based on that. I was personally weighing in favor of not adding the CI code, and just making sure that SDL and GLEW are runtime dependencies of Mindustry on every distribution platform possible.
You are correct of course; It is unreasonable to expect the average to troubleshoot a missing SDL or GLEW on their system. In this case, they are essentially out luck. Like I said, the point I was trying to drive across was to question if it was worth the CI hassle for such a possibly insignificant subset of users (which could be mitigated/shrunk even further with proper dependency management). But, this isn't too relevant now, so I digress. Of course, like I've mentioned previously, it's ultimately your decision which solution to go with.
This is honestly the best solution in my opinion: linking to the system libraries first and only falling back to the bundled ones if the system ones can't be found. It should be a simple change, which I can just add to this PR. This should be possible with just a Not to mention, right before you stated you weren't against loading system libraries first, you said (emphasis mine):
Which (to me at least) implied that you are strongly against changing the library code. Regardless, I still do personally believe this is the best solution. Let me breakdown the solutions (to the issue of how to implement system library linking on top of this PR/commit I've made) that we've come up with so far, on a basic level:
Note that I only listed pros and cons that are relative to the other solutions. In other words, these pros and cons are on top of the overall pros and cons of Arc linking to system libraries listed above. Again, I think overall, linking to system GLEW and SDL first and then linking to bundled if GLEW and SDL aren't found on the system is subjectively the best solution. As I've explained, I believe the highest priority is to have the option to link to system libraries and for that to be the default; Thus, the other two solutions (and probably any other solutions we haven't considered yet) can be considered valid as well. I'm sorry for the incredibly long response. I wanted to respond to every point you made and try to clarify my ideas better. Hopefully I didn't miss anything. |
Why do you assume this? I, personally, am one of those people. Anyone who uses the bleeding edge builds for Mindustry is also one of those people. I think you missed my point earlier about GLEW and SDL installations, so I just want to make it clear: There is no reason to assume anyone has GLEW and SDL installed on their system by default. I, as a developer, do not have GLEW installed! There are 87,410 downloads of the latest version of Mindustry on itch.io for Linux, as of this writing. This does not include those who use the bleeding edge/Github jar files. This also does not include people who downloaded earlier versions, of which there are many. You have absolutely no reason to believe all these people have the right libraries installed. You are asking a "a possibly insignificant subset of users" (87 thousand people) to just download the game and hope (with absolutely no data) that they have GLEW and SDL installed! This is complete madness! The fact that you even suggested this as a possible course of action is utterly baffling to me. "This guy on Github made a PR to fix his niche problem so it's not my fault your game is crashing" is not something anyone cares about when a new update suddenly broke their game. Telling someone to install packages to fix their problems would get old very quickly.
This simply isn't possible, nor should it be necessary.
These are all essentially different variations of "it might fix some problems for someone", but I've yet to see any aside from the specific one you've pointed out in this issue, which, again, is so incredibly rare that it has taken years for anyone to bring it up. These benefits don't seem to be relevant if they don't actually fix any problems.
True, but only by 1.5 MB unzipped, only when distributed through a package manager, and only on top of Mindustry's 60+ MB of code and assets. In this day and age, this is completely insignificant, especially for a game.
In my experience, Overall, I still think these benefits are still mostly irrelevant.
Yes, but this PR in its current state will absolutely not be used, and I was not referring to it when I made my claim. By linking to the system and only system libraries, you are asking (as I mentioned above) 87 thousand people to go install two libraries, or be faced with a game that doesn't start with no error popup whatsoever. My point was that this PR would not fix wayland support if we went the initial route of linking to pre-bundled (and only pre-bundled) shared libraries for binary distributions.
Any changes you make to Arc need to be tested, and to be tested, they need to be run on desktop, and to be run on desktop, Arc needs the desktop natives. If I clone the repo and try to do anything useful with it at all, I will need to compile the library file.
No, it does not. If
...is to not have it be a requirement at all. I don't think there's any reason to remove the pre-generated libraries.
Your code doesn't load any bundled libraries; it only loads the system libraries, and as I've pointed out above, that is not an acceptable solution. More code has to be added to extract and load the bundled libraries, which is what my point was referring to. GLEW isn't loaded or bundled at all; what happens when it is not present on the system? The game crashes.
I never mentioned prioritizing the bundled versions, so I'm not sure why you bring that part up, but, regardless, both problems apply. If I want Arc to:
Then I must:
Yes, but again, this risk seems to have manifested in a grand total of one issue reported here.
I think, even for someone using Linux, you vastly overestimate what the average user is going to know about switching SDL versions to suit their game at all. This is such a niche use case that I don't think it's even worth considering.
Yes, that is unreasonable, but so is switching their SDL/GLEW version to make the game work. Software should just work without the need for people to tweak their library versions. As I see it, your point seems to be "with this extremely specific niche problem (1 of which has ever been reported), the user will have to do something extremely unreasonable, as opposed to something moderately unreasonable". Yes, you could say it's true, but... frankly, the real solution would be the user filing an issue, and the developer figuring out why the bundled version doesn't work for everyone out of the box. Nothing the user can do is a true "solution", it's just a workaround. Apologies if parts of this response sounded rude, but I want to make absolutely clear that only loading system libraries is not a solution, and I do not consider any change that could conceivably break the game for existing users acceptable. Assuming we choose the third solution (System + fallback):
As mentioned above, I don't think this will be enough. How will GLEW be loaded from the JAR? How will the bundled library extractor know whether to extract GLEW/SDL? How will the fallback mechanism work? This seems like quite a bit more work than just a single method call. |
|
I'm going to keep my responses to this comment short, since It seems we fundamentally agree on the idea of using system libraries (and now we also agree on how to go about doing it) so it's not productive to go into depth about this; I'd like to move forward on how to implement using system libraries first with a fallback to bundled. As an aside, this may not be to relevant now, but I just want to quickly clear up what I feel are some misconceptions and exaggerations about what I've said.
It seems that you are taking what I said as black and white. Like I said, I could've worded that idea better, but I never straight up assumed any of this was the case, and I tried my best to highlight that using explicit language (e.g. " I just proposed it as a possibility - one of three solutions (hence the table I made). I never said that we should for sure only use system libraries and that we should just assume everyone has glew and SDL installed since there's so few people that don't. I even said:
because I don't know for sure. Again, it's up to you. You also have to realize that I don't have the numbers and download statistics that you do. Thus, I don't see how it's baffling that I proposed an alternative solution to cover all bases, since I don't have the statistical context that you do (i.e. I have no way of knowing that 87k people downloaded Linux mindustry on itch.io) .
I don't see the point of comparing principal vs practical benefits; Because regardless, it's bad practice/principal to not use the system libraries if possible. It seems you even agree with me on this part, too:
Remember that my specific issue was just meant to be a personal example; Wayland support (e.g. Anuken/Mindustry#1393, which had many people wanting wayland support) is also another example, so it's not entirely fair to say it only affects one person, since lack of wayland support is caused by not using system libraries. Not to mention, your logic of "it only affects one person, so it's not really an issue" is just really bad principal in my opinion. It's still an issue regardless of how many it affects, and again, this isn't factoring in any other future issues that are caused by similar incompatibilities; You're essentially leaving it up to chance how many issues arise due to said incompatibilities. So, it's not entirely correct to just ignore the principal in the name of pure practicality. Even if the benefits of this PR are irrelevant, if they outweigh the drawbacks (which I think we've agreed that, with the correct solution, they do), it should still be implemented.
That's true; but again, my intention was to avoid compiling the sdl backend library file unless it was necessary, not just if you wanted to compile it without testing. But regardless, if you want to talk about practicality, it only adds about 30 seconds of compilation time on an average system anyway, and only on first clone or a clean; Hence why I listed this as a con in the pros/cons list in my last comment.
This makes so sense at all to me. If it's not required in the first place then why is it there? Let me try to explain this: It essentially does nothing in that case; The point of If it really was the case that I suspect what's really going on is that Regardless, the only thing my PR changes is doing this with GLEW as well (using
I meant that a solution that removes the bundled libraries completely (e.g. the second solution in the table) would require some CI code changes. Otherwise, a solution that still includes the bundled libraries (e.g. the last and, in my opinion, preferred solution) requires library loading code changes. So, they can be seen as mutually exclusive. I'll touch on this in more detail in the next section below.
Either way, what to do in terms of developer requirements is a separate issue that I think should be left for afterwards. The two solutions I can think of off the top of my head is to either have GLEW and SDL as build requirements (which is very common for projects of other languages at least) and thus listed under a set build instructions and dependencies or to add significant gradle build code to try to link to system libraries first, then fallback to linking to the pre-compiled ones if it can't (but this has it's own problems, like the possibility of mismatched SDL versions if it links to the bundled one). It's also important to note that whatever solution is decided on here, it will also apply to people who build and run mindustry from source (which includes those who download mindustry from their package manager). Like always, it's your decision in the end.
My point with this was simply that if the average user had to switch SDL versions for whatever reason, then in isolation, I would think it's much easier to use their package manager to switch up their SDL just once as opposed to having to mess with the mindustry jar every time mindustry is updated. I wanted to highlight this concern separately as it's quite relevant (emphasis mine):
This isn't entirely correct (which thankfully means less work needed); In this case you shouldn't have to change the CI at all just to bundle GLEW; You just need to do what you're already doing with SDL. That is, bundling GLEW is as trivial as adding a Now to the prevalent issue at hand:
The current library loading code is already setup to load SDL from the jar: Arc/backends/backend-sdl/src/arc/backend/sdl/jni/SDL.java Lines 120 to 138 in 8fdcdbf So, this code just needs to be adapted/duplicated to do the same thing but with GLEW. And like I just said above, bundling GLEW is as trivial as dropping a shared library file (
Breaking it down: we first call Again, right now this is all theory since we haven't done any testing, but I don't see this posing much of an issue; It should be pretty straightforward. |
I'm going to skim over the rest of this answer, because I think you may have misunderstood my point: It's there to configure things if you are explicitly trying to build the natives. If you are simply testing any PR unrelated to natives, it is not required. Your PR, by virtue of requiring the natives to build upon a clone, requires SDL to test anything useful. Without your PR (as it is now), SDL is not required, unless you explicitly want to rebuild the natives for some reason. As for your solution, I was a little confused, because your initial wording said (highlights mine):
...but what you're saying now sounds like several new load/loadLibrary calls, some copy-pasted/extracted code to mimic the SDL linking, and some try-catches, which is rather different.
This statement confuses me as well. Why isn't there a third solution of keeping things as they were, except with glew being linked dynamically, and both libraries being system-first? No significant buildscript changes would have to be done (see below), and nobody would need to wait 30 seconds or rebuild sdl-arc, as it would be bundled. Why rebuild it at all when it links to the system libraries first? I think the overall plan would be this:
|
I guess I didn't explain that very well; It's better if I just show what I mean in the next commit. My point was just that it should be straightforward and simple to implement. Before it slips my mind again, I should also mention that this needs to be done with all of GLEW's library dependencies (EGL, GL, GLU, OpenGL). However, this is a trivial change in the library loading code and also would just require adding a few more files to the
This might be difficult to explain, but: This goes beyond the scope of just jni; I originally did this because it's common and better practice, at least from my perspective - e.g. ABI incompatibility shenanigans, not to mention reasons I've listed previously. It's mainly for those that are building mindustry from source anyway (i.e. they would build Arc from source too by default), not just developers. However, after giving it some further thought, I think in this specific case it won't make much of a difference and I don't foresee any issues arising with what you're proposing (especially since we have explicit control over how the libraries are loaded). Just in case though, I'll revert the build script changes in a separate commit. So with what you're proposing, like you said, you only need SDL and GLEW on your system if you're explicitly trying to build the library file. |
GLU makes sense, but why bundle the rest? I'm not too familiar with how dynamic linking works, to be honest, but aren't GL/EGL always present on the system? I've always been linking GL dynamically with no bundled libraries up to this point, and it hasn't caused any problems. You also list OpenGL and GL separately - what is the distinction between the two? The GLEW guide page only mentions linking to Additionally: Will linking to EGL cause problems with those that don't use EGL? |
They're all the possible dependencies of GLEW (depending on configuration) according to its Since the
Probably, but not necessarily; All of these dependencies are part of
Including information from
(Note how for EGL support, you need My guess is that the GLEW guide only mentions linking to GLU and GL most likely because it is the minimum/default configuration for GLEW; Since there's Xwayland (and thus X11 is supported under Wayland), only X11 support is necessary; Thus, the minimum needed would be the OpenGL function dispatch library and the GLX library (which is essentially what However, even if you only wanted to support X11 (and thus configured GLEW that way), I believe you still need the library loading code to load EGL and have Arc's SDL backend link to EGL. Otherwise, Arc can't use a system GLEW with an EGL configuration, and thus you would lose wayland support/capabilities. Specifically, Arc couldn't function properly if it used a system GLEW that used EGL functions but Arc itself wasn't linked to EGL. Similarly, it would obviously be a problem if Arc was linked to EGL but the EGL library wasn't loaded, even if it wasn't using any of the EGL functions (i.e. if it was using a bundled GLEW with X11 support only).
No, it shouldn't (although I can't fully speak for the inner workings of jni and if it affects library loading in this regard). Linking to and actually using the functions a library are separate. In this case, if GLEW isn't configured for EGL, then you'd essentially be linking to an unused library - it wouldn't use the EGL API functions, regardless of if the EGL library was linked to it or not (the linker might even perform some optimization on this). This is essentially the opposite scenario of what I just described above: Like I mentioned previously, since it was linked at compile-time, the bundled library would need be present regardless of configuration; The only problem could be if GLEW was somehow configured for EGL but ran on a system that does not support EGL windows (e.g. native X11). |
Change backend-sdl task to link Arc's SDL backend library to a shared GLEW library instead of using a pre-compiled version. These changes fix many build and mindustry issues caused by not using the system's GLEW.
On the SDL backend on linux, try to load system libraries (SDL2, GLEW, and their dependencies) first. If that fails, load bundled ones.
|
I have gone ahead and made the source code changes. Arc's SDL backend should now link to system libraries when being generated; It will also try to load the system libraries first, then falling back to bundled ones if it fails. The final commit needed is to add/update the bundled libraries themselves, but I won't do this myself (unless you explicitly ask me to) out of security concerns. It should be fairly straightforward to do this; For each of the dependencies (SDL2, GLU, libglvnd, and GLEW):
You also need to re-generate I tested mindustry with my own pre-bundled libraries (I generated the libraries on my end using the above steps and put them in Once you've finalized the aforementioned commit, I suggest you build and test mindustry as well on your system, both with and without the system GLEW and SDL installed. You can tell if mindustry is using the system libraries or not using the |
Arastais
changed the title
This PR reworks the build system for Arc on Linux to essentially only use system libraries and no longer use pre-bundled library files. Along with the many advantages of using the system libraries on Linux, this PR generally improves the overall experience on arc-based games on Linux (i.e. mindustry) and fixes some build issues (more on that later).
Specifically, in this PR:
libSDL2.so(i.e. the SDL library) has been removed. Arc links to the system's SDL instead. When included in the mindustry jar, the includedlibSDL2.sois specifically used instead of the system SDL. Without any library file in the jar, mindustry uses the system SDL, which is definitely better suited. The most obvious advantage is that now the user/system has control over how SDL is configured.libsdl-arc64.sois removed. Instead, it is generated using the system libraries (namely SDL and glew as mentioned before). The generation task is also now a dependency for the Arc jar; This forces the Arc library file to be generated using the system's libraries on Linux during the build process for Arc. This makes sure the sdl backend is generated before being bundled and thus prevents a pre-bundled version of Arc to be included.Using the system's libraries is preferred (at least on Linux). I personally think this is especially true on linux since different users and distros/package managers may have a differently configured version (or a different version altogether) of the same library.
As an example, this PR inherently adds wayland support to Arc and thus fixes Anuken/Mindustry#1393/Anuken/Mindustry#8705 with no actual changes to mindustry's source code, simply because Arc is no longer using pre-bundled libraries with a fixed configuration. (wayland support in mindustry had become broken again with the regression in 3928486, since it removed the option of dynamic natives in the backend-sdl build.gradle); Similarly, this also fixes #131, which is an issue I was personally having before this PR - Arc wasn't using my system's specific version of glew with patches/configuration from my distro's package manager. Since arc now links to the systems libraries, users with a glew compiled with
SYSTEM=linux-egland a properly configured SDL can now just run mindustry under native wayland with no other configuration and no changes to mindustry itself. This PR may solve many other current and future runtime and build related issues on Linux, although it may not be immediately obvious as all issues on the mindustry repository are marked as resolved.In general, it is a very good idea to use the system's libraries when building arc on linux, and to simultaneously make sure that no pre-bundled library files are used.