guile: update to 3.0.9, fix ppc; guile2: old guile goes here

[barracuda156]

Description

Long due version update for guile, patch for ppc32.
Old 2.2.7 moved to a new port, guile2.

Type(s)

• bugfix
• enhancement
• security fix

Tested on

macOS 10.8.5 (x86_64)
Xcode 5.1.1

macOS 10.6.8 Server (ppc, x86_64)
Xcode 3.2.6

macOS 10A190 (ppc)
Xcode 3.2

Installation of both new guile and guile2 (old 2.2.7) tested on 10.6.8 Intel. Won’t bother repeat on PPC, takes forever.
New guile builds on PPC, at least ppc32.

[barracuda156]

@kencu The port has no maintainers, so I guess we have to decide anyway. guile is stuck on the very old version (and on PPC it didn’t even build as-is).

[barracuda156]

Looks like there is a problem with parsing the ending portion of the portfile where a part is commented out inside brackets. I will fix that today.

[barracuda156]

@kencu @catap Alternatively, we may either move an old guile to guile2 or the new one to guile3.

[catap]

@barracuda156 does any port depends on guile? And if so, can it be built with your update?

[barracuda156]

@barracuda156 does any port depends on guile? And if so, can it be built with your update?

@catap Not many; with a previous @2.9.9 (which in fact reports itself as @3.0) I have built gnotime and swig3-guile, with patching config to look for a correct guile version.

@kencu On a side note, why does the macos-10.15 bot report this error?

find: /tmp/mpbb: No such file or directory
[11](https://github.com/macports/macports-ports/runs/6059903717?check_suite_focus=true#step:9:11)
Error: Process completed with exit code 1.

What does that even mean? )

[barracuda156]

@kencu Okay, another ten hours, and I can confirm that --enable-shared works, and in fact is necessary for dependencies to build. I have enabled it in the portfile. gnotime has build successfully using guile @3.0.8.

[barracuda156]

@kencu @catap Alternatively, we may either move an old guile to guile2 or the new one to guile3.

That would be great! Usually, the way this works is the current port has the default name, in this case "guile" and an older version left behind to support some port that needs that version still has a versioned name like "guile18" or "guile2" or "guile22" or something like that.

In the case of guile, the last time somebody tried to roll out an older version (guile18) it turned out to be more complicated than expected https://github.com/macports/macports-ports/blob/master/lang/guile18/Portfile but perhaps all the new stuff in MacPorts since then will make it easier to make a guile2 port.

@kencu I agree that making an old version as guile2 appears to be a better idea. Technically it is straightforward, we literally copy the existing portfile into a new port. Should I do that? (Can I even commit a new port?)

[kencu]

@kencu I agree that making an old version as guile2 appears to be a better idea. Technically it is straightforward, we literally copy the existing portfile into a new port. Should I do that? (Can I even commit a new port?)

It's a bit more complicated. The new guile2 port has to be

1. installed somewhere other than the default place guile is installed
2. not collide with the guile18 or new guile port
3. still has to be able to be found by the ports that need to use it when they try to build.

[barracuda156]

@kencu I agree that making an old version as guile2 appears to be a better idea. Technically it is straightforward, we literally copy the existing portfile into a new port. Should I do that? (Can I even commit a new port?)

It's a bit more complicated. The new guile2 port has to be

1. installed somewhere other than the default place guile is installed
2. not collide with the guile18 or new guile port
3. still has to be able to be found by the ports that need to use it when they try to build.

@kencu Got it. Can someone take care of that? These are nuances specific to Macports internals, which I am not aware of. Or if there is a transparent step-wise instruction, I can do that, of course.
I will also appreciate if someone can test guile @3.0.8 on a relevant Intel system. I would imagine building on PPC is more problematic, and on Intel every option gonna work as-is, but better be safe.

[barracuda156]

Building guile @3.0.8 now on 10.6.8 Rosetta, will leave it to compile overnight, but so far all looks good.

[harens]

On a side note, why does the macos-10.15 bot report this error?

find: /tmp/mpbb: No such file or directory
[11](https://github.com/macports/macports-ports/runs/6059903717?check_suite_focus=true#step:9:11)
Error: Process completed with exit code 1.

What does that even mean? )

By default, when a GitHub action fails, any workflows that are still running are cancelled. Running more tests doesn't help much when there are already errors to fix. It's more useful to have the bots running on other PRs.

The macOS 11 build failed, so the 10.15 build was cancelled. Hence the obscure error :)

[barracuda156]

guile @3.0.8 successfully built on 10.6.8 Rosetta:

macmini:~ svacchanda$ port -v installed guile
The following ports are currently installed:
  guile @3.0.8_0 (active) requested_variants='' platform='darwin 10' archs='ppc' date='2022-04-21T02:27:46+0800'

Took a while, since in VM.

[barracuda156]

@kencu @harens Dependencies fixed, guile builds on bots now.

[barracuda156]

@mascguy @kencu Do we have reservations with this update? Or should it rather be turned into guile-devel or guile3? I am perfectly fine with either of the options, as you guys see optimal for Macports.

[kencu]

I am in charge of nothing, but unless the above comments about making a guile2 port happen are done then many things will break I think.

[barracuda156]

I am in charge of nothing, but unless the above comments about making a guile2 port happen are done then many things will break I think.

What do you think a painless transition could be? Maybe a technically easier, though perhaps less consistent, would be just introduce guile3 for the time-being, until everything is confirmed to build with it? In such a case no extra PRs required immediately, nothing gonna break.

[barracuda156]

Both OpenBSD and NetBSD decided to keep all three major versions of Guile as separate ports:
https://github.com/NetBSD/pkgsrc/tree/trunk/lang
https://github.com/openbsd/ports/tree/master/lang

[catap]

@barracuda156 just an idea split it into commits:

• rename glue -> glue2
• updated dependencies to use glue2
• introduce glue3

[barracuda156]

@barracuda156 just an idea split it into commits:

• rename guile -> guile2
• updated dependencies to use guile2
• introduce guile3

@mascguy @kencu WDYT?

[mascguy]

@barracuda156 just an idea split it into commits:

• rename guile -> guile2
• updated dependencies to use guile2
• introduce guile3

@mascguy @kencu WDYT?

Sounds reasonable, but I haven't had a chance to review all of the details and prior discussion.

@kencu?

[kencu]

@barracuda156 just an idea split it into commits:

• rename glue -> glue2
• updated dependencies to use glue2
• introduce glue3

this was always the recommended plan, but turned out to be harder than it looks when previously tried.

see my june 10 comment above, for one example

[kencu]

oh, whether the newest guile is numbered or not is a small thing. Parallel installs of the various guile versions was the thing thst takes a bit of coding for someone

[barracuda156]

oh, whether the newest guile is numbered or not is a small thing. Parallel installs of the various guile versions was the thing thst takes a bit of coding for someone

What example do we have to avoid conflicts in such cases?

[catap]

@barracuda156 I suggest to start to figure out: did glue2 and glue3 had compatibily API? If no => separated ports. If yes => you may simple update the port. If ABI is different => you must increase revision of all ports which is depends on it.

[barracuda156]

@barracuda156 I suggest to start to figure out: did glue2 and glue3 had compatibily API? If no => separated ports. If yes => you may simple update the port. If ABI is different => you must increase revision of all ports which is depends on it.

As the very minimum, dependents may need patches to configure scripts to recognize guile 3.x (I have done that locally for some).

[mascguy]

@mascguy Should I rebase to resolve conflicts, or too early anyway?

Don't worry about it, I'll take care of it when I have time

[dglb]

@dglb Did you try this PR, which does introduce guile 3.0.x?

I did it and it's working: I can compile mu +guile.
mu @1.10.7_0+emacs+guile (active)

thank's

I hope it will be soon in the "real" macports tree.

[pmetzger]

This is closing in on two years old. I would like to suggest that we close it for now, and that when someone has time to work on the problem, they can reopen it.

[barracuda156]

@pmetzger We do not gain anything by having all this wasted. In the worst case guile3 can be added as a new port, which will not require any changes to existing ones.

Then switching to the new guile3 can be done separately.

@mascguy Or you still intend to complete this in the present form, which is more demanding, admittedly?

[pmetzger]

@barracuda156 Nothing is wasted. It's simply a change of notation on the pull request. Nothing prevents continuing work, changing the state back to open, opening new PRs, etc.

I do want to emphasize that the PR queue is meant for things that can be pulled now. The proliferation of drafts that are never progressed is not really what the queue is for. If one is working on a project, one can do so at will without having to invoke the queue. One can even run the CI in one's own fork of the repo at will.

However on the objeject level here, given the length of time involved, I think your approach of creating a guile3 port and then moving things over one by one seems least intrusive.

[pmetzger]

As explained above, I'm closing this for now. It can be re-opened when someone wants to, or a guile3 port can be started to make the transition easier, or some other method can be chosen.

[lemzwerg]

It's great that someone is working on the transition to guile 3, thanks for that!

The configure script of LilyPond's development version no longer checks for guile-2; you have to explicitly enforce it. I thus wonder whether there is any progress with this patch.

[barracuda156]

@lemzwerg There are two options here:

1. Someone steps in to help with moving ports to guile 3.x, whatever possible, the rest are switched to use guile2. This is what has been partially done here, but it got stuck for months, which was the reason the PR got closed.
   I cannot handle everything myself and test on every existing system. Presumably maintainers of the ports relying on guile should be interested in moving to the current version, but nobody aside of @mascguy and myself joined in.

A technical complication is that some number of guile-related ports are likely to be just broken (unrelated to this PR), and that will prevent CI from passing. I have no write access to the repo, so cannot help here. Someone should make a judgement whether specific ports are fixable, and if not, merge the PR regardless.

2. Instead, guile3 can be made as a new port without switching anything to using it (at least immediately). This will not require fixing archaic broken ports or testing anything besides guile3 itself. This can be easily accomplished – we have everything for that already here – but we need some consensus on the matter.
   @mascguy @pmetzger @catap Any opinions?

[lemzwerg]

Alas, I'm not a real Mac user at all, just helping maintain the LilyPond port; I have only sporadic access to an old Lion box, which is not really helpful here.

BTW, Perry already voiced his opinion above to proceed with your option 2.

[lemzwerg]

I think it's better if you restrict guile 3 to the lilypond-devel subport.

[lemzwerg]

Ping!

[barracuda156]

BTW, Perry already voiced his opinion above to proceed with your option 2.

@lemzwerg I will try to get this sorted again, ok. In a course of a few days – I am busy with other stuff atm.

[mascguy]

Sergey, I'm seeing an issue with guile-3.0, which prevents successful use by dependents:

$ guile-config-3.0 compile
error: ("/opt/local/bin/pkg-config" "--cflags" "guile-3.0") exited with non-zero error code 127

Whereas guile-2.2 works fine:

$ guile-config-2.2 compile
-D_THREAD_SAFE -I/opt/local/include/guile/2.2 -I/opt/local/include

Just a friendly reminder: What ultimately stopped progress on this, is issues with Guile 3.x. (And back when I investigated 6+ months ago, it also appeared to affect the Guile 3.x cask from brew. So it's not a MacPorts-specific issue.)

Haven't returned to it since, so there's a chance it's been fixed by upstream. But if not, then we need to engage with upstream, to get it fixed.

[barracuda156]

@mascguy I think that at least on our side chances of it getting fixed increase substantially with having a port to begin with. This of course does not mean that other ports should be switched to the new Guile without verifying if that works, as it happens with some other languages )
But introducing it in a non-destructive way should be beneficial.

[mascguy]

@mascguy I think that at least on our side chances of it getting fixed increase substantially with having a port to begin with. This of course does not mean that other ports should be switched to the new Guile without verifying if that works, as it happens with some other languages ) But introducing it in a non-destructive way should be beneficial.

Well, the problem is that it's pointless to add a Guile 3.x port, until/unless it actually works. And the failure mentioned earlier is fatal, blocking any potential use by dependents.

[mascguy]

@mascguy I think that at least on our side chances of it getting fixed increase substantially with having a port to begin with. This of course does not mean that other ports should be switched to the new Guile without verifying if that works, as it happens with some other languages ) But introducing it in a non-destructive way should be beneficial.

Well, the problem is that it's pointless to add a Guile 3.x port, until/unless it actually works. And the failure mentioned earlier is fatal, blocking any potential use by dependents.

Although you're right, it's a chicken-and-egg issue. So we can certainly add a Guile 3.x port in isolation, so long as the port clearly states that it's not ready for consumption.

How does that sound?

[barracuda156]

@mascguy Yes, as the first step it will be good, and done that way it is safe.

[lemzwerg]

Another ping – guile is now available as version 3.0.10...

[barracuda156]

Another ping – guile is now available as version 3.0.10...

I need to build it locally first. It takes a lot of time, but I will try to make it in coming days.

[pmetzger]

@lemzwerg You might try making a pull request of your own.

[lemzwerg]

@pmetzger Sorry, but I have no access to a Mac right now, and I'm actually not a Mac user at all – I just try to take care of LilyPond, which needs guile 3 for current development versions.

[barracuda156]

@lemzwerg I will try to make it in coming days. As discussed above, it will not be in a form of this (closed) PR, but rather as a new guile3 port.

[graywolf]

Comment from the peanut gallery: I have working guile30 Portfile. The pkg-config example from above works, all tests do pass (this required 10 patches that I will be sending upstream later; I was surprised how Linux and Hurd centric Guile currently is). I will write an email to the mailing list with few questions and then work on preparing a pull request which will include guile -> guile22 transition and guile_select port.

FWIW I intend to put myself into the maintainers field.

[barracuda156]

@graywolf Oh, great, please handle this.

It would be nice if you pick fixes for legacy systems from my portfile (though if it is problematic, I can do a separate PR for those). I.e. from here: https://github.com/barracuda156/powerpc-ports/blob/009a424aaa540181c7f8ffc9fd3aef0889ccf58c/lang/guile-3.0/Portfile
Plus two patches referred.

[graywolf]

First, a disclaimer: I am new to both macports and (more importantly) to apple ecosystem (I simply got handed a macbook pro at work, not by choice). So I might not see some details that seem obvious to other people.

I think I would prefer to get the guile rework merged in the form I currently have and then fix portability issues later. I think I managed to keep the Portfile pretty simple, so I would like to complicate it only when necessary and as little as possible.

However few questions if I might:

1. What is the repository (powerpc-ports)? My impression from the MacPorts Guide document is that ppc and ppc64 are supported architectures, so why is the separate repository required?
2. Would you know why dprintf is misdetected?
3. Would you happen to know why Oresolve-primitives -Ocps is required on i386 and ppc? Same with the -O1.
4. Since you apparently already did the work (I see guile-2.2), why did you not sent it to this repository? (There is no obligation of course, I am just curious.)

[barracuda156]

@graywolf

What is the repository (powerpc-ports)?

powerpc-ports is just a name I use for the fork because my focus is on powerpc. You can fork a repo, but change its name. Name and contents of other branches in my repo is irrelevant, this PR was done against MacPorts master.

Since you apparently already did the work (I see guile-2.2), why did you not sent it to this repository?

Well, as you can see, the PR here was not merged. I kept the upgrade locally.

Would you know why dprintf is misdetected? Would you happen to know why Oresolve-primitives -Ocps is required on i386 and ppc? Same with the -O1.

If the flags are present in earlier guile ports, they are just borrowed. If not, then they were added to fix some errors. Since many month passed, now I cannot say why something was done. For sure nothing was added just for the sake of making a port more complicated.
Unless there are strong objections regarding specific flags, I would retain what I had, since these are confirmed to produce a working build. guile takes many hours to build on PowerPC, since it has to make a full bootstrap. I am not ready to spend several days of compilation time to check if alternative combinations of flags gonna work or not.

[pmetzger]

@graywolf I wouldn't worry about PPC support anyway. @barracuda156 can fix that up after you have a draft that works correctly for a new Guile. Just do your best based on the MacPorts documentation and existing models, and make a pull request.

[barracuda156]

Yeah, if it was not explicit, I am not expecting or asking anyone to test this for ppc or introduce specific fixes.
I merely meant to copy-paste what was already done. Nothing beyond that.

[graywolf]

So lets consider #25319 starting point for the discussion.