awk detection is broken on Musl-based Linux distros #132
Comments
|
@jperkin responded on IRC on Sunday, March 17th, 2024:
To be continued. |
|
Just to document my latest finding. It looks like the path for AWK is decided in ... with 607 LOCALBASE?= /usr/pkg
608 TOOLBASE:= ${LOCALBASE} |
netbsd-srcmastr
pushed a commit
that referenced
this issue
Mar 25, 2024
Changelog:
Version 4.3.1+dbg-1.6 (2022-01-22)
Allow --profile to take an argument to specify callgrind for json PR #132 (Jayson Messenger)
Fix bug in info var PR #113
Typos in make.1 Issue #110
Support gcc-10 Issue #109
Document debugger commands
Debugger commands are in separate .c files (not inlined, .h included)
I'd like to thank Marten Cassel for his sponsorship.
Version 4.3.1+dbg-1.5 (2020-03-15) Ron Frankel
Rebase code on GNU Make 4.3.
This was made possible by Thomas Kupper (boretom) who did all the heavy lifting while rocky looked on in amazement.
Thomas also added OS/X CI testing, and tested on the BSD-ish platforms.
With this release, we've started redoing the texinfo documentation in sphinx and put on readthedocs. Since sphinx supports output into LaTeX, TeXinfo, pdf, epub, and info as well as HTML, the TeXinfo document will be removed when the sphinx documentation is more stable. I expect the in next release to work on this.
Similarly, you'll see that README.md has been redone and includes screencasts. The install docs have also been corrected and expanded.
New and Changed Features
--search parent -- I really like this one.
--tasks -- the definition of a "tasks" has been simplified. Now, it is simply a target that has a description comment (#:) before it. After many years of using this myself, I highly encourage people to start using description comments more. Output from running remake --tasks is nicer because we use spaces to align columns rather than tabs.
Debugger Changes
Commands with file expansion now use glob(), not word_expand() (Thomas did this too). Previously source used to POSIX.1-2008 wordexp(), but this is not available on BSD-ish systems. glob() is more general, and GNU make ships with its own glob() function when none is provided in the underlying OS library.
load command added eval command removed. eval never worked and it attempted to be the same thing as load; load is the gdb name.
$(debugger) function fixed. This function gave an virtual memory exhausted on exit. This has been fixed. The required parameter for this function, a tag name, is now shown on entry
info tasks has been added. It is basically the same thing as remake --tasks
Description lines are now shown on in info targets and list commands
The debugger now disallows any "running" command inside post-mortem debugging
Help as shown inside the debugger has been greatly expanded and more closely matches the sphinx docs. We now show in help text the short command name and any aliases attached to the command
|
It seems mk/tools/tools.Linux.mk tries to find "gawk" specifically, and otherwise will not assign anything. That whole file basically assumes GNU tooling. (You kind of already partly answered your own question there in #130 (comment) .) A dependency may indeed be specified in mk/tools/replace.mk where you quote, in certain contexts only. (If you're looking for a single place where everything is revealed, well, a lot of the pkgsrc infrastructure isn't like that. It's more complicated.) |
ricardobranco777
pushed a commit
to ricardobranco777/pkgsrc
that referenced
this issue
Mar 31, 2024
Also update bundled PEAR package Archive_Tar to 1.5.0. PEAR 1.10.15 (2024-03-09) * PR NetBSD#132: cleanup uneeded test * PR NetBSD#135: Fix PHP Deprecated: Calling get_class() without arguments Archive_Tar 1.5.0 (2024-03-16) * PHP Version: PHP 5.2.0 or newer * PEAR Package: PEAR Installer 1.9.0 or newer
sskras
changed the title
|
Confirmed on the following distros, attaching the logs inline: Quoting @jperkin's IRC response and inviting for assistance:
Surely the file does not exist yet: # ls /usr/pkg/
ls: /usr/pkg/: No such file or directory
My impression is that nawk was not bootstrapped. That's the root problem I see.
Jonathan, I still doesn't understand that part. Sounds too abstract, maybe even ambiguous to me.
The only awk references I found in Which @dhgutteridge explained above:
|
|
Debian 12 also provides non-GNU implementation by default, the From 05-bootstrap-builds-nawk-on-Debian-12.log: --- /root/debug/src/github.com/NetBSD/pkgsrc.git/debug/logs/03-bootstrap-misses-nawk-on-Alpine-Linux.log
+++ /root/debug/src/github.com/NetBSD/pkgsrc.git/debug/logs/05-bootstrap-builds-nawk-on-Debian-12.log
@@ -1,16 +1,16 @@
===> bootstrap command: bootstrap/bootstrap
-===> bootstrap started: Fri Apr 19 20:26:12 EEST 2024
+===> bootstrap started: Sat Apr 20 08:29:37 AM EEST 2024
Working directory is: /root/debug/src/github.com/NetBSD/pkgsrc.git/work
...
-===> running: (cd /root/debug/src/github.com/NetBSD/pkgsrc.git/work/bmake && /bin/sh make-bootstrap.sh)
...
-===> running: /bin/sh /root/debug/src/github.com/NetBSD/pkgsrc.git/work/bin/install-sh -c -o root -g root -m 755 /root/debug/src/github.com/NetBSD/pkgsrc.git/work/bmake/bmake /root/debug/src/github.com/NetBSD/pkgsrc.git/work/bin/bmake
+===> running: (cd /root/debug/src/github.com/NetBSD/pkgsrc.git/work/bmake && /usr/bin/sh make-bootstrap.sh)
...
+===> running: /usr/bin/sh /root/debug/src/github.com/NetBSD/pkgsrc.git/work/bin/install-sh -c -o root -g root -m 755 /root/debug/src/github.com/NetBSD/pkgsrc.git/work/bmake/bmake /root/debug/src/github.com/NetBSD/pkgsrc.git/work/bin/bmake
===> Building libnbcompat
-===> running: /bin/sh /root/debug/src/github.com/NetBSD/pkgsrc.git/work/bin/install-sh -d -o root -g root /root/debug/src/github.com/NetBSD/pkgsrc.git/work/libnbcompat
-===> running: (cd /root/debug/src/github.com/NetBSD/pkgsrc.git/work/libnbcompat; /bin/sh ./configure -C --prefix=/usr/pkg --infodir=/usr/pkg/info --mandir=/usr/pkg/man --sysconfdir=/usr/pkg/etc --enable-bsd-getopt --enable-db && /root/debug/src/github.com/NetBSD/pkgsrc.git/work/bin/bmake -j1)
+===> running: /usr/bin/sh /root/debug/src/github.com/NetBSD/pkgsrc.git/work/bin/install-sh -d -o root -g root /root/debug/src/github.com/NetBSD/pkgsrc.git/work/libnbcompat
+===> running: (cd /root/debug/src/github.com/NetBSD/pkgsrc.git/work/libnbcompat; /usr/bin/sh ./configure -C --prefix=/usr/pkg --infodir=/usr/pkg/info --mandir=/usr/pkg/man --sysconfdir=/usr/pkg/etc --enable-bsd-getopt --enable-db && /root/debug/src/github.com/NetBSD/pkgsrc.git/work/bin/bmake -j1)
configure: creating cache config.cache
-checking build system type... x86_64-pc-linux-musl
-checking host system type... x86_64-pc-linux-musl
+checking build system type... x86_64-pc-linux-gnu
+checking host system type... x86_64-pc-linux-gnu
checking for gcc... gcc
checking whether the C compiler works... yes
checking for C compiler default output file name... a.out
@@ -265,13 +267,11 @@
checking whether ln -s works... yes
checking for ranlib... ranlib
checking for gawk... no
-checking for mawk... no
-checking for nawk... no
-checking for awk... awk
+checking for mawk... mawk
checking for ar... ar
checking for fparseln in -lutil... no
-checking for grep that handles long lines and -e... /bin/grep
-checking for egrep... /bin/grep -E
+checking for grep that handles long lines and -e... /usr/bin/grep
+checking for egrep... /usr/bin/grep -E
checking for _Bool... yes
checking for stdbool.h that conforms to C99... yes
checking for alloca.h... yes
...
ranlib libnbcompat.a
+===> Bootstrapping awk
+===> running: /usr/bin/sh /root/debug/src/github.com/NetBSD/pkgsrc.git/work/bin/install-sh -d -o root -g root /root/debug/src/github.com/NetBSD/pkgsrc.git/work/awk
+===> running: (cd /root/debug/src/github.com/NetBSD/pkgsrc.git/work/awk && /root/debug/src/github.com/NetBSD/pkgsrc.git/work/bin/bmake -j1 -f Makefile CC="cc" CFLAGS="")
+cc -c awkgram.tab.c
+cc -c b.c
+cc -c main.c
+cc -c parse.c
+cc -c proctab.c
+cc -c tran.c
+cc -c lib.c
+cc -c run.c
+cc -c lex.c
+cc awkgram.tab.o b.o main.o parse.o proctab.o tran.o lib.o run.o lex.o -lm -o a.out
+===> running: /usr/bin/sh /root/debug/src/github.com/NetBSD/pkgsrc.git/work/bin/install-sh -c -o root -g root -m 755 /root/debug/src/github.com/NetBSD/pkgsrc.git/work/awk/a.out /root/debug/src/github.com/NetBSD/pkgsrc.git/work/bin/nawk
===> Bootstrapping pkgtoolsI wonder what makes the bootstrap fail to reach (or just skip) this step on the topic distros. |
|
OK, found the answer in 737 # Debian/Ubuntu's awk is mawk, and mawk does not understand
738 # some regexps used in pkgsrc/mk.
739 if [ -f /etc/debian_version ]; then
740 need_awk=yesSo it's tied to the hardcoded OS name. Waiting for the suggestions on how to proceed in a clean way :) |
sskras
changed the title
netbsd-srcmastr
pushed a commit
that referenced
this issue
Apr 29, 2024
Changes in 2.2.5
#117: Document that empty for Concurrently waits forever
#120: Add ConcurrentlyE.
#123: Fix failing concurrentlyE tests in older GHCs.
#124: Allow hashable 1.4
#126: Semigroup and Monoid instances for ConcurrentlyE
#120: Add ConcurrentlyE
#138: expose internals as Control.Concurrent.Async.Internal
#131: Fix typos in docs
#132: waitAny(Catch): clarify non-empty input list requirement
#142: Add cancelMany
#135, #145, #150: Support for GHC 9.4, 9.6, 9.8
Document that empty for Concurrently waits forever.
netbsd-srcmastr
pushed a commit
that referenced
this issue
Apr 29, 2024
0.2.5.4 * Fix autolink parsing regression (#151). This affects autolinks with doubled internal line-ending punctuation characters. 0.2.5.3 * Fix rebase_relative_paths extension so it works with URLs with non-ASCII characters (#148). Previously these would not be properly detected as absolute URIs. 0.2.5.2 * Improve autolinks extension (#147). The autolinks extension was interacting badly with explicit links, To fix this we had to make autolink parsing a bit stricter than the GFM spec does. They allow unbalanced ) except at the end of a URL (which is defined as: followed by optional final punctuation then whitespace or eof). With this change, we don't allow unbalanced ) at all in raw URLs. This should not be a big problem in practice. * Protect against quadratic generated table size explosion (Michael Howell). This commit adds a limit to the number of auto-completed cells around 200,000. The result is, in these original samples: 0.2.5.1 * Add test/alerts.md to extra-source-files in cabal file. 0.2.5 * Add support for alerts extension, supporting GitHub-style alerts (#132). https://docs.github.com/en/get-started/writing-on-github/getting-started-with-writing-and-formatting-on-github/basic-writing-and-formatting-syntax#alerts New module Commonmark.Extensions.Alerts [API change]. * Do not accept footnote labels with line breaks (Michael Howell). * Parse [^ ] and [^] as links (Michael Howell). This is consistent with most other CommonMark parsers, even when they have support for footnotes turned on.
netbsd-srcmastr
pushed a commit
that referenced
this issue
May 6, 2024
2024-05-05 -- 0.9.8 >>>>>>>>>>>>> SECURITY >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * Fixed: [CVE-2024-34402] Protect against integer overflow in ComposeQueryEngine (GitHub #183, GitHub #185) * Fixed: [CVE-2024-34403] Protect against integer overflow in ComposeQueryMallocExMm (GitHub #183, GitHub #186) >>>>>>>>>>>>> SECURITY >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * Changed: Require CMake >=3.5.0 (GitHub #172) * Added: CMake option URIPARSER_SHARED_LIBS=(ON|OFF) to control, whether to produce a shared or static library for uriparser and that alone, falls back to standard BUILD_SHARED_LIBS if available, else defaults to "ON" (GitHub #169, GitHub #170) * Improved: Document that scheme-based normalization a la section 6.2.3 of RFC 3986 is a responsibility of the application using uriparser (GitHub #173, GitHub #174) * Improved: Document supported code points for functions uriEscape(Ex)W (GitHub #171, GitHub #175) * Infrastructure: Update Clang from 15 to 18 (GitHub #161, GitHub #187) * Infrastructure: Adapt to breaking changes in Clang packaging (GitHub #160) * Infrastructure: Get sanitizer CFLAGS and LDFLAGS back in sync (GitHub #161) * Infrastructure: Pin GitHub Actions to specific commits for security (GitHub #165) * Soname: 1:31:0 — see https://verbump.de/ for what these numbers do 2022-10-05 -- 0.9.7 * Fixed: Multiple issues with IPv6 and IPvFuture literal parsing (GitHub #146, GitHub #150) Thanks to Scallop Ye for the report and the pull request! * Fixed: Fix symbol visibility for -DBUILD_SHARED_LIBS=OFF (GitHub #139, GitHub #141); thanks to Mariusz Zaborski for the report! * Fixed: For MinGW, use size_t for inet_ntop declaration and fix macro checks for both MinGW and mingw-w64 (GitHub #131) * Fixed: Compiler warnings (GitHub #132, GitHub #152) * Improved: Use name UriConfig.h rather than generic config.h for the config header file to avoid name clashes and also include it through "UriConfig.h" with quotes rather than <UriConfig.h> so that it is found in quote path locations (GitHub #149) Thanks to Gaspard Petit for bringing this up! * Improved: Document need for UriConfig.h in UriMemory.c (GitHub #136) * Infrastructure: Add (support for) Visual Studio 17/2022 (GitHub #152) * Infrastructure: Drop (support for) Visual Studio <=14/2015 (GitHub #152) * Infrastructure: Update Clang from 13 to 15 (GitHub #143, GitHub #151) * Infrastructure: Make MinGW with 32bit Wine on Ubuntu 20.04 possible (GitHub #142, GitHub #144, GitHub #145) * Soname: 1:30:0 — see https://verbump.de/ for what these numbers do
The main thing we would avoid changing (active users and all...) is Enterprise Linux. Perhaps you can tell us if GNU awk is specifically identifiable on distributions like CentOS @sskras (perhaps installed as "gawk" in some directory)? Otherwise we can set need_awk=yes on all Linux... |
|
Thanks @alarixnia. I think I would rather:
On CentOS 9 it's this: Not sure if that helps. |
|
The same issue happens on Adelie Linux 1.0-beta5, also Musl-based distro (but no Busybox here): |
sskras
changed the title
netbsd-srcmastr
pushed a commit
that referenced
this issue
Oct 21, 2024
1.8.2 (2024-09-24) What's Changed * Drop commented-out line by @olleolleolle in #108 * Add Ruby 3.1 & 3.2 to CI matrix by @tricknotes in #109 * Fix/redos by @ooooooo-q in #114 * Raise HTTPStatus::BadRequest for requests with invalid/duplicate content-length headers by @jeremyevans in #120 * Bump actions/checkout from 3 to 4 by @dependabot in #121 * Improve CI by @hsbt in #123 * Fix WEBrick::TestFileHandler#test_short_filename test not working on mswin by @KJTsanaktsidis in #128 * Fix bug chunk extension detection by @jeremyevans in #125 * Fix CI. by @ioquatix in #131 * Merge multiple cookie headers, preserving semantic correctness. by @ioquatix in #130 * Test on macos-latest by @byroot in #132 * Require CRLF line endings in request line and headers by @jeremyevans in #138 * Prefer squigly heredocs. by @ioquatix in #143 * Only strip space and horizontal tab in headers by @jeremyevans in #141 * Treat missing CRLF separator after headers as an EOFError by @jeremyevans in #142 * Return 400 response for chunked requests with unexpected data after chunk by @jeremyevans in #136 * Fix reference to URI::REGEXP::PATTERN::HOST by @casperisfine in #144 * Prevent request smuggling by @jeremyevans in #146 New Contributors * @tricknotes made their first contribution in #109 * @ooooooo-q made their first contribution in #114 * @KJTsanaktsidis made their first contribution in #128 * @byroot made their first contribution in #132 * @casperisfine made their first contribution in #144
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Forking the issue #130 (comment).
Currently I test only the OpenMPTCProuter distro, but OpenWRT probably suffers in the same way.
The distro provides BusyBox version of AWK which seems to have a mix of features from
nawk,gawkand probably some other implementations too: https://wiki.alpinelinux.org/wiki/AwkMy idea was that the
boostrap/bootstrapscript should either pick up the OS-native version (/usr/bin/awk -> ../../bin/busybox) or bootstrap its' own version (/usr/pkg/bin/nawk).The bootstrapping seems to to ignore the former and require the latter. The problem is that the latter isn't built yet at the moment it gets required:
(I trimmed down details from all earlier steps except for the last three ones)
The text was updated successfully, but these errors were encountered: