Unable to build libgda on darwin due to itstool error

[lavoiesl]

To Reproduce
Steps to reproduce the behavior:

nix-env -i libgda

...
2 warnings generated.
  CC       browser-stock-icons.lo
  CC       text-search.lo
  ITMRG  gda-browser-5.0.appdata.xml
  ITMRG  gda-browser-5.0.desktop
  CC       tool-utils.lo
  CC       config-info.lo
  CCLD     libbrowser.la
  CCLD     gda-browser-5.0
make[3]: Leaving directory '/private/var/folders/qb/12dl_nvs3hl9m69qrqlxy3_40000gn/T/nix-build-libgda-5.2.9.drv-0/libgda-5.2.9/tools/browser'
Making all in doc
make[3]: Entering directory '/private/var/folders/qb/12dl_nvs3hl9m69qrqlxy3_40000gn/T/nix-build-libgda-5.2.9.drv-0/libgda-5.2.9/tools/browser/doc'
make[3]: Nothing to be done for 'all'.
make[3]: Leaving directory '/private/var/folders/qb/12dl_nvs3hl9m69qrqlxy3_40000gn/T/nix-build-libgda-5.2.9.drv-0/libgda-5.2.9/tools/browser/doc'
Making all in help
make[3]: Entering directory '/private/var/folders/qb/12dl_nvs3hl9m69qrqlxy3_40000gn/T/nix-build-libgda-5.2.9.drv-0/libgda-5.2.9/tools/browser/help'
  GEN      cs/cs.mo
  GEN      es/es.mo
  GEN      gl/gl.mo
  GEN      id/id.mo
  GEN      pt_BR/pt_BR.mo
  GEN      sl/sl.mo
  GEN      zh_CN/zh_CN.mo
  GEN    cs/
/nix/store/rr6hg4mq1q2banx6b24ij1bskpm4sphw-itstool-2.0.6/bin/itstool: line 19: from: command not found
/nix/store/rr6hg4mq1q2banx6b24ij1bskpm4sphw-itstool-2.0.6/bin/itstool: line 20: from: command not found
/nix/store/rr6hg4mq1q2banx6b24ij1bskpm4sphw-itstool-2.0.6/bin/itstool: line 25: import: command not found
/nix/store/rr6hg4mq1q2banx6b24ij1bskpm4sphw-itstool-2.0.6/bin/itstool: line 26: import: command not found
/nix/store/rr6hg4mq1q2banx6b24ij1bskpm4sphw-itstool-2.0.6/bin/itstool: line 27: import: command not found
/nix/store/rr6hg4mq1q2banx6b24ij1bskpm4sphw-itstool-2.0.6/bin/itstool: line 28: import: command not found
/nix/store/rr6hg4mq1q2banx6b24ij1bskpm4sphw-itstool-2.0.6/bin/itstool: line 29: import: command not found
/nix/store/rr6hg4mq1q2banx6b24ij1bskpm4sphw-itstool-2.0.6/bin/itstool: line 30: import: command not found
/nix/store/rr6hg4mq1q2banx6b24ij1bskpm4sphw-itstool-2.0.6/bin/itstool: line 31: import: command not found
/nix/store/rr6hg4mq1q2banx6b24ij1bskpm4sphw-itstool-2.0.6/bin/itstool: line 32: import: command not found
/nix/store/rr6hg4mq1q2banx6b24ij1bskpm4sphw-itstool-2.0.6/bin/itstool: line 33: import: command not found
/nix/store/rr6hg4mq1q2banx6b24ij1bskpm4sphw-itstool-2.0.6/bin/itstool: line 34: import: command not found
/nix/store/rr6hg4mq1q2banx6b24ij1bskpm4sphw-itstool-2.0.6/bin/itstool: line 36: PY3: command not found
/nix/store/rr6hg4mq1q2banx6b24ij1bskpm4sphw-itstool-2.0.6/bin/itstool: line 39: syntax error near unexpected token `('
/nix/store/rr6hg4mq1q2banx6b24ij1bskpm4sphw-itstool-2.0.6/bin/itstool: line 39: `    def ustr(s, encoding=None):'
make[3]: *** [Makefile:722: cs/cs.stamp] Error 2
make[3]: Leaving directory '/private/var/folders/qb/12dl_nvs3hl9m69qrqlxy3_40000gn/T/nix-build-libgda-5.2.9.drv-0/libgda-5.2.9/tools/browser/help'
make[2]: *** [Makefile:1098: all-recursive] Error 1
make[2]: Leaving directory '/private/var/folders/qb/12dl_nvs3hl9m69qrqlxy3_40000gn/T/nix-build-libgda-5.2.9.drv-0/libgda-5.2.9/tools/browser'
make[1]: *** [Makefile:923: all-recursive] Error 1
make[1]: Leaving directory '/private/var/folders/qb/12dl_nvs3hl9m69qrqlxy3_40000gn/T/nix-build-libgda-5.2.9.drv-0/libgda-5.2.9/tools'
make: *** [Makefile:737: all-recursive] Error 1

Additional context
It looks like the itstool is being called with the wrong python.

Weirdly enough, the shebang looks right. If I call the tool directly, it works:

$ /nix/store/rr6hg4mq1q2banx6b24ij1bskpm4sphw-itstool-2.0.6/bin/itstool
msgid ""
msgstr ""
"Project-Id-Version: PACKAGE VERSION\n"
"POT-Creation-Date: 2019-11-19 15:46+0100\n"
"PO-Revision-Date: YEAR-MO-DA HO:MI+ZONE\n"
"Last-Translator: FULL NAME <EMAIL@ADDRESS>\n"
"Language-Team: LANGUAGE <[email protected]>\n"
"MIME-Version: 1.0\n"
"Content-Type: text/plain; charset=UTF-8\n"
"Content-Transfer-Encoding: 8bit\n"

It's worth noting that it was working a month ago when I activated darwin support for libgda: #71200

It was also working fine last week, but I’m not sure exactly what changed in the mean time.

Metadata

• system: "x86_64-darwin"
• host os: Darwin 19.0.0, macOS 10.15.1
• multi-user?: no
• sandbox: no
• version: nix-env (Nix) 2.3.1
• channels(seb): "nixpkgs-20.03pre202062.84d582f4c6d"
• nixpkgs: /Users/seb/.nix-defexpr/channels/nixpkgs

Maintainer information:

attribute: libgda, itstool
module:

[lavoiesl]

Adding --without-help to the configure flags fixes it since it skips the gda browser help generation.

[worldofpeace]

It appears we're using python.withPackages which is a wrapped python as a shebang, and that's not allowed on darwin so it interprets it as bash 😦

@FRidh We'd have to revert 8d92646.

[worldofpeace]

Hmm, it seems libxml2Python uses python2. Seems it was reverted to python2 because of breakage

• 860b22f

Should we add libxml2Python3 @FRidh?

[FRidh]

Sigh, it's always Darwin.

I suggest using --without-help as suggested by @lavoiesl. Darwin maintainers need to work on a way to support these wrappers instead of us having to find ways for things to work on Darwin.

[lavoiesl]

I would assume that help documentation is available online anyway, so it's a reasonable workaround.

[worldofpeace]

Sigh, it's always Darwin.

I suggest using --without-help as suggested by @lavoiesl. Darwin maintainers need to work on a way to support these wrappers instead of us having to find ways for things to work on Darwin.

@FRidh I believe it's fundamentally disallowed, shell scripts cannot be intrepreters on the shebang in darwin. So going forward we'd need to have makeWrapper produce binary executables.
I've already make changes in gtk-doc for this exact reason, to allow it to run on darwin.

[FRidh]

Yes, it's fundamentally not possible on Darwin. I know of this limitation and what needs to be done to improve the situation, however, we should not need to keep working around it.

[lavoiesl]

It's weird, because it seems to work fine for me on zsh:

$ sw_vers
ProductName:	Mac OS X
ProductVersion:	10.15.1
BuildVersion:	19B88

$ $(echo $SHELL) --version
zsh 5.6.2 (x86_64-apple-darwin18.0.0)

$ ls -la $(realpath $(which itstool))
-r-xr-xr-x  1 seb  admin  68734 Jan  1  1970 /nix/store/rr6hg4mq1q2banx6b24ij1bskpm4sphw-itstool-2.0.6/bin/itstool

$ head -n1 $(which itstool)
#!/nix/store/721drf5czkl3pr0d5993fxl3dsaivdca-python3-3.7.5-env/bin/python -s

$ cat /nix/store/721drf5czkl3pr0d5993fxl3dsaivdca-python3-3.7.5-env/bin/python
#! /nix/store/4l15aq4m0hzkhn23qhgg7sbj750cb4wr-bash-4.4-p23/bin/bash -e
export NIX_PYTHONEXECUTABLE='/nix/store/721drf5czkl3pr0d5993fxl3dsaivdca-python3-3.7.5-env/bin/python3.7'
export NIX_PYTHONPATH='/nix/store/721drf5czkl3pr0d5993fxl3dsaivdca-python3-3.7.5-env/lib/python3.7/site-packages'
export PYTHONNOUSERSITE='true'
exec "/nix/store/9a5bl4gvzdm69fvhwbwln5ld6vn55iyx-python3-3.7.5/bin/python"  "$@"

$ itstool
msgid ""
msgstr ""
"Project-Id-Version: PACKAGE VERSION\n"
"POT-Creation-Date: 2019-11-20 11:58+0100\n"
"PO-Revision-Date: YEAR-MO-DA HO:MI+ZONE\n"
"Last-Translator: FULL NAME <EMAIL@ADDRESS>\n"
"Language-Team: LANGUAGE <[email protected]>\n"
"MIME-Version: 1.0\n"
"Content-Type: text/plain; charset=UTF-8\n"
"Content-Transfer-Encoding: 8bit\n"

It seems to be a limitation introduced by bash:

$ bash -c itstool
from: can't read /var/mail/__future__
from: can't read /var/mail/__future__
^C^C^C^C^C^C^C^C^C

[lavoiesl]

Ha! Changing the shebang to /usr/bin/env fixes it.

$ head -n1 `which itstool`
#!/usr/bin/env /nix/store/721drf5czkl3pr0d5993fxl3dsaivdca-python3-3.7.5-env/bin/python -s

All work fine:

$ zsh -c itstool
$ bash -c itstool
$ fish -c itstool

[lavoiesl]

Another option is to convert the wrapper script to am executable: http://www.datsi.fi.upm.es/~frosal/sources/shc.html

[FRidh]

About wrapping, have a look at #11133.

[vbgl]

The “shared-mime-info” package also fails to build on darwin due to “itstool” being broken.

[stale]

Thank you for your contributions.
This has been automatically marked as stale because it has had no activity for 180 days.
If this is still important to you, we ask that you leave a comment below. Your comment can be as simple as "still important to me". This lets people see that at least one person still cares about this. Someone will have to do this at most twice a year if there is no other activity.
Here are suggestions that might help resolve this more quickly:

1. Search for maintainers and people that previously touched the
   related code and @ mention them in a comment.
2. Ask on the NixOS Discourse. 3. Ask on the #nixos channel on
   irc.freenode.net.

[FRidh]

This was fixed for 20.03.