Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Proxy settings are ignored by xbps-install #152

Open
misirca opened this issue Jul 9, 2019 · 10 comments · May be fixed by #373
Open

Proxy settings are ignored by xbps-install #152

misirca opened this issue Jul 9, 2019 · 10 comments · May be fixed by #373
Labels
bug Something isn't working

Comments

@misirca
Copy link

misirca commented Jul 9, 2019

In some cases xbps-install seems to totally ignore proxy settings.

Note: this is not a sudo related issue as previously described here.

Note: Trials are made behind a corporate NTLM proxy, therefore outbound connections are made through a (hopefully well configured) CNTLM daemon.

Sanity check

Environment variables are set:

$ sudo env | grep proxy
http_proxy=127.0.0.1:3128
https_proxy=127.0.0.1:3128

wget downloads files, NTLM authentication is nicely handled by CNTLM

$ sudo wget www.free.fr
--2019-07-09 15:42:52--  https://www.free.fr/freebox/index.html
Connexion à 127.0.0.1:3128… connecté.
requête Proxy transmise, en attente de la réponse… 200 OK
Taille : 217424 (212K) [text/html]
Sauvegarde en : « index.html.1 »
index.html.1        100%[===================>] 212,33K  1,31MB/s    ds 0,2s    
2019-07-09 15:42:52 (1,31 MB/s) — « index.html.1 » sauvegardé [217424/217424]

Wireshark log

13	Computer	Parentproxy	TCP	74	60560 → 8080 [SYN] Seq=0 Win=64240 Len=0 MSS=1460 SACK_PERM=1 TSval=2763940829 TSecr=0 WS=128
14	Parentproxy	Computer	TCP	78	8080 → 60560 [SYN, ACK] Seq=0 Ack=1 Win=65535 Len=0 MSS=1460 SACK_PERM=1 TSval=1312957041 TSecr=2763940829 WS=64
15	Computer	Parentproxy	TCP	66	60560 → 8080 [ACK] Seq=1 Ack=1 Win=64256 Len=0 TSval=2763940832 TSecr=1312957041
16	Computer	Parentproxy	HTTP	367	GET http://www.free.fr/ HTTP/1.1 , NTLMSSP_NEGOTIATE

The issue

The following command seem to ignore proxy settings

$ sudo xbps-install -S -d
[DEBUG] XBPS: 0.56 API: 20190621 GIT: UNSET
[DEBUG] Processing configuration directory: /etc/xbps.d
[DEBUG] Processing system configuration directory: /usr/share/xbps.d
[DEBUG] Parsing configuration file: /usr/share/xbps.d/00-repository-main.conf
[DEBUG] [repo] `https://alpha.de.repo.voidlinux.org/current' stored successfully
[DEBUG] /usr/share/xbps.d/00-repository-main.conf: added repository https://alpha.de.repo.voidlinux.org/current
[DEBUG] Parsing configuration file: /usr/share/xbps.d/xbps.conf
[DEBUG] rootdir=/
[DEBUG] metadir=//var/db/xbps
[DEBUG] cachedir=/var/cache/xbps
[DEBUG] confdir=/etc/xbps.d
[DEBUG] sysconfdir=/usr/share/xbps.d
[DEBUG] syslog=true
[DEBUG] bestmatching=false
[DEBUG] Architecture: x86_64
[DEBUG] Target Architecture: (null)
[DEBUG] Repository[0]=https://alpha.de.repo.voidlinux.org/current
[*] Updating `https://alpha.de.repo.voidlinux.org/current/x86_64-repodata' ...
[DEBUG] st.st_size: 1898638
[DEBUG] st.st_atime: 08 Jul 2019 12:43
[DEBUG] st.st_mtime: 08 Jul 2019 09:17
[DEBUG] url_stat.size: -1
[DEBUG] url_stat.atime: 01 Jan 1970 00:00
[DEBUG] url_stat.mtime: 01 Jan 1970 00:00
ERROR: [reposync] failed to fetch file `https://alpha.de.repo.voidlinux.org/current/x86_64-repodata': Connection refused
[DEBUG] [rpool] `https://alpha.de.repo.voidlinux.org/current' failed to fetch repository data: Connection refused
[DEBUG] [rpool] checking `https://alpha.de.repo.voidlinux.org/current' at index 0
[DEBUG] [rpool] `https://alpha.de.repo.voidlinux.org/current' registered.
[DEBUG] [repo] `https://alpha.de.repo.voidlinux.org/current' public key already stored.

Wireshark log, no connection is made to parent proxy, neither to CNTLM daemon.
TCP SYN attempt is dropped by the firewall

8	Computer	DNSserver	DNS	87	Standard query 0x0243 A alpha.de.repo.voidlinux.org
9	Computer	DNSServer	DNS	87	Standard query 0x9e51 AAAA alpha.de.repo.voidlinux.org
10	DNSServer	Computer	DNS	128	Standard query response 0x0243 A alpha.de.repo.voidlinux.org CNAME a-hel-fi.m.voidlinux.org A 95.216.76.97
11	DNSServer	Computer	DNS	205	Standard query response 0x9e51 AAAA alpha.de.repo.voidlinux.org CNAME a-hel-fi.m.voidlinux.org SOA ns-cloud-c1.googledomains.com
12	Computer	95.216.76.97	TCP	74	33066 → 443 [SYN] Seq=0 Win=64240 Len=0 MSS=1460 SACK_PERM=1 TSval=3671962237 TSecr=0 WS=128
13	95.216.76.97	Computer	TCP	54	443 → 33066 [RST, ACK] Seq=1 Ack=1 Win=64240 Len=0

Quick and dirty workaround

I found today that the problem can be temporarily 'fixed' by forcing all TCP connections through the corporate proxy using proxychains
ie: App -> Proxychains -> CNTLM proxy -> Corporate Proxy -> Internet

sudo proxychains4 xbps-install -Syu
[proxychains] config file found: /etc/proxychains.conf
[proxychains] preloading /usr/lib/libproxychains4.so
[proxychains] DLL init: proxychains-ng 4.14
[*] Updating `https://alpha.de.repo.voidlinux.org/current/x86_64-repodata' ...
[proxychains] Strict chain  ...  127.0.0.1:3128  ...  alpha.de.repo.voidlinux.org:443  ...  OK

Wireshark logs, now connection begins with a proxy negociation.

8	Computer	Parentproxy	TCP	74	60952 → 8080 [SYN] Seq=0 Win=64240 Len=0 MSS=1460 SACK_PERM=1 TSval=2767636075 TSecr=0 WS=128
9 	Parentproxy	Computer	TCP	78	8080 → 60952 [SYN, ACK] Seq=0 Ack=1 Win=65535 Len=0 MSS=1460 SACK_PERM=1 TSval=1316652196 TSecr=2767636075 WS=64
10	Computer	Parentproxy	TCP	66	60952 → 8080 [ACK] Seq=1 Ack=1 Win=64256 Len=0 TSval=2767636076 TSecr=1316652196
11	Computer	Parentproxy	HTTP	302	CONNECT alpha.de.repo.voidlinux.org:443 HTTP/1.0 , NTLMSSP_NEGOTIATE

So finally I can't tell if this is really an unexpected behavior or maybe the consequence of a paranoid corporate proxy?
@Duncaen
Copy link
Member

Duncaen commented Jul 9, 2019

The code 0 where the http_proxy environment variable is used expects fetchParseURL to be successful and handle if there is no scheme:// and set it to http.
But fetchParseURL actually stops parsing if it couldn't find a scheme 1 and returns NULL so the url is never parsed and the "default" is unused.

I guess fetchParseURL needs to be changed to allow scheme less urls, but this might require some more changes at other places to error out without a scheme.

@Duncaen Duncaen added this to the 0.58 milestone Jul 28, 2019
@Duncaen
Copy link
Member

Duncaen commented Apr 23, 2020

This is not done, it requires bigger changes to the URL parsing to allow scheme less URLs and needs to be handled correctly in every place where URLs are parsed.

@elpres
Copy link

elpres commented Apr 23, 2020

I would be willing to test it with the current installation images, but at the moment we are working remotely and not from behind the corporate proxy. So I will only be able to do any testing once the lockdown is over, which will take a few more weeks.

@elpres
Copy link

elpres commented Sep 10, 2020

Good news! The VM I just set up with the current ISO version (void-live-x86_64-musl-20191109.iso) does work behind the proxy just fine. I was able to update the system as well as install new packages. The only intervention on my part was to set the relevant environment variables. This means that the problem originally reported in #75 which lead to the creation of this issue seems to have been resolved. The network situation didn't change since then, the same proxy is still in place.

Are there any more specific tests you would like me to do?

@CMB
Copy link
Contributor

CMB commented Feb 4, 2021

Maybe I should open another ticket for this; xbps-fetch has a
similar problem. The only reason I noticed is that I tried building packages in
an IPv6-only VM that uses a proxy for IPv4 connectivity.
xbps-src was unable to download anything, and when I tried running
xbps-fetch manually under strace, it tried to connect directly rather
than through the proxy.

@CMB
Copy link
Contributor

CMB commented Feb 4, 2021

Correction: I'm not seeing this on a dual-stacked host, just on an IPv6-only host.
If I try to run xbps-fetch on my dual-stacked host here at home,
with the *_PROXY environment variables set, it works as advertised in
the manpage. I see requests made by xbps-fetch in my squid logs.

The setup for my IPv6-only-host is a bit different. It's on another network.
It proxies through tinyproxy on a nearby dual-stacked system, rather than squid.

My test case is the following:

export ALL_PROXY='my-proxy-hostname:8123'
export HTTP_PROXY='my-proxy-hostname:8123'
curl https://github.com/index.html
xbps-fetch https://github.com/index.html

That's a good smoke-test, because github has no IPv6 in 2021.
For the IPv6-only host, curl succeeds and xbps-fetch fails.

@Duncaen Duncaen linked a pull request Feb 7, 2021 that will close this issue
Open
@leahneukirchen
Copy link
Member

I think for a HTTP proxy you need HTTP_PROXY='http://my-proxy-hostname:8123'

@CMB
Copy link
Contributor

CMB commented Feb 17, 2021 via email

@leahneukirchen
Copy link
Member

Then perhaps we just need better error handling if HTTP_PROXY does not make sense to xbps.

@66Ton99
Copy link

66Ton99 commented May 8, 2024

It is not a problem of xbps-install.
It is nuances of sudo and can be solved by adding Defaults env_keep += "ftp_proxy http_proxy https_proxy no_proxy" into /etc/sudoers - https://wiki.archlinux.org/title/Sudo#Environment_variables

I think it can be closed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Libfetch update Duncaen/xbps
7 participants
@leahneukirchen @CMB @66Ton99 @elpres @Duncaen @misirca and others