Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Change libraries XBPS depends on #323

Open
ericonr opened this issue Aug 20, 2020 · 13 comments
Open

Change libraries XBPS depends on #323

ericonr opened this issue Aug 20, 2020 · 13 comments

Comments

@ericonr
Copy link
Member

ericonr commented Aug 20, 2020

Moving XBPS off of LibreSSL (or even OpenSSL, depending on the resolution here) is an interesting change, for reasons of API simplicity, bootstrapping, and others. Some potential options are BearSSL and mbedtls. Custom signing and verification code might need to be added as well.

Moving off of libarchive has also been brought up, but I don't believe there have been specific suggestions for libraries. We would probably need at least xz and zstd support. As a bonus, we could find a threaded compression and decompression library.

@ericonr ericonr changed the title Remove library dependencies from XBPS. Change library dependencies in XBPS Aug 20, 2020
@ericonr ericonr changed the title Change library dependencies in XBPS Change libraries XBPS depends on Aug 20, 2020
@the-maldridge
Copy link
Member

I vote BearSSL, as well as dropping libarchive. Both have deeper than desirable bootstrapping chains in the chroot- set.

@sgn
Copy link
Member

sgn commented Aug 21, 2020

I also support BearSSL. mbedtls requires python3 in hostmakedepends, which will lengthen bootstrapping chain.
Re: libarchive, if we gonna remove it, we'll add tar back into bootstrap chain, since bsdtar is built together with libarchive? And we'll need at least gzip, xz, bzip2, and zstd to de-compress inside chroot.

@the-maldridge
Copy link
Member

Not sure what happened there, I lost the tab, most of my message, and the ticket closed. It wasn't important.

@ericonr
Copy link
Member Author

ericonr commented Aug 21, 2020

Would moving to separate backends for libzstd and liblzma be worth it, then? They are the libraries used by libarchive itself.

@sgn Since bsdtar isn't code we have to maintain, I think it would be ok to keep it as a bootstrap package. However, gzip, xz, bzip2 and zstd are already bootstrap packages, since they are dependencies of libarchive, so removing libarchive from bootstrap is equally possible.

@ericonr
Copy link
Member Author

ericonr commented Aug 21, 2020

It makes for a bigger bootstrapping/dependency chain, but another option includes using https://skarnet.org/software/s6-networking together with BearSSL. Some of the code, like for cert directories, will have to be present in XBPS, and I don't think depending on another library for some of it would be too bad (I might not have thought of all the potential issues).

Otherwise, it has some interesting code we can duplicate for interacting with things like certs.

@pullmoll
Copy link
Member

If we don't have bsdtar in the base chroot, we will have to touch quite a few templates again to either add bsdtar to hostdepends or switch (back) to tar and add that, and fix the common/ scripts which handle archives.
That's why I don't like the idea to remove libarchive from the bootstrap packages too much, if at all.

@ericonr
Copy link
Member Author

ericonr commented Aug 21, 2020

That's why I don't like the idea to remove libarchive from the bootstrap packages too much, if at all.

I don't think it needs to be. It's a small dependency that sits above all the other archive libraries, that we'd need anyway if we were to switch to GNU tar. bsdtar as a one-size-fits-all tool is pretty good.

One reason we'd switch from libarchive for XBPS specifically would be to enable better access to library functionality, such as threaded compression and decompression, and, ideally, cleaner memory handling, once the libarchive layer is removed.

@ericonr
Copy link
Member Author

ericonr commented Aug 29, 2020

I'm facing this problem in #324: earlephilhower/bearssl-esp8266@0c27e41

BearSSL apparently can't natively decode an RSA key as stored in a PEM file. Using a program like below:

#include <stdio.h>
#include <sys/mman.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <unistd.h>
#include <stdlib.h>
#include <string.h>

#include <bearssl.h>

void dummy(void *dest_ctx, const void *src, size_t len)
{
    br_x509_decoder_context *dest = dest_ctx;
    br_x509_decoder_push(dest, src, len);
}

int main(int argc, char **argv)
{
    if (argc < 2) {
        fputs("too few arguments", stderr);
        return 1;
    }

    int key_file = open(argv[1], O_RDONLY);
    if (key_file < 0) {
        fprintf(stderr, "error opening %s: %m\n", argv[1]);
        return 1;
    }
    unsigned char *buffer = mmap(NULL, 800, PROT_READ, MAP_PRIVATE, key_file, 0);
    if(buffer == NULL) {
        fputs("mmap error", stderr);
        return 1;
    }
    close(key_file);

    br_pem_decoder_context pem;
    br_x509_decoder_context x509;
    br_pem_decoder_init(&pem);
    br_x509_decoder_init(&x509, 0, 0);
    br_pem_decoder_setdest(&pem, dummy, &x509);
    size_t len = 800;
    while (len > 0) {
        size_t pushed = br_pem_decoder_push(&pem, buffer, len);
        printf("pushed %lu bytes\n", pushed);
        buffer += pushed;
        len -= pushed;

        switch(br_pem_decoder_event(&pem)) {
            const char *name;

            case 0:
                break;
            case BR_PEM_BEGIN_OBJ:
                name = br_pem_decoder_name(&pem);
                puts(name);

                break;
            case BR_PEM_ERROR:
                fputs("br_pem_error!", stderr);
                break;
            case BR_PEM_END_OBJ:
                if (len != 0) {
                    fputs("didn't finish file", stderr);
                } else {
                    fputs("finished file!", stderr);
                }
                break;
        }
    }

    fprintf(stderr, "err code: %d\n", br_x509_decoder_last_error(&x509));
}

on the following key, which is what LibreSSL decodes for us in the rsa_verify_hash function in lib/verifysig.c:

-----BEGIN PUBLIC KEY-----
MIICIjANBgkqhkiG9w0BAQEFAAOCAg8AMIICCgKCAgEAvrSzBZMvgvOCI3AXbOja
2rKRkJSTM3c/EjTIgCgDXgum93BPCtY8McFVoCU4OiXHGfTmq39BVNpLvLHL9Kk1
p278SBhXVOtbB2EVmDKnvfIDETldLGze7rZNRJdtuN2mZ/TVrUB9S0yQc+Iuf4hv
3+D97VIdTJHA7ASr0428ppEGJGwSShY2XJm9D5I0EuGRWc14MEG7dIKJiYiM0nE4
tYo2/vH4IFTHdnVA3gYiZyDnbuCAR/8ESfUTU13SNCOdbudf3D5Bcy+UiMDJI3Ye
4MFKBrT9ZhZ+GsXBZY482lqiZi6CL5ptc9IQFf8/eKZa8jatkiVFVgrKeNRjOTxN
fWSu2nkxGNX+bhXYthiGWmJEY8cCAPyFN+Lv5RetK256gdcb2skmEqefv2zPC+wV
WBbdH5bD4bZjn0N6Zl81rv5RzDvnvf+vD14aFUbZ8QFqu75PbL4w6mYM4ldM/g0R
6NXE8Az9Bwx2tDfYeKuutw1EpPm2YvFyTUb1coyAuTGRyAap1UTHvg9lhPIJmhFQ
J5kCg1qD7A31Wl0Rlne6hgGo0ZZNJ5cJM/v/zSTKJcuFgwo7H0hOGil6Dfo89B4j
G96ACyPS+DVKPFXRYwj/AkbHpaQ2f1FMAoSpBquDqHh3ukk71KX6j19t0iF8DQLr
vtFSSdIjDA02lweV9NaQpWsCAwEAAQ==
-----END PUBLIC KEY-----

yields an error code of 36: https://www.bearssl.org/apidoc/bearssl__x509_8h.html#a5a1ef832c53c92ed31ea4ced41d830a1

@ericonr
Copy link
Member Author

ericonr commented Aug 29, 2020

Also, BearSSL apparently doesn't do TSL 1.3, according to the front page.

@ArsenArsen
Copy link
Contributor

Current version is 0.6. It is now considered beta-quality software: it successfully passes extensive test suites, and while not all intended features are present, new features should imply no breaking changes in API or ABI1. There is no such thing as bug-free code, and I won’t claim that there is none in BearSSL; only that I looked real hard. Use in production applications is, in any case, at your own risk.

I don't support moving to BearSSL. It might be better code, it might be a better API, but when it comes to an SSL library, it's more important that it's hard to break, this leaves two primary options:

  • OpenSSL: lots of eyes on it, especially after Heartbleed
  • LibreSSL: quite a competent set of people behind it, but it requires more patching, I've put it here also because it was already there.

@aurieh
Copy link

aurieh commented Sep 8, 2020

I have to agree with @ArsenArsen here. XBPS, being a critical system component, should not rely on a relatively little-known and beta-quality (according to its front page) TLS/crypto library.

@CameronNemo
Copy link
Contributor

CameronNemo commented Sep 22, 2020

@sgn

mbedtls requires python3 in hostmakedepends, which will lengthen bootstrapping chain.

It only really needs GNU Make and a C99 toolchain to get a functional library. Python and Perl are required for generating and running tests, but it is fairly straightforward to disable them.

See here: void-linux/void-packages#25033

@ArsenArsen
Copy link
Contributor

ArsenArsen commented Nov 19, 2020

We are planning on using xbps on a system we are working on, and I was looking into compatibility between Open- and LibreSSL in various software packages, and OpenSSL (unsurprisingly) requires a lot less patching, so I looked into whether xbps compiles with OpenSSL 1.1.1g, and bfad1af (./configure --enable-rpath --enable-fulldebug --enable-debug) compiles with no changes needed.
In my opinion, the best course of action, assuming Void switches to OpenSSL in its entirety, is to just switch to OpenSSL (or supporting both open- and libressl) for xbps.

I am not sure whether the benefits of OpenBSDs rigorous development process outweigh the downsides of it not being OpenSSL itself anymore.

EDIT: reconfigured and rebuilt xbps on master to run the tests, got these results:

===> Expected failures
libxbps/shell/obsoletefiles_test:multiple_obsoletes_with_alternatives_unordered  ->  expected_failure: not fixed yet: 1 != 0 (1 != 0)  [0.178s]
xbps-alternatives/main_test:cc_alternatives_removal  ->  expected_failure: https://github.com/void-linux/xbps/pull/253: 1 != 0 (1 != 0)  [0.164s]
xbps-alternatives/main_test:replace_file_with_alternative  ->  expected_failure: https://github.com/void-linux/xbps/pull/185: 1 != 0 (1 != 0)  [0.175s]
xbps-install/behaviour_tests:update_unpacked  ->  expected_failure: currently just configures the package: 'configure' != 'update'  [0.104s]
===> Failed tests
libxbps/shell/obsoletefiles_test:replace_package_same_files  ->  failed: 2 != 0 (2 != 0)  [0.154s]
xbps-install/behaviour_tests:reproducible  ->  failed: 2 != 0 (2 != 0)  [0.084s]
===> Summary
Results read from /home/arsen/xbps/result.db
Test cases: 244 total, 0 skipped, 4 expected failures, 0 broken, 2 failed
Start time: 2020-11-19T19:29:02.717288Z
End time:   2020-11-19T19:29:06.068760Z
Total time: 29.407s

@ericonr ericonr mentioned this issue Dec 12, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

7 participants