Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

lib: add ability to template repo urls with {arch} #574

Open
wants to merge 34 commits into
base: master
Choose a base branch
from
Open

lib: add ability to template repo urls with {arch} #574

wants to merge 34 commits into from

Conversation

classabbyamp
Copy link
Member

@classabbyamp classabbyamp commented Oct 9, 2023 •
edited
Loading

{arch} will be replaced with XBPS_TARGET_ARCH (if set) or XBPS_ARCH

not sure if this should be part of the public API, I can make it not if desired.

a little unsure about the memory management bits so I leave that up to the experts...

valgrind results 

==23432== Memcheck, a memory error detector
==23432== Copyright (C) 2002-2022, and GNU GPL'd, by Julian Seward et al.
==23432== Using Valgrind-3.21.0 and LibVEX; rerun with -h for copyright info
==23432== Command: bin/xbps-query/xbps-query -LMd --repository=http://0.0.0.0:8000/{arch}/{arch}/ --repository=/foo/bar/{arh}
==23432== 
[DEBUG] XBPS: 0.60 API: 20200423 GIT: ae11e157
[DEBUG] Processing configuration directory: /usr/etc/xbps.d
[DEBUG] Processing system configuration directory: /usr/share/xbps.d
[DEBUG] Parsing configuration file: /usr/share/xbps.d/00-repository-main.conf
[DEBUG] [repo] `https://repo-default.voidlinux.org/current' stored successfully
[DEBUG] /usr/share/xbps.d/00-repository-main.conf: added repository https://repo-default.voidlinux.org/current
[DEBUG] Parsing configuration file: /usr/share/xbps.d/10-repository-multilib-nonfree.conf
[DEBUG] [repo] `https://repo-default.voidlinux.org/current/multilib/nonfree' stored successfully
[DEBUG] /usr/share/xbps.d/10-repository-multilib-nonfree.conf: added repository https://repo-default.voidlinux.org/current/multilib/nonfree
[DEBUG] Parsing configuration file: /usr/share/xbps.d/10-repository-multilib.conf
[DEBUG] [repo] `https://repo-default.voidlinux.org/current/multilib' stored successfully
[DEBUG] /usr/share/xbps.d/10-repository-multilib.conf: added repository https://repo-default.voidlinux.org/current/multilib
[DEBUG] Parsing configuration file: /usr/share/xbps.d/10-repository-nonfree.conf
[DEBUG] [repo] `https://repo-default.voidlinux.org/current/nonfree' stored successfully
[DEBUG] /usr/share/xbps.d/10-repository-nonfree.conf: added repository https://repo-default.voidlinux.org/current/nonfree
[DEBUG] Parsing configuration file: /usr/share/xbps.d/20-repository-debug.conf
[DEBUG] [repo] `https://repo-default.voidlinux.org/current/debug' stored successfully
[DEBUG] /usr/share/xbps.d/20-repository-debug.conf: added repository https://repo-default.voidlinux.org/current/debug
[DEBUG] Parsing configuration file: /usr/share/xbps.d/void-virtualpkgs.conf
[DEBUG] Parsing configuration file: /usr/share/xbps.d/xbps-arch.conf
[DEBUG] /usr/share/xbps.d/xbps-arch.conf: native architecture set to x86_64
[DEBUG] Parsing configuration file: /usr/share/xbps.d/xbps.conf
[DEBUG] processing templated repository: http://0.0.0.0:8000/{arch}/{arch}/
[DEBUG] Native architecture is x86_64
[DEBUG] Target architecture is x86_64-musl
[DEBUG] Repository[0]=http://0.0.0.0:8000/x86_64-musl/x86_64-musl/
[DEBUG] Repository[1]=/foo/bar/{arh}
[DEBUG] Repository[2]=https://repo-default.voidlinux.org/current
[DEBUG] Repository[3]=https://repo-default.voidlinux.org/current/multilib/nonfree
[DEBUG] Repository[4]=https://repo-default.voidlinux.org/current/multilib
[DEBUG] Repository[5]=https://repo-default.voidlinux.org/current/nonfree
[DEBUG] Repository[6]=https://repo-default.voidlinux.org/current/debug
got 1 A and 0 AAAA records
[DEBUG] [repo] `/foo/bar/{arh}/x86_64-musl-repodata' open repodata No such file or directory
ERROR: No such file or directory: /foo/bar/{arh}/x86_64-musl-repodata
got 1 A and 1 AAAA records
got 1 A and 1 AAAA records
got 1 A and 1 AAAA records
got 1 A and 1 AAAA records
got 1 A and 1 AAAA records
[DEBUG] xbps_pkgdb_unlock: pkgdb_fd -1
[DEBUG] [pkgdb] released ok.
   -1 http://0.0.0.0:8000/x86_64-musl/x86_64-musl/ (RSA maybe-signed)
   -1 /foo/bar/{arh} (RSA maybe-signed)
   -1 https://repo-default.voidlinux.org/current (RSA maybe-signed)
   -1 https://repo-default.voidlinux.org/current/multilib/nonfree (RSA maybe-signed)
   -1 https://repo-default.voidlinux.org/current/multilib (RSA maybe-signed)
   -1 https://repo-default.voidlinux.org/current/nonfree (RSA maybe-signed)
   -1 https://repo-default.voidlinux.org/current/debug (RSA maybe-signed)
==23432== 
==23432== HEAP SUMMARY:
==23432==     in use at exit: 9,857 bytes in 25 blocks
==23432==   total heap usage: 20,175 allocs, 20,150 frees, 3,789,037 bytes allocated
==23432== 
==23432== 10 bytes in 2 blocks are indirectly lost in loss record 1 of 11
==23432==    at 0x4842794: malloc (in /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so)
==23432==    by 0x4AE2FBA: strndup (strndup.c:43)
==23432==    by 0x488BB3F: parse (format.c:508)
==23432==    by 0x488C1FC: xbps_fmt_parse (format.c:569)
==23432==    by 0x488682D: xbps_repo_format (repo.c:236)
==23432==    by 0x4880803: xbps_init (initend.c:169)
==23432==    by 0x10B987: main (main.c:362)
==23432== 
==23432== 25 bytes in 3 blocks are indirectly lost in loss record 2 of 11
==23432==    at 0x4842794: malloc (in /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so)
==23432==    by 0x4AE2FBA: strndup (strndup.c:43)
==23432==    by 0x488C1BD: xbps_fmt_parse (format.c:561)
==23432==    by 0x488682D: xbps_repo_format (repo.c:236)
==23432==    by 0x4880803: xbps_init (initend.c:169)
==23432==    by 0x10B987: main (main.c:362)
==23432== 
==23432== 96 bytes in 1 blocks are still reachable in loss record 3 of 11
==23432==    at 0x4842794: malloc (in /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so)
==23432==    by 0x486B009: _prop_array_alloc (prop_array.c:268)
==23432==    by 0x48869F4: xbps_repo_store (repo.c:332)
==23432==    by 0x10B8C0: main (main.c:294)
==23432== 
==23432== 96 bytes in 1 blocks are still reachable in loss record 4 of 11
==23432==    at 0x4842794: malloc (in /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so)
==23432==    by 0x486C5C9: _prop_dictionary_alloc (prop_dictionary.c:557)
==23432==    by 0x48805CE: xbps_init (initend.c:97)
==23432==    by 0x10B987: main (main.c:362)
==23432== 
==23432== 96 bytes in 1 blocks are still reachable in loss record 5 of 11
==23432==    at 0x4842794: malloc (in /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so)
==23432==    by 0x486C5C9: _prop_dictionary_alloc (prop_dictionary.c:557)
==23432==    by 0x48805E0: xbps_init (initend.c:100)
==23432==    by 0x10B987: main (main.c:362)
==23432== 
==23432== 128 bytes in 1 blocks are still reachable in loss record 6 of 11
==23432==    at 0x484774F: calloc (in /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so)
==23432==    by 0x486B3BD: _prop_array_expand (prop_array.c:297)
==23432==    by 0x486B467: _prop_array_add (prop_array.c:608)
==23432==    by 0x486BE29: prop_array_add (prop_array.c:713)
==23432==    by 0x486BE29: prop_array_add (prop_array.c:705)
==23432==    by 0x4870D43: prop_array_add_cstring (prop_array_util.c:252)
==23432==    by 0x48869DB: xbps_repo_store (repo.c:350)
==23432==    by 0x10B8C0: main (main.c:294)
==23432== 
==23432== 195 (160 direct, 35 indirect) bytes in 1 blocks are definitely lost in loss record 7 of 11
==23432==    at 0x484796C: realloc (in /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so)
==23432==    by 0x488C171: xbps_fmt_parse (format.c:552)
==23432==    by 0x488682D: xbps_repo_format (repo.c:236)
==23432==    by 0x4880803: xbps_init (initend.c:169)
==23432==    by 0x10B987: main (main.c:362)
==23432== 
==23432== 270 bytes in 6 blocks are still reachable in loss record 8 of 11
==23432==    at 0x4842794: malloc (in /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so)
==23432==    by 0x4AE244A: strdup (strdup.c:42)
==23432==    by 0x4880803: xbps_init (initend.c:169)
==23432==    by 0x10B987: main (main.c:362)
==23432== 
==23432== 280 bytes in 7 blocks are still reachable in loss record 9 of 11
==23432==    at 0x4842794: malloc (in /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so)
==23432==    by 0x486FDE2: _prop_string_alloc (prop_string.c:128)
==23432==    by 0x486FDE2: prop_string_create_cstring_nocopy (prop_string.c:187)
==23432==    by 0x4870E59: prop_array_set_cstring_nocopy (prop_array_util.c:253)
==23432==    by 0x4880814: xbps_init (initend.c:169)
==23432==    by 0x10B987: main (main.c:362)
==23432== 
==23432== 504 bytes in 1 blocks are definitely lost in loss record 10 of 11
==23432==    at 0x4842794: malloc (in /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so)
==23432==    by 0x4AC1589: open_memstream (memstream.c:76)
==23432==    by 0x4886819: xbps_repo_format (repo.c:231)
==23432==    by 0x4880803: xbps_init (initend.c:169)
==23432==    by 0x10B987: main (main.c:362)
==23432== 
==23432== 8,192 bytes in 1 blocks are still reachable in loss record 11 of 11
==23432==    at 0x484774F: calloc (in /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so)
==23432==    by 0x4AC15B3: open_memstream (memstream.c:83)
==23432==    by 0x4886819: xbps_repo_format (repo.c:231)
==23432==    by 0x4880803: xbps_init (initend.c:169)
==23432==    by 0x10B987: main (main.c:362)
==23432== 
==23432== LEAK SUMMARY:
==23432==    definitely lost: 664 bytes in 2 blocks
==23432==    indirectly lost: 35 bytes in 5 blocks
==23432==      possibly lost: 0 bytes in 0 blocks
==23432==    still reachable: 9,158 bytes in 18 blocks
==23432==         suppressed: 0 bytes in 0 blocks
==23432== 
==23432== For lists of detected and suppressed errors, rerun with: -s
==23432== ERROR SUMMARY: 2 errors from 2 contexts (suppressed: 0 from 0)

potential other useful template variables that I can think of:

  • {machine} for plain uname -m arch
  • {libc}

I can implement those if we want

closes #573

@classabbyamp classabbyamp mentioned this pull request Nov 1, 2023
Closed
Duncaen and others added 28 commits January 11, 2024 13:00
@Duncaen @classabbyamp
lib/compat/humanize_number.c: switch to freebsd version
36df78d
@Duncaen @classabbyamp
lib: add xbps_fmt* functions for string formatting
ca40961
@Duncaen @classabbyamp
bin/xbps-query: add -F/--format flag
b3ba0f2
@Duncaen @classabbyamp
tests: add tests for xbps_fmt* functions
0813ea7
@Duncaen @classabbyamp
lib/format.c: new humanize format !humanize[ ][.][width][minscale[max…
5972c11 
…scale]][i]
@Duncaen @classabbyamp
bin/xbps-query: document format flag
03b4ab1
@Duncaen @classabbyamp
lib/format.c: split/cleanup code
870b70c
@Duncaen @classabbyamp
lib/format.c: change escaped [{}] to \\[{}]
0982b22
@Duncaen @classabbyamp
lib/format.c: add some more escape sequences
c5388cd
@Duncaen @classabbyamp
bin/xbps-query: update/clean format documentation
574938d
@Duncaen @classabbyamp
bin/xbps-query: fix humanize "i" SI prefixs documentation
c95656e
@Duncaen @classabbyamp
lib/format.c: refactor a bit
16afd80
@Duncaen @classabbyamp
lib/format.c: add optional default to format string variables
eccfbeb
@Duncaen @classabbyamp
lib/format.c: simplify escape characters
cd41df8
@Duncaen @classabbyamp
lib/format.c: fix parsing empty default string
d324263
@Duncaen @classabbyamp
lib/format.c: fix xbps_fmts*
8b0663b
@Duncaen @classabbyamp
lib/format.c: handle errors from nexttok
902ad16
@Duncaen @classabbyamp
lib/json.c: add json printing stuff
5dfadfa
@Duncaen @classabbyamp
lib/format.c: add json value conversion
8efe558
@Duncaen @classabbyamp
bin/xbps-query: add --json flag as alternative to --format
9a95a39
@classabbyamp
lib/format: add \e escape, fix typos in format spec, remove dbg print
fb28e5c
@Duncaen @classabbyamp
bin/xbps-query: make --json compact if there is no indention
5640d57
@Duncaen @classabbyamp
bin/xbps-query: cleanup mode handling
c0422e2
@Duncaen @classabbyamp
bin/xbps-query: move --list-repo to main.c, list.c is exclusively pac…
6a5dae8 
…kage lists now
@Duncaen @classabbyamp
bin/xbps-query: add format string support to --list-repos
66c6216
@Duncaen @classabbyamp
bin/xbps-query: document the --json flag better
22cd693
@Duncaen @classabbyamp
lib/format.c: IWYU
95e8568
@Duncaen @classabbyamp
include: split xbps_fmt stuff into its own header
b5e66ba
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

repo reorg bikeshed
2 participants
@classabbyamp @Duncaen