I'm not sure about formatting guidelines, lemme know if any error is there regarding file formatting, max-line lengths etc.

Also lemme know if we need to force generate a new indexes for a remote repository (instead of syncing with xlocate.git, even if available)?

Using xbps repos is wrong, you can't assume that /xlocate is there.

The only sane way to support multiple repos is to allow the user to specify the git repos.

* Search for all the conf files defining the `repository` through `/etc/xbps.d` and `/usr/share/xbps.d` prioritizing the former.

Your implementation is broken, repository files can include multiple repositories. xbps-query -L should be used.

This makes -g obselete, so is removed. Shebang is changed to bash to utilize arrays and [[ checking builtin over the test command [.

So how are we supposed to create the actual index?

Using xbps repos is wrong, you can't assume that /xlocate is there.

Yes we're checking if there exists a git repository by git ls-remote, otherwise will fallback with xbps-query as usual :)

repository files can include multiple repositories.

Oh didn't knew about that, I thought the files are just shell scripts defining shell variables, like /etc/os-release, as they had = without spaces. xbps-query -L should be used I'll make sure to check about that.

So how are we supposed to create the actual index?

Oh if we mean about the official remote repository, then I was also worried about that post writing the changes, also stated in Also lemme know if we need to force generate a new indexes for a remote repository (instead of syncing with xlocate.git, even if available)?. I'll make sure to provide a better solution about that tomorrow 👀

What is number in front of result of xbps-query -L? I didn't found any mentions in the man page 👀

What is number in front of result of xbps-query -L? I didn't found any mentions in the man page eyes

     -L, --list-repos
         Lists repositories and the number of packages contained on them. If a
         repository is not available the number of packages will be -1.  The
         -v option can be used to show more detailed information of remote
         repositories.

Yes we're checking if there exists a git repository by git ls-remote, otherwise will fallback with xbps-query as usual :)

I'm talking about the urls, you can't assume the path of xlocate, xbps repositories are structured differently and mirros might put everything in more directories.

• https://alpha.de.repo.voidlinux/current/aarch64
• https://alpha.de.repo.voidlinux/current/musl
• https://alpha.de.repo.voidlinux/current
• https://some-mirror.com/voidlinux/...

Thank you!

I'm talking about the urls, you can't assume the path of xlocate, xbps repositories are structured differently and mirros might put everything in more directories.

Um, but those repository don't have prebuilt xlocate indexes attached to them (or I'm missing something), a musl package may contain some more or maybe less file(s), their prebuilt indexes aren't in https://alpha.de.repo.voidlinux/xlocate/xlocate.git (main/glibc indexes) I assume. I think we'll use xbps-query -Mio for building their index in that case. Correct me if there's anything wrong.

How do you come from https://alpha.de.repo.voidlinux/current or https://alpha.de.repo.voidlinux/current/musl to https://alpha.de.repo.voidlinux/xlocate and from https://some-mirror.com/voidlinux/current or https://some-mirror.com/voidlinux/current/musl to https://some-mirror.com/voidlinux/xlocate.

The main point is that its not really possible to get the repo and/or assume where the xlocate directory is located it.
It would be a lot simpler to just have the user specify the xlocate git remotes and then use them, instead of trying to come up with some scheme that assumes a bunch of things about configured repositories.

By replacing /<last-part-of-url> with /xlocate/xlocate.git 👀 I don't think there's a https://alpha.de.repo.voidlinux/current/xlocate/xlocate.git so it should just move on to xbps-query for the /musl or other subrepo?

Thats the point, that doens't work for musl while it should work, and you can't replace everything since that would break mirrors.

There is no point in trying to guess xlocate repos or enforcing some directory layout if the user just configures them.

so it should just move on to xbps-query for the /musl or other subrepo?

No, it should work like it currently does, simply don't use xbps repositories to guess locations of xlocate repositories.

Ok I had some misconceptions lately, the https://alpha.de.repo.voidlinux/xlocate/xlocate.git is xlocate repo only for glibc x86_64 right?

If it does we should just be able to pull from the guess (saving lot of cpu work), if not we just used xbps-query to build the indexes locally for rest of custom repo / sub repo, so we can search through all of it?

Otherwise -g currently does the same except it just delegates everything to xbps-query, taking much time and cpu to build full index of both large official repo and possibly small custom repo.

Ok I had some misconceptions lately, the https://alpha.de.repo.voidlinux/xlocate/xlocate.git is xlocate repo only for glibc x86_64 right?

Its generated from x86_64-glibc, but that doesn't matter, we use it for every architecture and libc.

Otherwise -g currently does the same except it just delegates everything to xbps-query, taking much time and cpu to build full index of both large official repo and possibly small custom repo.'

Right, and we still need that since that is used to generate /xlocate on the repository server.

Its generated from x86_64-glibc, but that doesn't matter, we use it for every architecture and libc.

So if we club index of all repositories (including subrepos) coming from a certain base url into one single commit, will it be better?

xlocate already contains all repos, I don't get the purpose of the PR, you can't derive the xlocate git repo from the repository urls and there are no other xlocate git repo to even add. If you want to have xlocate on you own repo then it would be simpler to allow to add additional git repositories urls to xlocate than trying to assume things based on configured xbps repos.

Is it better now, -g and -S will work as they were, with added allowance of defining multiple-repos with any of

• xlocate git repository url
• or local xbps repo path like hostdir/binpkgs
• or any remote xbps repository say mirrors

at env variable, and if one/more argument supplied to -g, it will build index only of that repo(s) instead of all.

Example:

It generates commit as:

  xtools on  master >>> git -C $HOME/.cache/xlocate.git log
commit c738d2ec1c86147e6be1fae1abf8d123158e9e90 (HEAD -> master)
Author: xupdategit <xupdategit@none>
Date:   Sat Mar 26 15:25:26 2022 +0530

    xupdategit: /home/animesh/Projects/void-packages/hostdir/binpkgs/xpack

commit 7cd8eb515eec512b04cd7aae29341dd4876dfb75
Author: xupdategit <xupdategit@none>
Date:   Sat Mar 26 15:25:16 2022 +0530

    xupdategit: https://alpha.de.repo.voidlinux.org/xlocate/xlocate.git

for each of the repository specified in the env variable. It also keeps track of the remote updates are present by hashing *-repodata for local repo or remote xlocate git's HEAD hash. As such if repo corresponding to 2nd commit has changes, it'll only update 2, 3, 4... and the first should be unaffected, making operation efficient to what's needed to be generated/pulled.