Draupnir: init at 2.0.0-beta.6 #274052

TheArcaneBrony · 2023-12-13T20:07:07Z

Description of changes

NOTE: This PR is a successor of #222939. The previous PR was closed by accident during a rebase operation.

Original description:

Draupnir is a hardfork of Mjolnir. Mjolnir package has been unmaintained due to upstream bugs. This package is mostly a drop-in replacement. This package also uses newer methods of handling dependencies.

Repository can be found at:
https://github.com/the-draupnir-project/Draupnir

Note on testing: package and module were (and still are) tested in our production environment, works fine as far as it's been used.
Note on replacing Mjolnir: Not sure whether this is appropriate, due to general usage of the bot changing. You can however fully automatically migrate from mjolnir to draupnir.

Things done

Add a 👍 reaction to pull requests you find important.

gaykitty · 2023-12-25T02:26:54Z

Result of nixpkgs-review pr 274052 run on x86_64-linux 1

2 packages blacklisted:

nixos-install-tools
tests.nixos-functions.nixos-test

1 package built:

draupnir

mweinelt

I see a lot of potential to improve the module and to bring it in line with today's standards.

I can see that it is a copy of the mjolnir module, but I'm not willing to accept a new module of similar quality, because it is hard to maintain.

This is especially relevant with RFC42 (settings & formatters) in mind

pkgs/servers/draupnir/default.nix

nixos/modules/services/matrix/draupnir.nix

TheArcaneBrony · 2024-01-20T18:53:02Z

Time for the almighty rebase on nixos:master... Please don't close the PR again...

TheArcaneBrony · 2024-01-20T19:01:44Z

🎊 It didn't close as merged!

TheArcaneBrony · 2024-01-20T19:02:42Z

(Renamed the package init commit)

TheArcaneBrony · 2024-01-27T07:52:34Z

(^ master rebase)

teutat3s · 2024-07-21T21:03:23Z

This now looks good to go for me. A co-authored-by on the test commit would be nice, but is not a requirement for me. Thanks for continuing to push this forward.

mweinelt

Went over everything except the test.

maintainers/maintainer-list.nix

mweinelt · 2024-07-21T16:09:52Z

pkgs/by-name/dr/draupnir/package.nix

+    preBuild = ''
+      # copy built modules to package...
+      echo "Copying built matrix-sdk-crypto-nodejs modules to package..."
+      cp -a ${matrix-sdk-crypto-nodejs}/lib/node_modules/* node_modules/


This could probably just be a symlink.

pkgs/by-name/dr/draupnir/package.nix

mweinelt · 2024-07-21T16:20:40Z

nixos/modules/services/matrix/draupnir.md

+{
+  services.draupnir = {
+    enable = true;
+    pantalaimon = {
+       enable = true;
+       username = "draupnir";
+       passwordFile = "/run/secrets/draupnir-password";
+       options = {
+        homeserver = "http://localhost:8008";
+        ssl = false;
+       };
+    };
+    settings = {
+      managementRoom = "!yyy:domain.tld";
+      protectedRooms = [
+        "https://matrix.to/#/!xxx:domain.tld"
+      ];
+    };
+  };
+}


Make sure to format this with nixfmt-rfc-style.

can nixfmt-rfc-style handle markdown files, or should i extract this nix excerpt into a separate file to process?

Extract, format, reinsert.

nixos/modules/services/matrix/draupnir.nix

Co-authored-by: teutat3s <[email protected]>

TheArcaneBrony · 2024-07-22T16:05:14Z

^ just renamed the maintainers commit and added the co author to the tests pr :)

teutat3s

What's the intention behind moving homeserverUrl? This now needs changing in the docs to fix the manual.

nixos/modules/services/matrix/draupnir.md

Scrumplex · 2024-08-05T09:00:28Z

pkgs/by-name/dr/draupnir/package.nix

+    yarnLock = src + "/yarn.lock";
+    packageJSON = ./package.json;


I don't think we need these? I can build the package without setting these.

This confused me as well to figure out, the package builds fine without these but the nixpkgs CI fails due to importing the package.json from source. I'm not sure what the yarnLock there is for though as it seems to be duplicated from offlineCache.yarnLock

I see, apparently this causes IFD. Disregard this then ^^

Unresolved this conversation, does anyone think it would be possible to address this in favor of #320528 and #327064?

No, that would be scope creep at this point.

Scrumplex · 2024-08-05T09:01:26Z

pkgs/by-name/dr/draupnir/package.nix

+      tests = {
+        inherit (nixosTests) draupnir;
+      };
+      updateScript = ./update.sh;


Is there anything special here, that nix-update-script couldn't handle? It can also deal with updating the Yarn cache hash in the derivation.

Suggested change

updateScript = ./update.sh;

updateScript = nix-update-script { };

Hashes are stored in a separate file, I think the reason was that nix-update-script couldn't handle the Yarn cache hash? I'd have to test this once a new Draupnir version comes out.

TheArcaneBrony · 2024-08-08T02:38:34Z

pkgs/by-name/dr/draupnir/package.nix

Question, does anyone know how to make better-sqlite3 work under nix? Seems adding sqlite as nativeBuildInput doesn't work, and this causes the roomStateBackingStore feature of Draupnir to cause a startup failure as better-sqlite3 fails to find it's bindings file.

share/draupnir/node_modules/better-sqlite3/ share/draupnir/node_modules/better-sqlite3/src/better_sqlite3.cpp share/draupnir/node_modules/better-sqlite3/src/better_sqlite3.hpp

Looks like it is here, but does not get build. Ideally we can build it outside of draupnir, as a dedicated package, and patch it in.

Yep, from earlier testing, seems to also be possible to rebuild it with a single command, and when I raised possibly making it a sub-package I was recommended not to.

diff

diff --git a/pkgs/by-name/dr/draupnir/package.nix b/pkgs/by-name/dr/draupnir/package.nix index fb2faad56350..81a8ee568fbd 100644 --- a/pkgs/by-name/dr/draupnir/package.nix +++ b/pkgs/by-name/dr/draupnir/package.nix @@ -3,7 +3,10 @@ , makeWrapper , nodejs , matrix-sdk-crypto-nodejs +, python3 , sqlite +, srcOnly +, removeReferencesTo , mkYarnPackage , fetchYarnDeps , nixosTests @@ -12,6 +15,7 @@ # docs: https://github.com/NixOS/nixpkgs/blob/master/doc/languages-frameworks/javascript.section.md#yarn2nix-javascript-yarn2nix let hashesFile = builtins.fromJSON (builtins.readFile ./hashes.json); + nodeSources = srcOnly nodejs; in mkYarnPackage rec { pname = "draupnir"; @@ -37,11 +41,30 @@ in yarnLock = src + "/yarn.lock"; packageJSON = ./package.json; + pkgConfig = { + "@matrix-org/matrix-sdk-crypto-nodejs" = { + postInstall = '' + # replace with the built package + cd .. + rm -r matrix-sdk-crypto-nodejs + ln -s ${matrix-sdk-crypto-nodejs}/lib/node_modules/@matrix-org/* ./ + ''; + }; + + better-sqlite3 = { + nativeBuildInputs = [ python3 ]; + postInstall = '' + # build native sqlite bindings + npm run build-release --offline --nodedir="${nodeSources}" + find build -type f -exec \ + ${removeReferencesTo}/bin/remove-references-to \ + -t "${nodeSources}" {} \; + ''; + }; + }; + #prebuild phase preBuild = '' - # copy built modules to package... - echo "Copying built matrix-sdk-crypto-nodejs modules to package..." - cp -a ${matrix-sdk-crypto-nodejs}/lib/node_modules/* node_modules/ echo "Adding version.txt..." mkdir -p deps/draupnir/ echo "${version}-nix" > deps/draupnir/version.txt

matrix-appservice-discord also seems to bundle it in.

Forgot to mention, I applied your patch in bd7179d

Fixes build with corepack upstream introduced by the-draupnir-project/Draupnir#472 Upstream may move to yarn v4, need to investigate options. (the-draupnir-project/Draupnir#475) Looks like the recommended option is to use a fixed-output derivation? Sources: - https://www.reddit.com/r/NixOS/comments/1f8tq94/its_2024_has_anyone_figured_out_how_to_build_a/ - https://github.com/NixOS/nixpkgs/blob/master/pkgs/applications/networking/cluster/tilt/assets.nix

TheArcaneBrony · 2024-09-15T23:07:20Z

pkgs/by-name/dr/draupnir/package.nix

+      mkdir -p deps/draupnir/
+      echo "${version}-nix" > deps/draupnir/version.txt
+
+      sed -i 's/corepack //g' deps/draupnir/package.json


(Copy paste from commit description)
fixup! draupnir: init at 2.0.0-beta.6 - ugly sed hack to drop corepack

Fixes build with corepack upstream introduced by the-draupnir-project/Draupnir#472
Upstream may move to yarn v4, need to investigate options. (the-draupnir-project/Draupnir#475)
Looks like the recommended option is to use a fixed-output derivation?

Sources:

https://www.reddit.com/r/NixOS/comments/1f8tq94/its_2024_has_anyone_figured_out_how_to_build_a/

https://github.com/NixOS/nixpkgs/blob/master/pkgs/applications/networking/cluster/tilt/assets.nix

Any suggestions? I'm sure this is an ugly, unnecessary hack, but I couldn't find any documentation on the topic.

…x-sdk-crypto-nodejs and better-sqlite3 Co-Authored-By: div72 <div72@localhost>

TheArcaneBrony requested a review from alyssais as a code owner December 13, 2023 20:07

github-actions bot added 6.topic: nixos 8.has: module (update) 6.topic: lib The Nixpkgs function library labels Dec 13, 2023

ofborg bot added 8.has: package (new) 11.by: package-maintainer 10.rebuild-darwin: 1-10 10.rebuild-darwin: 1 10.rebuild-linux: 1-10 labels Dec 13, 2023

h7x4 added 8.has: documentation 8.has: module (new) labels Dec 26, 2023

mweinelt requested changes Jan 4, 2024

View reviewed changes

TheArcaneBrony requested review from piegamesde and Janik-Haag as code owners January 5, 2024 18:02

github-actions bot removed the 8.has: documentation label Jan 5, 2024

Janik-Haag removed request for piegamesde and Janik-Haag January 6, 2024 02:46

ofborg bot removed the 10.rebuild-darwin: 1 label Jan 20, 2024

TheArcaneBrony changed the title ~~Draupnir: init at 1.85.1~~ Draupnir: init at 1.86.0 Jan 20, 2024

github-actions bot removed the 6.topic: lib The Nixpkgs function library label Jan 20, 2024

ofborg bot added the 2.status: merge conflict label Jan 20, 2024

TheArcaneBrony force-pushed the master branch from 28ee30d to a0c9de0 Compare January 20, 2024 18:56

TheArcaneBrony force-pushed the master branch from a0c9de0 to cd3ce15 Compare January 20, 2024 19:02

ofborg bot removed the 2.status: merge conflict label Jan 20, 2024

TheArcaneBrony force-pushed the master branch from cd3ce15 to 38a2e8f Compare January 27, 2024 07:52

mweinelt reviewed Jul 21, 2024

View reviewed changes

TheArcaneBrony and others added 4 commits July 22, 2024 15:50

maintainers: add RorySys

4fffcb7

draupnir: init at 2.0.0-beta.4

1e396ac

nixos/draupnir: init

bbadc5e

draupnir: add tests

ee8cbf8

Co-authored-by: teutat3s <[email protected]>

TheArcaneBrony force-pushed the master branch from 3b7380e to ee8cbf8 Compare July 22, 2024 16:04

TheArcaneBrony added 10 commits July 22, 2024 16:12

fixup! draupnir: init at 2.0.0-beta.4 - Remove hook call echos

4ac0923

fixup! draupnir: init at 2.0.0-beta.4 - set fetchYarnDeps name

4727e58

fixup! nixos/draupnir: init - remove dead code

cbdb027

fixup! nixos/draupnir: init - Moderation bot, not tool

8c6092f

fixup! nixos/draupnir: init - witout -> without

0ed8bbc

fixup! nixos/draupnir: init - clean up null stripping

65b7eff

fixup! nixos/draupnir: init - move homeserverUrl to global

dfc2fb1

fixup! nixos/draupnir: init - drop module config reference

e3adb0c

fixup! draupnir: add tests - move homeserverUrl

1fac8d3

fixup! nixos/draupnir: init - remove parial assertion

a13cc66

teutat3s reviewed Jul 23, 2024

View reviewed changes

nixos/modules/services/matrix/draupnir.md Outdated Show resolved Hide resolved

Scrumplex reviewed Aug 5, 2024

View reviewed changes

TheArcaneBrony added 2 commits August 8, 2024 02:24

fixup! nixos/draupnir: init - Update docs somewhat

0c8581d

fixup! draupnir: init at 2.0.0-beta.4 - Add sqlite as native build input

dbf6ae4

TheArcaneBrony commented Aug 8, 2024

View reviewed changes

wegank removed the 12.approvals: 1 label Sep 8, 2024

TheArcaneBrony commented Sep 15, 2024

View reviewed changes

TheArcaneBrony changed the title ~~Draupnir: init at 2.0.0-beta.4~~ Draupnir: init at 2.0.0-beta.6 Sep 15, 2024

fixup! draupnir: init at 2.0.0-beta.4 - Switch to pkgConfig for matri…

bd7179d

…x-sdk-crypto-nodejs and better-sqlite3 Co-Authored-By: div72 <div72@localhost>

TheArcaneBrony mentioned this pull request Sep 16, 2024

Migrate to modern yarn with pnpm linking the-draupnir-project/Draupnir#562

Open

winterqt mentioned this pull request Sep 18, 2024

draupnir: init at 2.0.0-beta.6 #342893

Draft

13 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Draupnir: init at 2.0.0-beta.6 #274052

Draupnir: init at 2.0.0-beta.6 #274052

TheArcaneBrony commented Dec 13, 2023

gaykitty commented Dec 25, 2023

mweinelt left a comment

TheArcaneBrony commented Jan 20, 2024

TheArcaneBrony commented Jan 20, 2024

TheArcaneBrony commented Jan 20, 2024

TheArcaneBrony commented Jan 27, 2024

teutat3s commented Jul 21, 2024

mweinelt left a comment

mweinelt Jul 21, 2024

mweinelt Jul 21, 2024

TheArcaneBrony Jul 22, 2024

mweinelt Jul 23, 2024 •

edited

Loading

TheArcaneBrony commented Jul 22, 2024

teutat3s left a comment •

edited

Loading

Scrumplex Aug 5, 2024

TheArcaneBrony Aug 6, 2024

Scrumplex Aug 6, 2024

TheArcaneBrony Aug 24, 2024

mweinelt Sep 16, 2024

Scrumplex Aug 5, 2024

TheArcaneBrony Aug 6, 2024

TheArcaneBrony Aug 8, 2024

mweinelt Aug 8, 2024

TheArcaneBrony Aug 18, 2024

div72 Sep 14, 2024

TheArcaneBrony Sep 18, 2024

TheArcaneBrony Sep 15, 2024

	updateScript = ./update.sh;
	updateScript = nix-update-script { };

Draupnir: init at 2.0.0-beta.6 #274052

Are you sure you want to change the base?

Draupnir: init at 2.0.0-beta.6 #274052

Conversation

TheArcaneBrony commented Dec 13, 2023

Description of changes

Things done

gaykitty commented Dec 25, 2023

mweinelt left a comment

Choose a reason for hiding this comment

TheArcaneBrony commented Jan 20, 2024

TheArcaneBrony commented Jan 20, 2024

TheArcaneBrony commented Jan 20, 2024

TheArcaneBrony commented Jan 27, 2024

teutat3s commented Jul 21, 2024

mweinelt left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mweinelt Jul 23, 2024 • edited Loading

Choose a reason for hiding this comment

TheArcaneBrony commented Jul 22, 2024

teutat3s left a comment • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mweinelt Jul 23, 2024 •

edited

Loading

teutat3s left a comment •

edited

Loading