update to mirage 4.2.0 & mirage-xen 8.0.0 #140

palainp · 2022-03-30T07:15:54Z

~~With these changes it compiles with mirage 4.0 and runs on qubes 4.1.~~
EDIT: This PR is now a general update to mirage 4.2+mirage-xen 8.0.0 and update of qubes-builder + docker scripts.

xaki23 · 2022-05-27T03:08:14Z

as part of the whole mess around (the lower parts of) QubesOS/qubes-issues#6162 and Solo5/solo5#516 i can confirm this branch works at least as much as any other, if not better.

tl;dr: LGTM, please merge

hannesm · 2022-05-27T13:03:01Z

config.ml


 let () =
-  register "qubes-firewall" [main $ default_random $ default_monotonic_clock]
+  register "qubes-firewall" [main $ default_random $ default_monotonic_clock $ default_time]
    ~argv:no_argv


Suggested change

~argv:no_argv

as discussed in Solo5/solo5#516 (comment) and following, we don't need the no_argv anymore

The command that permits to remove this line is in the README.md both for new VM creation:

qubes-mirage-firewall/README.md

Line 85 in ef2419b

qvm-features mirage-firewall no-default-kernelopts 1

and for VM updates:

qubes-mirage-firewall/README.md

Line 99 in ef2419b

qvm-features mirage-firewall no-default-kernelopts 1

It's ok to be removed. 👍

Do you think I should also add the command elsewhere?

How's the user story when installing an RPM, is there some kind of qvm-features executions already in some script?

The arguments for the kernel come from the grub.cfg file : https://github.com/marmarek/qubes-builder-mirage/blob/master/template_scripts/grub.cfg
Regarding other commands to properly setup the template VM, there is a post-install script included in the .rpm file, I still need to find if we can provide specific commands to set kernel to grub2-pvh.

the RPM postinstall script currently doesnt really work for mirage VMs in general. (the rpm installs though and allows the user to adjust the required prefs/features)
the problem is there does not seem to be a sane path to have https://github.com/QubesOS/qubes-linux-template-builder (which is used to roll the rpm) use a custom install script. and/or a "template parameter file" to be included in the template rpm, which could be used by both an in-rpm install script and alternate tooling for installing templates.

from my pov, not a blocker for this PR, it doesnt really make anything worse than it has been since forever.

…ubes

palainp · 2022-07-27T12:34:23Z

Thanks @hannesm for the mirage update. I manually pinned mirage.4.2 until the merge in opam-repository and updated the memory_pressure.ml + config.ml for using mirage-xen.8.0.0. It seems to works correctly with the new quick_stat estimation.
For @xaki23 if you want to try before opam-repository is updated, you may:

opam update -yu && \
opam pin functoria.4.2.0 git+https://github.com/mirage/mirage.git#main -y && \
opam pin functoria-runtime.4.2.0 git+https://github.com/mirage/mirage.git#main -y && \
opam pin mirage-runtime.4.2.0 git+https://github.com/mirage/mirage.git#main -y && \
opam pin mirage.4.2.0 git+https://github.com/mirage/mirage.git#main -y

before compiling with the last commit.
EDIT: the various pins are no longer needed since the new mirage version is actually out, opam update -yu should be enough.

config.ml

palainp · 2022-08-12T06:27:27Z

As @Szewcson I have a correct running here with:

[user@dom0 Desktop]$ sha256sum /var/lib/qubes/vm-kernels/mirage-test/vmlinuz 
588e921b9d78a99f6f49d468a7b68284c50dabeba95698648ea52e99b381723b  /var/lib/qubes/vm-kernels/mirage-test/vmlinuz

[2022-08-12 08:20:21] Solo5: Xen console: port 0x2, ring @0x00000000FEFFF000
[2022-08-12 08:20:21]             |      ___|
[2022-08-12 08:20:21]   __|  _ \  |  _ \ __ \
[2022-08-12 08:20:21] \__ \ (   | | (   |  ) |
[2022-08-12 08:20:21] ____/\___/ _|\___/____/
[2022-08-12 08:20:21] Solo5: Bindings version v0.7.3
[2022-08-12 08:20:21] Solo5: Memory map: 64 MB addressable:
[2022-08-12 08:20:21] Solo5:   reserved @ (0x0 - 0xfffff)
[2022-08-12 08:20:21] Solo5:       text @ (0x100000 - 0x31bfff)
[2022-08-12 08:20:21] Solo5:     rodata @ (0x31c000 - 0x386fff)
[2022-08-12 08:20:21] Solo5:       data @ (0x387000 - 0x544fff)
[2022-08-12 08:20:21] Solo5:       heap >= 0x545000 < stack < 0x4000000
[2022-08-12 08:20:21] 2022-08-12 06:20:21 -00:00: INF [qubes.rexec] waiting for client...
[2022-08-12 08:20:21] 2022-08-12 06:20:21 -00:00: INF [qubes.gui] waiting for client...
[2022-08-12 08:20:21] 2022-08-12 06:20:21 -00:00: INF [qubes.db] connecting to server...
[2022-08-12 08:20:21] 2022-08-12 06:20:21 -00:00: INF [qubes.db] connected
...

@TommyTran732 can you copy/paste your vm configuration and the last part of the qubes log to help understanding the crash please?

TommyTran732 · 2022-08-14T11:25:09Z

As @Szewcson I have a correct running here with:

[user@dom0 Desktop]$ sha256sum /var/lib/qubes/vm-kernels/mirage-test/vmlinuz 
588e921b9d78a99f6f49d468a7b68284c50dabeba95698648ea52e99b381723b  /var/lib/qubes/vm-kernels/mirage-test/vmlinuz

[2022-08-12 08:20:21] Solo5: Xen console: port 0x2, ring @0x00000000FEFFF000
[2022-08-12 08:20:21]             |      ___|
[2022-08-12 08:20:21]   __|  _ \  |  _ \ __ \
[2022-08-12 08:20:21] \__ \ (   | | (   |  ) |
[2022-08-12 08:20:21] ____/\___/ _|\___/____/
[2022-08-12 08:20:21] Solo5: Bindings version v0.7.3
[2022-08-12 08:20:21] Solo5: Memory map: 64 MB addressable:
[2022-08-12 08:20:21] Solo5:   reserved @ (0x0 - 0xfffff)
[2022-08-12 08:20:21] Solo5:       text @ (0x100000 - 0x31bfff)
[2022-08-12 08:20:21] Solo5:     rodata @ (0x31c000 - 0x386fff)
[2022-08-12 08:20:21] Solo5:       data @ (0x387000 - 0x544fff)
[2022-08-12 08:20:21] Solo5:       heap >= 0x545000 < stack < 0x4000000
[2022-08-12 08:20:21] 2022-08-12 06:20:21 -00:00: INF [qubes.rexec] waiting for client...
[2022-08-12 08:20:21] 2022-08-12 06:20:21 -00:00: INF [qubes.gui] waiting for client...
[2022-08-12 08:20:21] 2022-08-12 06:20:21 -00:00: INF [qubes.db] connecting to server...
[2022-08-12 08:20:21] 2022-08-12 06:20:21 -00:00: INF [qubes.db] connected
...

@TommyTran732 can you copy/paste your vm configuration and the last part of the qubes log to help understanding the crash please?

Sorry for the late reply, where do you get those logs?

I am using the same VM configuration in the README. It works when I build Mirage 3.10, but when I replace the vmlinuz with the 4.2 version it just crashes shortly after it launches.

palainp · 2022-08-14T12:35:15Z

The logs come from tail -f /var/log/xen/console/guest-mirage-test.log. With a correct running with mirage 3.10 (post solo5+pvh) I don't see what can be your issue now, so the logs will certainly be useful :)

TommyTran732 · 2022-08-25T08:57:23Z

I figured out what was wrong. I forgot to set qvm-features mirage-firewall no-default-kernelopts 1 :D
For some reason it will happily run with the 3.10 kernel but it crash with 4.2. Anyhow, this was user error on my part. The PR is good to go I think :P

TommyTran732 · 2022-08-26T09:43:00Z

@palainp Mirage 4.2.1 just came out a few hours ago. Is there any chance you can update this PR? (Would be a great opportunity to bump the ocaml docker image version too) :D

palainp · 2022-08-29T09:41:13Z

@TommyTran732 @Szewcson: The build should be fine with mirage 4.2.1 now. I ran into troubles for the compilation, maybe you will have to correctly clean the repository. The shasum has not been changed because the mirage update only affects the build system.

hannesm · 2022-08-30T14:29:23Z

Thanks for all your hard work! :)

palainp · 2022-08-30T14:45:38Z

Thanks @hannesm! I hope it can also be built fine with orb at robbur. The next step will be to work on #115 :)

hannesm · 2022-08-30T14:55:57Z

Yes, see https://builds.robur.coop/job/qubes-firewall/build/4ead4483-4236-4b87-9498-b86ed931df8b -- indeed I just sent you a mail how to approach #115

palainp added 5 commits March 30, 2022 03:12

update to mirage 4.0.0 & mirage-xen 7.0.0

a99d7f8

bump lower bound for mirage-xen

3cce2a5

update qubes-builder script for mirage 4.0

dbe068c

fix opam option

6f257c5

fix kernel name

f33db2b

palainp mentioned this pull request Apr 4, 2022

Why isn't the Mirage Qubes Firewall available as a Qubes Template in the upstream repos #115

Open

hannesm reviewed May 27, 2022

View reviewed changes

no_argv not needed anymore with no-default-kernelopts for the VM in Q…

7718c95

…ubes

hannesm mentioned this pull request Jul 26, 2022

With solo5, firewall runs out of memory after a while and can't recover #120

Closed

use the new quick_stat+trim from mirage-xen 8.0.0

68ab4f3

palainp changed the title ~~update to mirage 4.0.0 & mirage-xen 7.0.0~~ update to mirage 4.2.0 & mirage-xen 8.0.0 Jul 27, 2022

TommyTran732 reviewed Aug 8, 2022

View reviewed changes

config.ml Show resolved Hide resolved

palainp added 2 commits August 9, 2022 14:16

update docker build for mirage 4.2

e73c160

must make depend before building solo5 with make tar

ba1b044

update to mirage 4.2.1

df4f7bf

hannesm merged commit 61767ef into mirage:master Aug 30, 2022

This was referenced Aug 30, 2022

Build fails since security improvements in git protocole #139

Closed

compile problems. #131

Closed

palainp deleted the mirage4 branch August 30, 2022 14:36

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

update to mirage 4.2.0 & mirage-xen 8.0.0 #140

update to mirage 4.2.0 & mirage-xen 8.0.0 #140

palainp commented Mar 30, 2022 •

edited

Loading

xaki23 commented May 27, 2022

hannesm May 27, 2022

palainp May 27, 2022

hannesm May 27, 2022

palainp May 27, 2022

xaki23 Jun 21, 2022

palainp commented Jul 27, 2022 •

edited

Loading

palainp commented Aug 12, 2022

TommyTran732 commented Aug 14, 2022 •

edited

Loading

palainp commented Aug 14, 2022

TommyTran732 commented Aug 25, 2022 •

edited

Loading

TommyTran732 commented Aug 26, 2022 •

edited

Loading

palainp commented Aug 29, 2022

hannesm commented Aug 30, 2022

palainp commented Aug 30, 2022

hannesm commented Aug 30, 2022

update to mirage 4.2.0 & mirage-xen 8.0.0 #140

update to mirage 4.2.0 & mirage-xen 8.0.0 #140

Conversation

palainp commented Mar 30, 2022 • edited Loading

xaki23 commented May 27, 2022

hannesm May 27, 2022

Choose a reason for hiding this comment

palainp May 27, 2022

Choose a reason for hiding this comment

hannesm May 27, 2022

Choose a reason for hiding this comment

palainp May 27, 2022

Choose a reason for hiding this comment

xaki23 Jun 21, 2022

Choose a reason for hiding this comment

palainp commented Jul 27, 2022 • edited Loading

palainp commented Aug 12, 2022

TommyTran732 commented Aug 14, 2022 • edited Loading

palainp commented Aug 14, 2022

TommyTran732 commented Aug 25, 2022 • edited Loading

TommyTran732 commented Aug 26, 2022 • edited Loading

palainp commented Aug 29, 2022

hannesm commented Aug 30, 2022

palainp commented Aug 30, 2022

hannesm commented Aug 30, 2022

palainp commented Mar 30, 2022 •

edited

Loading

palainp commented Jul 27, 2022 •

edited

Loading

TommyTran732 commented Aug 14, 2022 •

edited

Loading

TommyTran732 commented Aug 25, 2022 •

edited

Loading

TommyTran732 commented Aug 26, 2022 •

edited

Loading