Only add/modify what's in .src.ini, leaving all the rest untouched

[LorenzoBettini]

Hi

I've just started exploring chezmoi_modify_manager to handle my KDE dotfiles (currently, I'm using the Ansible-specific ini module).

What I'd like to do is have a .src.ini file with only my custom sections and key-values; I'd like these to be merged into the target (if they don't exist, then add them; if they exist, then replace them). So, what's missing in my .src.ini should not be removed from the target. I think this would be a real merging behavior.

I seem to understand that's not currently possible, is that?

I should manually ignore all the sections I don't want to handle (and that's easy: just ignore a whole section); however, for single key-values in a section I should ignore them one by one, is that right?
Or am I missing something?

Thanks in advance

[VorpalBlade]

Hi,

Your suggested behaviour is indeed a different behaviour than what is implemented. The approach currently is "merge everything unless rules say otherwise".

After thinking for a couple of minutes about the different approaches it isn't obvious to me that one is better than the other, with one exception: With "only handle some sections", it becomes difficult to handle re-adding files from the system to update the chezmoi source state. This is something that currently works fairly well and transparently with only minor annotations needed for when you are doing things with set or keyring-transform.

Your suggested workaround of manually ignoring sections could work, but sounds annoying to maintain to be honest. It would be compatible with re-adding files though, so that is a plus.

So I'm wondering why you want to only manage some sections instead of just ignoring only the problematic sections (like recently opened files or window positions). Is there a reason why you think that approach is better? (I'm not necessarily arguing against you, I'm interested to learn and see if there is potential for improvement.)

[LorenzoBettini]

Hi!

Maybe I'm biased by my current Ansible approach, where I only specify my customizations.
I might also be misled by "merge": I think "merging" has the semantics I described. In that respect, mentioning "merging" as a feature might be misleading as well ;)

I haven't considered the re-adding feature (which is a cool feature of chezmoi itself), so, if re-adding would be hard to implement in my view, then it's not worthwhile to investigate it further.

I was thinking about customizing shortcuts: I'd only want to specify my custom ones without having in the source the full list (e.g., see [kwin] in .config/kglobalshortcutsrc). Maybe I shouldn't care about that: in the end, it does not harm to have the full list.

I guess it's just a matter of excluding the known sections that KDE applications continuously modify. The number of ignored sections shouldn't be long (I also looked at your documentation and examples).

I'll try to port my dotfiles accordingly.

I only started the discussion to be sure I hadn't missed such a feature.

[VorpalBlade]

I might also be misled by "merge": I think "merging" has the semantics I described. In that respect, mentioning "merging" as a feature might be misleading as well ;)

The merging semantics implemented by Chezmoi_modify_manager (chmm for short, I should have picked a shorter name in retrospect):

1. If a key or section is ignored, use the value from the config on the system (sys)
2. If a transform is specified for a key or section apply it (special rules apply for each transform, see docs for each).
3. Otherwise pick the value from the chezmoi source state (src)
4. If a key or section is only in sys, but not in src, remove it (unless ignored)

There are additionally a few corner cases in the actual implementation (keys before the first section, sections entirely missing, etc) that are properly handled to do the "obviously expected thing".

I do believe this qualifies as a merging algorithm.

I haven't considered the re-adding feature (which is a cool feature of chezmoi itself), so, if re-adding would be hard to implement in my view, then it's not worthwhile to investigate it further.

Chmm has special support for adding new and readding existing ini-files. Use chezmoi_modify_manager -a or chezmoi_modify_manager -s for this, not the default chezmoi add.

I was thinking about customizing shortcuts: I'd only want to specify my custom ones without having in the source the full list (e.g., see [kwin] in .config/kglobalshortcutsrc). Maybe I shouldn't care about that: in the end, it does not harm to have the full list.

I guess it's just a matter of excluding the known sections that KDE applications continuously modify. The number of ignored sections shouldn't be long (I also looked at your documentation and examples).

Take a look at https://github.com/VorpalBlade/chezmoi_modify_manager/blob/main/doc/examples.md in general for examples of how to configure many KDE settings (not really updated for KDE 6 yet though, but so far I found that not much changed there with regards to ignores, the bigger issue is when you have a mix of KDE 5 and KDE 6 systems, see #98 for more thoughts on that).

My approach for shortcuts specifically is to just manage the entire file. If there is a KDE upgrade that changes things, I just re-add the file to update it in the src state. There is a specific transform for the shortcuts file though, since that is one of the things KDE likes to randomly reformat without reason (see the examples again).

My current file looks like:

#!/usr/bin/env chezmoi_modify_manager

source auto

ignore regex "ActivityManager" "switch-to-activity-.*"
transform regex ".*" ".*" kde-shortcut

{{ if (.custom.is_work) }}
set "khotkeys" "{5cef61b9-6823-4258-85d3-bab6a0bd47de}" "Ctrl+Alt+Y,none,Some work specific command I'm not telling you about" separator="="
{{ end }}
add:remove "khotkeys" "{5cef61b9-6823-4258-85d3-bab6a0bd47de}"

This does trigger a warning currently: [chezmoi_modify_manager WARN] Overlapping regex matches for ActivityManager/switch-to-activity-4bbc8a8d-b350-48fc-9f33-8190f1772eea, first action taken. I'm considering changing the behaviour here such that the order of evaluation is guaranteed (I didn't want to commit to a specific behaviour for this yet). In that case the warning could go away.

[LorenzoBettini]

I started with Dolphin.
In this case, I had to do a more involved exclusion, because I only want these:

[MainWindow]
MenuBar=Enabled
ToolBarsMovable=Disabled

[Open-with settings]
CompletionMode=1

[VersionControl]
enabledPlugins=Dropbox

and I had to do

#!/usr/bin/env chezmoi_modify_manager

source auto

ignore section "KPropertiesDialog"
ignore section "General"
ignore section "KFileDialog Settings"
ignore section "DetailsMode"
ignore section "$Version"
ignore section "PreviewSettings"
ignore "Open-with settings" "History"

maybe it's just a matter of doing the hard work the first time ;)

[LorenzoBettini]

And it's still giving me headaches: after opening Dolphin, if I try to re-apply I always get:

--- a/.config/dolphinrc
+++ b/.config/dolphinrc
@@ -21,6 +21,7 @@
 [MainWindow]
 ToolBarsMovable=Disabled
 
+MenuBar=Enabled
 [Open-with settings]
 CompletionMode=1
 History=kate

so I guess I have to disable that section completely as in the example; but maybe that's one of the things I'd want to keep ;)

[VorpalBlade]

So what is happening here is that a new line from the source state is being added at the end of the section. That "end of section" includes the final blank line in that section. Normally that is not a problem (dolphin will reformat the file next time it runs).

But if it happens every time: Perhaps dolphin is instead removing the line (maybe it moved somewhere else in a newer version?). Otherwise I'm not quite sure what is going on.

[VorpalBlade]

I went looking in my Dolphin. I don't have any "MenuBar" line at all. That doesn't mean much, so I went digging in the dolphin GUI settings. I can't find any settings to enable a menu bar at all.

That might be a KDE 6 thing though, because on my work laptop with KDE 5.27 I do have a menu bar in dolphin and it has a "MenuBar=Disabled" in the dolphinrc (weird! why do I have a menu bar then?)

So is it possible this is related to also upgrading to KDE 6 at the same time?

[VorpalBlade]

Interesting, I found out I could get the menu bar back by pressing Ctrl+M. This adds the line [General] ModifiedStartupSettings=true to the config. Presumably the menu bar is now stored elsewhere in KDE 6.

[VorpalBlade]

• It looks like it is now stored in ~/.config/session/dolphin_dolphin_dolphin. But regardless if what I do I still have MenuBar=Disabled in that file. So that isn't it either. There are a few other dolphin files there too, but in fact those files haven't changed since 2022.
• ~/.local/share/dolphin/dolphinstaterc now exists in KDE 6, but it doesn't seem to be stored there either, at first glance.

However if I diff the file after making a copy and changing the state of the menu bar setting I get:

--- /home/arvid/.local/share/dolphin/dolphinstaterc	2024-03-14 18:16:34.136788163 +0100
+++ /home/arvid/.local/share/dolphin/dolphinstaterc.2	2024-03-14 18:16:27.819730063 +0100
@@ -63,7 +63,7 @@
 HDMI-2 eDP-1 XPosition 1920x1080 eDP-1=274
 HDMI-2 eDP-1 YPosition 1920x1080 HDMI-2=29
 HDMI-2 eDP-1 YPosition 1920x1080 eDP-1=114
-State=AAAA/wAAAAD9AAAAAwAAAAAAAACtAAACmfwCAAAAAvsAAAAWAGYAbwBsAGQAZQByAHMARABvAGMAawAAAAAA/////wAAAAAA////+wAAABQAcABsAGEAYwBlAHMARABvAGMAawEAAAAuAAACmQAAAFMA////AAAAAQAAAAAAAAAA/AIAAAAB+wAAABAAaQBuAGYAbwBEAG8AYwBrAAAAAAD/////AAAAAAD///8AAAADAAAHgAAAASj8AQAAAAH7AAAAGAB0AGUAcgBtAGkAbgBhAGwARABvAGMAawEAAAAAAAAHgAAAAI0A////AAAG0gAAApkAAAAEAAAABAAAAAgAAAAI/AAAAAEAAAACAAAAAQAAABYAbQBhAGkAbgBUAG8AbwBsAEIAYQByAQAAAAD/////AAAAAAAAAAA=
+State=AAAA/wAAAAD9AAAAAwAAAAAAAACtAAACfvwCAAAAAvsAAAAWAGYAbwBsAGQAZQByAHMARABvAGMAawAAAAAA/////wAAAAAA////+wAAABQAcABsAGEAYwBlAHMARABvAGMAawEAAABJAAACfgAAAFMA////AAAAAQAAAAAAAAAA/AIAAAAB+wAAABAAaQBuAGYAbwBEAG8AYwBrAAAAAAD/////AAAAAAD///8AAAADAAAHgAAAASj8AQAAAAH7AAAAGAB0AGUAcgBtAGkAbgBhAGwARABvAGMAawEAAAAAAAAHgAAAAI0A////AAAG0gAAAn4AAAAEAAAABAAAAAgAAAAI/AAAAAEAAAACAAAAAQAAABYAbQBhAGkAbgBUAG8AbwBsAEIAYQByAQAAAAD/////AAAAAAAAAAA=
 eDP-1=eDP-1
 eDP-1 HDMI-2=HDMI-2
 eDP-1 HDMI-2 XPosition 1920x1080 HDMI-2=2500

Eww. Not sure what to do about that. It appears to be base64 encoded binary settings.

[LorenzoBettini]

I think the problem with menubar is that the default value is true, so it's always removed if I set that property to true (in the original file it was set to false). I guess that's the default behavior of KDE not to set properties to have their default values. So it's not a problem I'd say: it's just a matter of not specifying that property in my source and everything's fine (I want the menu bar)

[VorpalBlade]

See my replies to your previous comment (Github discussions are a bit weird, they have two levels). I don't believe that to be the case at all. In fact it looks like it moved from dolphinrc to a separate file and is encoded as base64 binary data... I don't think there is much I can reasonably do about that.

[LorenzoBettini]

Yes, I had seen your previous answers, but I confirm that if I remove that property and I apply the changes on another computer, the property is correctly removed and the menu bar appears (previously it wasn't shown on this other computer). I'm talking about plasma 6 on Arch.

[LorenzoBettini]

I'm doing some other experiments with Plasma 6. On a fresh VM, it looks like the rc files in .config only store configurations, so all the related state information, as you also found, is stored elsewhere. When I wrote the initial messages in this discussion, I started from existing rc files (after upgrading from Plasma 5 to 6), so I guess most of the things I had to exclude are simply remainders of the existing Plasma 5 behaviors. Maybe now things are much simpler and there's nothing much to exclude :)

[VorpalBlade]

That would be nice. Though there are still some other programs that have the same issue, for example PrusaSlicer (if you are into 3D printing).

I will have to evaluate this at some point, though I'll wait until I'm able to fully transition to Plasma 6 (work laptop lagging behind).

[LorenzoBettini]

Another example: if I only want to set the fixed font, I can write the "modify_..." as follows:

#!/usr/bin/env chezmoi_modify_manager

source auto

# Only set the font, so ignore everything...
ignore regex ".*" ".*"
# and list the single setting
set "General" "fixed" "Hack Nerd Font,10,-1,5,400,0,0,0,0,0,0,0,0,0,0,1" separator="="
# Note: the contents of private_kglobals.src.ini is fictitious

[VorpalBlade]

That does work, though see issue #99 about evaluation order. It is currently not guaranteed (and you are relying on regex matches being evaulated last). If I change it, it will probably be to "section matches first, then first match wins". So put your set before your ignore to be future proof.