Dev questions

[TristanTrim]

Hi : )

rnote seems like the most promising note taking app for my usage, and I want to start developing on it. My first few goals are:

• Allow zooming in and out further
• Allowing a setting to have the pen stroke scale with the zoom level
• Make "links" to specific locations in notes for faster navigation within large notes and between notes.
• Allow searching text ( both user written and in imported docs )
• access pdf outlines
• circular pen context menu / gestures for faster tool switching
• When zoomed out very far, have optional outline stopping from writing over top of smaller text.
• do a PR with a drawing created with rnote

If anyone is interested in any of those I can explain them in more detail. Also if there are other discussions or issues that are relevant I'd be happy to hear about them.

I've forked the repo and got it compiled and running with help from the BUILDING.md, and changed the config vars in camera.rs:

-    pub const ZOOM_MIN: f64 = 0.2;
-    pub const ZOOM_MAX: f64 = 6.0;
+    pub const ZOOM_MIN: f64 = 0.002;
+    pub const ZOOM_MAX: f64 = 600.0;

So I'm running into two issues rn: Building the app is super slow (>5 min) and zooming to the extremes causes rnote to chug and crash. I'm hoping someone can give me some insights, but if not, that's ok, I might just use this as a journal of my dev endeavors.

About building the app: I'm using the following commands to build and run:

time flatpak-builder -v --user --force-clean flatpak-app build-aux/com.github.flxzt.rnote.Devel.yaml
flatpak-builder --run flatpak-app build-aux/com.github.flxzt.rnote.Devel.yaml rnote

And it seems like each time it is deleting and re-cloning all the dependencies and recompiling 500+ files which seems unnecessary for having changed 2 lines of code... So I'm hopeful I'm doing something wrong and this is not intended workflow.

About zoom chugging: I'm assuming it's two different issues:

When zooming in I think it might be rendering an entire page ( or some other amount of content ) at an insanely high resolution, even though only a very tiny fragment of that content is being viewed. I still need to verify that that's actually what it is, but I'm thinking it could possibly be solved by figuring out what determines how much is rendered and telling it to render less, mostly just rendering what is currently on screen. Although I'm less invested in getting zooming in working well than zooming out.

When zooming out I think an insanely high number of elements (grid dots, strokes, letters) are all each contributing a tiny amount to pixels that are much larger than them. Again, I need to confirm that's what's really happening, but if it is, then I think a good solution would be dropping smaller elements from rendering, and introducing cached "images" of smaller elements, so instead of using 500 individual text characters on a page to calculate that a fuzzy grey block should be shown, just using a cached image of a fuzzy grey block.

Thanks for all the dev work and thanks for anyone who can help me understand what I'm doing.

[flxzt]

Hey! Glad to see interest. I'll do my best to try to answer your questions:

Building the app is super slow (>5 min)

Personally I am building without flatpak (directly on the host with meson) most of the time, and when I do build through flatpak, I am using the Vscode flatpak extension which reuses compilation artefacts between builds. Both options greatly reduce the build time to maybe 10-15s instead of minutes by reusing compilation artefacts. I think the flatpak extension does that by telling cargo to use a different target directory outside of the flatpak build directory, but I am not 100% sure.

zooming to the extremes causes rnote to chug and crash.

Difficult to tell what's happening without any logs/debugging, but you might hit texture dimension limits (of cairo/gsk) at some point.

mostly just rendering what is currently on screen.

This optimization is already implemented, Rnote uses a R-tree internally for bounding box queries and only rerenders those that are in view. Large strokes are also only partially rendered, at least for some of the Stroke types. Check out the implementations of the Content trait to see how this is implemented. The rendering itself is triggered in StrokeStore::regenerate_rendering_in_viewport_threaded() which gets called whenever the viewport size/offset/zoom changes.

then I think a good solution would be dropping smaller elements from rendering, and introducing cached "images" of smaller elements

Rnote does use cached textures while zooming, and only rerenders the elements after a short timeout after a zoom. But rendering something simpler when it is clear that the content is not recognizable would make sense to me.

[TristanTrim]

directly only the host with meson

Thanks, I haven't looked at meson yet. I tried using just "cargo build" directly, but that was a bad time.

I'll look at the Content trait and regenerate_rendering_in_viewport_threaded. I'm thinking I'll try to add in functionality for caching renders for extreme zoom out. For refrence, This is the kind of workflow I'm looking to have:

So I want to be able to quickly zoom out very far and zoom back in somewhere else in order to reference different parts of the textbook and my notes, and draw connections between them, sorta like a really large mind map.

Thanks for working on such a cool app. Wish me luck : )

[TristanTrim]

Update on this. I figured out how to build faster. I just needed to use meson as is detailed in the BUILDING.md but then cd into the build dir and call ninja.

I've got a hacky modification to zooming working. I can zoom in and out further, and zoom is 2x the amount you pinch your fingers, which makes it much faster. It seems like rnote loads the entire file into memory when opening, which is reasonable, but my goal is to have my entire notes from each class in one file, which makes them very large. I'd like to have lazy loading so only the pages being viewed and edited are loaded into memory, but as a work-around, I just threw a ton of swap space on my computer. I'd be willing to put together a pull request for these things, but I think they are far from ready for use by other users. Definitely need to be configurable through the settings, which I have not implemented.

I think I'll wait on looking at Content trait and regenerate_rendering_in_viewport_threaded for zoom caching until I have a bit more time and a bit more experience with rust.

@flxzt The next thing I want to work on though, is adding the ability to save and jump to locations in the document. To that end I found where "Return to Origin Page" is in the code and copied it in crates/rnote-ui/data/ui/canvasmenu.ui, rnote-ui/src/appwindow/appwindowactions.rs, but when I recompile with ninja, the menu appears unchanged. Do you have any idea what could be wrong? Is there another location I need to edit, or some config update I need to run? Thanks : )

[flxzt]

Regarding the zooming part: My intuition is that the actual limiting factor would be the work to render many objects that are in the view when zoomed out very far. Implementing LOD together with precaching rendering could be a good way to solve that.

Ah, yes, that can be unexpected. Rnote currently always loads the .ui files installed into the data dir (so /usr/share/rnote/ most of the time). You need to install the application to the system whenever you make changes to the ui xml files.

[TristanTrim]

That was such a confusing error. Thanks for the help!

Oh, yeah, I was thinking about precaching rendering. I'm not sure the best strategy. I'm thinking maybe a separate precache for zooming in, zooming out, and panning. Then as the user starts to pan or zoom, the precache uses the direction of motion to decide what direction to precache in. I think figuring out when to dump things from the cache might also be important for good memory vs cpu performance. It might be worth saving some of the more zoomed out caches to disk since they could be quite expensive to render... but it actually already works pretty well on my (not old) laptop. : )

[TristanTrim]

Oh, adding onto this, I'd like to be able to verify some performance speculation before optimizing the wrong thing. I'm not sure how I'd go about that, and it's probably going to be a long time before I actually have time to look at it, but I'd love to hear thoughts anyone has on that.