Setting specific frame glitches with onTapGesture

[s1m-0n]

I'm having some issues with getting a big schematic in SVG resized and keeping the onTapGestures unaffected. I created a scrollView around the SVGView and I'm adding the typical gestures (drag, zoom, pinch etc.) I also want to have certain parts of the schematic selectable but that seems to break partially when I use the .frame modifier to scale out the SVG to fit the screen. I can zoom in perfectly but the selection just works on the top left part of the SVGView.
I tried the .scaleEffect modifier which retains the tab gestures perfectly but as that only affects the view layer the content size of the scrollview is way too big and not adjusted to the scaled schematic.

Does anyone have any idea what is going on?