onDragEnd event being fired early on mobile

[gjakubik]

When using @use-gesture/react, my onDragEnd events are firing after only a few onDrag events, even when touch is held down much longer.

Here is the code that handles actual drag events. I am developing a google-maps like element where each element in the data object is being updated to reflect a drag movement. This code works as intended on desktop.

const [dragStartX, setDragStartX] = useState(apiArgs.x);
const [dragStartY, setDragStartY] = useState(apiArgs.y);
const [absStartX, setAbsStartX]   = useState(apiArgs.x);
const [absStartY, setAbsStartY]   = useState(apiArgs.y);
const [lastLocX, setLastLocX]     = useState(apiArgs.x);
const [lastLocY, setLastLocY]     = useState(apiArgs.y);

const handleDragStart = (xy: number[]) => {
  console.log('starting');
  setDragStartX(xy[0]);
  setDragStartY(xy[1]);
  setAbsStartX(xy[0]);
  setAbsStartY(xy[1]);
}

const handleDrag = (xy: number[]) => {
  if (data?.image_map && ((lastLocX !== dragStartX) || (lastLocY !== dragStartY))) {
    console.log('moving');
    const diffX = xReversedScale(xy[0]) - xReversedScale(dragStartX);
    const diffY = yReversedScale(xy[1]) - yReversedScale(dragStartY);
    for (let elem of data.image_map) {
      elem.x = elem.x + diffX;
      elem.y = elem.y + diffY;
    }
    setDragStartX(xy[0]);
    setDragStartY(xy[1]);
  }
}

const handleDragEnd = (xy: number[]) => {
  console.log('ending');
  const diffX = xReversedScale(xy[0]) - xReversedScale(absStartX);
  const diffY = yReversedScale(xy[1]) - yReversedScale(absStartY);

  setApiArgs((apiArgs) => ({
    x: apiArgs.x - diffX,
    y: apiArgs.y - diffY,
    height: apiArgs.height,
    width: apiArgs.width,
  }));

  setDragStartX(xy[0]);
  setDragStartY(xy[1]);
  setLastLocX(xy[0]);
  setLastLocY(xy[1]);
}

const bind = useGesture({
  onDrag: state => handleDrag(state.xy),
  onDragStart: state => handleDragStart(state.xy),
  onDragEnd: state => handleDragEnd(state.xy),
})

However, on mobile, the onDragEnd event fires after only a few onDrag events, even when many more should be occurring. Here is the output of console.log() from the above code on a chrome emulation of a touch device. (This is also true for actual touch devices as well, based on app behavior)

For reference, here is the output from desktop after dragging around a bit:

Just in case the element doing the listening has something to do with it, here is the code for that:

<rect
    width={width}
    height={height}
    style={{touchAction: 'none'}}
    fill='transparent'
    {...bind()}
/>

Any insight as to why this event would be getting fired early only on touchscreens would be much appreciated.

[dbismut]

Weird. Would you be able to provide a monomial repro sandbox? I'll take a look.

[gjakubik]

Thanks for the advice!

Am I not allowed to bind twice? Those circles are the showing images, and they need to capture any drag events that start with the mouse or touch on top of an image circle. This is especially important for mobile where the amount of whitespace is much lower. I tried removing that second binding to see if it was the issue, and that was not it.

As for your notes, those are both super helpful and I will look into them to make everything more performant. I am actually planning to add pinch and mouse wheel zoom as well, so I figured useGesture was the way to go. I was especially unaware of the fact that react-spring updates outside of the render cycle, which could give me a nice performance boost.

I tried using useDrag in the way you specified just to see if it would have a different effect, and I am still having the same problem.

const bind = useDrag((state) => {
  handleDrag(state.xy, state.first, state.last);
})

const handleDrag = (xy: number[], first: boolean, last: boolean) => {
  if (first) {
    handleDragStart(xy);
    return
  }
  if (last) {
    handleDragEnd(xy);
    return
  }
  if (data?.image_map && ((lastLocX !== dragStartX) || (lastLocY !== dragStartY))) {
    console.log('moving');
    const diffX = xReversedScale(xy[0]) - xReversedScale(dragStartX);
    const diffY = yReversedScale(xy[1]) - yReversedScale(dragStartY);
    for (let elem of data.image_map) {
      elem.x = elem.x + diffX;
      elem.y = elem.y + diffY;
    }
    setDragStartX(xy[0]);
    setDragStartY(xy[1]);
  }
}

I updated my attempts in the temp github repo. If you have any other ideas of what could possibly be happening, let me know. If not, I may have to attempt to redo everything with the vanilla js TouchEvent API, which would be a bit of "drag" :).

My questions are:

• What settings do I have control of that would cause different behavior on touch than on desktop?
• Are there certain gesture options that could be set or unset to attempt to change this?
• Am I just using this in an application where it isn't meant to be used (like with multiple elements being bound)?

[dbismut]

Am I not allowed to bind twice? Am I just using this in an application where it isn't meant to be used (like with multiple elements being bound)?

You can bind wherever you want and how many times you want ;) bind() accepts arguments that you would recover from state.args so your handler can even perform custom logic depending on which element triggered the drag.

After looking at the code closely I see that you're using state.xy, which is not a bad thing per se, but defeats a lot of the features from this lib. Generally you would be looking at using movement or offset. Anyway, I was just curious to see if this was on purpose, but looks like it! Just let me know why you're not using movement (in your case offset don't make sense).

I am actually planning to add pinch and mouse wheel zoom as well, so I figured useGesture was the way to go.

You're right, it's the way to go! useDrag uses the exact same logic, so it's not surprising (even reassuring!) that moving to useDrag doesn't help!

What settings do I have control of that would cause different behavior on touch than on desktop?

There's something you haven't answered from my previous message is why doesn't circle have the touch-action property set? This is generally solves 90% of user problems when dealing with mobile vs desktop differences. I think I've written this explanation in different places, but use-gesture uses PointerEvents and these events get canceled on mobile when the user actually scrolls the body. This is why touch-action is so important as it tells the browser that the element can be manipulated and that scroll shouldn't happen when touch is initiated from that element.

But still, there's a escape hatch in use-gesture, as it can happen that you want to be able to drag while scrolling at the same time (not in your case, but for the sake of completeness here's how).

You would just do:

useGesture(handler, { drag: { pointer: { touch: true } } })

In that case the lib will use TouchEvents which don't get canceled when scrolling.

Hope it helps. Your code is pretty complex so I don't want to dive into it 😅

[gjakubik]

Thank you so much for the detailed reply!

As to your points:

Anyway, I was just curious to see if this was on purpose, but looks like it! Just let me know why you're not using movement (in your case offset don't make sense).

In this case, the backend has a different coordinate system than that which comes back from the DOM, so a transformation using a visx linear scale has to be done on both coordinates before the correct distance to be applied to the data can be calculated. I guess I could change it to find the last point by using the current xy and the movement, but right now I am just saving it in state. Definitely lots of performance improvements to be had here by saving a local copy of the data and cutting out transforming it both ways for updates and then renders, along with using less state to handle it all.

There's something you haven't answered from my previous message is why doesn't circle have the touch-action property set?

This was a great suggestion and I did try it. Sadly, it didn't work. Even then, dragging not starting on the circles should have had this property set, and the issue was still occurring. I will examine this extra closely to make sure that I am locking my browser window in place so that scrolling can't override this. That seems to be the most likely issue to me.

Your code is pretty complex so I don't want to dive into it 😅

That totally makes sense and I would not expect you to at all. You have already been more helpful than I ever would have hoped from tossing my question into the internet void!

Thanks again for your help and I will update here with new things I tried and if they worked or not. If anything else comes to your mind and you post it here I will definitely be checking back on this thread, but certainly don't feel inclined to dive any deeper. The time you have already spent giving me advice has been very valuable to me 😄

[dbismut]

Sure! Always happy to help. As this is a bit weird though I'll try to make a small repro sandbox tomorrow using svg.

Btw, have a look at the transform option from the lib which serves the purpose of mapping screen coordinates to custom ones. If that doesn't work out let me know what's missing.

[gjakubik]

Sure! Always happy to help. As this is a bit weird though I'll try to make a small repro sandbox tomorrow using svg.

That would be amazing! I have really been pulling my hair out on this one 😅

Btw, have a look at the transform option from the lib which serves the purpose of mapping screen coordinates to custom ones. If that doesn't work out let me know what's missing.

I will take a look at that when I get the chance. Since we are using visx to render the graphics and it has a built in function for this as well, using that has been sufficient for now.

Another observation I made just now is that the scroll seems to be correctly locking on mobile because the scrollbar disappears when dragging. This makes me think that touch-action is working as intended. If I drag as slowly as possible, I can get about 9 events to fire before it ends. Dragging faster gives less events before the drag is ended. Are there some sort of bounds that are being enforced?