Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Gizmo Normal Snapping #2539

Merged
merged 41 commits into from
Jun 5, 2024
Merged

Gizmo Normal Snapping #2539

merged 41 commits into from
Jun 5, 2024

Conversation

max-mrgrsk
Copy link
Collaborator

Summary:
This pull request addresses two main features:

  1. Gizmo Normal Snapping (gizmo should allow snapping normal to #2445):

    • ability for the gizmo to snap to normals.
    • interactive features for the gizmo, such as hover and click events for all axes.
  2. Point-and-Click Improvements (Point and Click improvements #2035):

    • "Reset View" functionality.

Key Updates:

  1. Gizmo Interaction Enhancements:

    • Enabled Raycaster only when the mouse is over the gizmo canvas to optimize performance.
    • Added visual feedback by scaling gizmo handles on hover and sending axis information on click.
    • Implemented the updateCameraToAxis method in cameraControls to adjust camera vantage and up vector based on the selected axis.
  2. View Reset:

    • Temporarily integrated "Reset View" option within the gizmo.
    • Utilized the default_camera_look_at and default_camera_get_settings endpoints to manage camera settings.

Next Steps:

  • Review the gizmo updates, including the interactive handle and scaling effects.
  • Remove the temporary reset handle if the current implementation is ok.
  • Turn around selected part / assembly instead of world 0

Implementation Details:

  • Most logic is implemented within src/clientSideScene/CameraControls.ts.
  • Method calls are integrated in src/components/Gizmo.tsx.

References:

Please review the changes and provide any additional comments or suggestions.

nice and clickable
@max-mrgrsk max-mrgrsk linked an issue May 27, 2024 that may be closed by this pull request
Copy link

qa-wolf bot commented May 27, 2024

QA Wolf here! As you write new code it's important that your test coverage is keeping up.
Click here to request test coverage for this PR!

Copy link

vercel bot commented May 27, 2024

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name Status Preview Updated (UTC)
modeling-app ✅ Ready (Inspect) Visit Preview Jun 5, 2024 11:57am

@jessfraz
Copy link
Contributor

having some playwright scrrenshot tests for this would be great!

max-mrgrsk and others added 3 commits May 27, 2024 21:57
@Irev-Dev
Copy link
Collaborator

having some playwright scrrenshot tests for this would be great!

Maybe, I think there might be a way to test it without screenshots. But tests would be good ofc!

Comment on lines 786 to 790
let vantage = { x: 0, y: 0, z: 0 }
let up = { x: 0, y: 0, z: 0 }

if (axis === 'x') {
vantage = { x: distance, y: 0, z: 0 }
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we change this so to something like

    const vantage = this.target.clone()
    let up = { x: 0, y: 0, z: 0 }

    if (axis === 'x') {
      vantage.x += distance
      up = { x: 0, y: 0, z: 1 }
// ...

And then for the default camera look at keep the users's target

await this.engineCommandManager.sendSceneCommand({
      type: 'modeling_cmd_req',
      cmd_id: uuidv4(),
      cmd: {
        type: 'default_camera_look_at',
        center: this.target,
        vantage: vantage,
        up: up,
      },
    })

Sorry if if this is different from what I originally asked for, but think this will be better as it respects the user's current target, and just rotates the camera to a different perspective.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

now I can see that too ! so much better to code with viewport on :DDD

Comment on lines 828 to 848
async resetCameraPosition(): Promise<void> {
await this.engineCommandManager.sendSceneCommand({
type: 'modeling_cmd_req',
cmd_id: uuidv4(),
cmd: {
type: 'default_camera_look_at',
center: { x: 0, y: 0, z: 0 },
vantage: { x: 0, y: -128, z: 64 },
up: { x: 0, y: 0, z: 1 },
},
})
await this.engineCommandManager.sendSceneCommand({
type: 'modeling_cmd_req',
cmd_id: uuidv4(),
cmd: {
type: 'zoom_to_fit',
object_ids: [], // leave empty to zoom to all objects
padding: 0.2, // padding around the objects
},
})
}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry I couldn't help but make a commit, just cause I wanted to see this working.

All I did is change the vantage to where the camera is on boot up vantage: { x: 0, y: -128, z: 64 }, The most important thing here is the ratio of y and z (double y).

But immediately followed up by a zoom to fit means it will look exactly the same as when you refresh the page.

Note: default_camera_get_settings is not needed because we already subscribe to the camera payload of zoom_to_fit

Copy link
Collaborator Author

@max-mrgrsk max-mrgrsk May 29, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

since we fixed my viewport issue today, I could test this one...

In my case it does not work. The camera resets to weird position somewhere far away.

My findings:

1. Unexpected Camera Position: After executing both commands, the camera ended up in a strange position, far from where it should have been. The coordinates before resetting were something like {x: 1719.28, y: -1096.2126, z: 2252.1978} (zoomed and default rotation), but after resetting, they became {x: 1719.28, y: 4877.849, z: -696.26904}, instead of staying the same. Weird math...

2. Testing Commands Individually: When I tested each command (default_camera_look_at and zoom_to_fit) separately, they worked fine on their own. The issue only appeared when both commands were sent one after the other.

3. Pause Between Commands: I suspected that the rapid change from large coordinate values to small ones and back to large values might be causing the issue. I tried adding a short pause (1000ms) between the two commands to give the server time to process the changes, but it didn't help.

4. Eliminating the Coordinate Jump: Instead of using fixed small values for the default_camera_look_at command, I used the current target position for the center and adjusted the vantage point relative to this target. This way, there wasn't a large jump in coordinate values. For example:

center: this.target,
vantage: { x: this.target.x, y: this.target.y - 128, z: this.target.z + 64 }

instead of:

center: { x: 0, y: 0, z: 0 },
vantage: { x: 0, y: -128, z: 64 },

This approach worked and the camera reset correctly without any issues.

but in case of moving camera super far away you might need to click reset 2 times :DD

5. Further Investigation: I found it strange that using the this.target position worked, but hardcoded values did not. It turns out that the vantage value can be any value; the angle may not be default, but the zoom-in will work as expected. The problem lies with the center value. Any value other than this.target causes an offset in the zoom_to_fit command.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay this sounds like a good work around, but maybe the zoom to fit should be robust against this kind of thing?

Maybe I'll mess with it and raise something with the engine team because I'm not sure it should require us to finagle like this.

Copy link
Collaborator

@Irev-Dev Irev-Dev left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great.

One small tweak of https://github.com/KittyCAD/modeling-app/pull/2539/files#r1616427476

And now that it's interactive it would be good to get some e2e tests in I'll follow up with a few more details.

-- edit: never mind, realised this is essentially what I mentioned in #2539 (comment), it just didn't click sorry.

@Irev-Dev
Copy link
Collaborator

Irev-Dev commented May 28, 2024

Okay so here's my idea for tests, the stub in fc37031, there should be some notes in the readme about playwright (but feel free to updates them if anything is missing or stale)

And the following explanation

Screen.Recording.2024-05-28.at.11.51.19.AM.mp4

@Irev-Dev
Copy link
Collaborator

Also from your description, I don't get

Turn around selected part / assembly instead of world 0

Could you expand on what you mean here?

@max-mrgrsk
Copy link
Collaborator Author

@Irev-Dev, thank you so much! The video was super helpful! ^^

Awesome update:

type: 'zoom_to_fit',
        object_ids: [], // leave empty to zoom to all objects
        padding: 0.2, // padding around the objects

I couldn't test it because I don't see any model to zoom in on. Do you have any updates on fixing my issue (probably related to tokens)? I could better explain and test orientation ideas if I had something on screen. Anyway, if it works for you, then I can remove the "reset view" handle.

const vantage = this.target.clone()
    let up = { x: 0, y: 0, z: 0 }

    if (axis === 'x') {
      vantage.x += distance
      up = { x: 0, y: 0, z: 1 }

This matches what I was proposing:

Turn around selected part/assembly instead of world 0

Thanks for implementing it!

Sorry I couldn't help but make a commit, just cause I wanted to see this working.

Awesome, thanks!

Nest steps:
I'll do a bit of refactoring for gizmo today, and focus on playwright tests

@Irev-Dev
Copy link
Collaborator

Oh right, yeah it's going to be hard for you to finish the test if you can't get the stream/token working.

@max-mrgrsk
Copy link
Collaborator Author

max-mrgrsk commented Jun 2, 2024

@franknoirot, thank you for the tips, they helped a lot!

@Irev-Dev here are my findings regarding the recent issues we've faced with our Playwright tests, both locally and in our CI environment.

Local Testing:
Running the tests locally many times, I discovered that the timeouts needed to be extended slightly to allow the camera sufficient time to rotate. Increasing the timeouts made the tests stable on my local machine.

CI Testing:
However, when testing in the CI environment, I encountered persistent failures with Playwright. Initially, I increased the timeout from 0.2 to 0.5 seconds.

await page.waitForTimeout(400)
await page.mouse.move(clickPosition.x, clickPosition.y)
await page.waitForTimeout(100)
await page.mouse.click(clickPosition.x, clickPosition.y)
await page.waitForTimeout(500)

This adjustment allowed the tests to pass on Mac, but they continued to be flaky on Ubuntu.

Upon reviewing the trace screenshots from the Ubuntu tests, I noticed that after clicking on the gizmo, the camera appears to rotate, but the values in the debug output remain unchanged, and the gizmo itself does not update. This suggests that the gizmo correctly sends new coordinates to the server, and the server updates the server-side camera, but fails to notify the client-side camera.

page@f6cfcf4ab91087d895d568ee63ceaf21-1717276988414
mouse over works (gizmo handle scaled up)

page@f6cfcf4ab91087d895d568ee63ceaf21-1717276988531
click works (server side camera moved)
client side camera fails to move (gizmo stays in place)

it explains this error message:
Screenshot 2024-06-02 at 22 06 43

This should not happen because in the updateCameraToAxis method in CameraControls.ts at line 812, there is a call to:

await this.engineCommandManager.sendSceneCommand({
  type: 'modeling_cmd_req',
  cmd_id: uuidv4(),
  cmd: {
    type: 'default_camera_get_settings',
  },
});

For some reason, this call does not always execute properly on Ubuntu.

Temporary Solution:
To address this, I added an extra line in my test to repeat this command after clicking on the gizmo:

await u.sendCustomCmd({
  type: 'modeling_cmd_req',
  cmd_id: uuidv4(),
  cmd: {
    type: 'default_camera_get_settings',
  },
});
await u.waitForCmdReceive('default_camera_get_settings');

This modification helped the tests pass, but we need to investigate why this issue occurs and double-check the behavior of the default_camera_get_settings command, especially in the CI environment on Ubuntu, to understand why the client-side camera is not being updated properly.

Comment on lines +4901 to +4907
await u.sendCustomCmd({
type: 'modeling_cmd_req',
cmd_id: uuidv4(),
cmd: {
type: 'default_camera_get_settings',
},
})
Copy link
Collaborator

@Irev-Dev Irev-Dev Jun 5, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Making a not of this for as issue #2603

Copy link
Collaborator

@Irev-Dev Irev-Dev left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I got rid of some of the longer timeouts in a3313e2

🤞 the tests are okay, if not we can change it back. As a rule of thumb I'm okay with timeouts of 100, as a way to wait for things to tick along, but if they're more than 100 it's normally a sign that we're not waiting on the right thing, (having said that there are still bigger timeout in the tests, we'll use them where needed). So what I did instead of the timeout is waiting for responses to the websocket commands.

The problem with timeouts when used to wait for something async to happen is the timeout is either too short and you have flakey tests (a real PITA) or you make them conservatively long and most of the time you wouldn't have needed to wait that long, but for the odd test that's running slow, it's slow enough for that case, but now you've slowed down that test every-time it runs to the slowest possible case. And the time it takes for tests to run can really ad up.

Your solution with the camera not updating is a good one, that's a bit of a mystery as to what's happening there. I'd rather not forget about it so I made #2603

@Irev-Dev Irev-Dev enabled auto-merge (squash) June 5, 2024 04:24
@Irev-Dev
Copy link
Collaborator

Irev-Dev commented Jun 5, 2024

Sorry I'm getting unrelated failures now, I'll get this fixed up and merged soon.

@Irev-Dev Irev-Dev merged commit dd3601e into main Jun 5, 2024
16 checks passed
@Irev-Dev Irev-Dev deleted the 2445-gizmo-snapping branch June 5, 2024 12:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

gizmo should allow snapping normal to
5 participants