Open XR support

[Aggror]

Feature Request: Implement OpenXR Support in Stride 3D

Summary
We propose advancing the OpenCollective project for OpenXR by detailing the requirements and improvements needed to integrate robust OpenXR support in Stride 3D. OpenXR provides a unified API for VR/AR devices, making it a critical feature for modern immersive applications.

Objectives

1. Implementation Goals

   • Enable OpenXR support similar to existing VR backends.
   • Verify compatibility with VR headsets and mobile devices.
   • Ensure integration aligns with the current VR abstraction layer, requiring potential small adjustments.

2. Bare Minimum Functionalities

   • Basic head tracking and controller input.
   • Rendering compatibility for OpenXR-enabled devices.
   • Workflow adjustments to support OpenXR on mobile platforms.

3. Current Developer Issues

   • Lack of proper documentation and examples for using OpenXR with Stride 3D.
   • Vulkan support requires fixes for a seamless VR experience.
   • Developers report challenges with the VR abstraction layer's alignment with OpenXR requirements.

4. Testing and Platform Support

   • Test OpenXR implementation on popular VR headsets (e.g., Oculus, HTC Vive).
   • Test mobile compatibility for AR/VR applications.
   • Focus on Vulkan support (DX12 support deemed unnecessary for this phase).

5. Documentation Needs

   • Comprehensive OpenXR integration documentation.
   • Guides on using OpenXR for VR/AR workflows in Stride 3D.
   • Examples for common use cases (headset setup, rendering adjustments, etc.).

Benefits

• Unified VR/AR Development: OpenXR eliminates the need for multiple SDKs, simplifying development for multiple VR platforms.
• Cross-Platform Compatibility: Mobile and headset support expands Stride 3D's utility for AR/VR developers.
• Future-Proofing: OpenXR is the standard for XR development, ensuring Stride 3D stays competitive in immersive tech.

Related Links

• OpenXR for Stride on OpenCollective
• GitHub Discussion: Implement OpenXR for Stride

Next Steps

1. Test and validate the OpenXR implementation on supported devices.
2. Fix Vulkan support issues to ensure compatibility.
3. Create detailed documentation and guides for developers.
4. Gather feedback from the community to prioritize additional improvements.

Open Collective Project
https://opencollective.com/stride3d/projects/openxr-support

[tebjan]

Just some thoughts:

As we started integrating OpenXR into Stride, our primary focus was on DX11, with Vulkan functioning at approximately 90% completion. A minor issue involving the Silk wrapper was encountered towards the end of the project, which should be relatively straightforward to resolve.

Our progress hit constraints related to time and budget, which led to the incomplete state of the Vulkan and no D12 backend. To advance this integration, we should prioritize addressing these remaining issues.

Regarding the bare minimum functionalities for OpenXR support, it would be particularly interesting to look at compatibility with mobile platforms, android and iOS.

Furthermore, it would be good if Happenstance could articulate their specific requirements, use cases, and expectations. This would aid in aligning the development efforts with the needs of the community.

[Keepsie]

The OpenXR support in Stride has some functionality, but there are issues. Crashes and "not implemented" tags will probably become bigger headaches as more people start experimenting. It needs to be polished and tested properly.

Vulkan is a must-have, especially since it's key for desktop, Android, and the emerging iOS XR scene. I don't know about the benefits of DX12 with VR.

The VR documentation on the site has been pretty decent for me. However:
I can't get the camera to rotate, so I cannot do any snap or smooth turning which is basically a critical bug.

I took a break from it for a bit, focusing on our current project. But I'll be diving back in during my free time each week.

I see Stride as the top VR choice outside of Unity. Super excited about its potential. Once our current project wraps up, I'm looking to jump into the next one with Stride, which is why I'm trying to get everything squared away now.

Overall, I need to dive into the backend more. I’m too ignorant about how things work so, I can only help so much on that front.

[tebjan]

Yes, good points:

• Fix Vulkan
• DX12 not necessary
• Check VR workflow with OpenXR and mobile

[Starkium]

Agreed with above about vulkan for mobile. Another big need is going to be occlusion culling. In this case it'll need to be a software occlusion method since mobile will be very gpu constrained already. These were the biggest things that were affecting UE5 devs for a while.

There's plenty of other features that go a long way for XR, but I think the above are likely the bare minimum to get started. I'll update my post if something else comes to mind.

[Aggror]

Hi @MaximilianEmel, Thank you for your interst in this topic. I did reach out to @tamamutu for this topic 1 month ago (3rd october) and he was interested in taking a look as well. However, I didn't hear from him since then. I send him a reminder on the 31st of oct, but no response on that either.
Would it be possible for you to wait 1 more day before you start working on it? It would be such a shame if there is actually some work done on it already. I just send them a message for confirmation.

[Aggror]

@MaximilianEmel You are good to go. I will now reserve this topic on your name.

[Eideren]

Hiding this comment as Maximilian is no longer working on this from what we could gather, if anyone wants to take over feel free !

[Starkium]

checking in, any news on progress?

[Starkium]

uh, hello?

[vpenades]

So... no VR on quest so far?