How to leverage VisionOS using SwiftUI, many of the tools and frameworks have been extended over the last few years. Making AR more accessible across devices and platforms.
This is all about the Immersive Space in VisionOS. Both Windows and volumes let you display content within their bounds. To go outside of the window you must use the immersive space.
Spaces are a container to present your UI
Get Started with Spaces
- Extending the “Hello World” app to explore the solar system. Begin with defining an ImmersiveSpace, while one ore more space can be created within an app. You can only have one open at a time.
- Opening a space replaces the shared space with your Full Space, an implicitly defines it’s on coordinate system. Based on the user’s position
Display Content
- It is a scene type, so you can place any SwiftUI view in it. Anything you place uses the same system you are used to, but since the origin is the users feet, you will want to adjust by using RealityView. RealityView has built in support for asynchronous loading. You use ARKit anchors of placement.
- Remember that RealityKit uses different system than SwiftUI. Check out Build Spatial experience to RealityKit (WWDC23)
- You should define a id or value type or both. For controlling you have .openImmersiveSpace and .dismissImmersiveSpace actions from the @Environment, the system will automatically animate the animation. They are asynchronous so you can act on them when they succeed.
- Sequence of windows in a Scene will decide which one shows up first.
- You should use Model3D for asynchronous loading of your assets
Managing your Space
- Scene management and coordinate conversion are key to understand.
- Scene phases should be handled to let user know what is doing one, for when an alert pops up or other interactions.
- You can do things like changing scale to let user know
- To do repositioning, items between swiftUI and 3D content take a look at this graphic
- Note that the window and the 3D object use the Y axis differently. You can use transform(in: .immersiveSpace) you will get a conversion.
- There may be conversions you need to handle between private immersive space vs. Group Immersive Space
- Immersion Styles
- This is different presentation for how your environment looks; Mixed, progressive, or Full
- By default Mixed, you can use a scene modifier .immersionStyle to define the types your scene supports.
- Progressive style allows you to see people around you and have some interaction, you can choose the level of progression, but if you go all the way it will take you to full immersion
- By pressing the digital crown you go back to passthrough
Customization
- A Scene Manifest is used to launch your app directly to full immersion – This is in the Info.plist by setting the UIApplicationPreferredDefaultSceneSessionRole to UISceneSessionRoleImmersiveSpaceApplication
- If you want to toggle back to a window set UISceneInitialImmersionStyle to UIImmersionStyleMixed
- .preferredSurroundingEffect to set the effects when you change to Full Immersion
- Hiding hands means you can show virtual hands instead – you use RealityView to do this. The hands are an entity, using ARKit and Hand tracking API. Check for hand tracking anchors and updates. By using Transform to the anchor. Look at session “Evolve your ARKit app for spatial experiences” from WWDC 23.