RealityKit was introduced in 2019.
Will be going thru “Hello World” sample to explain the concepts.
RealityKit and SwiftUI
- RealityKit let’s you add 3D elements to your Windows and Views from SwiftUI
- By putting models in a separate window, using the new volumetric window – it will be a fully 3d object viewable from any angle. You can define a specific size, which will be consistent, (sized in meters). To create an immersive space, this is a scene type, not just a window View. You create with ImmersiveSpace(id:) { RealityView }. Not that Open Immersive Space is Asynchronous
- Sessions to watch – Meet SwiftUI for spatial computing … and … Take SwiftUI to the next dimension
- Go beyond the Windows with SwiftUI – takes you into the various types of spaces in more detail.
Entities and Components
- Entities are a container object, it must have components to do “something”, comments are things like models and transformers. (Really good description on this)
- Check out Explore Materials in Reality Composer Pro
- Transforms places things in a space. Properties are used for scale and motion.
- There are functions to convert between RealityKit and SwiftUI coordinate systems
- You can create your own components
RealityView
- This is a SwiftUI view to place entities in your app. You add entities to a content instance.
- You can connect sate to component properties so that you can express connection between SwiftUI state and the model
- These are observable so they only change when objects that they depend on change
- Convert function will change from entity space to swiftUI space.
- This is useful for scaling, etc.
- You can subscribe to RealityView events in SwiftUI via .subscribe(to:) to run code based on events.
- You can also attach SwiftUI views to entities – check out enhance your spatial computing app with RealityKit
Input, Animation, and Audio
- You can add gestures to realityKit views, you must have input target and collision components.
- You can add these components via Reality Composer Pro
- USDZ files can reference other usd files
- Default shape for collision component is cube, you should match it as best you can to your object’s shape.
- You can add a component (HoverEffectComponent) to make your app react to where you are looking. This is done outside of the apps process for privacy.
- Built in type of animations: From-To-By, Orbit, and Time Sampled
- RealityKit sounds are spatial by default. No additional reverb is added to ambient sources, channel is great for background music.
Custom Systems
- You can combine existing functionality in different ways to create components or systems.
- Work with Reality Composer pro content in Xcode – tells you more about this
- Systems are code that act on entities and components …. This allows your structure code that implements your apps behavior
- Registering a system in your app, makes it available across your app.
- You can filter the entities via a entity query so only those matching entities are impacted by the system.