Make Features Discoverable with TipKit

Create a Tip

  • Title and Message – with direct action labels as title, easy to remember instructions in the message
  • Don’t use for promotional or error messages.
  • Add the TipsCenter.shared. ?? in the init() of the @main for the app.
  • Add icons and color match with the application.  You may wish to add an action button if there are settings.
  • Set treatment and placement.
    • Pop-over – points to element or button to direct the user  (.popoverMiniTip)
    • Inline – adjust the apps UI so no UI is blocked.

Eligibility rules

  • Don’t be spammy or irrelevant.  If the feature is already discovered don’t spam them.  If users are very infrequently using your app, you may not want to show tips either.
  • Rules are
    • Parameter based rule – persistent based on State and Boolean
    • Event-Based rules – define an action that must have occurred before they are shown.
    • Donate the event
    • You can create custom events using associated types and use that for the rule


Display and dismissal 

  • Don’t show it forever, don’t show multiple at once
  • You can do a DisplayFequency (.daily, .hourly, custom duration) you can also do .immediate
  • You can do .ignoresDisplayFrequence(true) for a specific tip 
  • If the feature is used it should be dismissed… You can then call the invalidate function with user performed action
  • You can also set a .maxDisplayCount to not annoy the user with the same tip over and over.
  • Tips can be synced via iCloud, so you don’t want to show the same tip on different devices

Test tips

  • You can work around eligibility rules.  Use TipsCenter.showAllTips() or .showTips with tip iD, or .hideTips to prevent, or .hideAllTips()
  • .resetDatastore() can be used to clean it up 
  • You can also use com.apple.TipKit.showAllTips in your launch arguments (or any of the other commands)

Running your iPad and iPhone apps in the Shared Space

Majority of apps will run fine in VisionOS (check it out in the simulator)

Built-in Behaviors

  • Built on iOS foundations, more than likely your app will just work.
  • iPod and iPad apps will be in light mode in landscape mode.  iOS only would be portrait mode.  If you support rotation you will see rotation button.  Mini and max size will cause bounce.
  • Can use trackpad or game controller when using your app, along with touch.
  • Local authentication is forwarded to Optic ID

Functional Differences

  • There’s no notation of rotating the entire device, so you may define the default orientation – UIPreferredDefaultInterfaceOrientation to you info.plist 
  • UISupportedInterfaceOrientations – will decide if you need a rotation button
  • UIRequiredDeviceCapabilities – will decide if your app can be on the device.  
  • Gestures – work differently
    • Eyes & Hands, maximum of two simultaneous inputs (each hand is a single button)
  • ARKit – ARSession
    • Significantly updated to handle new architecture and privacy.
    • Existing AR views won’t work on this device, you will have to do work on this.
  • Location support – same as iPad
  • Look to Dictate – allows your to quickly adjust around the screen.  You will get a microphone in search fields as an example.
    • API is available, but disabled by default of iOS and iPad app
    • .searchDictationBehavior to enable in SwiftUI
  • Use availability checks to make sure it is available for you try to use it.
  • xrOS Device (Designed for iPad) new target

Choose your experience

  • Most apps don’t need changes at all, you can rebuild against xrOS SDK but not required.
  • Only In iOS SpriteKit and Storyboards
  • You get additional immersion with xrOS SDK – Volume, etc. 
  • You will get system look and feel when using xrOS SDK in the simulator.  Ornaments anchor on side and bottoms.

Check out the Meet SwiftUI for Spacial Computing to see how apps will look and learn what you need.

Bring Widgets to New Places

New location for widgets -First introduced in iOS 14, added to Lock Screen iOS 16

New locations are Desktop on Mac, iPad lock screen, standby Mode on Phone, stack mode on WatchOS

Transition oto content margins

  • Padding automatically applied to prevent getting to close to the edges.   This replaces safe area … .contentMarginsDisabled() will do like ignore safe area.

Add a removable background to widgets

  • You need to remove for background for Lock Screen, etc.
  • Add containerBackground(for: .widget) {  // your original color here in the closure }
  • Smart Stack will use this approach.
  • If you have no background to remove (or one that doesn’t make sense) you .containterBackgroundRemovable(false)

Dynamically adjust layout

  • Change layout when background is removed, this is good for reading from far away in standby mode
  • Also blends better in Lock Screen.

Prepare for vibrant rendering mode

  • This is for Lock Screen mode
  • Use .widgetRenderingMode environment variable to see which mode you are in.

Meet ActivityKit

This is all about Live Activities

Overview

  • Allow for glanceable ways of keep track of task or event.  Have a defined start and end time.
  • Can use either background or push notifications
  • Dynamic Island can display two live activities at any time.  They will show minimal presentation.
  • You can press to expand to more information 
  • Now supported on StandBy Mode and on iPad 
  • It leverages Widget enhancement to add buttons or toggles.
  • Use ActivityKit framework – it is declarative, very similar to using Home Screen Widgets.  Only available after a discreet user activity.  They are user-moderated like notifications. It must support both Lock Screen and Dynamic Island.

Lifecycle

  • The App goes thru different stages,
    • Request
    • Update
    • Observer activity state
    • End
  • Request app is in foreground, configure initial content and configuration.
    • Setup the update based on information and provide information on state (like when will it be stale).  Reliance is also optional.
    • Define if it is pushType for remote verses local
  • Update
    • Based on key things in the application you can send an update, you can also display alerts if the event is critical.
  • Activity state changes can happen at any time during the Live Activity’s lifecycle (started, finished, dismissed and Failed) are the possible values.  Based on the state changes, you can do cleanup and other activities.
  • End
    • Create a final content and provide that to the item
    • Change the policy 

Building Live Activity UI

  • Need a widget configuration to describe the live activity.
  • Need to define all three Dynamic Island UI
    • Add CompactLeading, compactTrailing, Expanded and minimal views.
  • When more than live activity is running, the system will decide which apps are shown in the dynamic island.
  • Expanded presentation has multiple areas defined by the system. .leading, .trailing, .bottom space.

Design and Build apps for watchOS 10

Goals is to surface timely information, communicate at a glance, take advantage of the screen, and make things consistent.

Design Principals

  • Timely and relevant: in the moment 
  • Focused and highly specialized for brief interaction
  • Unique to Apple Watch – utilize the Digital Crown – this is becoming a new focus item for apple (should be backed up by touch)
  • Consider the journey – people will take from moment they raise their wrist – Smart Stack is an example of this.

Navigation

  • Updated in NavigationSplitView
    • Borrowed from 2 column layout from weather on iPad – the list is now a button on the top leading screen – great if you have a source list with detailed views.  Start with the detailed view when user starts your app.  Try with no title
    • Transition between detail view and source list is animated.  Great for showing comparative data.  (Should add number of cards in my view for card app).
    • Same API on watch as other platforms.
  • TabView updates
    • Now it is designed to be vertical.  Can be expanded inline so that you can stay on a single screen by default but support localized text, etc.
    • Activity is a great example of this.  (Uses the  .tabvViewStyle(.verticalPage))
    • Combine it with .containerBackground((color, for: .tabView) for seamless blending
    • Tabview will automatically expand a list if you add it to the grouping.
    • Animation to scale information as it moves to new locations.  You can now drive animation based on the tabView.  You  can use .matchedGeometryEffect to do so.
  • NavigationStack (core paradigm)
    • Use a NavigationStack if the other two don’t address your appropriate workflow.

Layout System Updates

  • Dial based views
    • Great for dense info delivered at a glance, you can also provide 4 different controls 
  • Infographic Views
  • List views
    • Scroll thru context to find what you need
  • If you use these layout patterns then your toolbar placement, and the like don’t need extra padding.  It just works.

Color and Materials

  • Four background materials – Ultra Thin, Thin, Regular and Thick. 
  • Full screen background gradient 
  • You can use Primary, Secondary, Tertiary, Quaternary prominence layers – with vibrant versions to ensure legibility 

Meet watchOS 10

Widgets are surfaced automatically, 

Be focused and specialized

Design for glances

Based on shape of display, the new design language has three foundational patterns,

  • Dial
  • Infographic
  • List

Use background information to convey additional information – this is key to the migration of the widgets.  This is backgroundContent – not just a visual flourish 

New material applications (this is to align with xrOS or VisionOS) – 

  • Translucency established hierarchy 

New navigation patterns – 

  • Use the crown to do things without covering the screen.
  • Vertical patination, is now used for multiple tabs in an app. Use this instead of horizontal pagination.
  • User object permanence of items across pages 
  • If you extend beyond the height of the display 
  • Source List is a new pattern. See World clock to see what this means

Meet SwiftData

Code focused, using macros, to naturally work with SwiftUI

Using the model Macro

  • @Model – new model schema from swift Code
  • Simply decorate the class with @Model 
  • Attributes inferred from properties 
  • Relationships are inferred from the model
  • You can use @Attribute or @Relationship on your properties to drive uniqueness and delete behavior
    • For example 
    • @Attribute(.unique) var name: String
    • @Relationship(.cascade) var bucketList: [BucketListItem]? = []
  • Check out Model your schema with Swift Data

Working with Data

  • Model Container
    • Persistence backend or customize with migration options
  • Model Context
    • Watches for changes
    • Does tracking, fetching, saving and undoing changes.
      • This is usually an Environment feature for the context.
    • Predicate and FetchDescriptor 
    • Does strongly typed construction
    • Can do related objects to be pre-cached 
    • Basic CRUD is supported with simple context.save()
    • Dive deeper into SwiftData session

Use SwiftData with SwiftUI

  • Seamless integration  with SwiftUI
  • Automatically fetching and updating UI
  • Simple code example
import SwiftData
import SwiftUI

struct ContentView: View {
	@Query(sort: \.startDate, order: .reverse) var trips: [Trip]
	@Environment(\.modelContext) var modelContext

	var body: some View {
		NavigationStack() {
			List {
				ForEach(trips) { trip in 
					// …
				}
			}
		}
	}
}
  • Since this is Observable, you don’t need to use @State – it just works.
  • Check Migrate to SwiftData

What’s new in VisionKit

This API is all about lifting subjects from images, i.e. pulling out your favorite pet from a picture to be used elsewhere 

Subject Lifting

  • Simple long press on an image, you can share the subject.  Now you can lift any subject and turn it into a sticker.  This is automatically for most code from last year.
    • Just set an appropriate interaction type, .automatic does a combination of al types
    • .imageSegmentation will only do Subject lifting
    • .automaticTextOnly – same as iOS 16 behavior – test selection and data detectors

Visual Look Up

  • Allows users to know about pets, nature, landmarks, art and Media.  In iOS17 adding Food, Products and Signs & Symbols
    • This helps with laundry tag information!
    • Works for 6 languages to start with.
  • Analysis is entirely on the device. Includes feature extraction.
    • One you look up the object  then it goes to server for more information.
    • Add .visualLookup to your analyzer configuration 
    • Badges are displayed on subjects for lookup etc. 

Data Scanner and Live Text – allows for OCR 

  • Optical flow – can now do high speed scanning and comes for free when recognizing text.  
  • Currency Support
    • Find and interact with monetary by setting textContentType to .currency in the initializer, just like email or phone numbers
    • Provides both a Bounds and a transcript.  Code example in video for totaling a receipt.
  • Live Text – more regions:
    • Can detect document structure, like list detection (iOS16), but now can also detect Tables   Will even merge cells if needed.
    • Context-aware data detectors – will pull additional information from surrounding areas, like name and address
    • You can do .transcript last year, you can now do ranges, selected text, Attributed and identify when text changes.

Expanded platform support

  • All about the Mac – Catalyst support being rolled out
  • Catalyst –
    • Simple recompile for LiveText, Subject Lifting and Visual Look Up, no QR support yet ( you can do it via Vision Framework)
  • Native Mac API
    • ImageAnalyzer
    • ImageAnalysisOverlayView – subclass of NSView, just add above image content (adding as SubView of ContentView is simpler)
    • Basically the name as iOS (except .machine-readable is a NOOP
    • Contents rect – needs to know the rectangle bounds that it needs.
    • If you use NSView you can just set trackingImageView on the overlay and the system will do it for you
  • Contextual Menus (Mac)
    • You can combine items in the same menu, this provides a much better right click menus based on selected item.
    • Items are identified by tags, there is a struct that defines them.  Code examples are in the session.

What’s new in SwiftUI

SwiftUI in more places

  • Containers have been updated to address VisionOS – you can add .volumetric to a WindowView
  • RealityView
  • ImmersiveSpace {} Scene type allows for full emersion or mixed to create a AR view
  • Redesigned UX on watchOS 10, is based on existing SwiftUI views.  Get new transitions
    • .containerBackground modifier will allow for transitions in push and pop
    • ToolbarItem(placement: .topBarTrailing and .bottomBar}
    • Date Picker and Selection in List have been added to Apple Watch
  • Widgets on iPad, Home Screen, Desktop, and interactive controls
  • New framework updates for SwiftUI – Focus on major updates to MapKit – (will need to see how I can reformat my Map integrations – Meet MapKit for SwiftUI), ChartKit has scrolling charts, donuts, and pie charts.

Simplified Data Flow

  • Start with data model created in SwiftData
    • @Observable models use SwiftUI patterns but for DataFlow, if you pass a model thru intermediate views, they won’t get invalidated if they don’t use the property in the model
    • Confirm session Discover Observation with SwiftUI
  • SwiftData are represented entirely by code. Will receive persistence and observable.
    • Add a .modelContainer to you App View
    • Add @Query to model in your views (tells swift data to do a fetch from database).  Also works for document based apps.
  • Document Groups – automatically get sharing, renaming and more
  • Inspectors – create side bars or sheets based on platform.
  • Dialog customization
    • Help Links – 
  • Lists and Tables for fine tuning them.
    • Tables – Column order and visibility, and new Disclosure Table rows (for grouping)
    • Sections now have programmatic expansion.
    • Style formatting for smaller lists or tables.

Extraordinary animations

  • Animations can be improved with KeyFrames – triggered by state change. (Runs a set of animations in parallel)
  • Phase animators (simpler than KeyFrame – step thru a sequence of animations)
  • Haptic feedback uses new SensoryFeedback API – add a modifier .sensoryFeedback check HIG for Playing Haptics
  • .visualEffects modifier – don’t need geometry reader, to allow for an animation based on position on the screen.
  • You can do text interpolation with .foregroundStyle so you can do cool metal shaders using ShaderLibrary.
  • Sliders now have .symbolEffect modifier (can do it at a single item or the entire view).  Check out (Animate Symbols in your app)
  • You can do .textScale to a text view.
  • .typesettingLanguage modifier will allow for text that requires more space like some languages (Thai)

Enhanced interactions

  • .scrollTransition modifier will allow effects to items in a scroll view.
  • .containerRelativeFrame modifier allows for making things relative to other screen parts
  • .scrollTargetLayout()  
  • .scrollPosition allows your do some thing based on position within the scroll view.
  • .allowedDynamicRange modifier is now supported to show full fidelity, use sparingly (must be a performance hog)
  • New Accessibility options. You can add .accessibilityZoomAction so voice over can access other actions like swipe actions.
  • Color can look up customer colors defined in your asset Catalog.
  • Menus now have improved visual styles
  • New BorderShape styles for buttons 
  • .onKeyPress – allows for actions based on key presses (with modifiers)

What’s new in Swift

Swift Project updates

Expressive Code

  • If-else/switch can now be used inline – this simplifies initialization and other conditions
  • Improved error messages and more directly aligned to where the problem is.  (This will come in handy in SwiftUI)
  • Swift Generics allows for type identification preservation, this has been extended to handle # multiple argument types.  So no longer will you be limited to the number of arguments to be passed. Now use <each Result>
  • Macro System –
    • Macros are APIs so you can import module that defines them
    • @Observable macros is key to turn a class into fully observable in your code. (Expand on Swift Macros session)

Swift everywhere

  • Low ceremony code to make it easier to scale
  • Future of Foundation – starting a rewrite of Foundation for Apple and non-Apple platforms.  This is moving the implementation of Swift to Swift.
    • Drives major performance updates across calendar (20%), date formatting (150%), and JSON coding (200-500%)
    • New opt-in capabilities which focus on code ownership. Check it out on the swift forums
  • You can now interoperate between C++ and Swift now. There is a Swift Forum that is being driven by a WorkGroup focused on this.
  • CMake now support Swift and can e used to mix and match C++ and Swift make files, there is a sample Repo to help you get started
  • Actors and Concurrency
    • This section explained synchronization, how the actors and tasks behave

Case Study