What’s new in privacy

Understand and allowing control by users of what your collect is key for your users.  Their privacy pillars align with GDRP and extend in to on-device processing and security protections.

New Tools

  • There are new APIs in the Embedded Photo Picker, the Screen Capture Picker, a Write-Only calendar access, Oblivious http, and Communication Safety
  • The photo picker allows the API to only access a subset of pictures, you can embed this picker into your app in iOS17 and macOS Sonoma
    • If you use the new embedded picker, you will not have to request permissions
    • Embed the Photos Picker in your App – check this session
    • The new permissions dialog makes it very obvious what can be shared, and will periodically remind users what your app has access to
  • Screen Capture Picker
    • Prior to Sonoma you had to provide the whole screen, in Sonoma yet will not present a window picker on your behalf
    • You will be able to record selected content for the duration.  
    • A screen sharing menu bar item will be displayed.
    • What’s new in ScreenCaptureKit – session
  • Calendar can now be set to only add items since it has a lot of private information from a users perspective
    • EventKit will allow permission by default to create events. 
    • If you create your own API for creating entries you will have a new UI to request write only permission
    • If you need full access you can ask once for upgrade – you will not be able to ask again
    • For write only, you will be transition to this by default and if you use an old versions of EventKit it will only be asking for write
    • Discover Calendar and EventKit –
  • Oblivious HTTP API – hides client IP address from your server
    • This will also hide data from network operators 
    • This may add additional challenges for your app, so you can now use OHTTP to help protect app usage – by separating the who from the what.  This lightweight standard protocol allow the network operator to only see data to the relay – not the end point.
    • This is already used by Private relay
  • There are additional consideration is you use this, depending on your apps architecture
  • Communication Safety – to address sensitive content
    • This is the hide nudity protection from children.  It has been expanded beyond messages, to Airdrop, Contact posters, photos picker, and Phone app.  
    • It is available for all users, not just kids
    • This on device technology is available for apps to deal with sensitive data.

Platform Changes

  • Mac App data protection, advanced data protection, safari private browsing and Safari app extensions
    • Locations on Disk – have systems managed permission – like Desktop, Documents, and Downloads, some apps have data in other locations like ~/Library or ~/Library/Container
      • You can now control that a user must give permission to allow an app to access data in a container from a different developer
      • Use App Sandbox to protect data by your users
      • You can ask for permission by default if you make no changes – it is valid for as long as your app is open, otherwise the permission will be reset.  You should provide a meaningful purpose string.
        • Use NSOpenPanel – outside of your process to allow a user to find the data before a user is prompted
        • For backup or disk management tool that have already been provided Full Disk assess will not need to ask
        • If you have the same signature across apps, you should  have access to the data from your own other apps.  You can specific a NSDataAccessSecurityPolicy to change from “Same Team” access.
    • Advanced Data Protection – (added in 2022) to provide E2E encryption. If you use CloudKit you can get ADP for your app, when the user enabled ADP in their iCloud
      • Use CKAsset and Encrypted variants for all data types in your app.
      • Use the encryptedValues API to simplify the impact in your app
    • Safari Private Browsing – enables protection from finger printing and advanced tracking protections  (you can also turn this on for normal browsing)
      • By default in private mode, known tracking methods are not allowed, you can see the blocking in the web inspector
      • Tracking information on links is automatically stripped away
    • There is a new permission model will be also users to decide on a per site basis if an extension can run and/or when in private browsing mode.

Spatial Input Model

  • To achieve these goals hand and eyes are processed by an internal system component. So your app only get’s the tap or touch notice.

What’s new in App Store Connect

Monetize your app

  • Think about pricing and in app purchases
  • StoreKit for SwiftUI allows you to simplify adding in App Subscriptions to your app
  • Setting pricing needs to consider international pricing issues – there are 9 price points, based on a base region, You can have the store automatically adjust pricing to manage currency exchange changes. 

Manage testers

  • TestFlight makes it easy to test on all platforms and testers on each platform.  You get data about your testers to see how engaged they are, if they installed, sessions, crashes, and feedback, and now which device they installed on.
  • You can put release controls in place to send specific builds to internal teams vs. external teams.
  • Using Xcode cloud you can easily upload TestFlight what to test, can do this from a custom build script or directly from a local text file
  • Sandbox testing can now support family sharing, by combing test accounts to a family group.  Up to 6 accounts are in a single family
  • You can also add on-device to modify renewal rates, test interrupt purchases, clear purchases, and see the family grouping

Build your store presence

  • You can configure your product page, with privacy nutrition labels, and you can now add in information related to VisionOS, for surrounding, hands and head movement.
  • Release schedule on the app, can support Pre-Orders on a regional basis via soft launches.  This is on the availability page.
  • Product page optimization lets you see which ones your users like best, these now persist across app releases.

Automate with APIs

  • Allows for customizing workflows, syncing with internal systems, and dealing with customer reviews and responses.  Adding support for Game Center later this year
    • This is for leaderboards and achievements
    • You can do scores and achievement unlocks
    • Manage fraudulent activities
    • Match based on custom rules
  • For API authentication
    • You can generate marketing and customer services API Keys 
    • You can also create a user-based key 

Verify app dependencies with digital signatures

Automatically verify integrity of your dependencies, thru signature verification. This is trying to address the supply chain security attacks that have been happening.

Just like the discussion on privacy reporting, it is ultimately each developer’s responsibility to ensure all your signatures are correct and you are not using maliciously modified code.

Dependency Signatures

  • This is code signing is done by linking the binary, info.plist and privacy manifest to your developer identity.  This is done via a CDHash and your Dev. Identify from your Certificate. This hash allows for validation that the object has not been modified and was signed by your key.
  • New Signature verification will automatically handle the dependencies in your project
    • It tracks identities of frameworks in your project.  
    • Note the chain of trust will help across changes in certificates, etc.
  • A revoked certificate will trigger an alert to allow you to resolve or remove the framework from your application.

App developers

  • Demo with Backyard Birds app.- for any XCFrameworks you can see the signing information 
  • Notice the Kind and Status. When you build the app all signatures are validated in the app.  
  • You will get an error if they don’t match and get an alert with an option to accept the change or trash the framework.  Be careful with any self signed certificate.
  • You should start using only signed SDKs

SDK Authors

  • For SDK Authors, it becomes important to sign your SDKs so that developers can confirm that the code has not been manipulated 
  • You can either use Apple Developer program (ADP) or Self Signed certificates.
  • For ADP you should use Apple Distribution and Development Certificates
    • Apple is attestation is provided by Apple for these certificates
    • Validation is handled automatically
  • You can sing your code with codesign —timestamp -v —sign “YOUR CERTIFICATE Apple Distribution: (XXXX)” App.xcframework
    • This is using self signed certificate if you are a ADP member
  • You should start signing right away either with ADP or self-signed

Update your app for watchOS 10

While this is a code along session, at the time of my viewing there was no link provided, I did search the forums and the link here was provided, but it gave me a page not found.  Hopefully by the time others see this blog post the link will work.

Check out the Meet watchOS 10 a Design and Build apps for WatchOS 10 sessions.

An initial compile of a WatchOS 9 app against watchOS 10, you will get many features automatically, if you were using standard controls and options.  Shifting to NavigationSplitView can be achieved by switching from NavigationStack, moving your contents into the detail View builder, while adding a list view for navigation.

Breaking up long views to Vertical Tab Views make it easier to break up content in to logical aspects.

Adding in ToolBar and setting materials to provide additional information, makes it easy to help users.

Take SwiftUI to the next dimension

This will review how to use a handful APIs to use Windows, Volumes, and Full Space

Volumes

  • This is used to emphasize 3D content.  You get a fixed scaled container, which maintains the same size regardless of distance.
  • You just use .windowStyle(.volumetric) to get started…. It’s that simple

3D views and layout

  • Model3D API is used to load usdz objects and more.  It is similar to asyncImage 
  • There is a a phase to landing a Model3D,
    • These include .empty, .failure, .success
  • You should use .scaleToFit() modifier so that the model can it within the volume
  • By default alignment is back flush – so you want to use a .frame(depth: , alignment) modifier 
  • And than use .overlay to attach Labels
  • Using a TimelineView you can add .animation to the 3D View so it has motion

RealityView

  • For more complex scenes or experiences you should use RealityView instead of Model3D
  • Using Entity/Component system – check out “Build Spatial Experience with RealityKit” and “Enhance your spatial computing app with RealityKit”
  • Adding Attachments allow you to provide annotations relative to entities. You need to use any washable value for the .tag assigned to attachments
    • .look(at: ) allow you to position and orient items 

3D gestures

  • To interact with these entities you need to configure them for input, this is achieved by adding both InputTargetComponent and CollisionComponent so that you can add spatial tap gestures
  • There is also a .targetedToEntity so that gestures will fail if they are not on the specific entity.  Use .onEnded to retrieve the coordinates of the gesture.
  • You can add a Model3D as an attachment to a RealityView entity , don’t forget to add a .tag
  • You can combine gestures in a Gesture Modifier to provide more functionality

Simplify distribution in Xcode and Xcode Cloud

This is all about the CI/CD pipeline and speeding up the iteration process.

Express TestFlight Distribution

  • .xcarchive is a optimized release build of your app with debug symbols and it is repackaged for distribution
  • Xcode allows you to use the Achieve so that it either goes to your team only, without going to the App Store, or build a workflow that will ultimately be released to the App Store.
  • Nice to see the Internal test option which really makes it quick and easy to test with your team, one button click!  You can use App Store Connect to add in What to test information and receive screenshot feedbacks.
  • Xcode Cloud allows you to build workflows to customize the CI/CD process, this is now discoverable in the “Integrate” menu.  You can have workflows build based on various tags, branches, or other rules.  You can use Git Commit messages to include notes to testers. 

Automating notarization 

  • You can notarize your app so you can distribute without using the App Store, by uploading to the notary service, this will check for malware, etc.
  • To notarize in Xcode, you can go to Organizer and select an archive and click Distribute – choose Direct distribution
  • If you use Notarize post-action in Xcode cloud it can be used to automatically notarize your app for direct distribution 

Model your Schema with SwiftData

Utilize schema Macros

  • Watch meet SwiftData and How to build an App with SwiftData
  • Sample trip app is used with multiple relationships against the main @Model class
  • Schema Macros allow you to customize the behavior of Schemas
    • @Attribute(.unique) – allows for making a value is Unique – this will cause an update instead of a new items
    • You can provide constrains on primitive value types, (Numeric, String, or UUID)
    • You can also decorate a To-One relationship
  • To rename a variable – it will be seen as a new property so you can use @Attribute(originalName: “old_name”) var newName: String
    • This will ensure a simple migration 
    • @Attribute also provides support for External Data, Transformable, Spotlight Integration and Has Modifier
  • Working with Relationship – will auto setup implicit inverse relationships with default delete rule,  You can add @RElationship(.cascade) to delete them verses, just nulling them out.
  • To add non-persistent – just add @Transient and the value will be calculated at use time, must have a default logical

Evolving schemas

  • To handle updates to your schemas between releases, you use VersionSchema to define distinct releases
  • See SchemaMigrationPlan to do ordering of all updates needed to migrate the data
    • Define each migration phase  – either Lightweight (which requires no code), or Custom (which requires you to write code for things like-dedupe.
  • Annotate migrations so that when you build your plan, it will do the migration for you
  • You setup the Model with current Schema and a Migration plan in your app and it will upgrade data when needed.

Meet Object Capture for iOS

Intro for Object Capture for iOS – allows you to use Computer Vision to create lifelike 3D models which can be used in Object Capture API on Mac, but now you can do on device reconstruction for iOS.  There is a sample application to learn how to do this.   To me this looks like an update from the app shared earlier this year / late last year.

The code is available in the developer documentation, but you should certainly bookmark this page – https://developer.apple.com/documentation/Updates/wwdc2023

More objects with LiDAR

  • Performs best with extra details, but improved objects with low texture objects by using LiDAR, system augments models based on the point cloud to create objects
  • Still avoid transparent or reflective objects

Guided Capture

  • This automatically captures images with LiDAR provides feedback and guidance on how to capture.
  • Capture dial indicates which areas that has images – kinda like when you scan your face
  • You will get notified if there is not enough light.  Also use diffused light to minimize reflection 
  • You want a consistent distance when scanning and keep the object within the frame
  • Don’t forget to flip objects if it is rigid, if the object is repetitive, it may be problematic to do flipping.
  • There is now an API to tell you if the object is captured enough for flipping. And will recommend which way you should flip.

iOS API

  • imageCapture API is what you want to look up to find more information.  This is basically a State Machine to capture between ready, detecting, capturing, finishing, and finally completed
  • The APIs are in RealityKit and SwiftUI (https://developer.apple.com/documentation/realitykit/objectcapturesession) 
  • You should capture a space for where images are stored during Initialize phase
  • Your app will require to create your own UI to control the capture session for the user.  
  • The detection phase allows you to identify the bounding of the object so that it knows what you’d like to capture
  • Capturing will generate a point cloud to show you progress, one you are finished, you will need to create your own UI to complete the capture, or generate new captures to additional passes, or Flipping the object.
  • The Finishing process will wait for all data to be saved and then automatically move to Completing state
  • Completed state will then allow you to do On Device reconstruction, if completing fails you will have to create a new session.
  • Creating a 3D Model is the “Reconstruction API” – This is a PhotogrammetrySession which is pointed to the images to process and generate a usdz model file.  More on this wwdc21
  • Mac will also use LiDAR data, and supports higher resolution than the iOS device.  You can just use Reality Composer Pro on the Mac and won’t have to write any code.

Reconstruction enhancements 

  • Mac performance has been improved, along with providing an estimate processing time
  • You can create poses of images to pre configured and optimized poses.
  • You can also customize the number of Triangles, with a new Custom detail level

Inspectors in SwiftUI: Discover the details

Inspector

  • This show views of selected content.  For example a keynote side bar showing you about a selected item
  • This is available on macOS, iPadOs, and iOS, which includes programatic control of presentated state and width.
  • It is an structural API like NavigationStack or .popover
  • To add an inspector – you have a bool binding, and add the content in the trailing viewBuilder
  • Inspector uses Group style by default, they are not resizable by default, but you can add .inspectorColumnWidth( with an ideal value ) changes are handled by the system and retained across app launches.
  • It has different behaviors based on content it is attached to, this impacts toolbar placement and overall UI characteristics. On MacOS it is a bit easier.  If you are using a splitView it should be placed in the detail section.

Presentation customization

  • In iOS16.4 there are new features and they are also enabled for Inspectors
  • .presentationBackground( ._: )
  • .presentationBackgroundInteraction(._:) 
  • .presentationCornerRadius(_:)
  • .presentationContentInteraction(_:)
  • .presentationCompactAdaptation(_:)

Get started with privacy manifests

This is about simplifying creation of Privacy Nutrition Labels and holding developers accountable for the APIs and SDKs they use.

You are responsible for all code in your app.  I can image a fight brewing now between SDK creators and developers, who are ultimately responsible for the usage of these SDKs.

I highly recommend you check out this presentation yourself at https://developer.apple.com/wwdc23/10060

Privacy manifests

  • Third party developers can provide information about their SDK to be included in your app.
  • Create a PrivacyInfo.xcprivacy file in Xcode which defines the data collected and linked to the app, and how it is being used (Should update this for my own apps).
  • Check App privacy details on the App Store Documentation 

Privacy Report 

  • This pulls together all the information in one place.  It will aggregate across all content.  Right click on context menu for a App Archive, and choose Generate Privacy Report
    • This only works if you have created PrivacyInfo.xcprivacy files and they must be included in the archive

Tracking domains

  • Control network connections from your app.
  • Some SDKs may default to tracking or depend on you to request permission and will assume yes.
    • If you specify domains in your privacy manifest it will automatically disallow tracking unless the user agrees.
    • If the same domain has both tracking and non-tracking functions – split the domain to track this
  • Xcode Instruments will show you domains that are used to track across websites.  So you should run this instrument on your code to confirm that you are OK, Then you can declare the value in your privacy manifest
  • Fingerprinting is NEVER allowed

Required reason APIs

  • For important use cases while avoiding finger printing, Apple has created groups of categories  of APIs with a list of approved reasons
  • For example NSFileSystemFreeSize (disk space) check Required APIs in developer APIs
  • Documentation links to a feedback form if you have a valid reason
  • You must clearly state why you use these APIs in your Privacy Manifest
  • Check the Privacy-impacting SDKs in the developer documentation

Starting in fall 2023, apple will start sending informational emails to developers.  Starting in Spring 2024 they must reviewed and addressed. Also Privacy manifests will be expected at that point.