Instead of creating a unique page for each session I watched, I’ve decided to group them. After the Keynote today, I started updating my various devices, got all but the development mac updated. I think my old development watch may have just gone passed the supported level too, so it is stuck on watchOS 26. The mac should be completely upgradable, so not sure what is going on there.
Platform State of the Union
Apple Intelligence
Apple Foundation Models Plus Google Gemini
Apple used both their models and Gemini to build a new Apple foundation model, that runs on device. Uses foundation model framework to include both image input and server models.
API can now integrate with the cloud provider of your choice. Don’t get held back by infrastructure costs.
A single API to support any model you need…and surface your app (via App Intents) to Apple Intelligence.
using these models, add privacy and no infrastructure costs. There is a sample app for building origami. Using foundation models to help design new folding projects.
Multimodal prompts – yes! About time, adding images and text to do much more natural interactions.
Server models, are necessary for more complex workflows and can call Claude or Gemini, or provide their own Swift Package. Using Apple’s models, will be free for developers. My personal choice would be to use Apple’s models to reduce costs and increase privacy.
Foundation Models framework Utilities
Custom skills and more. This is a new OSS package for extending your AI.
Dynamic Profiles – new declarative APIs in the Foundation Model framework.
Cool that switching models will share the same transcript.
Added evaluation framework to test your models and improve oyur app.
Later this summer the framework will be made Open Source so that it will be available for Server deployments.
Core AI
New framework built into the platform. Swift API, memory Safe, with fine grain model tuning and more. Allows to trace your tensor model back to the python code in debugging mode.
App Intents
Now expose your app to Siri AI. This leverages the Spotlight semantic search so users can find out information from your app. Use IndexEntity protocol (so it can be searched) and Entity Schema (so it can be reasoned on ).
If you support sendMessage – it can now DO things with your app.
view Annotation allows siri to look at items within the View and react based on them.
Liquid Glass improvements
Ok let’s be real, they took a lot of good feedback over the last year. I am sure some people will complain, but I think they have addressed most, if not all, of the major design challenges. The outline feature will make things easier.
They also added resizability features, they are certainly telling me, as a developer, that they are introducing a new screen – foldable iPhone anyone?
Re-orderable containers – Will need to add that to my ForEach of my Vinyl Trackr app.
Swipe actions within any container. Very cool.
Shared implementation across SwiftUI, AppKit, and UIKit
Finally, adding better alignment across all three frameworks.
Moved State to lazy under the hood, so you don’t have excess object creation.
New Capabilities
SO any I will have to review. But Spatial Preview model, is one that should really help with VisionOS and Future glasses (anyone).
Swift
Available on Linux, Windows, Android, and Web. With major interoperability added to Java, C, and C++.
Full stack Swift! Gotta happen. Swift 6.4 should improve the view debugging.
Apple Silicon Only
The migration is considered done. Removing support for old design, all OS 27.X will use new desing.
Developer productivity
Last year added AI coding assistants to Xcode – it moves really fast, so added new releases even faster.
- Added MCP support and will supoort Github and Figma, agents from Google, OpenAI and Anthropic. (Added today to Xcode 26
- Xcode 27 will add the following:
- Daily Experience – 30% smaller, Apple Silicon only, background doc downloads. Settings saved to iCloud (YEAH!).
- Customize the toolbar and Themes (and you can set different themes by project).
- Xcode Cloud – No appstore connect setup needed.
- Previews – easier to do multiple variants
- Device Hub – replaces simulator and allows both devices and simulators to be see in one place. (This seems to do a lot of what KickStart Command center did).
- Intelligence
Transforming how you build apps. Added agents to every layer of the experience. every response will be grounded in Apple’s data and swift.
Adding support for /plan and diagrams to work thru designs before writing any code. With multi-threaded interactions to see what is being built, while allowing you to refine.
Validate App via tests and API explorations. Can do different localizations, color modes, and device sizes.
Localization looks at context for translations.
Use apple’s own expertise for the skills, and then will leverage a third party for LLMs. Also supports Plugins, from the community – like skills, tools (MCP), and Agents (Agent client protocol). This will reach out to a git repo to pull that agent in.
Apple Apps
Reality composer 3 – major updates and updates to Game Developers and the Game Porting Toolkit.
— At this point I ran out of steam. Will be watching it again.
Rev up your CarPlay app
Basically what’s new in CarPlay for iOS27.
Apps in CarPlay
Audio, communication, navigation, driving task, EV charging, fueling, parking, public safety, and Quickly Food ordering. And Live Activities and Widgets. Added Voice-based conversational, and browse/play videos.
Video Apps
Playback is only available when the car is not moving. It will play audio while the car is moving.
CarPlay framework
Works with both CarPlay and CarPlay Ultra. Lots of UI updates for lists, voice control, and more. using the existing Framework means you don’t have to deal with different screens or layouts. It will do it for you automatically.
Lists can show images, and card elements now have Thumbnails with overlays, titles, and other information. You can set preferred presentation to video or audio on playback configuration.
There are additional overlays for sports teams and score. And other formats for Details header for movies and more.
Nice walkthru of the carPlay simulator.
Navigation apps
Have more control can now show panels for your own UI and provide more information for your app. Helpful for EV charging stop recommendations and driver assistance features. Also used in Route Sharing apps. Can be used so that an EV can suggest waypoints for charging along the way. Driver must pair with the vehicle to allow route sharing.
What’s new in Wallet
Passes are an essential part for a lot of people’s day. New pass-style, barcodes, new featured actions. And new developer tools.
Poster Generic
Great for membership cards, etc. Pretty much a simple json file, with a QR code. (But very graphically rich, with only a single footer field.)
Barcodes
There are four new types being supported – EAN-13, Code 39, Codabar, and Interleaved 2 of 5 (ITF). Don’t really know the difference here.. but I am sure they are important for different use cases.
Providing multiple barcodes that a pass can have in priority order, then if the user is on an older versions where the older bar codes can be shown. If you must have a format, use a credential field to make sure people can use it.
Featured actions
There are semantic URLs to allow for new information and actions for your pass. This will do things like, viewing offers, and they will be displayed below the case. You can have up to two featured actions.
Pass Designer
A new mac app WYSIWYG editor for building passes, and also new swift on server package to provide the package for sending to devices. Checked for the download on the developer site, but didn’t show up yet.
Find your accessory with Bluetooth Channel Sounding
This is a feature for developer who utilize nearby bluetooth accessories. This sessions is really designed for people who build hardware devices.
With channel sounding you can actually measure distance to a device. This is interesting and I am sure it is used in hacking of devices. You can use core Bluetooth and the accessory is paired and setup.
Not really going to look at this, but my goal this year is to go thru all the videos.
What’s new in Shortcuts
This was very exciting, the ability to use AI (LLM) to build new shortcuts is something that seemed almost magical during the keynotes. And the fact that the macStories team did it just a few weeks ago means the time was right.
Updates on Automations and new types
You can make shortcuts run automatically when things occur. For example when you leave from work. There are three new types: Screenshot (when it is saved), Keyboard (when an external keyboard is connected), and Notification (when a notification is received from a specific app).
Example was to auto turn on porch lights when a delivery is nearby. Really cool model, since you can make it easy parse based on best practices for design a notification, then an automation can take advantage of it.
Use model
This allows you to use both Cloud, Cloud Pro, and On-Device models. Giving a short cut to take a large amount of data to do things. If a model doesn’t produce what you inspect, you can inspect the models context (show content action) to see what it did and why it responded it that way.
There is a good session in WWDC25 on App Intents
Storage
You can now save content within a shortcut to persist information between runs, and a global value to share across multiple shortcuts. Set, Get, and Delete (Stored Content) to a variable.
This is a great way to build up knowledge or memory across model based shortcuts. It can even store an App Entity. Stored values sync across devices.