Additional Sessions from Day 1

Instead of creating a unique page for each session I watched, I’ve decided to group them. After the Keynote today, I started updating my various devices, got all but the development mac updated. I think my old development watch may have just gone passed the supported level too, so it is stuck on watchOS 26. The mac should be completely upgradable, so not sure what is going on there.

Platform State of the Union

Apple Intelligence

Apple Foundation Models Plus Google Gemini

Apple used both their models and Gemini to build a new Apple foundation model, that runs on device. Uses foundation model framework to include both image input and server models.
API can now integrate with the cloud provider of your choice. Don’t get held back by infrastructure costs.
A single API to support any model you need…and surface your app (via App Intents) to Apple Intelligence.

using these models, add privacy and no infrastructure costs. There is a sample app for building origami. Using foundation models to help design new folding projects.

Multimodal prompts – yes! About time, adding images and text to do much more natural interactions.

Server models, are necessary for more complex workflows and can call Claude or Gemini, or provide their own Swift Package. Using Apple’s models, will be free for developers. My personal choice would be to use Apple’s models to reduce costs and increase privacy.

Foundation Models framework Utilities

Custom skills and more. This is a new OSS package for extending your AI.
Dynamic Profiles – new declarative APIs in the Foundation Model framework.
Cool that switching models will share the same transcript.

Added evaluation framework to test your models and improve oyur app.

Later this summer the framework will be made Open Source so that it will be available for Server deployments.

Core AI

New framework built into the platform. Swift API, memory Safe, with fine grain model tuning and more. Allows to trace your tensor model back to the python code in debugging mode.

App Intents

Now expose your app to Siri AI. This leverages the Spotlight semantic search so users can find out information from your app. Use IndexEntity protocol (so it can be searched) and Entity Schema (so it can be reasoned on ).
If you support sendMessage – it can now DO things with your app.
view Annotation allows siri to look at items within the View and react based on them.

Liquid Glass improvements

Ok let’s be real, they took a lot of good feedback over the last year. I am sure some people will complain, but I think they have addressed most, if not all, of the major design challenges. The outline feature will make things easier.
They also added resizability features, they are certainly telling me, as a developer, that they are introducing a new screen – foldable iPhone anyone?
Re-orderable containers – Will need to add that to my ForEach of my Vinyl Trackr app.
Swipe actions within any container. Very cool.

Shared implementation across SwiftUI, AppKit, and UIKit

Finally, adding better alignment across all three frameworks.
Moved State to lazy under the hood, so you don’t have excess object creation.

New Capabilities

SO any I will have to review. But Spatial Preview model, is one that should really help with VisionOS and Future glasses (anyone).

Swift

Available on Linux, Windows, Android, and Web. With major interoperability added to Java, C, and C++.
Full stack Swift! Gotta happen. Swift 6.4 should improve the view debugging.

Apple Silicon Only

The migration is considered done. Removing support for old design, all OS 27.X will use new desing.

Developer productivity

Last year added AI coding assistants to Xcode – it moves really fast, so added new releases even faster.

  • Added MCP support and will supoort Github and Figma, agents from Google, OpenAI and Anthropic. (Added today to Xcode 26
  • Xcode 27 will add the following:
    • Daily Experience – 30% smaller, Apple Silicon only, background doc downloads. Settings saved to iCloud (YEAH!).
    • Customize the toolbar and Themes (and you can set different themes by project).
    • Xcode Cloud – No appstore connect setup needed.
    • Previews – easier to do multiple variants
    • Device Hub – replaces simulator and allows both devices and simulators to be see in one place. (This seems to do a lot of what KickStart Command center did).
  • Intelligence
    Transforming how you build apps. Added agents to every layer of the experience. every response will be grounded in Apple’s data and swift.

Adding support for /plan and diagrams to work thru designs before writing any code. With multi-threaded interactions to see what is being built, while allowing you to refine.

Validate App via tests and API explorations. Can do different localizations, color modes, and device sizes.

Localization looks at context for translations.

Use apple’s own expertise for the skills, and then will leverage a third party for LLMs. Also supports Plugins, from the community – like skills, tools (MCP), and Agents (Agent client protocol). This will reach out to a git repo to pull that agent in.

Apple Apps

Reality composer 3 – major updates and updates to Game Developers and the Game Porting Toolkit.
— At this point I ran out of steam. Will be watching it again.

Rev up your CarPlay app

Basically what’s new in CarPlay for iOS27.

Apps in CarPlay

Audio, communication, navigation, driving task, EV charging, fueling, parking, public safety, and Quickly Food ordering. And Live Activities and Widgets. Added Voice-based conversational, and browse/play videos.

Video Apps

Playback is only available when the car is not moving. It will play audio while the car is moving.

CarPlay framework

Works with both CarPlay and CarPlay Ultra. Lots of UI updates for lists, voice control, and more. using the existing Framework means you don’t have to deal with different screens or layouts. It will do it for you automatically.

Lists can show images, and card elements now have Thumbnails with overlays, titles, and other information. You can set preferred presentation to video or audio on playback configuration.

There are additional overlays for sports teams and score. And other formats for Details header for movies and more.

Nice walkthru of the carPlay simulator.

Navigation apps

Have more control can now show panels for your own UI and provide more information for your app. Helpful for EV charging stop recommendations and driver assistance features. Also used in Route Sharing apps. Can be used so that an EV can suggest waypoints for charging along the way. Driver must pair with the vehicle to allow route sharing.

What’s new in Wallet

Passes are an essential part for a lot of people’s day. New pass-style, barcodes, new featured actions. And new developer tools.

Poster Generic

Great for membership cards, etc. Pretty much a simple json file, with a QR code. (But very graphically rich, with only a single footer field.)

Barcodes

There are four new types being supported – EAN-13, Code 39, Codabar, and Interleaved 2 of 5 (ITF). Don’t really know the difference here.. but I am sure they are important for different use cases.

Providing multiple barcodes that a pass can have in priority order, then if the user is on an older versions where the older bar codes can be shown. If you must have a format, use a credential field to make sure people can use it.

Featured actions

There are semantic URLs to allow for new information and actions for your pass. This will do things like, viewing offers, and they will be displayed below the case. You can have up to two featured actions.

Pass Designer

A new mac app WYSIWYG editor for building passes, and also new swift on server package to provide the package for sending to devices. Checked for the download on the developer site, but didn’t show up yet.

Find your accessory with Bluetooth Channel Sounding

This is a feature for developer who utilize nearby bluetooth accessories. This sessions is really designed for people who build hardware devices.

With channel sounding you can actually measure distance to a device. This is interesting and I am sure it is used in hacking of devices. You can use core Bluetooth and the accessory is paired and setup.

Not really going to look at this, but my goal this year is to go thru all the videos.

What’s new in Shortcuts

This was very exciting, the ability to use AI (LLM) to build new shortcuts is something that seemed almost magical during the keynotes. And the fact that the macStories team did it just a few weeks ago means the time was right.

Updates on Automations and new types

You can make shortcuts run automatically when things occur. For example when you leave from work. There are three new types: Screenshot (when it is saved), Keyboard (when an external keyboard is connected), and Notification (when a notification is received from a specific app).

Example was to auto turn on porch lights when a delivery is nearby. Really cool model, since you can make it easy parse based on best practices for design a notification, then an automation can take advantage of it.

Use model

This allows you to use both Cloud, Cloud Pro, and On-Device models. Giving a short cut to take a large amount of data to do things. If a model doesn’t produce what you inspect, you can inspect the models context (show content action) to see what it did and why it responded it that way.
There is a good session in WWDC25 on App Intents

Storage

You can now save content within a shortcut to persist information between runs, and a global value to share across multiple shortcuts. Set, Get, and Delete (Stored Content) to a variable.

This is a great way to build up knowledge or memory across model based shortcuts. It can even store an App Entity. Stored values sync across devices.

WWDC 26 – Day 1

Good Morning!

Loved the stop motion, and “Good Morning!” Energy has never been stronger, with over 1,000 submissions every hour! (Wonder how many are vibe coded).

Nice to see Tim in front of the Apple Park Rainbow! And of course there is the video from the moon trip! Apple Intelligence and Siri. And time to Craig to “get us started”.

Focus in three area

  1. Platform improvement
  2. Trust and Safety updates
  3. Big leap forward on Apple Intelligence and Siri.

Yay! VisionOS was mentioned.

They motored north to name it! Summer of love, (love the 60’s graphics!). Golden Gate! Perfect name for the new one.

Platform improvements

Expect this to be underpinned with AI and improved reliability. So taking existing features and sweating the details. Time for the Liquid Glass improvements. A few highlights:

  • Design – So Liquid Glass is upfront. So “listened” to users and developers. Updating the foundation, to improve readability, adding clarity and tinting to personalize the UI. No changes to existing apps. More uniformed toolbar, to keep test and heading legible. Sidebars now expand to the leading edge (about time). Tighter corner radius.
    • App Icons – integrating layers into the icons artwork, to make them even sharper. This means icons will be more work.
  • Optimized system to address responsiveness. System animations, faster launches (up to 30%), faster content loading (up to 70%), 5x improvement on external drive access.
    • CPU Scheduler updated to make older devices feel faster. (All the way to iPhone 11, meaning that older devices are still supported).
    • Network transitions (are more seamless), via smart switching.
    • Search – rebuild spotlight, photos, and mail to improve this (I bet this is done for AI access). Prioritize new content for indexing, but more detailed search overall.
  • Features
    • Photos- iCloud shared albums can now support Android users and full resolution
    • Health – Perimenopause tracking and information
    • AirPods – customer EQ (yeah!)
    • Apple VisionPro – panos can become spacial scenes and can be used as an environment
    • Maps – improved flyover for improved 3d Maps!

Trust and safety

Apple builds products to empower people and stay safe. Long standing commitment to safety for kids and teens, is import to apple. So building a trusted safety for parents and kids.

  • Putting responsibility on Parents and using expert health research. Including establishing boundaries around technology use. Age based permissions guided by parental supervision. Using family media plan (guide for parents), and collaboration with researchers and advancing the science.
    • Create a child account, that tailors device to their age. You can covert existing accounts. Addresses, content they can see, who they can talk to, when they have access and more.
    • Manage downloads, App Store age groups and ask for permission from parents via messages. Added “Websites” for kids to browse. On by default for kids under 13.
    • Communication safety now includes gore.
    • Screen Time – new time allowances to allow categories, Games, entertainment, and social media. Default times based on research, but parents can adjust.
    • Can setup schedule for different times of the day, I.e. in school only allow specific apps, etc.
    • Redesigned screen time, to make it easier to use and understand.
  • Developers updates to parental controls, and every app “should” enable the APIs. Wonder if my Vinyl Trackr needs this since an album cover could have gore or sex.
  • New website for child safety.

Intelligence

AI has a profound value, but some are pursing AI for the sake of AI. Apple wants to make it helpful and intuitive products for everyone. AI must be centered around you and your needs. So deep integration around your personal context, and design with privacy at every level.

Architecture

Centered around you and your apple products, using Apple Foundation models, and deep collaboration with google and their Gemini model. Adapted to run on device and servers using Apple Privacy Cloud Compute. Image understanding, generation, and editing photos.
More powerful version for most capable system to generate and understand speech and more.
System orchestrator understanding across your system and using Spotlight’s new semantic index. Added in Broad world knowledge and use Private cloud compute to bring back your answer. And then App Actions to do things. On Screen Awareness, makes it more powerful.
Privacy first design! Unlike those who ask you to turn off features to be private. Apple is not stored or accessible to apple or anyone else.

Siri

All new Siri – Mike Rockwell – VP Siri Engineering. Called Siri AI. Uses new Apple Intelligence and used via “hey Siri”. A significantly more capable assistant and more conversational. And new dedicated Siri app allows you to refer back to them, with more tools like Write with Siri and more. Live demo and showing both hand and talking (call back to the marketing demos).
Dynamic Island what a nice touch. On screen awareness was cool. The demo with world knowledge was cool.
The use of intents (or what aver the new name is going to be) finally shows off what they demoed two years ago.
The most advanced products have improved audio, with expressive talking. System wide dictation improvements with punctuation , capitalization, etc. Nice to hear it is also extended to CarPlay.
iOS – swipe down for island. macOS – added into Spotlight or Control click of context windows. iPadOS or dedicated Siri app across platforms will keep history privately synced via iCloud. Allowing you to switch between devices for conversations. WatchOS will also support, and visionOS takes advantage of visualization and you put it anywhere in your space – look at it to talk talking (very cool!).
Visual Intelligence – tap shutter button to see what you see and you can then ask it questions, etc. Including nutritional info, and tab splitting. On Mac OS, you use a keyboard shortcut to select something. Multi event adding to calendar – nicely demoed. (Also coming to vision OS – baed on what you are looking at.)
Write with Siri – visually anywhere you type. Easy drafts to get started. Will know how you usually communicate with specific people. Can also do writing feedback. (Auto proofreading!).

Apps

New architecture makes apps more powerful via Apple Intelligence and Siri AI.

  • Safari – simpler to manage tabs, can use AI to organize them into topics, and added new tabs to a topic as you work. Can also use it to monitor a sit and let you know when (notify me) things are available, I.e. out of stock items.
    • More personal, describe an extension will help build new safari extensions for you!
    • Passwords app – uses AI to auto update accounts which are vulnerable.
  • Messages – use AI to understand context of conversation to add one tap suggestions, like find photos, reminders, etc.
  • Email – suggestions
  • Calendar – to add events to your schedule (via natural language).
    • Easily to edit events.
  • Phone – all context to proactively surface information across your apps, i.e. confirmation code when talking to the airline. Looks at how you are calling, not what you are saying. And runs only on device.
  • Home -Accessory notifications, to help you understand related notifications as a single activity. Now cameras can be used to analyze and describe what happened in the video. Easier to search for what was captured.
  • Shortcuts – Describe a shortcut, will use AI to reason and assemble the steps for you.
  • Image Playground – entirely new version – with high quality images using Private Cloud Compute. Bet this is using the Gemini models. Can also improve your image modifications via touch or speech. Now integrated across the system. Developers can now use these new features via Image Playground API.
  • Photos – Three new feature clean up, extend, and (based on Vision Pro) Spatial Reframing (this is very very cool). Uses on device spatial models and image generation on Private Cloud Compute. Works on almost every picture in your library.

Big disclaimers

EU won’t get many of the features, nor in China.

Boxes

Agentic Codes

Xcode coding assistant – can auto localize your app. (Well that killed my app).
Device Hub to improved testing.

Back to Tim

Thank you Craig and all of our presenters. Personal note – highlights, sharing, and seeing what you create with them. No mentioned of new CEO, not surprised.

Getting Ready for WWDC And Building my HomeLab

It’s been an eventful year, and WWDC is right around the corner (tomorrow as I write this). every year since 2018, I’ve taken an Education-Vacation to align with WWDC. In 2018 and 2019, I was lucky enough to actually get a ticket to go in person. I’ve tried the last few years, but no luck so far.

Picture of author at the WWDC conference in 2019. It's a selfie with the glowing logo of WWDC19 over the left shoulder, and a presentation stand.

WWDC 2019 was amazing, and I had a great time meeting other developers and talking to Apple engineers. It was also the year that the first developer build bricked my MacBook Pro, and I had to schedule time to have the Apple team to re-image my machine as if the Beta came from the factory! What an experience that was.

Well this year, I will be squirreled away at home for 8-12 hours a day, going thru videos and setting up my development devices to try and take advantage of the new APIs and features that Apple will be releasing tomorrow. I hope that they will make my Apps more competitive in the App Store.

In the mean time, I have been doing a lot more work with AI models at my day job. I’ve been working for a year at Atlassian, as a Principal in their Advisory Innovation Lab. My job is to help build those solutions that our consulting teams will use with large enterprise customers. I am primarily focused on AI governance, which is a huge and important things for enterprises right now. And as such, I’ve been spending a lot of time doing some amazing things to help build out systems and processes for the safe, secure, and appropriate usage of AI tools.

This approach helped me focus on setting up my own private AI stack at home, and over the last few weeks, I’ve been building a set of processes so that I can ask questions about my code and gain insights to patterns that have been alluding me.

Today, I have finally finished a web-based chat setup using Ollama, LM Studio, Open WebUI, Qdrant, and custom python code. I can now load up a chat with any of my major projects, it automatically indexes any code changes to update the RAG model I have setup, and I use qwen2.5-coder-32b-instruct with a large context window.

The one challenge I am still working on is how I can take advantage of a great set of Swift skills developed by Paul Hudson over at hackingwithswift.com. Once I can figure that part, I am hoping to add Apple’s latest Swift developer documentation to my RAG in order to ensure that I am always using the latest APIs in my discussions.

Like my good friend Andy Piper, I don’t like the idea of my code or data being consumed by large enterprises for their advantage. If they want to pay for, I am sure we can come up with a financial arrangement, but until then, it is my code and my ideas.

New App Idea – Work Active

Not sure if I will keep the name, but I am really excited about a new app idea. It will be my second native watchOS app! My goal is to help people remember to do 5 minutes of simple walking an hour, during their work day. It can be as simple as slow marching in place, but the idea is movement.

I was listening to a NPR story this morning, about a study that they did with a university that showed the health impacts of sitting for 8-9 hours a day was very bad. And while I have a standing desk, that actually is not as healthy as you’d think. Standing for 2 hours a day can lead to blood clots and other health issues.

The most impactful thing you can do, is take a five minute, slow walk, once an hour. Even sitting, if you shuffle your feet, you can have improvements.

So I am noodling what I can do to make this a simple, one time purchase app.

Now I just need a great UI!

Slow but Steady Growth for Vinyl Trackr!

A few weeks back I released an app I’ve been working on for over a year. It is a privacy first Vinyl Trackr, meaning all the data is yours, and it is only stored in your iCloud for syncing across devices. If you don’t have iCloud setup, it will only exist on your device (thanks to the magic of SwiftData!).

So far I’ve only gotten one review, and it made me sad, since the person had bought the app to “Share” with his wife. He installed and then wondered why she couldn’t see the data on her device. I am guessing it was not clear what I meant by private.

While this is disappointing, I am happy to report that a few more people have bought the app, and I am hoping that my next update will expand the base even more. (As an indie developer, any reviews have a huge impact to the reach of my apps).

I spent a lot of time trying to make sure that the app scratched my personal itch, and watching where the purchases are coming in form. The data shows that more interesting is coming from France and Germany, so those will be the first two languages I expand the app to.

I released an app a year ago that does internationalization, so I am hoping that it will be good enough to help me gain additional traction in those markets.

Two Weeks Out, and Excitement is Building

It’s two weeks until all the detailed videos for developers will be made available during Apple’s Virtual WWDC week, and I’ve already setup my plan for the week.

I’ve signed up for a few of the group Q&A sessions on the topics that interest me the most, and I’ve been trying to clean up my code base so that I can start up my summer work to add new features based on what Apple releases.

My hope is that Apple spends this summer cleaning up their features and code base, adds more capabilities to SwiftData, and shows that they are still interested in VisionOS. That’s about it for me.

The myth of unlimited storage

I’ve been blogging and podcasting for over 20 years now, and my hosting provider has always said unlimited storage. Well this week, I was notified that that really means only 20GB of storage. As you can imagine in the 20 years I may have used more than that (actually had 60GB of content, backups, and edits stored on the site).

So I’ve been going through my blog and with the help of ffmpeg compressing so much of my site. I also took my edits offline (and moved them over to my 2TB cloud storage provider.

So now after a weekend of unplanned maintenance I will be under 15GB of storage on the site. Not bad!

My AirPods Review

History of AirPods

I’ve been using AirPods since they were first released, and I can say I love them! I have owned almost every version of them. The only ones that I have skipped so far are the AirPods 4.

When the first pair were released in Dec. of 2016, I immediately ordered a pair. Before that, I had spent 100s of dollars on various wired and bluetooth headsets from companies like Bose, Sennheiser, and Jabra.

I had been worried that the initial AirPods would fall out of my ears, like the EarPods did. But by removing the wire, Apple had fixed that issue. And I’ve never looked back.

AirPods Max

I also picked up the AirPods Max a few years back, they were the perfect replacement for my Bose Quiet Comfort over the ears headphones. But this isn’t about them.

Are the AirPods Pro 3 worth it

I could easily write a one word review – YES! But, let’s break it down.

Fit – I’ve bought multiple third-party foam based ear tips to improve the fit of the AirPods Pro and AirPods Pro 2, but ultimately I always went back to the default tips. The default tips, tend to wear out on my ears, and none of them really fit perfectly. After a few weeks, the fit test would always show one ear having problems with fit. Part of this is because I have non-standard ear canals, i.e. one is larger than the other. The medium tips fit well in the right, and the small fits in the left. Also, we all have oils that build up in the ear which will break down the rubber tips. Right now, after almost non-stop usage, they are still fitting perfectly with the same tip in each ear.

Noise Cancelling – You can’t beat the sound isolation of a good set of over the ear headsets like the AirPods Max or Bose Quiet Comfort line, BUT the AirPods Pro 3, with the good fit from above, have finally fixed that for me. The improved sound quality and fit get’s rid of bubble head. If you’ve used over the ear headset, you know what I mean. That feeling that the world is somewhat too isolated and you are floating around in a sound proof bubble. I also have noticed that over the last few days when I am in Adaptive mode, I can turn the sound down a lot lower than before and still have excellent sound levels, as well as noise cancelling. It’s perfect for listening while I am out taking a walk.

Battery Life – while you can’t really test battery life when you get new tech, my test on Saturday was 8-10 hours with no problem in transparency (or as I like to call it – hearing aid mode). That’s an amazing feat. Over the years, I’ve been around way to much loud noise, but I can tell you the exact point in my life when my hearing started messing up. I was at Prince’s Musicology Tour on April 23rd, 2004. What an incredible show! There was someone behind me to the left, who screamed at the top of their lungs the entire show. For two weeks after that show, I had ringing in that ear. Since that show I have tenitus and drop outs in my audio spectrum. So, what does this have to do with battery life? My AirPods Pro 2 have about 4-5 hours of transparency mode, and so getting thru an entire day with AirPods Pro 3 in transparency mode, is a game changer.

Verdict

I may have to but two more pair to replace my two pair of AirPods Pro 2 that I keep with me when traveling. Currently, I am rotating thru all three pair across the days.

PhotoDome for visionOS

Over the last few months I’ve had the joy of helping beta test an app that was released last week – PhotoDome for Apple’s Vision Pro.

The developer of this app, and I are both members of a slack community focused on developing for the Apple ecosystem. He had posted a question to see if any one with the Apple Vision Pro (AVP) would be willing to test early builds. As an early adopter, obviously by the fact that I have the AVP, I immediately signed up.

Recently on my podcast Games At Work dot Biz, we talked about another application that I was a member of the TestFlight – Cassette. As I mentioned in that episode, I don’t tend to talk about apps that I am testing, unless the developer specifically asks for public feedback. I find that providing feedback to the developer is valuable, but publishing feedback about unfinished code is not.

Now that it is available, I can say that I really enjoy PhotoDome. Similar to the value of Cassette, it provides me with a way of find old memories that I haven’t seen in a long time.

It also provides a truly immersive experience with lots and lots of options for how you may wish to view the content. You are able to pick what content you want to see.

The developer did a great job of creating a sky box so that you can focus on your photos, or you can just be out in the void. In my view above, I picked a nice grid of various Christmas cards from over the years.

I think that people who don’t have a AVP may not understand the interface, since it is hard to describe, but PhotoDome allows you to get a random set of image, from various collections, and present them in multiple different playful ways.

You can have a simple grid (as you see above), moving lanes, random, and one of my favorite – bubbles. The bubbles bounce of each other, etc.

I am hoping for fully spatial video support for a total immersive overload!

I highly recommend, if you have a AVP, that you check it out. We are in the early days of creative new experiences in AR/VR, and supporting the developers who are willing to try this out is a way that we get more and more apps and experiences!. If you have picked it up, I’d love to hear about your experience with it.