{"id":3036,"date":"2023-06-06T21:11:55","date_gmt":"2023-06-07T01:11:55","guid":{"rendered":"https:\/\/michaelrowe01.com\/?p=3036"},"modified":"2023-06-08T09:11:47","modified_gmt":"2023-06-08T13:11:47","slug":"whats-new-in-visionkit","status":"publish","type":"post","link":"https:\/\/michaelrowe01.com\/index.php\/blog\/whats-new-in-visionkit\/","title":{"rendered":"What\u2019s new in VisionKit"},"content":{"rendered":"\n<figure class=\"wp-block-image size-large\"><img data-recalc-dims=\"1\" loading=\"lazy\" decoding=\"async\" width=\"660\" height=\"369\" data-attachment-id=\"3085\" data-permalink=\"https:\/\/michaelrowe01.com\/index.php\/blog\/whats-new-in-visionkit\/attachment\/%ef%a3%bfwwdc23-4\/\" data-orig-file=\"https:\/\/i0.wp.com\/michaelrowe01.com\/wp-content\/uploads\/2023\/06\/%EF%A3%BFWWDC23-4.png?fit=1882%2C1053&amp;ssl=1\" data-orig-size=\"1882,1053\" data-comments-opened=\"1\" data-image-meta=\"{&quot;aperture&quot;:&quot;0&quot;,&quot;credit&quot;:&quot;&quot;,&quot;camera&quot;:&quot;&quot;,&quot;caption&quot;:&quot;&quot;,&quot;created_timestamp&quot;:&quot;0&quot;,&quot;copyright&quot;:&quot;&quot;,&quot;focal_length&quot;:&quot;0&quot;,&quot;iso&quot;:&quot;0&quot;,&quot;shutter_speed&quot;:&quot;0&quot;,&quot;title&quot;:&quot;&quot;,&quot;orientation&quot;:&quot;0&quot;}\" data-image-title=\"\uf8ffWWDC23-4\" data-image-description=\"\" data-image-caption=\"\" data-large-file=\"https:\/\/i0.wp.com\/michaelrowe01.com\/wp-content\/uploads\/2023\/06\/%EF%A3%BFWWDC23-4.png?fit=660%2C369&amp;ssl=1\" src=\"https:\/\/i0.wp.com\/michaelrowe01.com\/wp-content\/uploads\/2023\/06\/%EF%A3%BFWWDC23-4.png?resize=660%2C369&#038;ssl=1\" alt=\"\" class=\"wp-image-3085\" srcset=\"https:\/\/i0.wp.com\/michaelrowe01.com\/wp-content\/uploads\/2023\/06\/%EF%A3%BFWWDC23-4.png?resize=1024%2C573&amp;ssl=1 1024w, https:\/\/i0.wp.com\/michaelrowe01.com\/wp-content\/uploads\/2023\/06\/%EF%A3%BFWWDC23-4.png?resize=300%2C168&amp;ssl=1 300w, https:\/\/i0.wp.com\/michaelrowe01.com\/wp-content\/uploads\/2023\/06\/%EF%A3%BFWWDC23-4.png?resize=768%2C430&amp;ssl=1 768w, https:\/\/i0.wp.com\/michaelrowe01.com\/wp-content\/uploads\/2023\/06\/%EF%A3%BFWWDC23-4.png?resize=1536%2C859&amp;ssl=1 1536w, https:\/\/i0.wp.com\/michaelrowe01.com\/wp-content\/uploads\/2023\/06\/%EF%A3%BFWWDC23-4.png?w=1882&amp;ssl=1 1882w, https:\/\/i0.wp.com\/michaelrowe01.com\/wp-content\/uploads\/2023\/06\/%EF%A3%BFWWDC23-4.png?w=1320&amp;ssl=1 1320w\" sizes=\"auto, (max-width: 660px) 100vw, 660px\" \/><\/figure>\n\n\n\n<p>This API is all about lifting subjects from images, i.e. pulling out your favorite pet from a picture to be used elsewhere&nbsp;<\/p>\n\n\n\n<p>Subject Lifting<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Simple long press on an image, you can share the subject.&nbsp; Now you can lift any subject and turn it into a sticker.&nbsp; This is automatically for most code from last year.\n<ul class=\"wp-block-list\">\n<li>Just set an appropriate interaction type, .automatic does a combination of al types<\/li>\n\n\n\n<li>.imageSegmentation will only do Subject lifting<\/li>\n\n\n\n<li>.automaticTextOnly &#8211; same as iOS 16 behavior &#8211; test selection and data detectors<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n\n\n\n<p>Visual Look Up<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Allows users to know about pets, nature, landmarks, art and Media.&nbsp; In iOS17 adding Food, Products and Signs &amp; Symbols\n<ul class=\"wp-block-list\">\n<li>This helps with laundry tag information!<\/li>\n\n\n\n<li>Works for 6 languages to start with.<\/li>\n<\/ul>\n<\/li>\n\n\n\n<li>Analysis is entirely on the device. Includes feature extraction.\n<ul class=\"wp-block-list\">\n<li>One you look up the object&nbsp; then it goes to server for more information.<\/li>\n\n\n\n<li>Add .visualLookup to your analyzer configuration&nbsp;<\/li>\n\n\n\n<li>Badges are displayed on subjects for lookup etc.&nbsp;<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n\n\n\n<p>Data Scanner and Live Text &#8211; allows for OCR&nbsp;<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Optical flow &#8211; can now do high speed scanning and comes for free when recognizing text. &nbsp;<\/li>\n\n\n\n<li>Currency Support\n<ul class=\"wp-block-list\">\n<li>Find and interact with monetary by setting textContentType to .currency in the initializer, just like email or phone numbers<\/li>\n\n\n\n<li>Provides both a Bounds and a transcript.&nbsp; Code example in video for totaling a receipt.<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n\n\n\n<figure class=\"wp-block-image size-large\"><img data-recalc-dims=\"1\" loading=\"lazy\" decoding=\"async\" width=\"660\" height=\"371\" data-attachment-id=\"3086\" data-permalink=\"https:\/\/michaelrowe01.com\/index.php\/blog\/whats-new-in-visionkit\/attachment\/language-support\/\" data-orig-file=\"https:\/\/i0.wp.com\/michaelrowe01.com\/wp-content\/uploads\/2023\/06\/Language-support.png?fit=1920%2C1080&amp;ssl=1\" data-orig-size=\"1920,1080\" data-comments-opened=\"1\" data-image-meta=\"{&quot;aperture&quot;:&quot;0&quot;,&quot;credit&quot;:&quot;&quot;,&quot;camera&quot;:&quot;&quot;,&quot;caption&quot;:&quot;&quot;,&quot;created_timestamp&quot;:&quot;0&quot;,&quot;copyright&quot;:&quot;&quot;,&quot;focal_length&quot;:&quot;0&quot;,&quot;iso&quot;:&quot;0&quot;,&quot;shutter_speed&quot;:&quot;0&quot;,&quot;title&quot;:&quot;&quot;,&quot;orientation&quot;:&quot;0&quot;}\" data-image-title=\"Language-support\" data-image-description=\"\" data-image-caption=\"\" data-large-file=\"https:\/\/i0.wp.com\/michaelrowe01.com\/wp-content\/uploads\/2023\/06\/Language-support.png?fit=660%2C371&amp;ssl=1\" src=\"https:\/\/i0.wp.com\/michaelrowe01.com\/wp-content\/uploads\/2023\/06\/Language-support.png?resize=660%2C371&#038;ssl=1\" alt=\"\" class=\"wp-image-3086\" srcset=\"https:\/\/i0.wp.com\/michaelrowe01.com\/wp-content\/uploads\/2023\/06\/Language-support.png?resize=1024%2C576&amp;ssl=1 1024w, https:\/\/i0.wp.com\/michaelrowe01.com\/wp-content\/uploads\/2023\/06\/Language-support.png?resize=300%2C169&amp;ssl=1 300w, https:\/\/i0.wp.com\/michaelrowe01.com\/wp-content\/uploads\/2023\/06\/Language-support.png?resize=768%2C432&amp;ssl=1 768w, https:\/\/i0.wp.com\/michaelrowe01.com\/wp-content\/uploads\/2023\/06\/Language-support.png?resize=1536%2C864&amp;ssl=1 1536w, https:\/\/i0.wp.com\/michaelrowe01.com\/wp-content\/uploads\/2023\/06\/Language-support.png?w=1920&amp;ssl=1 1920w, https:\/\/i0.wp.com\/michaelrowe01.com\/wp-content\/uploads\/2023\/06\/Language-support.png?w=1320&amp;ssl=1 1320w\" sizes=\"auto, (max-width: 660px) 100vw, 660px\" \/><\/figure>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Live Text &#8211; more regions:\n<ul class=\"wp-block-list\">\n<li>Can detect document structure, like list detection (iOS16), but now can also detect Tables &nbsp; Will even merge cells if needed.<\/li>\n\n\n\n<li>Context-aware data detectors &#8211; will pull additional information from surrounding areas, like name and address<\/li>\n\n\n\n<li>You can do .transcript last year, you can now do ranges, selected text, Attributed and identify when text changes.<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n\n\n\n<p>Expanded platform support<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>All about the Mac &#8211; Catalyst support being rolled out<\/li>\n\n\n\n<li>Catalyst &#8211;\n<ul class=\"wp-block-list\">\n<li>Simple recompile for LiveText, Subject Lifting and Visual Look Up, no QR support yet ( you can do it via Vision Framework)<\/li>\n<\/ul>\n<\/li>\n\n\n\n<li>Native Mac API\n<ul class=\"wp-block-list\">\n<li>ImageAnalyzer<\/li>\n\n\n\n<li>ImageAnalysisOverlayView &#8211; subclass of NSView, just add above image content (adding as SubView of ContentView is simpler)<\/li>\n\n\n\n<li>Basically the name as iOS (except .machine-readable is a NOOP<\/li>\n\n\n\n<li>Contents rect &#8211; needs to know the rectangle bounds that it needs.<\/li>\n\n\n\n<li>If you use NSView you can just set trackingImageView on the overlay and the system will do it for you<\/li>\n<\/ul>\n<\/li>\n\n\n\n<li>Contextual Menus (Mac)\n<ul class=\"wp-block-list\">\n<li>You can combine items in the same menu, this provides a much better right click menus based on selected item.<\/li>\n\n\n\n<li>Items are identified by tags, there is a struct that defines them.&nbsp; Code examples are in the session.<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n","protected":false},"excerpt":{"rendered":"<p>This API is all about lifting subjects from images, i.e. pulling out your favorite pet from a picture to be used elsewhere&nbsp; Subject Lifting Visual Look Up Data Scanner and Live Text &#8211; allows for OCR&nbsp; Expanded platform support<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"jetpack_post_was_ever_published":false,"hide_page_title":"","_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_publicize_message":"","jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":true,"jetpack_social_options":{"image_generator_settings":{"template":"highway","default_image_id":0,"font":"","enabled":false},"version":2}},"categories":[2,3],"tags":[107,682,680],"class_list":["post-3036","post","type-post","status-publish","format-standard","hentry","category-blog","category-personal-softwareandit","tag-day-2","tag-visionos","tag-wwdc23"],"aioseo_notices":[],"jetpack_publicize_connections":[],"jetpack_featured_media_url":"","jetpack_shortlink":"https:\/\/wp.me\/p2aMa8-MY","jetpack-related-posts":[],"jetpack_sharing_enabled":true,"_links":{"self":[{"href":"https:\/\/michaelrowe01.com\/index.php\/wp-json\/wp\/v2\/posts\/3036","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/michaelrowe01.com\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/michaelrowe01.com\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/michaelrowe01.com\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/michaelrowe01.com\/index.php\/wp-json\/wp\/v2\/comments?post=3036"}],"version-history":[{"count":3,"href":"https:\/\/michaelrowe01.com\/index.php\/wp-json\/wp\/v2\/posts\/3036\/revisions"}],"predecessor-version":[{"id":3088,"href":"https:\/\/michaelrowe01.com\/index.php\/wp-json\/wp\/v2\/posts\/3036\/revisions\/3088"}],"wp:attachment":[{"href":"https:\/\/michaelrowe01.com\/index.php\/wp-json\/wp\/v2\/media?parent=3036"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/michaelrowe01.com\/index.php\/wp-json\/wp\/v2\/categories?post=3036"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/michaelrowe01.com\/index.php\/wp-json\/wp\/v2\/tags?post=3036"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}