The WWDC 2019 Platforms State of the Union Highlights

Here are some things that were mentioned during the Platforms State of the Union at WWDC 2019.

macOS Catalina

• DriverKit - kernel extensions but in userspace

Explicit User Authorizations for:

• Keystroke recording
• Screen capture
• Screen recording
• User Data
  • Document
  • Desktop
  • Downloads
  • iCloud Drive
  • Removeable media
  • Network volumes

watchOS 6

• Independence from the iPhone
• Audio Streaming
• Extended Runtime
• SwiftUI support

Privacy

Sign In with Apple

• Real User Indicator based on account history (determine if another verification step should be performed)
• Works on the Web and across all Apple platforms

ML

• image saliency (user interest heat maps for auto-cropping)
• text recognition (printed text transcription)
• word embeddings (get synonyms of words for search purposes)
• speech api on-device (recognize speech)
• speech saliency (get the cadence and pitch of spoken words)

CoreML

• Personalized, background-updated machine learning models personalized on each user's device.

CreateML

• macOS app for building models without code with templates, and testing them with various inputs from built-in sensors on iOS and Mac.

Siri Shortcuts

• Shortcut phrase suggestions (Add to Siri)
• Conversational (parametrized) shortcuts
• Shipped with iOS and iPadOS
• Automation (smart shortcut triggers)
• Multi-step, multi-app shortcuts

ARKit

• Apple Pay integration (instant buy)
• Simultaneous front and back camera usage
• Motion capture
• People occlusion

RealityComposer

• WYSIWYG interface for laying out 3D AR experiences for Apple devices
• An iOS and Mac application

RealityKit

• An engine for 3D AR experiences
• Multi-threaded renderer
• Synchronizes with camera data for realistic results

Metal

• Supported fully by the iOS Simulator on Mac
• Metal Memory Debugger to understand resource allocation
• Metal Indirect Compute Encoding (offloading more tasks from the CPU to the GPU)
• Better raytracing support
• Peer Group API (GPUs can share data with each other without clogging the slower PCI system bus)

Thanks for reading!

Comments

[Maegan Wilson]

As a user, I think Sign In with Apple is going to be great for privacy.