loading...

The WWDC 2019 Platforms State of the Union Highlights

solomon profile image Solomon Originally published at blog.solomonvictorino.com ・2 min read

Here are some things that were mentioned during the Platforms State of the Union at WWDC 2019.

macOS Catalina

  • DriverKit - kernel extensions but in userspace

Explicit User Authorizations for:

  • Keystroke recording
  • Screen capture
  • Screen recording
  • User Data
    • Document
    • Desktop
    • Downloads
    • iCloud Drive
    • Removeable media
    • Network volumes

watchOS 6

  • Independence from the iPhone
  • Audio Streaming
  • Extended Runtime
  • SwiftUI support

Privacy

Sign In with Apple

  • Real User Indicator based on account history (determine if another verification step should be performed)
  • Works on the Web and across all Apple platforms

ML

  • image saliency (user interest heat maps for auto-cropping)
  • text recognition (printed text transcription)
  • word embeddings (get synonyms of words for search purposes)
  • speech api on-device (recognize speech)
  • speech saliency (get the cadence and pitch of spoken words)

CoreML

  • Personalized, background-updated machine learning models personalized on each user's device.

CreateML

  • macOS app for building models without code with templates, and testing them with various inputs from built-in sensors on iOS and Mac.

Siri Shortcuts

  • Shortcut phrase suggestions (Add to Siri)
  • Conversational (parametrized) shortcuts
  • Shipped with iOS and iPadOS
  • Automation (smart shortcut triggers)
  • Multi-step, multi-app shortcuts

ARKit

  • Apple Pay integration (instant buy)
  • Simultaneous front and back camera usage
  • Motion capture
  • People occlusion

RealityComposer

  • WYSIWYG interface for laying out 3D AR experiences for Apple devices
  • An iOS and Mac application

RealityKit

  • An engine for 3D AR experiences
  • Multi-threaded renderer
  • Synchronizes with camera data for realistic results

Metal

  • Supported fully by the iOS Simulator on Mac
  • Metal Memory Debugger to understand resource allocation
  • Metal Indirect Compute Encoding (offloading more tasks from the CPU to the GPU)
  • Better raytracing support
  • Peer Group API (GPUs can share data with each other without clogging the slower PCI system bus)

Thanks for reading!

Posted on by:

Discussion

markdown guide
 

As a user, I think Sign In with Apple is going to be great for privacy.

 

For sure! It'll also help complete the loop for applications that can't store data with iCloud but still want to be at home on Apple's platforms.