2019 Google Developers ML Summit Taipei

Author: Corey Lai
Date: 2019.03.29

1. Opening Google ML 產品概觀

Lecturer : 上官林傑 / Eric ShangKuan
Developer Relations Program Manager
Google

Tensorflow

Data Flow Graphs 計算框架

• 彈性直覺的
• 自動化區別分類
• 支援多線程、佇列、異步處理
• 在多CPU、GPU、TPU下進行訓練
• 運行於多種平台上

Tensorflow Lite

Design for Speed

Kaggle

Google Company of Dataset
Kaggle是一個數據建模和數據分析競賽平台。企業和研究者可在其上發布數據，統計學者和數據挖掘專家可在其上進行競賽以產生最好的模型。這一眾包模式依賴於這一事實，即有眾多策略可以用於解決幾乎所有預測建模的問題，而研究者不可能在一開始就了解什麼方法對於特定問題是最為有效的。

MLKit for Firebased

• 專門給予行動裝置功能強大並簡單使用的SDK
• 可以在Local端以及雲端進行運算
• 提供方便的APIs幫助您使用您客製化的 Tensorflow Lite 模型於App中

Actions on Google

在Google Assistant上開發第三方App

GDG

GDG 開發者社群

• GDG 計畫 Google Developer Group
• 全球925個GDG中台灣有7個GDG聚會
• GTop meetup.com/pro/gdg

2. TensorFlow 2.0 Updates

Lecturer : 佐藤一憲 / Kaz Sato
Developer Advocate, Cloud Platform 8 years
Google

Use Case

Cleaning House Lemon

洗衣工廠開發Ai自動辨識 透過tensorflow在沒有經驗的工程師底下開始學習開發，花費2~3年開發辨識衣服類型，以分類洗衣方式，沒有任何專家的協助下，透過自我學習的方式學會使用Tensorflow.
https://www.limitlessiq.com/news/post/view/id/4768/

Air Cognizer

學習天氣規則以及影響 劇烈天氣影響
for studying the effect of extrme weather, Using TensorFlow Lite to Predict Air Quality.
https://medium.com/tensorflow/air-cognizer-predicting-air-quality-with-tensorflow-lite-942466b3d02e

Ranking Tweets with TensorFlow

非常困難使用 學習曲線過高
Numpy python .. model pythonic
high level api make it simple
貢獻你的 simply api

Split Network Architecture For Timelines Model

2.0 Update

• Simplified intergrated Api
• Converter script to upgrade from 1.x
• 1.x compatible module included

Deploy anywhere Server , Edge devices

2.0 訓練開發流程

Traning -> read and preproccess data -> Tensorflow flow hub -> tf.keras -> premade Est

tf.data feature column -> model building keras -> traning eager execute -> saveing model

Flexibility for every use case

tf.keras.models.Sequential ([... adding model])

TensorrFlow Probability

• A toolbox for probabilistic modelling
• Simple Linear Regression
• Simple Linear Regression with Probability
• Fit Mormal Distributaion

Tensorboard 可視覺化學習

Tracking Metrics Model/Graph what-if model inspection, Histograms, Embedding

TenslorFlow Dataset Allow you to download use dataset (import tensorflow_datasets as tfds)
single line to donwload data set

tsdf.load(“mnist”)

Tensorflow Extends

TFX Config
Air Runtime -> Kubeflow runtime -> your own favorite orchestrator

Need to spending much time on data validation
Tensorflow model analysis

Tensorflow.js

Build and deploy ML models using JS on Browser, Node, and other platform

Tensorflow Lite

ML edge computing with Tensorflow Lite

Use Case

Speech Detection	Video Segmentation	Edge Detection
Google Keyboard	QIY	Google Photo

Focused on Speed

Incredible inference performance
CPU on MobileNet -> CPU Quantization -> Flow OpenGL -> Edge TPU 62x fixed point

Edge TPU
A google designed ASIC that let you run inference on device

Coral Accelerator
TPU Google Edge TPU

Tensorflow Lite for Micocontrollers smallest cpu tiny models on tiny computers
Live Edge TPU Demo

3. TensorFlow and Cloud TPU

Lecturer : 魏澤人 / Tzer-jen Wei
Google Developers Experts (Machine Learning)

TPU

TPU 速度快上GPU 70倍


Cloud TPU v3

• 每秒 420 萬億次浮點運算
• 128 GB HBM

Cloud TPU v2 Pod Alpha

Cloud TPU Pod

• 11.5 petaflops 每秒 11.5 千兆次浮點運算
• 4 TB HBM
• 2-D toroidal mesh network 2-D 環面網狀網路

Edge TPU (Coral)

Cloud TPU

your vm -network- > host vm(runs data feed) -pci v3 x32-> TPU board(runs traning code) = 8 TPU Core

data from cloud storage

Cloud TPU Pod (不提供)

vm or container -> computerional Graph -> cloud bigtable (互相連結)

TPU System Architecture

Complexity Matrix Multiplication	History
Native $O(n^3)$ Solvay Strassen $O(n^{2.807})$ Coppersmith-Winograd $O(n^{2.3737})$ Fastest Know $O(n^{2.37})$ TPU $O(3n-2)$

Cloud TPU Programming
矩陣運算高效能
Transferring data between Cloud TPU and Host Memory slow
Cloud TPU interconnect and the on chip hbm are faster
input traning data is streamed to an infeed quene on Cloud TPU

適用: 模型有很多矩陣運算
不適用: 運算多變時運算效能會降低

to TPU

Estimator API

estimator.train()

Convert to TPU
convert tf.estimator.E to tf.contrib.tpu.TPUEstimator

Or Using Keras
GPRC 連接 TPU_ADDRESS

Numpy Array

infeed_manager = TPUDatasetInfeedManage(dataset)
TPU Process numpy, dataset
numpy 過大 建議使用 dataset

Tensorboard TPU

協助監控問題: 效能監控、速度、HOST、IDLE、TPU、IDLE、PCI-E

4. Cloud AI: A New Wave for Developing Machine Learning Applications

Lecturer : 林書平 / Harry Lin
Customer Engineer Google Cloud

AI Journey On Google Cloud

Three Way for ML Development on GCP

your data + your model = build your own models
your data + our model = AutoML
our data + our model = call our proceptions api

Cloud TPUs Compute, Computer Engine
Tensorflow, Keras, H20, XGBoost, weka, skit , pytorch

Cloud Dataproc = Spark (Managed Hadoop)
Kubernetes = Kubeflow (Managed Kubernetes)

Introducing Cloud AutoML

New Deep Learning VM Image

• Fast Prototyping
• CPU, GPU and TPU Support
• Performance Optimized for Google Cloud
• Flexibility

Data Parallelism vs Model Parallelism

Data Parallelism : to proccess multiple mini-batches in different work simultaneously

Model Parallelism : to run different model operations on multiple devices simultaneously

Hyperparameter Tuning

「超參數」包含了管理訓練程序本身的資料。

• Automatic hyperparameter turning service
• google developer black-box search (bayesian optimisation) algorithm
• in addition to random search and gird search
• support numic, discrete and categorical param
• early stopping and resumability

訓練應用程式在訓練模型的過程中會處理三種資料類別：

• 「輸入資料」(也稱做訓練資料) 是個別記錄 (樣本) 的集合，其中包含了對機器學習問題來說很重要的特徵。系統在訓練期間會使用這種資料設定模型，以準確地預測類似資料的新樣本。不過，輸入資料中的值一律不會直接成為模型的一部分。
• 模型的「參數」是指選定的機器學習技術用來調整資料的變數。舉例來說，深層類神經網路 (DNN) 由處理節點 (神經元) 組成，每個處理節點會包含將資料傳輸到網路時對資料執行的作業。DNN 經過訓練後，每個節點會包含一個權重值，可告知您的模型其對於最終預測的影響程度。那些權重值是模型參數的範例。從許多方面來說，模型的參數「是」模型，用於區別您的特定模型和其他處理類似資料的相同類型模型。
• 您的「超參數」是管理訓練程序本身的變數。舉例來說，在設定深層類神經網路的過程中，會需要決定輸入層和輸出層之間要使用多少節點的隱藏層，以及每層要使用多少節點。這些變數與訓練資料沒有直接關聯，屬於設定變數。請注意，參數在訓練工作中會有所變更，而超參數在工作期間通常會保持不變。

Google Cloud CI/CD

Use Case

How a Japanese cucumber farmer is using deep learning and TensorFlow

日本小黃瓜農透過 Tensorflow 訓練資料分類小黃瓜大小

Tensorflow-based Cucumber Selector

https://cloud.google.com/blog/products/gcp/how-a-japanese-cucumber-farmer-is-using-deep-learning-and-tensorflo

Disney Shop Search

「Cloud AutoML 的技術讓我們能夠輕鬆建構視覺模型，以 Disney 角色、產品類別和顏色來標註我們的產品。我們將這些註解整合至我們的搜尋引擎，在 shopDisney 中提供更多的相關搜尋結果、快速查詢功能和產品建議，成功改善顧客的使用體驗。」

- Disney 消費性產品與互動媒體技術長暨資深副總裁 Mike White

Photo dataset -> AutoML Vision -> Generate Predictions

迪士尼與Google合作設計，父母無須知道卡通角色名稱而是透過文字敘述形容網站進而提供相關資料
https://www.shopdisney.com/search?query=red%20car

5. Bringing modern AI to your apps with ML Kit for Firebase

Lecturer : 李致緯 / Richard Lee
CTO / Co-founder Polydice

ML @ iCook

• Content Ranking 訂閱服務 推薦最棒食譜
  • As VIP Subscription
• Chat-Based Agent
  • Action on Google

ML in Pratice

• Traning Cost is high
• From data science to app service is messy

Traning Cost is high

• Two Type Problem
  • General :OCR / Face detection etc...
  • Domain specific :Recipes labels etc...

You Should Only Spend Time in Domain Problem

From lab to app

• After traning a model in lab ,you may
  • Set up a prediction API
    • What if devices are offline?
• Embed model in your app
  • How to update?

Google Solution ML kit for firebase

Firebase is

• Google’s Mobile
• Platform that help you
• quickly develop
• high-quality

Pre-built general purpose models

Face Recognition

• Landmark detection - POIs on face
• Contour - shape of face
• Classification - smile/eye open

Image Labeling

• detect what’s in images
  • with confidence levels
• Knowledge Graph EntityID

Landmark Detection

• Online Only
• Detect where the is
• Return Knowledge Graph
• Longitude & latitude

Compare Cloud vs On device

在裝置離線下只能辨識出類型
Device: Food / Cuisine
在裝置上線下可辨識出更多細節
Cloud: + ingredient/ salad/ seafood/ recipe/ …

Model delopyment based on TF Lite

App ML Considerations

• Should I bundle model with app?
  • Increase initial download size
• Should I update model ?
  • Download over celllar or wifi

Pro Tips: Use with other Firebase APIs

• Analytics
  • Evaluate performance
• Remote Config
  • Dynamically update setting
• Combined
  • A/B test your model

Summary - ML Kit

• Google’s solution for ML in practice
• State-of-art pre-trained models
• Model depoly done easy

Wanna use as API?
Cloud Vision API ⇔ Firebase