Photo by Alex Blăjan on Unsplash
Intro
A while back I created a couple of apps to test drive some new (to me) frameworks:
Medium
Continuing in that tradition I decided to write a similar app using AWS Amplify and React Native. Thanks to AWS Amplify I ended up with a much more secure and private version of my previous apps with virtually no additional work from me.
The app we will build allows a user to take a photo which is uploaded and stored securely and privately. The photo is then analyzed using Amplify Predictions which returns a list of labels. The labels are then stored along with the image key so that the user can view a list of all of the images and the associated labels.
Here is a little preview of what we are building:
The finished app is available here:
cbetz
/
react-native-amplify-image-labeler
React Native app that stores and labels images powered by AWS Amplify
Here is a list of frameworks and libraries we will use:
- React Native
- AWS Amplify Authentication
- AWS Amplify API (GraphQL)
- AWS Amplify DataStore
- AWS Amplify Storage
- AWS Amplify Predictions
- react-native-camera
- react-native-paper
- react-native-vector-icons
Ready? Let's get to it.
Environment and AWS Setup
First, let's get your environment and the Amplify CLI setup. Follow the instructions here. At this point your environment is setup and you've linked your AWS account to the Amplify CLI.
React Native + Amplify Project Setup
Second, we need to initialize our fullstack project including React Native and Amplify. First create a new React Native project following the instructions here. You can change RNAmplify to whatever name you'd like; I used RNAmplifyImageLabeler. Stop when you reach the bottom of the page, don't proceed to Connect API and database to the app as we are going to do something different next.
Amplify API and Datastore
Next we are going to add our API and Datastore. Under the covers these steps will create and setup both AWS App Sync and DynamoDB. We are going to use GraphQL for our API. Amplify makes this easy for us:
amplify add api
Answer the prompts as follows:
? Please select from one of the below mentioned services: GraphQL
? Provide API name: rnimagelabeler
? Choose the default authorization type for the API API key
? Enter a description for the API key: demo
? After how many days from now the API key should expire (1-365): 365
? Do you want to configure advanced settings for the GraphQL API No, I am done.
? Do you have an annotated GraphQL schema? No
? Choose a schema template: Single object with fields (e.g., “Todo” with ID, name, description)
? Do you want to edit the schema now? Yes
This should open a file, schema.graphql, in your editor. If not, you can find it in your project at amplify/backend/api/rnimagelabeler/schema.graphql
You should see a Todo model:
type Todo @model {
id: ID!
name: String!
description: String
}
Change this to:
type Image @model {
id: ID!
key: String!
labels: [String]
}
An image object will have an id, a key that corresponds to the file key, and a list of labels that will store the labels found by Amplify Predictions.
Let's push this backend to AWS:
amplify push
You'll see another series of prompts:
? Are you sure you want to continue? Yes
? Do you want to generate code for your newly created GraphQL API Yes
? Choose the code generation language target javascript
? Enter the file name pattern of graphql queries, mutations and subscriptions src/graphql/**/*.js
? Do you want to generate/update all possible GraphQL operations - queries, mutations and subscriptions Yes
? Enter maximum statement depth [increase from default if your schema is deeply nested] 2
When complete, you should see
✔ Generated GraphQL operations successfully and saved at graphql
✔ All resources are updated in the cloud
Awesome! Let's test it. Open up App.js.
Let's add a list to display some of the image info and a button to add a new image. We will build upon this to actually deal with images in a later step. Change App.js to this:
fetchImages() uses the auto-generated listImages to get a list of images from our API. addImage() is a temporary function to test adding a new image with a key value of the timestamp and labels set to a list of test labels.
Now let's run it on an iOS simulator:
npx react-native run-ios
Or, you can run on an Android emulator:
npx react-native run-android
You should see something like this:
We just successfully tested creating and fetching image objects using Amplify API and DataStore!
Amplify Authentication
Let's add authentication to our app. We want to make sure that only the owner of the images is able to view their images. We need to make sure we secure the image objects in the database as well as the images we upload (covered in the next step). We also need a way for users to register for our app, login, reset their password, etc. Lucky for us Amplify makes this ridiculously easy!
First let's add auth to our backend:
amplify add auth
And follow the prompts (you can choose a different sign in method if you'd like):
Do you want to use the default authentication and security configuration? Default configuration
How do you want users to be able to sign in? Email
Do you want to configure advanced settings? No, I am done.
Let's push our changes:
amplify push
If you're asked if you'd like to continue just hit Enter.
Now let's add to our app. This is the really cool part. Add an import to App.js:
import { withAuthenticator } from 'aws-amplify-react-native';
And change the last line of App.js...
export default withAuthenticator(App);
...and save. You should see your simulator or emulator refresh and voilà! We now have a complete auth flow with 2 lines of code. 🤯
Let's update our Image model to use Owner Based Authorization. This specifies create, read, update, and delete operations are private to the owner. Change amplify/backend/api/rnimagelabeler/schema.graphql to
type Image @model @auth(rules: [{ allow: owner }]) {
id: ID!
key: String!
labels: [String]
}
We now have to update our API to use Amazon Cognito instead of the API Key we initially setup:
amplify update api
Walk through the series of prompts:
? Please select from one of the below mentioned services: GraphQL
? Select from the options below Update auth settings
? Choose the default authorization type for the API Amazon Cognito User Pool
Use a Cognito user pool configured as a part of this project.
? Configure additional auth types? No
Push the changes we just made to the backend:
amplify push
Once that is complete go back to your simulator (or emulator), tap Sign Up, create an account, then sign in. Where did our test Image object go? Now that we're using owner based auth we don't see any as we don't own any. Before we create any more image objects let's add the ability to upload and display actual images.
Amplify Storage
We need a way to use the device's camera and upload to Storage (S3). There are many react native camera libraries out there but we are going to use react-native-camera. Let's add it now:
yarn add react-native-camera
Then update pods if you're building an iOS app:
npx pod-install
There are some additional steps around adding permissions that we need to do. Follow instructions for iOS and/or Android.
Next, add storage:
amplify add storage
And follow the prompts:
? Please select from one of the below mentioned services: Content (Images, audio, video, etc.)
? Please provide a friendly name for your resource that will be used to label this category in the project: rnamplifyimagelabelerstorage
? Please provide bucket name: rnimagelabelerdceb74205c75476884aef48a88a40517
? Who should have access: Auth users only
? What kind of access do you want for Authenticated users? create/update, read, delete
? Do you want to add a Lambda Trigger for your S3 Bucket? No
Push to the cloud:
amplify push
Before we go any further we have to change index.js to get around this issue
Error: No credentials, applicationId or region
#5918
Error: No credentials, applicationId or region with the latest aws-amplify ad aws-amplify-react-native version.
"aws-amplify": "^3.0.11",
"aws-amplify-react-native": "^4.1.2",
"aws-sdk": "^2.681.0",
Auth.signIn succeeds but throws console error and every time it showing the warning
https://github.com/aws-amplify/amplify-js/pull/5629 fix not working in the latest version.
@ashika01 Can you let me know if you have any updates or whether I need to change anything from my side
Change
Amplify.configure(config);
to
Amplify.configure({
...config,
Analytics: {
disabled: true,
},
});
Now, let's update App.js to take a picture with the device camera, upload it to S3, and save the info to the database:
We've added a few things. onShowCameraPressed() sets the newly created showCamera state, which allows us to present the camera view to the user. Inside the camera view we have a button, Label Image, that is bound to takePicture(). takePicture() sets some camera options and takes a photo. In order to get the image in a format we can upload to S3, we fetch the image file and take the blob from the response. We then grab the filename to use as the key and use Storage.put() to upload the image. We specify { level: 'private' } to ensure the image is only viewable by the owner of the image. We then add the image to the database with the S3 key and dismiss the camera view.
We need a way to display the image back to the user using the S3 key. Lucky for us Amplify has S3Image that we can pass the key and level="private" along with some basic styling. That's all we need!
Now let's test this on a real device since we want to use the camera. For iOS, run:
npx react-native run-ios --device
You will probably see an error similar to this:
error Failed to build iOS project. We ran "xcodebuild" command but it exited with error code 65. To debug build logs further, consider building your app with Xcode.app, by opening RNAmplifyImageLabeler.xcworkspace.
Since we are testing on a device we need to add a team to our project. Open up XCode (change the path to whatever you named your project):
open ios/RNAmplifyImageLabeler.xcworkspace/
Make sure your app project is selected on the left:
Then click the Signing & Capabilities tab and choose a Team:
Back to the terminal. You should be able to build and run the app now:
npx react-native run-ios --device
Go ahead and take a picture. You should see something like the following:
Sweet! We are still adding the test labels to each image. Let's add image labeling next.
Amplify Predictions
Amplify Predictions offers language translations, text to speech, audio transcriptions, and much more. We are going to use it to label objects in our images. Under the covers, this utilizes Amazon Rekognition
First, add it:
amplify add predictions
Then follow the prompts:
? Please select from one of the categories below Identify
? What would you like to identify? Identify Labels
? Provide a friendly name for your resource rnamplifyimagelabeler
? Would you like use the default configuration? Default Configuration
? Who should have access? Auth users only
Next run:
amplify push
Now we need to Configure our app by changing index.js to
import { AppRegistry } from 'react-native';
import App from './App';
import { name as appName } from './app.json';
import Amplify from 'aws-amplify';
import Predictions, { AmazonAIPredictionsProvider } from '@aws-amplify/predictions'
import config from './aws-exports';
Amplify.configure({
...config,
Analytics: {
disabled: true,
},
});
Amplify.addPluggable(new AmazonAIPredictionsProvider());
AppRegistry.registerComponent(appName, () => App);
Next update App.js to label the images:
Now after we upload the image to S3, we call Predictions.identify() with the image key, the level (private), and type: "ALL" (you can change type as you see fit). We then map the image label names and pass them into our addImage() to save them to our database. Let's fire up our app again:
Awesome! We allow our users to register for our app, save images privately to the cloud, identify labels, and display a list back to them. From here we can add all sorts of functionality: removing images, displaying labels based on confidence (we are showing all so far), searching the list of images based on labels, adding bounding boxes to the image to outline the objects that are labeled, etc, etc. If anyone ends up using this as the basis for an app, I'd love to hear about it!
Before we go, let's clean up our app a bit.
React Native Paper
Paper is "Paper is a collection of customizable and production-ready components for React Native, following Google’s Material Design guidelines."
In our case we are going to use Cards and AppBar to spruce up our app.
Run
yarn add react-native-paper
Then
yarn add react-native-vector-icons
On iOS linking isn't working (at least for me), so I followed these instructions. Add the following line to ios/Podfile:
pod 'RNVectorIcons', :path => '../node_modules/react-native-vector-icons'
Then
npx pod-install
Finally, add the following to the end of ios/<project name>/Info.plist, inside the closing </dict>:
<key>UIAppFonts</key>
<array>
<string>MaterialCommunityIcons.ttf</string>
</array>
Now let's change App.js a bit:
Let's do a fresh build and test:
npx react-native run-ios --device
Beautiful!
Let me know if you run into any issues and I'll do my best to help. I'm thinking about re-creating the above as either a live coding session or a prerecorded video. Let me know if you're interested!
Top comments (0)