loading...

GraphQL Tutorial - How to Manage Image & File Uploads & Downloads with AWS AppSync & AWS Amplify

dabit3 profile image Nader Dabit Updated on ・10 min read

How to create & query images and files using GraphQL with AWS AppSync, AWS Amplify, and Amazon S3

How to upload files and images with GraphQL

Storing and querying for files like images and videos is a common requirement for most applications, but how do you do this using GraphQL?

One option would be to Base64 encode the image and send as a string in the mutation. This comes with disadvantages like the encoded file being larger than the original binary, the operation being computationally expensive, and the added complexity around encoding and decoding properly.

Another option is to have a separate server (or API) for uploading files. This is the preferred approach and the technique we will be covering in this tutorial.

To view or try out the final example project, click here.

How it all works

You typically would need a few things to make this work:

  1. A GraphQL API
  2. A storage service or database for saving your files
  3. A database to store the GraphQL data including a reference to the location of the file

Take for example the following schema for a product in an E-commerce app:

type Product {
  id: ID!
  name: String!
  description: String
  price: Int
  image: ?
}

How could we use this image field and make it work with our app to store and reference an image? Let's take a look at how this might work with an image stored in Amazon S3.

Using Amazon S3 there are two main types of access: private and public.

Public access means anyone with the file url can view or download it at any time. In this use case, we could reference the image url as the image field in the GraphQL schema. Since the image url is public anyway, we don't care who can view the image.

Private access means that only authenticated users can view or download the file. In this use case, we would only store a reference to the image key (i.e. images/mycoolimage.png) as the image field in the GraphQL schema. Using this key, we can fetch a temporary signed url to view this image on demand from S3 whenever we would like it to be viewed by someone.

In this tutorial, you'll learn how to do both.

Creating the client

In this tutorial I will be writing the client code in React, but you can use Vue, Angular, or any other JavaScript framework because the API calls the we will be writing are not React specific.

Create a new client project, change into the directory and install the amplify and uuid dependencies:

npx create-react-app gqlimages

cd gqlimages

npm install aws-amplify @aws-amplify/ui-react uuid

Public access

The first example we will create is a GraphQL API that has public image access.

The GraphQL type that we will be working with is a Product with an image field. We want this product's image to be public so it can be shared and visible to anyone viewing the app, regardless if they are signed in or not.

The GraphQL schema we will use is this:

type Product @model {
  id: ID!
  name: String!
  description: String
  price: Int
  image: String
}

How could we implement the API for this?

For mutations

  1. Store the image in S3
  2. Send a mutation to create the Product in the GraphQL API using the image reference along with the other product data

For Queries

  1. Query the product data from the GraphQL API. Because the image url is public, we can just render the image field immediately.

Creating the services

To build this API, we need the following:

  1. S3 bucket to store the image
  2. GraphQL API to store the image reference and other data about the type
  3. Authentication service to authenticate users (only needed in order to upload files to S3)

The first thing we will want to do is create the authentication service. To do so, we'll initialize an Amplify project and add authentication.

If you have not yet installed & configured the Amplify CLI, click here to view a video walkthrough.

amplify init

amplify add auth

? Do you want to use the default authentication and security configuration? Default configuration
? How do you want users to be able to sign in when using your Cognito User Pool? Username
? What attributes are required for signing up? Email

Next, we'll create the storage service (Amazon S3):

amplify add storage

? Please select from one of the below mentioned services: Content (Images, audio, video, etc.)
? Please provide a friendly name for your resource that will be used to label this category in the project: gqls3
? Please provide bucket name: <YOUR_UNIQUE_BUCKET_NAME>
? Who should have access: Auth and guest users
? What kind of access do you want for Authenticated users?
  ❯◉ create/update
   ◉ read
   ◉ delete
? What kind of access do you want for Guest users?
 ◯ create/update
❯◉ read
 ◯ delete
? Do you want to add a Lambda Trigger for your S3 Bucket? N

Finally, we'll create the GraphQL API:

amplify add api

? Please select from one of the below mentioned services (Use arrow keys): GraphQL
? Provide API name: (gqls3)
? Choose an authorization type for the API: API key
? Do you have an annotated GraphQL schema? N
? Do you want a guided schema creation? Y
? What best describes your project: Single object with fields
? Do you want to edit the schema now? Y

When prompted, update the schema located at /amplify/backend/api/gqls3/schema.graphql with the following:

type Product @model {
  id: ID!
  name: String!
  description: String
  price: Int
  image: String
}

Next, we can deploy the API using the following:

amplify push

? Do you want to generate code for your newly created GraphQL API Yes
? Choose the code generation language target javascript
? Enter the file name pattern of graphql queries, mutations and subscriptions src/graphql/**/*.js
? Do you want to generate/update all possible GraphQL operations - queries, mutations and subscriptions Yes
? Enter maximum statement depth [increase from default if your schema is deeply nested] 2

Next, we'll configure index.js to recognize the Amplify app:

import Amplify from 'aws-amplify'
import config from './aws-exports'
Amplify.configure(config)

Now that the services have been deployed, we need to update the S3 bucket to have a public /images folder so that anything stored in the folder can be viewed by anyone.

Warning: when making an S3 folder public, you should make sure that you never store any sensitive or private information here as the folder is completely open for anyone to view it. In this case, we are simulating an E-commerce app where we have public images for our products that would go on the main site.

Open the S3 console at https://s3.console.aws.amazon.com and find the bucket that you created in the previous step.

Next, click on the Permissions tab to update the bucket policy.

Update the policy to the following. You need to update the Resource field to your bucket's resource name (i.e. the arn:aws:s3:::gqlimages6c6fev-dev needs to be replaced with the name for your bucket):

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Principal": "*",
            "Action": "s3:GetObject",
            "Resource": "arn:aws:s3:::gqlimages6c6fev-dev/public/images/*"
        }
    ]
}

Interacting with the API from a client application

Now that the backend is created, how can we interact with it to upload and read images from it?

Here is the code that we could use to not only save files to our API, but also query and render them in the UI.

There are two main functions:

  1. createProduct - uploads the product image to S3 and saves the product data to AppSync in a GraphQL mutation
  2. listProducts - queries the GraphQL API for all products
import React, { useEffect, useState } from 'react';
import { Storage, API, graphqlOperation } from 'aws-amplify'
import { v4 as uuid } from 'uuid'
import { withAuthenticator } from '@aws-amplify/ui-react'

import { createProduct as CreateProduct } from './graphql/mutations'
import { listProducts as ListProducts } from './graphql/queries'
import config from './aws-exports'

const {
  aws_user_files_s3_bucket_region: region,
  aws_user_files_s3_bucket: bucket
} = config

function App() {
  const [file, updateFile] = useState(null)
  const [productName, updateProductName] = useState('')
  const [products, updateProducts] = useState([])
  useEffect(() => {
    listProducts()
  }, [])

  // Query the API and save them to the state
  async function listProducts() {
    const products = await API.graphql(graphqlOperation(ListProducts))
    updateProducts(products.data.listProducts.items)
  }

  function handleChange(event) {
    const { target: { value, files } } = event
    const fileForUpload = files[0]
    updateProductName(fileForUpload.name.split(".")[0])
    updateFile(fileForUpload || value)
  }

  // upload the image to S3 and then save it in the GraphQL API
  async function createProduct() {
    if (file) {
      const extension = file.name.split(".")[1]
      const { type: mimeType } = file
      const key = `images/${uuid()}${productName}.${extension}`      
      const url = `https://${bucket}.s3.${region}.amazonaws.com/public/${key}`
      const inputData = { name: productName , image: url }

      try {
        await Storage.put(key, file, {
          contentType: mimeType
        })
        await API.graphql(graphqlOperation(CreateProduct, { input: inputData }))
      } catch (err) {
        console.log('error: ', err)
      }
    }
  }

  return (
    <div style={styles.container}>
      <input
        type="file"
        onChange={handleChange}
        style={{margin: '10px 0px'}}
      />
      <input
        placeholder='Product Name'
        value={productName}
        onChange={e => updateProductName(e.target.value)}
      />
      <button
        style={styles.button}
        onClick={createProduct}>Create Product</button>

      {
        products.map((p, i) => (
          <img
            style={styles.image}
            key={i}
            src={p.image}
          />
        ))
      }
    </div>
  );
}

const styles = {
  container: {
    width: 400,
    margin: '0 auto'
  },
  image: {
    width: 400
  },
  button: {
    width: 200,
    backgroundColor: '#ddd',
    cursor: 'pointer',
    height: 30,
    margin: '0px 0px 8px'
  }
}

export default withAuthenticator(App);

To launch the app, run npm start.

To see the completed project code, click here and open the src/Products.js file.

Private Access

The next example we will create is a GraphQL API with a type that has a private image field.

This image can only be accessed by someone using our app. If someone tries to fetch this image directly, they will not be able to view it.

For the image field, we'll create a GraphQL type type that holds all of the information we need in order to create and read private files from an S3 bucket, including the bucket name and region as well as the key we'd like to read from the bucket.

The GraphQL type that we will be working with is a User with an avatar field. We want this avatar image to be private so it can be only be visible to someone signed in to the app.

The GraphQL schema we will use is this:

type User @model {
  id: ID!
  username: String!
  avatar: S3Object
}

type S3Object {
  bucket: String!
  region: String!
  key: String!
}

How could we implement the API to make this work?

For mutations

  1. Store the image in S3
  2. Send a mutation to create the User in the GraphQL API using the image reference along with the other user data

For Queries

  1. Query the user data from the API (including the image reference)
  2. Get a signed URL for the image from S3 in another API call

To build this app, we need the following:

  1. Authentication service to authenticate users
  2. S3 bucket to store image
  3. GraphQL API to store the image reference and other data about the type

Building the app

If you did not build the app in the previous example, go back and build the above project (create the authentication service, GraphQL API, and S3 bucket) in order to continue.

We can now update the schema located at /amplify/backend/api/gqls3/schema.graphql and add the following types:

type User @model {
  id: ID!
  username: String!
  avatar: S3Object
}

type S3Object {
  bucket: String!
  region: String!
  key: String!
}

Next, we can deploy the changes:

amplify push

? Do you want to update code for your updated GraphQL API Yes
? Do you want to generate GraphQL statements (queries, mutations and
subscription) based on your schema types? This will overwrite your cu
rrent graphql queries, mutations and subscriptions Yes

Interacting with the API from a client application

Now that the backend is created, how can we interact with it to upload and read images from it?

Here is the code that we could use to not only save files to our API, but also query and render them in the UI.

There are three main functions:

  1. createUser - (uploads the user image to S3 and saves the user data to AppSync in a GraphQL mutation)
  2. fetchUsers - Queries the GraphQL API for all users
  3. fetchImage - Gets the signed S3 url for the image in order for us to render it and renders it in the UI.
import React, { useState, useReducer, useEffect } from 'react'
import { withAuthenticator } from 'aws-amplify-react'
import { Storage, API, graphqlOperation } from 'aws-amplify'
import { v4 as uuid } from 'uuid'
import { createUser as CreateUser } from './graphql/mutations'
import { listUsers } from './graphql/queries'
import { onCreateUser } from './graphql/subscriptions'
import config from './aws-exports'

const {
  aws_user_files_s3_bucket_region: region,
  aws_user_files_s3_bucket: bucket
} = config

const initialState = {
  users: []
}

function reducer(state, action) {
  switch(action.type) {
    case 'SET_USERS':
      return { ...state, users: action.users }
    case 'ADD_USER':
      return { ...state, users: [action.user, ...state.users] }
    default:
      return state
  }
}

function App() {
  const [file, updateFile] = useState(null)
  const [username, updateUsername] = useState('')
  const [state, dispatch] = useReducer(reducer, initialState)
  const [avatarUrl, updateAvatarUrl] = useState('')

  function handleChange(event) {
    const { target: { value, files } } = event
    const [image] = files || []
    updateFile(image || value)
  }

  async function fetchImage(key) {
    try {
      const imageData = await Storage.get(key)
      updateAvatarUrl(imageData)
    } catch(err) {
      console.log('error: ', err)
    }
  }

  async function fetchUsers() {
    try {
     let users = await API.graphql(graphqlOperation(listUsers))
     users = users.data.listUsers.items
     dispatch({ type: 'SET_USERS', users })
    } catch(err) {
      console.log('error fetching users')
    }
  }

  async function createUser() {
    if (!username) return alert('please enter a username')
    if (file && username) {
        const { name: fileName, type: mimeType } = file  
        const key = `${uuid()}${fileName}`
        const fileForUpload = {
            bucket,
            key,
            region,
        }
        const inputData = { username, avatar: fileForUpload }

        try {
          await Storage.put(key, file, {
            contentType: mimeType
          })
          await API.graphql(graphqlOperation(CreateUser, { input: inputData }))
          updateUsername('')
          console.log('successfully stored user data!')
        } catch (err) {
          console.log('error: ', err)
        }
    }
  }
  useEffect(() => {
    fetchUsers()
    const subscription = API.graphql(graphqlOperation(onCreateUser))
      .subscribe({
        next: async userData => {
          const { onCreateUser } = userData.value.data
          dispatch({ type: 'ADD_USER', user: onCreateUser })
        }
      })
    return () => subscription.unsubscribe()
  }, [])

  return (
    <div style={styles.container}>
      <input
        label="File to upload"
        type="file"
        onChange={handleChange}
        style={{margin: '10px 0px'}}
      />
      <input
        placeholder='Username'
        value={username}
        onChange={e => updateUsername(e.target.value)}
      />
      <button
        style={styles.button}
        onClick={createUser}>Save Image</button>
      {
        state.users.map((u, i) => {
          return (
            <div
              key={i}
            >
              <p
                style={styles.username}
               onClick={() => fetchImage(u.avatar.key)}>{u.username}</p>
            </div>
          )
        })
      }
      <img
        src={avatarUrl}
        style={{ width: 300 }}
      />
    </div>
  )
}

const styles = {
  container: {
    width: 300,
    margin: '0 auto'
  },
  username: {
    cursor: 'pointer',
    border: '1px solid #ddd',
    padding: '5px 25px'
  },
  button: {
    width: 200,
    backgroundColor: '#ddd',
    cursor: 'pointer',
    height: 30,
    margin: '0px 0px 8px'
  }
}

export default withAuthenticator(App)

To launch the app, run npm start.

To view or try out the final example project, click here.

My Name is Nader Dabit. I am a Developer Advocate at Amazon Web Services working with projects like AWS AppSync and AWS Amplify. I specialize in cross-platform & cloud-enabled application development.

Posted on Jun 18 '19 by:

dabit3 profile

Nader Dabit

@dabit3

Web and mobile developer specializing in cross-platform & cloud-enabled application development.

Discussion

markdown guide
 

Nader, when I set my S3 bucket to public per instruction I got a warning in AWS S3 console:

This bucket has public access
You have provided public access to this bucket. We highly recommend that you never grant any kind of public access to your S3 bucket.

Does it mean that anyone can use this S3 bucket???
Please let me know
Thanks

 

Yes, if you set a bucket of folder in a bucket public, anyone can read from that bucket. I put a warning letting the readers know about this as it is not recommended by AWS security policy, but many people ask for or want this functionality so I showed how it could be done as well.

 

Can public users of the S3 bucket just read from it or write into it as well? How to make it more secure?

Ah, no they would only be able to read from it using the instructions here. To make it secure from reads as well, see the other example I provided in this tutorial.

 
[deleted]
 
  1. So there are two parts to accessing the S3Object, one from the bucket itself and two from the actual API. Typically the best security practice is to leave all images secure and only access them using a signed URL. The example I gave with private images is typically the use case I recommend. If we use @auth rules for owner, only the user uploading them image would be able to view it but in reality we want it to be available to any user of the app. Sure, we could set queries to null and allow anyone to access the location that of the image, but either way we ideally only want users accessing the image directly from our app to be successful.

  2. We actually have equal support for Angular & Vue. We now also have an advocate like me on our team who specializes in angular but does not write as much content, he's busy traveling around giving more workshops and talks. I think we see much more articles talking about React because I am very visible and active in that community, but in reality there is pretty much feature parity between the frameworks.

  3. I don't know the answer to this. If this is a feature you'd like, I'd suggest submitting an issue in the GitHub repo and we can see about putting it on our roadmap.

 
[deleted]

1 No, the @auth rules only apply to the GraphQL API not the S3 bucket for storage. The rules you mentioned will allow anyone to read from the database, but the a user still needs to be authorized to read from the S3 bucket in some way, either signed in or not, via the Amplify SDK (sends a signed request, gets a signed url that is valid for a set period of time)

4 Yes, we support multi auth now (starting last week) from the CLI -> aws-amplify.github.io/docs/cli-too...

5 You can update the API key by changing the expiration date in the local settings and run amplify push to update -> aws-amplify.github.io/docs/cli-too...

[deleted]
  1. Yes you can combine authorization rules. See details here

  2. Private access is built in to Amplify - See docs here referencing private access

  3. Yes, the process of storing would be the same, the only difference is you would need to deal with standard streaming / buffering protocols on the client that are agnostic to Amplify.

 

Just what I needed! Using multer with MongoDB simply did not cut it for me. Especially because you can only delete images/files locally and they are not deleted in S3! Looking forward to learning more about DynamoDB and what it can do for me especially when it comes to working with dynamic images NoSQL and graphql along with serverless. Thanks for this great post Nader!

 

Thanks Maria, glad you found it useful!

 

Absolutely! Since I got you here I'll ask a question I was going to do in person tonight, but asking here will leave me more time to ask other questions. As far as dynamically deleting an image or file from the client that results in its deletion on S3, would the Amplify S3Image component do the trick since it renders an Amazon S3 object key as an image, and therefore I as I understand it, would make it possible to identify the image in question for deletion? Because the other examples regarding deleting files only show the deletion of individual, hard coded file names. But if one were to use the key approach, files would be dynamically deleted, right? Or am I getting this all wrong? Thanks in advance!

 

Hi Nader, Thanks for this great tutorial. I got everything working except that the User uploaded images are not showing up (They are in S3 as I can tell) with "403 Fobidden" error in browser console even though I signed in. Can you tell what I missed?

Products images are shown without problem.

 
 

It looks like its possible for the file upload to S3 to succeed, but the graphql Mutation to fail. How do you deal with the zombie files?

 

You should check out Object Expiration for S3. As part of the mutation, you could then remove the Object expiration, or copy the file to another "persistent" bucket.

 
 

Hi Nader,

For the Private Access part, I have 403 to get images both for :

Storage.put( this.key, file, {
level: 'public',
contentType: 'image/*'
} )

Storage.put( this.key, file, {
level: 'private',
contentType: 'image/*'
} )

How to fix it ?

 

I have found the solution here :
itnext.io/part-2-adding-authentica...

We must just store only the document key and for each access use :
await Storage.vault.get(key) as string;

 

Hi Nadar,

I need your help as I need to submit the Image format in a byte from Graphql schema and need to submit this image in Dynamo DB.

type S3Object {
bucket: String!
key: String!
region: String!
}

input S3ObjectInput {
bucket: String!
region: String!
localUri: String
mimeType: String
}

can you please tell me the mutation for this as how it works ?

 

Hi Nader,

I followed the tutorial and able to upload a video file to the s3 bucket using the private access. However when i tried doing the same thing next day, i am getting credentials Error. Any idea what might have happened.

AWSS3Provider - error uploading CredentialsError: Missing credentials in config

 

Much needed tutorial. Thanks a lot Nader.

 

Should it not be:

type S3Object @model {
...
}

and

avatar: S3Object @connection

?

 

Thanks for the tutorial.

It always blows my mind how you're supposed to figure this out from barebones in the AWS and Amplify docs...