DEV Community: Bradley Schofield

Introducing Appwrite Migrations: Effortless data migration from your platforms

Bradley Schofield — Thu, 31 Aug 2023 10:14:27 +0000

Moving house is a chore. You have to pack, then hire a moving company, wonder if your giant sofa fits through the front door, and panic when you can’t find your favourite t-shirt as you unpack. Moving your data from platform to platform is much the same. You have to write scripts to transfer your data, translate any data to the destination platform’s data types, have backups in case there’s a failure, and fight your platforms who’re desperate to retain you as a customer. It’s all a huge nightmare.

Not anymore.

Introducing Migrations in Appwrite, a new service that allows you to easily move your data from all sorts of platforms over to Appwrite, including Firebase, NHost, and Supabase, with more to come! You can even transfer data between Appwrite instances, including Cloud to self-hosted, and the other way round. At the end of the day, it’s your data and we believe it’s your right to move it where you like.

In case we’re not saying it loud enough, here it is in a quote:

At the end of the day, it’s your data and we believe it’s your right to move it where you like.

Start your move

You can find the new Migrations service in Settings under the Migrations tab, where you'll be given two options:

Import project data

Clicking on the Import Data button will forward you to a wizard with a range of services you can import your data from, including Firebase, Supabase, and NHost.

All you need to do is fill in your credentials for the service you want to migrate. Each provider requires different credentials and setup processes, so make sure to check the Documentation for more details on how to set up and find the credentials to migrate your data.

After this, a quick check will occur while we scout out the project, check the credentials work, and tally how many resources the project has to migrate.

Do note that Firebase will not show how many resources, since they don’t allow counts through their APIs.

After, you can select the resources you want to import, and finally, you can click on Create. This will kick off the migration and redirect you back to the migrations tab, where you can watch your migration in real-time.

That's really how easy it is to migrate your data to Appwrite!

Self-hosted to Cloud and visa-versa

We also made it extremely easy to migrate your projects between Appwrite instances. This means self-hosted to Cloud, or Cloud to self-hosted. We want you to know that you own your data, and we believe this will make you feel more at ease to entrust us with your data.

Once you click Deploy to Cloud the console will generate an API key in the background and redirect you to Appwrite Cloud. Next, you'll be prompted to select an organization and select a project or create a new project.

Following that, the process is similar to other migrations. You'll be asked to select resources to import and kick off the migration by clicking Create. You'll once again be redirected over to Migrations to watch it take place.

The process to migrate Appwrite Cloud over to self-Hosted is much the same, except you will be asked to enter your self-hosted instance's console endpoint.

Note that for both Appwrite Cloud to self-hosted migration and visa-versa, your self-hosted project must be accessible from the internet so that Appwrite Cloud can reach it.

⏭️ What’s next?

Going forward we will be adding more features and support for new services as we want to make your transition to and from Appwrite as easy as possible. This feature is still experimental, so please give us any feedback or let us know of any bugs you encounter.

You can learn more about Migrations by heading over to our docs, where we have prepared guides for every service as well as the limitations of the service.

Appwrite Loves Open Source: Why I decided to support Starship

Bradley Schofield — Mon, 23 Jan 2023 10:21:00 +0000

Open-source is at the ❤️ of everything we do at Appwrite, and we want to enable and foster the open-source community that helped us grow to nearly 28k stars on GitHub.
However, open-source projects require much effort to maintain and grow. We use open-source tools daily to build Appwrite and want to help our community.
To give back, each Appwrite engineer gets to pick an open-source project for Appwrite to sponsor for one year.

Keeping track of where you are and your current tools can be time-consuming in the terminal.

This is why Starship is such a cool project. Starship is an excellent customisable prompt for any terminal, shell, and OS. Many developers run the same command when they change into a new directory. ls. Many of us also run other commands to tell what version of PHP, Rust, or other tools we use.

Starship adds extra information to your terminal prompt to tell you what technologies are currently in the directory you are using, what version they are, and what git branch you are on. These usually are commands we typically have to sit there typing ourselves, spending precious time just getting our bearings on what we are doing.

This extra information has even saved me a couple of times, reminding me of what branch I'm currently on (usually the wrong one 😅) and reminding me to maybe not push to it.

Not only that, but Starship is infinitely customisable; want to see what song is currently playing on your Spotify in your terminal? You can write a module for that. Want to see your current memory usage? There's a module for that.

This ease of customisation and ability to install on any environment I use, whether that be windows, macOS, Linux or even Android! It makes Starship a main stay in whatever device I'm using to develop.

Because this project had given me so much in saving time and time spent tinkering, when I was asked what project I would like to sponsor as a part of our Open Source initiative, it was a no-brainer to help support this project.

Open-source Software (OSS) Is Hard

Since Appwrite is open-source, we understand the challenges that OSS projects face. If you fall in love ❤️ with an open-source project (like we have), consider checking out ways to contribute. Most OSS projects happily accept contributions, whether in the form of commits, bug 🐛 reports, advocacy, or even monetary 💰 support. If you love Starship, consider joining us as a GitHub Sponsor. Or, if you're interested in contributing to Appwrite, check out our contribution guide.

Who are we?

Appwrite is an open-source Backend-as-a-Service (BaaS), packaged as a set of Docker micro-services to give developers of any background the tools necessary to build modern apps quickly and securely.

Check out Appwrite as the backend for your next Web, Flutter, or Server-side application. Here are some handy links for more information:

How to migrate Netlify GoTrue Users To Appwrite

Bradley Schofield — Thu, 22 Sep 2022 09:18:58 +0000

With Appwrite 1.0, we are excited to announce that you can import users from different platforms to Appwrite. One of these platforms is Netlify GoTrue. In this tutorial,
we will show you how to export users from a GoTrue based platform and import them into Appwrite.

First, you might be wondering what exactly Appwrite is. Appwrite is an open-source Backend-as-a-Service (BaaS) packaged as a set of Docker micro-services to give developers of any background the tools necessary to build modern apps quickly and securely.

For this tutorial, you will need NodeJS installed on your machine, as we will be using it to export your GoTrue users and import them into Appwrite. We will also provide a script at the end of this tutorial that you can copy and paste to get started.

TL;DR - GitHub to completed project

Warning: Users with passwords that are less than six characters will not work with Appwrite; we suggest sending an email to your users to reset their password in this case.

Now then, let's move on to our first stage.

Exporting Users from GoTrue

To export users from GoTrue, you need to connect to your Postgres database and dump the auth.users table. This can be done using the node-postgres node module.

Using the node-postgres module

First, create a project folder and install the node-postgres module using npm.

npm install pg

Next, create a file called main.js and add the following code.

const { Client } = require('pg')

// Connect to your Postgres database
const client = new Client({
    user: 'postgres',
    host: 'postgres_url',
    database: 'postgres',
    password: 'yourpasword',
    port: 5432,
  })

client.connect()

// Dump the auth.users table
client.query('SELECT * FROM auth.users', (err, res) => {
    if (err) {
        console.log(err.stack)
        client.end()
    } else {
        console.log('Recieved ' + res.rowCount + ' users')
        client.end()
        return res.rows
    }
})

This code will connect to your GoTrue database and dump the auth.users table. You can then use the res.rows variable to access the users. In the next section, we will add this code to import the users into Appwrite.

Importing Users to Appwrite

Now that we have exported the users from GoTrue, we can import them into Appwrite. To do this, we will be using the Appwrite Server SDK for NodeJS.

First, we are going to install the Appwrite SDK using NPM.

npm install node-appwrite

Once done, we will extend the code we wrote in the previous section to import the users into Appwrite.

Add the following at the top of the file, making sure to update the variables to match your Appwrite instance:

const SDK = require('node-appwrite')

const appwriteClient = new sdk.Client()

appwriteClient
    .setEndpoint('http://localhost/v1') // Your API Endpoint
    .setProject('projectKey') // Your project ID
    .setKey('Server API Key') // Your secret API key
    .setSelfSigned() // Use only on dev mode with a self-signed SSL cert
;

Then overwrite the code in the client.query callback with the following:

console.log('Received ' + res.rowCount + ' users')

for (const user of res.rows) { 
            console.log('Importing user' + user.email)

            await users.createBcryptUser(
                user.id,
                user.email,
                user.encrypted_password
            )
  }

console.log('Successfully Imported ' + res.rowCount + ' users')
client.end()
return res.rows

This code will loop through each user and create a new user in Appwrite using the users.createBcryptUser method. You can find more information about this method in the Appwrite API Reference.

With all that done, you can run the script using node main.js, and your users will be imported into Appwrite. Make sure you have a test account in GoTrue to validate that the import was successful.

Conclusions

This tutorial showed you how to export users from a GoTrue based platform and import them into Appwrite. We also provided a script that you can copy and paste to get started. If you have questions or comments, feel free to join our Discord server.

A complete version of this project can be found in the GitHub repository here.

More about Appwrite

Appwrite is an open-source Backend-as-a-Service (BaaS), packaged as a set of Docker micro-services to give developers of any background the tools necessary to build modern apps quickly and securely.

Chat with us on Discord, or, Learn more about Appwrite:
Check out Appwrite as the backend for your next web, Flutter, or server-side application. Here are some handy links for more information:
• Appwrite Contribution Guide
• Appwrite Discord
• Appwrite Github
• Appwrite Documentation

Level up Your Webhook Security With Appwrite 0.15

Bradley Schofield — Wed, 29 Jun 2022 13:28:26 +0000

Don’t Know What Appwrite Is?

Appwrite is an Open-Source BaaS, which sounds complex but is actually simple. We abstract most of the tedious, repetitive tasks away from you, such as Authentication, Databases, Storage and more. So instead of reinventing the wheel, you can concentrate on building your project.

Why Should I Validate Webhooks?

Webhooks send a lot of sensitive data to your backend. It takes just one person to find your webhook URL to send invalid and insecure data to your backend; this could result in someone telling your backend that someone created a session for example (when they didn’t) and could give them access to other user’s accounts. Another thing they could do is say that database entries were edited, which could poison your local copy on the backend allowing for fraudilent data. This is just the tip of the iceberg of what could happen if you don’t validate your webhooks.

How Do I Validate Webhooks, Then?

Validating webhooks couldn't be easier! You need to run the webhook payload through the same algorithm used to generate the signature in the header and then compare the two. If they match, the webhook is confirmed to come from your Appwrite instance.

In order to begin enabling validation your going to need to grab your webhook’s signature key, this can be found in the dashboard in your webhook’s properties page. For those of you wondering what a signature key is, it’s the key that is used to hash the payload and generate the ‘x-appwrite-webhook-signature’ header. Please note that you should NEVER share or release this key as it allows a third party to forge the payload signature. You should store this in a secure place on your backend, either in a .env file or a secrets manager. If this key ever gets leaked you can always change the key within your webhook’s settings.

To generate the hash to compare, you'll need to run the payload appended to the URL through an HMAC SHA1 Function using the signature key, then convert the result into Base64. After that you compare it to the ‘x-appwrite-webhook-signature’ header. Implementing this will be different depending on the language your backend is written.
For example, nodeJS will look like so:

const crypto = require('crypto');
let token = crypto.createHmac("sha1", process.env.WEBHOOK_SIG_KEY)
    .update(`https://yourwebhookurl/test${JSON.stringify(payload.body)}`) // Make sure there isn't a space between the URL and body.
    .digest().toString('base64');

if (token !== payload.headers['x-appwrite-webhook-signature']) {
    throw new Error('Failed authentication check.')
}

and in PHP, it may look like this:

<?php

$token = base64_encode(hash_hmac('sha1', 'https://yourwebhookurl/test' . $payload.body, getenv('WEBHOOK_SIG_KEY'), true));

if ($token != $payload.headers['x-appwrite-webhook-signature']) {
    throw new Error('Failed authentication check.');
}

Please remember that this code may differ from what you write in your backend.

📚 Learn More

Webhook validation is only the tip of the iceberg of what Appwrite 0.15 offers! Check out the official release notes, and stay tuned for more blog posts! In upcoming weeks we will describe all features in their separate blogposts to give them well-deserved spotlights. We can't wait to see what you build with Appwrite! 😎

Everything you need to know about Appwrite Functions 0.13

Bradley Schofield — Mon, 07 Mar 2022 15:44:09 +0000

Welcome to the Appwrite 0.13 Functions feature spotlight. Everyone at Appwrite is excited to see what you developers do with this new release!

We have a whole heap of updates to storage and functions including new storage adapters and large file support enabling the ability to stream larger files through Appwrite. This article will concentrate on the update to functions, but stay tuned for separate one that will talk about Storage.

First up is the main change that made all of the other changes possible, the new functions model! This new model is a complete rewrite of how we deal with functions, and if you want to learn more about the technical side of things, I heavily recommend reading our engineering blog post.

One breaking change, however, is the way you write your functions. Instead of writing a source file and executing it directly through the command line, we now import your file into Appwrite’s internal runtime server and execute a function you export. This means that you have to export your code as a function. This change will also come in handy with any communication handshakes, as the new execution model allows you to write global code that will only run once during the so-called cold start of a function.

The way this works is different from runtime to runtime, so we recommend that you checkout the updated functions documentation, however, I will show a small example with JavaScript.

Old function model was straight-forward, writing the code and logging the data on root level:

console.log(`Hello ${process.env.APPWRITE_FUNCTION_DATA}`);

The exact same code in new functions model will now be wrapper inside a function that is exported, and will use response object to return data:

module.exports = async (req, res) => {
    res.send(`Hello ${req.payload}`);
}

As you can see, we are exporting an anonymous function as your source code’s main module export. This function will receive two objects: request and response. The types for these can be found within our functions documentation.

We then use the response.send function to return data instead of console.log, this makes it a lot clearer what exactly is being sent back.

Input is also much clearer now, instead of environment variable, you get the payload as part of request object. No more searching for environment variable name in the documentation!

The module.exports function will be executed every time your function gets an execution request, but code outside of it will be executed when the runtime initializes. This usually happens on first execution after a runtime gets put to sleep or after the build is finished.

🏄 Synchronous Functions

Yes! Functions can now be executed synchronously allowing you to easily build upon Appwrite’s functionality and introduce your own API endpoints to use along side Appwrite’s API. Simply pass the sync parameter to the create execution endpoint and your request will wait until the execution is finished. Once finished, a request will return the result of execution and time taken to finish.

For instance, if we used the same “Hello World!” example as before but this time added the “sync” parameter to the Create Execution endpoint we would receive the following as a result:

{
  “status”:”completed”,
  “response”: ”Hello World!”,
  “time”: 0.0024
}

As you can see in the response, we get the status, output of the function and the time taken to execute the request. With a little bit of code in your frontend you can easily parse this result and even receive JSON from your function. You could even receive files if you encode them into base64 and decode them on the frontend.

This is just a fraction of what’s now possible with synchronous functions, so let your imagination run wild!

🚀 Speed Improvements

Another amazing benefit of the new functions model is that execution times have been significantly reduced compared to previous versions. In benchmarks we have seen a 98% reduction in warm start execution times, dropping averages from 0.1s to 0.015s! If you would like to see a deeper look into these benchmarks, make sure to checkout our engineering blog post.

📦 Automatic Dependency Management

With this functions update, we have added a new feature to Appwrite which allows runtimes to automatically install dependencies for functions. Appwrite can now detect a package.json, composer.json, pubspec.yaml and many others to download and cache their dependencies ready for execution. This all takes place in the new build stage that just got added to functions.

The build stage starts right after you create a new deployment and will begin caching dependencies for non-compiled languages or building compiled runtimes. This new stage makes functions even faster as they don’t have to wait for dependencies on a cold start.

For some runtimes, certain steps must be taken so Appwrite can detect the dependencies. Please read up on this topic in the updated functions documentation.

👷 Compiled Runtimes

Compiled Runtimes? Yes, that’s right. Appwrite now has support for compiled runtimes such as Rust, Swift or Flutter. These new runtimes take longer to build, but are faster than their interpreted counterparts giving you even more options than ever to utilize any language you like for Appwrite. Rust and Swift do not currently support Automatic Dependency Management, however, support for it is coming at a later update.

🧑‍💻 CLI 2.0

Appwrite CLI 2.0 is here, bringing even more features than before! CLI 2.0 is now built with Node instead of PHP and does not require docker to install. This gives you more flexibility with how you use the CLI.

CLI 2.0 also introduces new features allowing you to build an entire Appwrite powered application without having to touch the console once. Packing everything into the the new appwrite.json file allows you to easily move your entire project to a new Appwrite instance with just one deploy command.

We have also introduced function templates into CLI 2.0 so you can prototype and iterate on functions at speeds unparalled to previous versions!

Creating a new Appwrite project with a function is as simple as logging into Appwrite:

appwrite login

Then creating a new project:

appwrite init project

And finally creating a new function:

appwrite init function

This will create a new function with the template of the runtime you specified ready to be modified and deployed:

appwrite deploy function

As you can see, the CLI method of deploying functions has been greatly simplified and streamlined so you can deploy functions faster than ever!

🌅 Conclusion

We hope you enjoy these new updates to functions in Appwrite and we can not wait to see what you all get up to with them. If you have any feedback, questions or suggestions do not hesitate to join our Discord where most of the Appwrite community hangs out and are happy to help with any issues you may run in to!

Take your Serverless Functions to new speeds with Appwrite 0.13

Bradley Schofield — Mon, 07 Mar 2022 11:49:25 +0000

🙋 What are Cloud Functions?

Cloud functions are a way of extending a cloud provider’s services to execute your code and add functionality that did not previously exist. Quite a few services have this functionality! Some examples include AWS Lambda, Google Cloud Functions, and Vercel Functions.

Amazon led the charge into cloud functions when they introduced AWS Lambda back in 2014, with Google following up four years later, making Google Cloud Functions public for all in 2018. All of that brings us to today, where Appwrite is introducing generation 2 of Appwrite Functions with a significantly improved execution model.

💻 Architecture Overview

So, what’s changed compared to previous versions? Well, as noted, our execution model has been wholly re-envisioned with speed in mind.

We first need to know how the original execution model worked to understand the changes.

The diagram above shows how the original function execution model worked. It would do this for every single execution. So essentially with each execution we were spinning up a new Docker container. This flow takes plenty of time and can put quite a lot of stress on the host machine.

Now compare this to 0.13’s model:

The updated model is a lot more complicated (even though this is a significantly simplified graph) and is no longer spinning up a new runtime with every execution. Not only that, but each runtime now has a web server inside of it to handle executions. Instead of using command-line executions, we now use HTTP Requests, making executions much faster. Using this method of execution does mean a couple of changes for the users. For instance, users must enter their script's filename instead of the entire command. They also now have to export their function. More details on this can be found in our functions documentation.

📦 Dependency Management

Remember having to package your dependencies with your function code manually? Well, no more! With Appwrite 0.13, we have introduced a build stage into functions that automatically install any dependencies you need. The build stage is also used to build the compiled runtimes ready for execution. Specific steps may be required for some languages, so we recommend checking our updated functions documentation.

⏳ Benchmarks

Thanks to our new execution model, functions are now over 10-times as fast as before! We have even introduced the ability to use compiled languages for the first time in Appwrite, introducing Rust and an improved Swift runtime into the mix with some awe-inspiring execution times. Why don’t we check out some solid benchmark numbers comparing 0.12 to 0.13 in execution time and scale?

Our first test is a simple “Hello World!” response from NodeJS 17.0 using the asynchronous execution model. We use the asynchronous method to compare the two versions because 0.12 does not support synchronous functions, and comparing asynchronous with synchronous would not be fair. We use k6 as the benchmarking tool that runs on our local device for all scripts. To properly benchmark executions, we prepare a proxy that can freeze requests and count how many requests hit the proxy in a specific timespan. The flow looks as follows:

Create Appwrite function tag and activate it
Spin up a proxy server with request freezing enabled
Run the k6 benchmark for 60 seconds
Unfreeze proxy server
Wait for all executions to finish

With this setup, k6 will create as many executions as possible in 60 seconds. Only the first execution will start during this time but won’t finish until the proxy server is frozen. Such freezing ensures that executions don’t eat up CPU while benchmarking how many executions can Appwrite create.

After one minute, the k6 benchmarks are completed, and we unfreeze the proxy server, thus resuming the executions queue. We let all executions finish while tracking timing data on the proxy server.

The results of this benchmark were breathtaking! Appwrite 0.12 created 20700 executions, while version 0.13 created 20824 executions. A tiny improvement of only 0.6% was expected, as we did not refactor the creation process in this version. The stunning performance can be seen when comparing how long these functions took to execute. While Appwrite 0.12 ran at the rate of 360 executions per minute, with Appwrite 0.13, the rate was shocking 5820 executions per minute!

The second test we ran was using Appwrite in a real-world scenario to see how the average execution time was improved. For this test, we prepared the same script in six different runtimes. We used an example script of converting a phone number to the country code, as covered in the Open Runtimes examples GitHub repository. The script ran the following commands:

Fetch country phone prefixes database from Appwrite Locale SDK
Validate request payload
Find a match in prefix
Return country information

For benchmarking these functions, we used the same k6 technology, but this time in a simpler flow:

Create function deployment and activate it
Run the benchmark for 60 seconds
Wait for executions to finish
Calculate average execution time

We ran these scripts in six different runtimes (languages) in both 0.12 and 0.13 versions, and the results dropped our jaws! The most surprising result was in Dart, where we managed to run this script in less than one millisecond!

On that note, let’s start with Dart. Dart is a compiled language, meaning the result of a build is a binary code. This makes the execution extremely fast, as everything is ready for our server in zeros and ones. Due to bad support for compiled languages in 0.12, the average execution time of our function was 1895ms. Our expectations were pretty high when running the same script in 0.13, but an incredible drop to 0.98ms average execution time left us speechless for sure!

Let’s continue with another commonly used language, NodeJS. The same function that took 325ms to execute in 0.12 only took 1.45ms in 0.13! This result was the most surprising for me, as I didn’t expect such great results from an interpreted language.

To follow up NodeJS, we compared Deno, which had similar results averaging around 3ms in version 0.13. The gap was slightly smaller, as this function only took 145ms in 0.12.

We continued the test with PHP, a well-known language running numerous websites on the internet. While the 0.12 average stood at 106ms, in 0.13, the average dropped to 7ms.

We continued to run the test in Python and Ruby, both with similar results. In 0.12, Python took 254ms to execute, while Ruby averaged at 358ms. Believe it or not, in version 0.13, Python only took an incredible 11ms, with Ruby a little bit faster 9.5ms.

As you can see, the execution rate has significantly improved with this release, and we look forward to seeing how developers using Appwrite will utilize these new features.

💪 Engineering Challenges

To allow for synchronous execution and prioritize speed, we decided to depart from the task-based system that most of our workers use and instead create a new component to Appwrite called the executor. The executor would handle all orchestration and execution responsibilities and remove the Docker socket from the functions worker. The executor is an HTTP Server built with Swoole and Utopia using various Appwrite libraries to interact with the database.

One of the initial challenges was creating an orchestration library that would allow us to easily switch away from Docker down the line if we wanted to. This change would enable us to use other orchestration tools like Kubernetes or Podman and allow our users to run Appwrite using their favorite orchestration tools. Currently, we have two adapters for this library, Docker CLI and Docker API; however, we plan to grow this selection as time progresses.

The next challenge was that Swoole, unfortunately, had a few bugs in their coroutine cURL hook, which we use to communicate with the Docker API version of the Orchestration library. These bugs forced us to use the Docker CLI adapter, which caused higher wait times and rate-limiting problems, which caused instability with higher loads.

We came up with a solution to introduce a queue-like system for utilizing Docker. This change would make bringing up runtimes slightly slower, but actual executions always use a cURL request, so execution times would not be negatively affected by this change.

🌅 Conclusion

With these improvements, we hope that Appwrite can deliver even more to help you build your dream application without sacrificing speed or flexibility. We all look forward to seeing what everyone builds with these new features and improvements to function execution speed.

We encourage you to join our Discord server, where you can keep up to date with the latest of Appwrite and get help if you need it from the very same people that develop Appwrite!

Bidding farewell to Appwrite's Tasks Service 👋

Bradley Schofield — Wed, 09 Feb 2022 12:42:37 +0000

Appwrite's Tasks Service was a great way to send HTTP requests to remote URLs at specific intervals based on a CRON schedule. This allowed external services to react to events that take place within Appwrite and perform actions based on the request.

We deprecated Tasks in Appwrite 0.12 because Appwrite's Cloud Functions can now serve the same purposes as Tasks and can give you even more control than Tasks ever could! With Functions, you will be able to respond to events from Appwrite directly and easily recreate the Tasks functionality!!

🤔 What is Appwrite?
Appwrite is an open source backend-as-a-service that abstracts all the complexity involved in building a modern application by providing you with a set of REST APIs for your core backend needs. Appwrite handles user authentication and authorization, databases, file storage, cloud functions, webhooks, and much more! If anything is missing, you can extend Appwrite using your favorite backend language.

The removal of the Tasks Service is a breaking change, and you may need to account for this before you migrate from 0.11 to prevent the functionality of your application from being broken. In this article, we'll see how to migrate your existing Tasks to Functions.

👷‍♂️ Recreating the Tasks functionality using Functions

We will be using NodeJS for this example because it's quick and easy to write. I'll be making sure to explain what the code is doing so you can follow along and use this base even if you don't know JavaScript!

First, you want to create a new Function within Appwrite. Navigate to the Functions Service in the Appwrite dashboard, and then click New Function. Within the following popup, make sure to set the runtime to "Node 16.0" and give it a name of your choosing.

Next, create a folder somewhere on your computer to start writing code. Don't worry, you won't be writing too much code. By creating a folder to store everything, you make it easier to package it later. Now, create a new file called main.js. This file is where we will be holding all of our code, so let's paste the following code into the file (Don't worry, I'll break it down and explain it in a moment 🙂)

// Configure this section
const reqURL =  new URL(process.env["REQUEST_URL"]);
const method = 'GET';
const AuthCredentials = {
    username: process.env["USERNAME"],
    password: process.env["PASSWORD"]
}
const headers = { 
// You can also add additional custom headers here.
    'User-Agent': 'Appwrite-Server',
    'X-Appwrite-Task-UID': process.env['APPWRITE_FUNCTION_ID'],
    'X-Appwrite-Task-Name': process.env['APPWRITE_FUNCTION_NAME']
};
// End of configuration

const client = reqURL.protocol == 'https' ? require('https') : require('http');

if (AuthCredentials.username && AuthCredentials.password) {
    headers['Authorization'] = 'Basic ' + Buffer.from(`${AuthCredentials.username}:${AuthCredentials.password}`).toString('base64');
}

const req = client.request({
    hostname: reqURL.hostname,
    port: reqURL.port,
    path: reqURL.pathname,
    method: method,
    headers: headers
}, (res) => {
    if (!(res.statusCode >= 200 && res.statusCode < 300)) {
        throw new Error('Request failed with status code: ' + res.statusCode);
    }
})

req.end();

At its core, it's a function that makes a simple HTTP request. It may look a little complicated because I have opted to use no dependencies, so it is effortless to package. Let me break it all down:

const reqURL =  new URL(process.env["REQUEST_URL"]);
const method = 'GET';
const AuthCredentials = {
    username: process.env["USERNAME"],
    password: process.env["PASSWORD"]
}
const headers = { // You can add custom headers here.
    'User-Agent': 'Appwrite-Server',
    'X-Appwrite-Task-UID': process.env['APPWRITE_FUNCTION_ID'],
    'X-Appwrite-Task-Name': process.env['APPWRITE_FUNCTION_NAME']
};

The section above is the one you will be interested in the most. It holds all the configuration for this function and is the equivalent to the (now deprecated)Task dialogue from the Tasks Service. The only difference is that it's in code instead of the UI.

reqURL: This is the URL that we'll be making the request to. Make sure to specify the https:// or http:// protocol at the beginning. You'll see why later!

method: The HTTP method to use when making the request.

authCredentials: These will be credentials that will be used when making the request. They are encoded using the HTTP Basic Authentication method. You can leave username and password empty, if your endpoint does not require authentication.

headers: These are the headers that will be used when making the request. A few things of note here:

User-Agent will always be 'Appwrite-Server', just like in the old Tasks service.
X-Appwrite-Task-UID is now the Function's ID
X-Appwrite-Task-Name is the Function's Name Feel free to add additional headers here if you like.

Now the rest is all the inner workings of this function. If you don't care about this, you can skip this section.

👨‍💻 Breakdown of the Code

const client = reqURL.protocol == 'https' ? require('https') : require('http');

I was referencing this when I stated that you should remember to set protocol when defining your URL. This is because NodeJS uses different modules to make an HTTP or HTTPS request. This section of code simply automates the selection of the correct module.

if (AuthCredentials.username && AuthCredentials.password) {
    headers['Authorization'] = 'Basic ' + Buffer.from(`${AuthCredentials.username}:${AuthCredentials.password}`).toString('base64');
}

This section computes the Basic Authentication header if it's required and sets it.

const req = client.request({
    hostname: reqURL.hostname,
    port: reqURL.port,
    path: reqURL.pathname,
    method: method,
    headers: headers
}, (res) => {
    if (!(res.statusCode >= 200 && res.statusCode < 300)) {
        throw new Error('Request failed with status code: ' + res.statusCode);
    }
})

This final section makes the request, using all the previously declared variables with an arrow function to check if the status code was successful and throw an error if it isn't.

📦 Packaging the function

Packaging and using the function is simple! There are multiple ways to package an Appwrite function, and I'll go through the two most common ones.

🚀 Appwrite CLI

The Appwrite CLI is a powerful tool for interacting and developing with Appwrite. We highly recommend it as it makes many tasks to do with Appwrite a breeze, including packaging an Appwrite function! You can read more about the Appwrite CLI and how to install it in our docs.

Open your terminal and run the following command on Linux or macOS:

appwrite functions createTag \
    --functionId=YourFunctionID \
    --command='node main.js' \
    --code='/path/to/folder/with/function'

or on Powershell:

appwrite functions createTag `
    --functionId=YourFunctionID `
    --command='node main.js' `
    --code='/path/to/folder/with/function'

Make sure to replace YourFunctionID with the ID of the function you created earlier and replace /path/to/folder/with/function with the path to the folder you created to contain the main.js file.

Then go back to Appwrite and activate the tag you created using the command above.

Finally, you can navigate to the Settings tab of your function and set the CRON Schedule back to the one it was. It follows the same rules as the Tasks Service's CRON Schedule. You can also configure all the environment variables like the username, password and the endpoint in here.

Once you're happy with the configuration don't forget to click the Update button at the bottom.

💻 Manual Packaging

You can also manually package the code into a tarball. Open up a terminal and navigate to the folder where you created your main.js file earlier. Then run the following command:

tar -cf code.tar.gz main.js

This command should create a code.tar.gz file within the directory where your code is.

Next, you want to navigate to your Appwrite console, return to the Functions service, and click on the function you created earlier. Within the overview tag of the function, click the Deploy Tag button, then click on Manual inside this dialogue, and set the command to:

node main.js

Finally, set the Gzipped Code (tar.gz file) field to the code.tar.gz file you created earlier, click on the Create button, and make sure you click the Activate button next to the tag you just created.

Once you're happy with the configuration don't forget to click the Update button at the bottom.

🎉 Congratulations!

You just successfully migrated your Task over to Functions! Wasn't that easy? Not only that, but you've levelled up the possibilities of your task 💪 Now you can call the URL when an Appwrite event happens, giving you even greater control of what your task can do!

You can use the following resources to learn more and get help

Become a master of the database using Appwrite's new query API

Bradley Schofield — Tue, 11 Jan 2022 13:24:57 +0000

With Appwrite 0.12, we made Appwrite harder, better, faster, stronger with many new additions to the Database! One of these recent changes is the Query API which has been completely rewritten to give you unparalleled control over your database queries using Appwrite.

🤔 What is Appwrite?

If you're wondering what Appwrite is, you're missing out! Appwrite is open source backend-as-a-service that abstracts all the complexity involved in building a modern application by providing you with a set of REST APIs for your core backend needs. Appwrite handles user authentication and authorization, file storage, real-time APIs, cloud functions, webhooks and the subject of today, databases.

👀 So what changed?

In Appwrite 0.12, the syntax for queries has been rewritten to make it easier and provide greater control to developers. In all our SDKs we have introduced a new Query class to abstract all the logic behind query creation. The new Query class enables IDE auto-completion and makes it easier to develop your project without having to constantly refer to the documentation to see what queries are available to you.

🧑‍💻 How do I use the Query class?

Using the Query class is easy! When using an API endpoint that supports queries, you pass an array of Queries instead of the previously used strings.

For example, if we had a movies collection, and wanted to search for Avatar in the movie title, we can now use:

const { Client, Database, Query } = require('appwrite');

const client = new Client(); 
const database = new Database(client); 
database.listDocuments('movies', [
  Query.equal('title', 'Avatar')
]);

As you can see here, we are using the Query class's .equal function, which checks for values that exactly match the provided one within the attribute you specify ( title in this case ).

You can also search for multiple values within these queries, equivalent to a database OR command. Let's say we'd like to search for movies with Avatar or Scott Pilgrim vs the World in the title. The code would look like this:

database.listDocuments('movies', [
  Query.equal('title', ['Avatar', 'Scott Pilgrim vs the World'])
]);

Now, what about AND conditions? They are simple. Just add them to the main array of queries.

For instance, if we were to search for movies released after 2010 but before 2020, the query would look something like this:

database.listDocuments('movies', [
  Query.greater('year', 2010)
  Query.lesser('year', 2020)
]);

There's also the greaterEqual and lesserEqual queries. These are exactly the same as the greater and lesser queires, but also check for equality. For instance, we could rewrite the above query to also include results from 2010 and 2020 as follows:

database.listDocuments('movies', [
  Query.greaterEqual('year', 2010)
  Query.lesserEqual('year', 2020)
]);

We can also use the search query to search for a substring within a string. For instance, if we wanted to search for movies with Lord of the rings in the title, we could use:

database.listDocuments('movies', [
  Query.search('title', 'Lord of the rings')
]);

When using a search query, please ensure that you have created a Fulltext index for the attribute you're querying for.

There are also notEqual queries. This is the opposite of the equal function. If you wanted to search for movies that we not called Avatar, you could use:

database.listDocuments('movies', [
  Query.notEqual('title', 'Avatar')
]);

Not only that, but you can also combine AND with OR like this:

database.listDocuments('movies', [
  Query.greater('year', 2010)
  Query.search('actors', ['Adam Sandler', 'Steve Buscemi'])
]);

This code will get all the movies after 2010 that has either Adam Sandler or Steve Buscemi.

As you can see, the possibilities are endless with Queries, and there are loads of them to use. You can look at all possible operations in our querying guide.

To use queries, you must have correctly set indexes on your attributes so Appwrite can efficiently search through your records. You can learn more about supported indexes in Appwrite in our docs.

🎬 Conclusion

As you can see, the new Query API is undoubtedly easier to use however that's not the only advantage of this new API! It also allows new OR queries and is significantly faster in benchmarks. The revamped Query API is just one of the many changes in Appwrite 0.12! Check out the full changelog here.

We'd love to hear your feedback on how we can make Appwrite better. Feel free to hop onto our Discord server or start a Github discussion to let us know how you're using Appwrite!

📚 Learn more

You can use following resources to learn more and get help regarding Appwrite and it's services

🚀 Appwrite Github
📜 Appwrite Docs
💬 Discord Community

0.12 Migration Notes

Bradley Schofield — Tue, 04 Jan 2022 16:47:04 +0000

0.12 introduces a few breaking changes that you must mitigate before upgrading from any previous version of Appwrite. Please make sure to read these notes before migration.

Breaking Changes

➤ Nested documents no longer exist as they cause massive performance degradation

The deprecation of nested documents means that you now have to store nested documents as JSON within a String attribute. An example of code that uses nested documents is below:

let actorsCollection = await database.createCollection(
    'Actors', // Collection Name
    ['*'], // Read permissions
    ['user:amadeus', 'user:salieri'], // Write permissions
    [ // Rules
      {
        "label": "Name",
        "key": "name",
        "type": "text",
        "default": "Empty Name",
        "required": true,
        "array": false
      },
      {
        "label": "Age",
        "key": "age",
        "type": "number",
        "default": 0,
        "required": true,
        "array": false
      }
    ]
);

let moviesCollection = await database.createCollection(
    'Movies', // Collection Name
    ['*'], // Read permissions
    ['user:amadeus', 'user:salieri'], // Write permissions
    [ // Rules
      {
        "label": "Name",
        "key": "name",
        "type": "text",
        "default": "Empty Name",
        "required": true,
        "array": false
      },
      {
        "label": "Release Year",
        "key": "releaseYear",
        "type": "numeric",
        "default": 1970,
        "required": true,
        "array": false
      },
      {
        "label": "Actors",
        "key": "actors",
        "type": "document",
        "default": null,
        "required": false,
        "array": true,
        "list": [actorsCollection['$id']] // Name the collections unique IDs that are allowed in the attribute
      }
    ]
);
let response = await database.createDocument(
  moviesCollection['$id'], // Parent collection unique ID
  {
    "name": "Frozen 2",
    "releaseYear": 2019,
    "actors": [
      {
        "$collection": actorsCollection['$id'], // The actors collection unique ID
        "$permissions": {"read": ["*"], "write": ['user:amadeus', 'user:salieri']}, // Set document permissions
        "name": "Idina Menzel",
        "age": 35
      },
      {
        "$collection": actorsCollection['$id'], // The actors collection unique ID
        "$permissions": {"read": ["*"], "write": ['user:amadeus', 'user:salieri']}, // Set document permissions
        "name": "Kristen Bell",
        "age": 35
      }
    ]
  },
  ['*'], // Read permissions
);

Converting that code so it works without nested documents looks like so:

let moviesCollection = await database.createCollection(
    'Movies', // Collection Name
    ['role:all'], // Read permissions, notice how it's now 'role:all' instead of '*'
    ['user:amadeus', 'user:salieri'], // Write permissions
    [ // Rules
      {
        "label": "Name",
        "key": "name",
        "type": "text",
        "default": "Empty Name",
        "required": true,
        "array": false
      },
      {
        "label": "Release Year",
        "key": "releaseYear",
        "type": "numeric",
        "default": 1970,
        "required": true,
        "array": false
      },
      {
        "label": "Actors",
        "key": "actors",
        "type": "Text",
        "default": null,
        "required": false,
        "array": true,
      }
    ]
);
let response = await database.createDocument(
  moviesCollection['$id'], // Parent collection unique ID
  {
    "name": "Frozen 2",
    "releaseYear": 2019,
    "actors": [
      JSON.stringify({
        "name": "Idina Menzel",
        "age": 35
      }),
      JSON.stringify({
        "name": "Kristen Bell",
        "age": 35
      })
    ]
  },
  ['role:all'], // Read permissions, notice how it's now 'role:all' instead of '*'
);

As you can see, the code is more straightforward now since we don't have to create an entirely different collection for that data. This change makes Appwrite overall faster and makes you a quicker developer when using Appwrite.

➤ Numeric Values are now migrated to floats

When using the migration tool, numeric values are now migrated to floats. If you create new integers, they will stay as integers. It only affects documents that have been migrated using the tool.

If you use a dynamically typed language, you won't need to worry about migrating this change.

You will need to update your values from integers to your language's relevant float type for a language like flutter, swift or java.

➤ Wildcard and Markdown rules no longer exist

The Appwrite team have removed the markdown and wildcard attribute because there isn't any reason to have a specific type of them. To migrate from these two changes, we recommend using the Text attribute. The migration tool will automatically handle this change for you.

➤ The Appwrite SDK/HTTP API now requires custom ID's for certain endpoints

Appwrite SDK's / HTTP API requires you to pass a custom ID or unique() as the first param for many of the createX() functions. We recommend double-checking the updated documentation to see which ones were updated.

We recommend finding all the instances of a create function within your code, then checking the documentation for each to see if they have unique ID support. If it does, add a new parameter at the start, which will be a string and will contain unique(). This change will then return functionality to generating a random unique ID as before 0.12.

➤ The * permission has been renamed

* has been renamed role:all, which clarifies what it means and follows other role conventions. With this change, the database migration tool will automatically migrate the data already in Appwrite. However, your code interacting with Appwrite will need to be updated to deal with this change.

We recommend changing all instances of * for Appwrite permissions to role:all.

➤ Tasks service has been deprecated and removed

The Appwrite team has removed the Tasks service because its functionality can be easily replicated and improved with functions.

➤ CreateCollection() no longer accepts rules

Instead of directly adding rules inside CreateCollection(), you now need to use createXAttribute() to add rules to the collection after the collection has been created. The X means a type of attribute, and there are a few of them: there is string, bool and integer.

➤ User status is now a boolean
The Appwrite team has now made user status a boolean to simplify determining if a user is active or not.

➤ Order parameters have been renamed for listDocuments()

orderAttributes have been renamed to orderField
orderTypes have beben renamed to orderType

➤ listDocument filters now use a different syntax

We have updated the syntax for listDocument filters to make it more powerful and easier to use. Each SDK has a Query class to help you build queries.

➤ Permission Levels act differently when migrated

In 0.12 we introduced two different permission levels document-level and collection-level. In previous versions, permissions were nested, which meant you would have to satisfy both permission levels to access a document. In 0.12, we have changed this only to require one permission level that you select to access a document.

By default, when migrating, this is set to document-level permissions.

Upgrading your SDK's

Updating your SDK varies depending on your language. For most of them, you only need to run a command.

➤ Client SDK's

Web

Run the relevant command for your package manager:

NPM:

npm update appwrite

Yarn:

yarn upgrade appwrite

Flutter

pub upgrade appwrite

Apple

Update the Appwrite entry in your Package.swift file to:

.package(url: "git@github.com:appwrite/sdk-for-apple.git", from: "0.2.0"),

Android

Gradle
Update the Appwrite entry in your build.gradle(.kts) to:

implementation("io.appwrite:sdk-for-android:0.3.0")

Maven
Update the Appwrite entry in your pom.xml file to:

    <dependency>
        <groupId>io.appwrite</groupId>
        <artifactId>sdk-for-android</artifactId>
        <version>0.3.0</version>
    </dependency>

➤ Server SDK's

NodeJS

Run the relevant command for your package manager:

NPM:

npm update node-appwrite

Yarn:

yarn upgrade node-appwrite

PHP

Run the following command:

composer update appwrite/appwrite

Dart

Run the following command:

pub upgrade dart_appwrite

Deno

Update the version after the @ in the URL of your import statement like so:

import * as SDK from "https://deno.land/x/appwrite@2.0.0/mod.ts";

Note: We highly recommend pinning your versions with Deno since your code could break when we push an update with a breaking change or for a newer version of Appwrite.

Ruby

Run the following command:

gem update appwrite

Python

Run the following command:

pip install appwrite --upgrade

Make sure to update your requirements.txt file if you use one.

Kotlin

Gradle
Update the Appwrite entry in your build.gradle(.kts) to:

implementation("io.appwrite:sdk-for-kotlin:0.2.0")

Maven
Update the Appwrite entry in your pom.xml file to:

    <dependency>
        <groupId>io.appwrite</groupId>
        <artifactId>sdk-for-android</artifactId>
        <version>0.2.0</version>
    </dependency>

Apple

Update the Appwrite entry in your Package.swift file to:

.package(url: "git@github.com:appwrite/sdk-for-apple.git", from: "0.2.0"),

Dotnet

Update the Appwrite entry in your project's .csproj file to:

<PackageReference Include="Appwrite" Version="0.4.0" />

You can also upgrade the packages from the command line:

# Package Manager
Install-Package Appwrite -Version 0.4.0

# or .NET CLI
dotnet add package Appwrite --version 0.4.0

Serving the world with Appwrite's Locale API

Bradley Schofield — Fri, 16 Jul 2021 14:42:43 +0000

Appwrite strives to be as data agnostic as possible. This doesn't just determine our stance on other programming languages but it also defines our stance on other general languages! 🌍

With Appwrite we give you the tools to succeed writing multilingual projects with ease thanks to the Locale API, so let's get started! 🎉

🇩🇪 Setting a user's Locale

The most important API of the entire Locale Service is the setLocale() API on the client. This API allows you to manually set a user's locale. The best way to use this is to have a language selector somewhere in your application and to use the language List API to get a list of languages to show, then when a user selects a language call the setLocale() API with the country code like so:

sdk.setLocale('de'); // This will change the SDK locale to german

Setting a locale will make a few changes for a user's experience with an Appwrite back-end. Most notably it will change email templates into the user's native language for email verification and password resets:

On the left you can see what happens when we use setLocale and on the right you can see when we don't use setLocale. The email was automatically sent in the user's own language when setLocale was used. setLocale doesn't just affect emails it also affects all API's within the Locale Service, which can be very useful for many things. ❤️

🇺🇸 Getting a user's locale

This is what you will be using to determine the user's country using their IP Address, pairing this with front-end libraries such as Airbnb's Polygot allows you to use your user's home language and change your front-end language to it.

This API call also returns their country code which is ISO 639-1, their continent, a flag 🇬🇧 indicating if they're apart of the EU 🇪🇺 and their currency code which can help with displaying a user's local prices. 💰

GET /v1/locale returns a JSON Response like so:

{
    "ip": "127.0.0.1",
    "countryCode": "US",
    "country": "United States",
    "continentCode": "NA",
    "continent": "North America",
    "eu": false,
    "currency": "USD"
}

🇮🇳 Getting a country list

Dealing with things that require location such as shipping is hard enough without the fact that countries have different names in different languages and users are not always able to distinguish their own countries from a list that isn't in their language. That's where the countries list API comes in! 🦸

This API will return a list of countries with thename being in their own native language, how neat! Each of these also have their code attached which you will be able to use to still determine which country the user is selecting. Using the countries list API is makes it so you have one less thing to think about when it comes to complex localization tasks. 🧠

GET /v1/locale/countries/ returns a JSON Response like so:

"sum": 194,
"countries": [
 {
     "name": "Afghanistan",
     "code": "AF"
 },
 {
     "name": "Albania",
     "code": "AL"
 },
 {
      "name": "Algeria",
      "code": "DZ"
 },
 ...
]

🇪🇺 Getting a list of EU Countries

If you are dealing with countries that are within the EU you are required to deal with data differently and comply with the EU's GDPR Law even if your company is not based in the EU. Appwrite makes this easy thanks to the fact that Appwrite stores a actively updated list of EU Countries for you to easily reference to ensure you are following the correct laws. This can also be useful for redirecting EU users to a EU Specific site.

GET /v1/locale/countries/eu returns a JSON Response like so:

{
    "sum": 27,
    "countries": [
        {
            "name": "Austria",
            "code": "AT"
        },
        {
            "name": "Belgium",
            "code": "BE"
        },
        {
            "name": "Croatia",
            "code": "HR"
        },
        ...
   ]
}

📱 Getting a list of Phone Codes

This one can be particularly useful if you are dealing with any SMS or Shipping information. This API can be leveraged to make it easy for a user to select their phone code while signing for a account or entering shipping information. It also returns the Country Code and Full name with each phone code.

GET /v1/locale/countries/phones returns a JSON Response like so:

{
    "sum": 194,
    "phones": [
        {
            "code": "+1",
            "countryCode": "CA",
            "countryName": "Canada"
        },
        {
            "code": "+1",
            "countryCode": "US",
            "countryName": "United States"
        },
        {
            "code": "+7",
            "countryCode": "RU",
            "countryName": "Russia"
        },
        ...
    ]
}

💷 Getting a list of currencies

Getting a list of currencies can be especially useful for conversion sites or any e-commerce business since it allows people to select the currency they want to use. This list is defined by ISO 4217.

GET /v1/locale/currencies returns a JSON Response like so:

{
    "sum": 117,
    "currencies": [
        {
            "symbol": "$",
            "name": "US Dollar",
            "symbolNative": "$",
            "decimalDigits": 2,
            "rounding": 0,
            "code": "USD",
            "namePlural": "US dollars"
        },
        {
            "symbol": "€",
            "name": "Euro",
            "symbolNative": "€",
            "decimalDigits": 2,
            "rounding": 0,
            "code": "EUR",
            "namePlural": "euros"
        },
        {
            "symbol": "£",
            "name": "British Pound Sterling",
            "symbolNative": "£",
            "decimalDigits": 2,
            "rounding": 0,
            "code": "GBP",
            "namePlural": "British pounds sterling"
        },
        ...
    ]
}

🌐 Getting a list of languages

The list languages API will return a list of languages with a twist. The list it returns will have a nativeName attribute to each of the languages which will give you the language name in the native tongue of that language. For instance nativeName will contain italiano for Italian in a list it could be used to display it like: italian (italiano). Pretty neat right? This API enables you to show a language select for a user in their native language making it easier to select their one. All languages also have their code which is ISO 639-1 so you can easily pass it into setLocale().

GET /v1/locale/languages returns a JSON Response like so:

{
    "sum": 185,
    "languages": [
        {
            "name": "Afar",
            "code": "aa",
            "nativeName": "Afar"
        },
        {
            "name": "Abkhazian",
            "code": "ab",
            "nativeName": "Аҧсуа"
        },
        {
            "name": "Afrikaans",
            "code": "af",
            "nativeName": "Afrikaans"
        },
        {
            "name": "Akan",
            "code": "ak",
            "nativeName": "Akana"
        },
        ...
    ]
}

👋 Conclusion

As you can see the locale API is very useful for making an international site. Giving you access to all localised lists you need to bring to your application to all users of the internet. All responses by the Locale API are localised to the local language if setLocale() has been used or the X-Appwrite-Locale header is set making this very easy to use for all HTTP Clients.

✨️ Credits

Hope you enjoyed this article! We love contributions and encourage you to take a look at our open isuses and ongoing RFCs.

If you can help translate Appwrite into other languages please reach out to us in our support issue!

If you get stuck anywhere, feel free to reach out to us on our friendly support channels run by humans.

Here are some handy links for more information:

*Taken from internet world stats for 2020

Appwrite In Production: Backups and Restores

Bradley Schofield — Sat, 10 Jul 2021 07:16:50 +0000

Backing up and restoring data is an extremely important part of running servers. It's a virtual safety net against most bad things that can happen. Made a bad config change? Restore a backup. Messed up an update? Restore a backup. Corrupted Drives? Restore a backup.

Not only that, backups can also come in handy when migrating data to other systems, for instance migrating a development server into a production environment or vice versa 🔁

To make this entire process as easy as possible we've written this simple and handy guide to explain everything you need to know about backing up and restoring your Appwrite instance.

Appwrite is broken down into multiple sections and most of Appwrite is stateless. This means that Appwrite only actually has two main things you need to backup: one is Appwrite's Database (MariaDB) and the other is the Docker volumes which store functions data and uploads. The rest - Appwrite can automatically handle and regenerate.

Please note that all these commands need to be run within the same directory as Appwrite's docker-compose.yml

With all that said, lets begin! 😊

💾 Backing up the MariaDB Database

Due to the fact that Appwrite uses a Docker image of MariaDB it is extremely easy to dump the entire database with just one command and likewise to restore the dump.

Creating a Database backup is just one command:

docker-compose exec mariadb sh -c 'exec mysqldump --all-databases --add-drop-database -u"$MYSQL_USER" -p"$MYSQL_PASSWORD"' > ./dump.sql

Going into depth this command does a couple things:

Docker-compose launches a temporary shell onto the MariaDB container to start work
It runs mysqldump on the server with two specific options --all-databases and --add-drop-database these are important since they ensure that when the backup is restored old data doesn't get overlapped with new data.
The output of mysqldump is piped into a dump.sql file. This is your backup ✅️

✍️ Restoring the MariaDB Database

Restoring the database is similarly easy and also requires just one command to do so:

docker-compose exec -T mariadb sh -c 'exec mysql -u"$MYSQL_USER" -p"$MYSQL_PASSWORD"' < dump.sql

This command is very simple once you break it down:

Docker-compose launches a temporary shell onto the MariaDB container to start work
Using the mysql command we restore the dump through a pipe

💿 Backing up your Docker Volumes

Appwrite stores various things in Docker volumes. This includes your file uploads and Cloud Function data and docker volumes makes it easy for us to coordinate data between the central Appwrite container and our various Appwrite workers. Uploads are especially important to backup since they include all your app's file uploads, these commands can take a while to run depending on how much data you have to backup.

Before running these commands is it highly recommended to shutdown your Appwrite instance to ensure you get a complete backup.

To backup the functions volume the command is:

mkdir -p backup && docker run --rm --volumes-from "$(docker-compose ps -q appwrite)" -v $PWD/backup:/backup ubuntu bash -c "cd /storage/functions && tar cvf /backup/functions.tar ."

and to backup the uploads volume the command is:

mkdir -p backup && docker run --rm --volumes-from "$(docker-compose ps -q appwrite)" -v $PWD/backup:/backup ubuntu bash -c "cd /storage/uploads && tar cvf /backup/uploads.tar ."

Both these commands do similar things and when you break them down they are pretty simple.

Start a new Docker container. This Docker container has a few special options

--rm will delete the container once it's done running. The reason we want this is because this container is only being used to package up our backup and give it to the host machine.

--volume-from This flag special as it will mount all of the volumes of the container we give it. To get the container ID we want we use a $(docker-compose ps -q appwrite) to get the ID within the command

-v This flag is being used to mount a volume onto our new container which will give us access to a backup folder we created using the mkdir command at the start

ubuntu is the image we are basing our new container on

Finally with this command created we change directories into the normal Appwrite mount point for uploads and create a tarball which will be created in the backup directory where we will be able to access it.

Once these commands are run you should find a new backup folder which containsuploads.tar and functions.tar these are your backups. Keep them safe.

📝 Restoring your Docker Volumes

Restoring your Appwrite volumes is fairly simple as well. Move the backup folder you just created to your destination machine next to the docker-compose.yml file and simply run the following commands to restore the backup.

Please note that the Appwrite instance should be shutdown while running these commands.

Restoring functions volume:

docker run --rm --volumes-from "$(docker-compose ps -q appwrite)" -v $PWD/backup:/restore ubuntu bash -c "cd /storage/functions && tar xvf /restore/functions.tar --strip 1"

Restoring uploads volume:

docker run --rm --volumes-from "$(docker-compose ps -q appwrite)" -v $PWD/backup:/restore ubuntu bash -c "cd /storage/uploads && tar xvf /restore/uploads.tar --strip 1"

This command creates new temporary Docker container like the backup command but instead extracts the tar back into the functions and uploads endpoint which will restore the backup.

⭐️ In Conclusion

To create a full Appwrite backup you will want to backup MariaDB and the two volumes specified here. Once you have done that make sure to keep it safe, the best backup will store the backup in multiple locations including locally and in multiple clouds. As with any Cloud Native Application, make sure that you backup your Appwrite instance regularly to ensure that you're never in a situation where you have to lose data due to a server failure.

This entire process can be easily done to help migrate an Appwrite installation easily by simply copying the backup files onto another server and running the restore steps.

We hope you enjoyed this article! We love contributions and encourage you to take a look at our open isuses and ongoing RFCs.

If you get stuck anywhere, feel free to reach out to us on our friendly support channels run by humans 👩‍💻

Here are some handy links for more information:

Thank you to Taylor Vick for the server image.