DEV Community: ITNEXT

Deploy on AWS using React, Flask, and SQLite.

Tallan Groberg — Sun, 30 Jul 2023 20:22:35 +0000

Deploying to AWS can be difficult to learn. I hope to make this easy.

Credits: Professor Andrew Awdeorio

What you will get out of this

You will have a very simple web app live on the internet. Click here and here to see the end result.

Possible limitations

To ensure you don't waste your time. SQLite might not be the best option if the application that you hope to make with this is very data intensive.

While this article is set up for image upload functionality to be possible. This is not covered in this tutorial. You can store references to images very easily with the functionality I use in this tutorial. Checking out my tutorial on image uploads with firebase is a great option until I release a tutorial about that for this stack.

Important:

This tutorial does not go over best practices for securing your AWS account upon account creation. If you plan on building applications that hold sensitive user data hosted through AWS and not just this tutorial, I recommend learning about how to secure your AWS account before you start building them.

Table Of Contents.

Part 1: creating the database
Part 2: creating a flask app
part 3: Creating the react app
part 4: Setting up an AWS account
part 5: Deploying to AWS

Introduction

In this tutorial we are going to walk through how to make a deployment to AWS with a simple SQLite server. How to make a flask app capable of interacting with that server to store and retrieve data. And how to make a react app hosted through flask.

The application itself will be simple but you will able host powerful web applications after completing this tutorial.

Why this is useful

Completing this tutorial is a great way introduction to hosting on aws. Even if this isn't your typical stack this will show the concepts to make your next deploy easier.

It's a perfect introduction to scripting. I use commandline scripting in almost every type of project that I do becuase it helps me save so much time in my workflows.

This can also be used to integrate python apps into something that can be shared on the web. This is important for anyone who has a great python app but doesn't have a means of share that with the world.

What you will need.

A text editor. I will be using VS code.
a command line interface. I will be using macOS with unix and zsh. I believe that the type of command line wont matter as long as you understand the corrisponding command for your operating system or look them up as needed.

python3 installed on your host machine 3.10 or greater.

A github account we will be using a repo to move all of our code base to an ubuntu instance on AWS since this is currently on the free tier.

prerequisite skills.

Basic understanding of python.
basic knowledge of flask.
Basic understanding of react.
good understanding of unix command line.
- we will use shell scripting in order to test things. if you understand bash this should be straightforward. We will also use many linux commands and I believe I have done a great job of making this as beginner friendly as possible.
basic understanding of git.
basic understanding of emacs is a plus but not necessary.

Disclaimer

While I have done many hours of due diligence on this article. Mistakes may still be in here. If you notice anything that isn't right, I am very open to constructive criticism and feedback. I'm also very willing to help anyone who is having trouble to complete this tutorial. I want to see you succeed in deploying a web-app if you are willing to give me your time to learn this and I take not wasting your time very seriously.

Part 1: Making the database.

We are going to start by making a simple project directory.

Go to the directory where you want this project in the terminal and paste this code snippet.



mkdir webapp && cd webapp

next we have to install SQLite via homebrew if on macOS.



brew install sqlite3

you can make sure you have the right version by running this line in the terminal.



sqlite3 --version

output:
-- Loading resources from /Users/tallan/.sqliterc
3.32.3 2020-06-18 14:16:19 02c34......

Next we are making a very simple database with one table so that we can test if the database works properly.

To do this, we want to make a folder to keep 2 files. A small amount of dummy data and a schema.

from the terminal inside the project directory, we make a new folder /sql



mkdir sql && cd sql

From the terminal, now inside the /sql directory, we make 2 files to host each of these.



touch data.sql && touch schema.sql

now we make a folder for stock images.

I wont provide the images but I took screen shots from an AI photo generator and added them to the following folder and named them after the [fake people(https://this-person-does-not-exist.com/en) that I'm about to add to the database.



mkdir uploads

when you run this command you should have all the same filenames.



tree uploads

output:



uploads
├── alicewilliams.png
├── bobjohnson.jpeg
├── charliebrown.jpg
├── janesmith.jpg
└── johndoe.png

Let's add the simple table to the schema.sql file.

Even though we don't have any foreign keys, this will be good to remember in case we decide to do anything else with the database after the tutorial.



PRAGMA foreign_keys=ON;

CREATE TABLE developer (
    fullname TEXT CHECK(length(fullname) <= 40),
    email TEXT CHECK(length(email) <= 40),
    picture TEXT CHECK(length(picture) <= 64),
    password TEXT CHECK(length(password) <= 256),
    created DATETIME DEFAULT CURRENT_TIMESTAMP
);

Add some simple entries to the data.sql file



INSERT INTO developer(fullname, email, picture, password)
VALUES('John Doe', 'john_doe@gmail.com', 'johndoe.png', 'password');

INSERT INTO developer(fullname, email, picture, password)
VALUES('Jane Smith', 'jane_smith@yahoo.com', 'janesmith.jpg', 'password');

INSERT INTO developer(fullname, email, picture, password)
VALUES('Bob Johnson', 'bob_johnson@outlook.com', 'bobjohnson.jpeg', 'password');

INSERT INTO developer(fullname, email, picture, password)
VALUES('Alice Williams', 'alice_williams@hotmail.com', 'alicewilliams.png', 'password');

INSERT INTO developer(fullname, email, picture, password)
VALUES('Charlie Brown', 'charlie_brown@gmail.com', 'charliebrown.jpg', 'password');

We move on to the bash scripting portion so that we can make setting up, taking down and reseting the database easy.

make sure you move your terminal to the project directory.
from inside the sql folder



tree .

*output:



├── data.sql
├── schema.sql
└── uploads
    ├── alicewilliams.png
    ├── bobjohnson.jpeg
    ├── charliebrown.jpg
    ├── janesmith.jpg
    └── johndoe.png

Move back to the projects root directory.



cd ..

Make a new folder for your bash script and the bash script file while staying in this project directory.



mkdir bin && touch bin/db

I'm going to add these in parts with an explanation of code and will post the whole file at the end.

at the very top of the db file paste this line.



#!/bin/bash

this lets the computer know that this is a bash script, so when you run the file, it will be like running many terminal commands as once.

below that line we will add commands to echo the outputs so that it's easy to spot errors in these commands.



set -Eeuo pipefail
set -x

let's add a friendly usage message for when we don't add an argument.



usage() {
  echo "Usage: $0 (create|destroy|reset|dump)"
}

if [ $# -ne 1 ]; then
  usage
  exit 1
fi

The first block is a function making it so that if we run ./bin/db in the terminal, then there is text that is output to the terminal.

The second block is saying if the number of terminal arguments used to call this script are not equal to 1 then call the usage function and exit 1. notice that the call to the script from the terminal (./bin/db) is not considered an argument.

We have to run this line to enable permissions for db to access the command line.



chmod +x bin/db

Now you can run this script with this command.



./bin/db

if you don't run chmod +x bin/db before you call this script, you will get this error.

output:
bash: ./bin/db: Permission denied

If you run it after entering chmod +x bin/db into the terminal then you will get the following output.

Now we get to the commands themselves.

To create a database, we make an else if statement that will run a block of commands based on the argument given when we call the script.

This command will be ./bin/db create and the code that will run when we call this will be.



if [ "$1" = "create" ]; then
    mkdir -p var/uploads
    sqlite3 var/App.sqlite3 < sql/schema.sql
    sqlite3 var/App.sqlite3 < sql/data.sql
    cp sql/uploads/* var/uploads/
fi

Then in the terminal run.



./bin/db create

if everything was coded correctly, and you ran chmod +x bin/db and then ./bin/db create you should have a new folder var and a file called App.sqlite3 inside that folder.

To check, run this command in the terminal



tree var

output:



var
├── App.sqlite3
└── uploads
    ├── alicewilliams.png
    ├── bobjohnson.jpeg
    ├── charliebrown.jpg
    ├── janesmith.jpg
    └── johndoe.png

We want to make a command for each functionality stated by the usage message.

To do that we will add elif's before the closing fi and after the command for create.

the rest of the elif statement will look like this.



elif [ "$1" = "destroy" ]; then
    rm -rf var/App.sqlite3 var/
elif [ "$1" = "reset" ]; then
    ./bin/db destroy
    ./bin/db create
elif [ "$1" = "dump" ]; then
    sqlite3 var/App.sqlite3 .dump

The full db file will look like this.



#!/bin/bash

set -Eeuo pipefail
set -x

usage() {
  echo "Usage: $0 (create|destroy|reset|dump)"
}

if [ $# -ne 1 ]; then
  usage
  exit 1
fi

if [ "$1" = "create" ]; then
    mkdir -p var/uploads
    sqlite3 var/App.sqlite3 < sql/schema.sql
    sqlite3 var/App.sqlite3 < sql/data.sql
    cp sql/uploads/* var/uploads/
elif [ "$1" = "destroy" ]; then
    rm -rf var
elif [ "$1" = "reset" ]; then
    ./bin/db destroy
    ./bin/db create
elif [ "$1" = "dump" ]; then
    sqlite3 var/App.sqlite3 .dump
fi

Make sure that when you run the following terminal commands that you get the same outputs before moving onto the flask portion. You will be using these commands often.

From the project directory with no database created



./bin/db create

output:



+ '[' 1 -ne 1 ']'
+ '[' create = create ']'
+ mkdir -p var
+ sqlite3 var/App.sqlite3
+ sqlite3 var/App.sqlite3



./bin/db destroy

output:



+ '[' 1 -ne 1 ']'
+ '[' destroy = create ']'
+ '[' destroy = destroy ']'
+ rm -rf var



./bin/db reset

output:



+ '[' 1 -ne 1 ']'
+ '[' reset = create ']'
+ '[' reset = destroy ']'
+ '[' reset = reset ']'
+ ./bin/db destroy
+ '[' 1 -ne 1 ']'
+ '[' destroy = create ']'
+ '[' destroy = destroy ']'
+ rm -rf var
+ ./bin/db create
+ '[' 1 -ne 1 ']'
+ '[' create = create ']'
+ mkdir -p var/uploads
+ sqlite3 var/App.sqlite3
+ sqlite3 var/App.sqlite3
+ cp sql/uploads/alicewilliams.png sql/uploads/bobjohnson.jpeg sql/uploads/charliebrown.jpg sql/uploads/janesmith.jpg sql/uploads/johndoe.png var/uploads/



./bin/db dump

output:



+ '[' 1 -ne 1 ']'
+ '[' dump = create ']'
+ '[' dump = destroy ']'
+ '[' dump = reset ']'
+ '[' dump = dump ']'
+ sqlite3 var/App.sqlite3 .dump
-- Loading resources from /Users/tallan/.sqliterc
PRAGMA foreign_keys=ON;
BEGIN TRANSACTION;
CREATE TABLE developer (
    fullname TEXT CHECK(length(fullname) <= 40),
    email TEXT CHECK(length(email) <= 40),
    picture TEXT CHECK(length(picture) <= 64),
    password TEXT CHECK(length(password) <= 256),
    created DATETIME DEFAULT CURRENT_TIMESTAMP
);
INSERT INTO developer VALUES('John Doe','john_doe@gmail.com','johndoe.png','password','2023-07-23 22:43:59');
INSERT INTO developer VALUES('Jane Smith','jane_smith@yahoo.com','janesmith.jpg','password','2023-07-23 22:43:59');
INSERT INTO developer VALUES('Bob Johnson','bob_johnson@outlook.com','bobjohnson.jpeg','password','2023-07-23 22:43:59');
INSERT INTO developer VALUES('Alice Williams','alice_williams@hotmail.com','alicewilliams.png','password','2023-07-23 22:43:59');
INSERT INTO developer VALUES('Charlie Brown','charlie_brown@gmail.com','charliebrown.jpg','password','2023-07-23 22:43:59');
COMMIT;

Part 2: Making the flask App.

Ensure part 1 works properly before continuing to part 2.

We are going to want to make a custom python environment that is specific to this directory

This will mean we need to define a requirements.txt that's used for defining all the packages.

If you mess up at any point in trying to do this section run this command in the project directory and it will remove the python env folder that we are about to create.



rm -rf env

To ensure that we have python3 on your machine, run this command.



python3 --version

output:
Python 3.11.4

You may have a different version but, as long as this is 3.10 or greater you don't have to update to succeed in this tutorial.

This also assumes that you are not using a python version from Anaconda.

Another gotcha is that when we run this command it should be a blank output.



printenv PYTHONPATH

if you do have an output when you run this, you can run this command, but you will have to do so everytime you restart the terminal.



env --unset PYTHONPATH

If all the above checks passed, we can run this command.



 python3 -m venv env

This will have created a python environment local to this directory and this command will need to be ran every time we restart the terminal regardless of whether you had an ouput on the command before the one above. If we don't then the flask App will fail to start.

To activate the environment we run this command.



source env/bin/activate

There should be something that says (env) to the left most side of our textbar where we write to the terminal.

To double check, run this command and it should output the env directory first.



 which -a python

output:



/Users/tallan/Desktop/article-writing/tutorials/flask-app/project/env/bin/python
/usr/local/bin/python
/usr/bin/python

Now we want to install jinja2 since this is what will allow us to mount the DOM to the flask App with react. It can also be very useful for making simple webpages like forms that are used in addition to react since they can be more responsive and easier to make than full react components, when the page is a simple form or needs little to no database calls, I prefer to use it.



./env/bin/pip install jinja2

We want to add additional packages to our starter files. To do this we will use a requirements.txt file.

from the terminal make a requirements.txt file



touch requirements.txt

copy and paste the following code snippet into the file.
__requirements.txt_



arrow==1.2.3
astroid==2.15.0
attrs==22.2.0
beautifulsoup4==4.11.1
bs4==0.0.1
certifi==2022.12.7
charset-normalizer==3.0.1
click==8.1.3
dill==0.3.6
exceptiongroup==1.1.0
Flask==2.2.2
html5validator==0.4.2
idna==3.4
iniconfig==2.0.0
isort==5.12.0
itsdangerous==2.1.2
Jinja2==3.1.2
lazy-object-proxy==1.9.0
MarkupSafe==2.1.2
mccabe==0.7.0
packaging==23.0
platformdirs==2.6.2
pluggy==1.0.0
pycodestyle==2.10.0
pydocstyle==6.3.0
pylint==2.16.2
pytest==7.2.1
python-dateutil==2.8.2
PyYAML==6.0
requests==2.28.2
six==1.16.0
snowballstemmer==2.2.0
soupsieve==2.3.2.post1
tomli==2.0.1
tomlkit==0.11.6
typing_extensions==4.4.0
urllib3==1.26.14
Werkzeug==2.2.2
wrapt==1.14.1

When you run the following command, this will install all the packages in the requirements.txt file to the virtual environment.

with our env activated and the txt file with all the pages above copied. Let's run the following command.



pip install -r requirements.txt

We may get some depreciation warnings, that shouldn't be an issue.

Run the env activate command again.



source env/bin/activate

We are ready to make the flask App itself.

Lowercase app is a reserved word in flask and I didn't want to call this package tutorial and so I settled for uppercase App. be mindful that there is a distinction between the 2.

We are going to house most of the App in a folder of the same name.



mkdir App

we need an __init__.py file to initialize the package.



touch App/__init__.py

__init__.py



"""App package initializer."""
import flask

# app is a single object used by all the code modules in this package
app = flask.Flask(__name__)  # pylint: disable=invalid-name

# Read settings from config module (insta485/config.py)
app.config.from_object('App.config')

# Overlay settings read from a Python file whose path is set in the environment
# variable INSTA485_SETTINGS. Setting this environment variable is optional.
# Docs: http://flask.pocoo.org/docs/latest/config/
#
# EXAMPLE:
# $ export APP_SETTINGS=secret_key_config.py
app.config.from_envvar('APP_SETTINGS', silent=True)

# Tell our App about views and model.  This is dangerously close to a
# circular import, but Flask was designed that way.
# (Reference http://flask.pocoo.org/docs/patterns/packages/)
import App.views
import App.model

Next we make a config.py file for environment variables specific to the flask app.

Inside the App directory make a file called config.py



touch App/config.py

config.py



"""App development configuration."""

import pathlib

# Root of this application, useful if it doesn't occupy an entire domain
APPLICATION_ROOT = '/'

# Secret key for encrypting cookies
SECRET_KEY = b'FIXME SET WITH: $ python3 -c "import os; print(os.urandom(24))"'
SESSION_COOKIE_NAME = 'login'

# File Upload to var/uploads/
APP_ROOT = pathlib.Path(__file__).resolve().parent.parent
UPLOAD_FOLDER = APP_ROOT/'var'/'uploads'
ALLOWED_EXTENSIONS = set(['png', 'jpg', 'jpeg', 'gif'])
MAX_CONTENT_LENGTH = 16 * 1024 * 1024

# Database file is var/App.sqlite3
DATABASE_FILENAME = APP_ROOT/'var'/'App.sqlite3'

As the TODO suggests, run the following command and replace the SECRET_KEY assignment with the following command.



python3 -c "import os; print(os.urandom(24))"

example output:



b'\x9a\xbb4\x86\x1d7\xac\x1ad\x14\xd9:\xcc\xf3\xf4\r\xf3\xd7\xd3cd\xfc$\xae'

Now we have to make a static directory to hold css and placeholder images, I wont be providing any code for this but this is folder is where you can house it for your project. If you don't use any style within your react application or the jinja templates. This is the command.



mkdir App/static

Now lets make a folder for templates inside the App directory to act as a place to put html and jinja template files.

We will also make an index.html in the same command.



mkdir App/templates && touch App/templates/index.html

We'll add simple html to use for testing if our flask app is working.



<!DOCTYPE html>
<html lang="en">
Hello world!
</html>

Now we need to make a views model. This is where you will keep python functions that are called when a user visits a url.



mkdir App/views

Next we need to make an __init__.py to access our views functions app wide and an index.py file to hold our function specific to the '/' route.



touch App/views/index.py && touch App/views/__init__.py

Inside this index.py we will make our first flask function.



"""
index (main) view.

URLs include:
/
"""
import flask
import App


@App.app.route('/')
def show_index():
    """Display / route."""
    context = {}
    return flask.render_template("index.html", **context)

Inside the views/__init__.py we add the import.



"""Views, one for each App's page."""
from App.views.index import show_index

We will add a placeholder file model.py. Even though we wont add any code to it for the moment, eventually this file will house the code for the database connection.



 touch App/model.py

We still don't have a way for flask to understand that we have an App package to use. To fix this we will create a .toml file.

in the project directory add this file.



touch pyproject.toml

Add the following text to the pyproject.toml.



[build-system]
requires = ["setuptools>=64.0.0", "wheel"]
build-backend = "setuptools.build_meta"

[project]
name = "App"
version = "0.1.0"
dependencies = [
    "arrow",
    "bs4",
    "Flask",
    "requests",
]
requires-python = ">=3.10"

[tool.setuptools]
packages = ["App"]

Double check that you are in your python virtual environment,
now we have to install App into the virtual environment.



source env/bin/activate

and then



pip install -r requirements.txt && pip install -e .

You should receive the following output.

output:
a bunch of build info..
...
Successfully built App
Installing collected packages: App
Successfully installed App-0.1.0

if everything was done correctly you should be able to see hello world by running the following command.



flask --app App --debug run --host 0.0.0.0 --port 4000

and going to localhost:4000

Exit the terminal for the flask app by pressing control + c when you are finished checking that you see hello world at the local host.

In preparation to move onto the part where we connect the database to the App, we are going to make a run script so we don't have to type a long command in everytime we want to start the server.



touch ./bin/run

And add the following code to the script.



#!/bin/bash
set -Eeuo pipefail
set -x

export FLASK_ENV=development
export FLASK_APP=App
export FLASK_DEBUG=1

usage() {
  echo "Usage: $0"
}

if [ $# -ne 0 ]; then
  usage
  exit 1
fi



# If var/App.sqlite3 does not exist, print an error and exit non-zero.
    if [ ! -f var/App.sqlite3 ]; then
        echo "Error: var/App.sqlite3 does not exist. Run ./bin/db create."
        exit 1
    fi


flask --app App --debug run --host 0.0.0.0 --port 4000

Enable the script.



chmod +x bin/run

Run the script to ensure no errors were introduced.



./bin/run

You should now be able to visit localhost:4000 the same as before.

let's start by resetting the database.



./bin/db reset

Ensure that you can interact with the database from the terminal first.



sqlite3 var/App.sqlite3 "SELECT fullname, email FROM developer;"

output:
-- Loading resources from /Users/tallan/.sqliterc
fullname email

---------- ------------------
John Doe john_doe@gmail.com
Jane Smith jane_smith@yahoo.c
Bob Johnso bob_johnson@outloo
Alice Will alice_williams@hot
Charlie Br charlie_brown@gmai

Copy and paste the following code into the blank model.py file.



"""App model (database) API."""
import sqlite3
import flask
import App


def dict_factory(cursor, row):
    """Convert database row objects to a dictionary keyed on column name.

    This is useful for building dictionaries which are then used to render a
    template.  Note that this would be inefficient for large queries.
    """
    return {col[0]: row[idx] for idx, col in enumerate(cursor.description)}


def get_db():
    """Open a new database connection."""

    if 'sqlite_db' not in flask.g:
        db_filename = App.app.config['DATABASE_FILENAME']
        flask.g.sqlite_db = sqlite3.connect(str(db_filename))
        flask.g.sqlite_db.row_factory = dict_factory

        # Foreign keys have to be enabled per-connection.

        flask.g.sqlite_db.execute("PRAGMA foreign_keys = ON")

    return flask.g.sqlite_db


@App.app.teardown_appcontext
def close_db(error):
    """Close the database at the end of a request."""

    sqlite_db = flask.g.pop('sqlite_db', None)
    if sqlite_db is not None:
        sqlite_db.commit()
        sqlite_db.close()

Before we can display anything to the browser, we have to account for how to handle images.

Let's make a file in the views folder called images.py



touch App/views/images.py

Add the following code to this file.



import flask
from flask import  send_from_directory
import App


@App.app.route('/uploads/<path:filename>', methods=['GET'])
def download_file(filename):
    """Download a file."""
    return send_from_directory(App.app.config['UPLOAD_FOLDER'],
                               filename, as_attachment=True)

Now add this function to the view/__init__.py so that we can find files in the uploads folder.

view/__init__.py



"""Views, one for each app's page."""
from App.views.index import show_index
from App.views.images import download_file

In this file we have a way for flask to get reference images based on the string names stored in the database.

Now change the views/index.py to have a database call for all users except one.



"""
index (main) view.

URLs include:
/
"""
import flask
import App


@App.app.route('/')
def show_index():
    """Display / route."""
        # Connect to database
    connection = App.model.get_db()

    # Query database
    logname = "John Doe"
    cur = connection.execute(
        "SELECT  fullname, email, picture "
        "FROM developer "
        "WHERE fullname != ?",
        (logname, )
    )
    devs = cur.fetchall()


    context = {"devs": devs}
    return flask.render_template("index.html", **context)

And change the index.html to display that information with jinja.



<!DOCTYPE html>
<html lang="en">
    <h1>developer</h1>
    {% for dev in devs %}
    <p>{{dev.fullname}}, </p>
    <p>{{dev.email}} </p>
    <img src="{{ url_for('download_file', filename=dev.picture) }}" alt="image" width="100" height="100">
    <br />
    {% endfor %}
</html>

If our development server is on by running ./bin/run, when we visit localhost:4000 we should see the names of the developers from the server except John Doe.

This completes the flask portion of this tutorial.

Part 3: Making the react app.

We are going to make our react app in a slightly different way that normal so that it will make the instantiation on AWS easier. Doing it this was will make it so that we can run the front-end and back-end with a single process.

Let's jump in.

Make sure that you have a version of node greater than 19.8.1



node --version

and npm version greater than 9.5.1.

this should have installed npm as a package with your node install.



npm --version

Now let's make an App/js folder to keep react components.



mkdir App/js

And a Index.js file to keep our dom root div.



touch App/js/Index.js

And add the following code to this file.



import React from "react";
import { createRoot } from "react-dom/client";
import App from "./App";

const root = createRoot(document.getElementById("root"));


root.render(<App />);

And the App component which will be the starting point of all the react code.



touch App/js/App.js

add the following code to the App.js file.



import React from "react";

const App = () => {
  return (
    <div>
        Hello React!
    </div>
  );
};

export default App;

Next we need a package json to define all of the packages for webpack and react to use.



touch package.json

Add this code to the package.json file.



{
    "name": "App",
    "version": "1.0.0",
    "description": "App front end",
    "main": "App/js/Index.jsx",
    "author": "awdeorio",
    "license": "MIT",
    "repository": {},
    "devDependencies": {
      "@babel/core": ">=7.21.3",
      "@babel/plugin-transform-runtime": ">=7.21.0",
      "@babel/preset-env": ">=7.20.2",
      "@babel/preset-react": ">=7.18.6",
      "@babel/runtime": ">=7.21.0",
      "@cypress/grep": "^3.1.5",
      "@types/react": ">=18.0.28",
      "@types/react-dom": ">=18.0.11",
      "babel-loader": "^9.1.2",
      "eslint": ">=8.36.0",
      "start-server-and-test": "^2.0.0",
      "tmp": "^0.2.1",
      "webpack": ">=5.76.2",
      "webpack-cli": ">=5.0.1"
    },
    "dependencies": {
      "latest-version": "^7.0.0",
      "moment": ">=2.29.4",
      "prop-types": ">=15.8.1",
      "react": "^18.2.0",
      "react-dom": "^18.2.0",
      "ts-loader": ">=9.4.2",
      "typescript": ">=5.0.2"
    },
    "engines": {
      "node": ">=18.0.0"
    }
  }

Next, we want to make a package-lock.json from the package.json.

To do this run the following command.



npm i --package-lock-only

Now run this command to make a node modules folder.



npm ci .

Now we want to make sure that we have webpack with the right version.



npx webpack --version

The output should look something like this but results may very slightly.

output:



 System:
    OS: macOS 11.7.6
    CPU: (8) x64 Intel(R) Core(TM) i7-4850HQ CPU @ 2.30GHz
    Memory: 276.08 MB / 16.00 GB
  Binaries:
    Node: 20.3.0 - /usr/local/bin/node
    Yarn: 1.22.19 - /usr/local/bin/yarn
    npm: 9.6.7 - /usr/local/bin/npm
  Browsers:
    Chrome: 115.0.5790.102
    Safari: 16.4.1
  Packages:
    babel-loader: ^9.1.2 => 9.1.3 
    ts-loader: >=9.4.2 => 9.4.4 
    webpack: >=5.76.2 => 5.88.2 
    webpack-cli: >=5.0.1 => 5.1.4

We want create a root.html and make a root div for the react-dom to mount onto.



touch App/templates/root.html

App/templates/root.html



<!DOCTYPE html>
<script type="text/javascript" src="{{ url_for('static', filename='js/bundle.js') }}"></script>
<html lang="en">
    <body>
        <!-- Plain old HTML and jinja2 nav bar goes here -->


        <div id="root">
          Loading ...
        </div>
        <!-- Load JavaScript -->
        <script type="text/javascript" src="{{ url_for('static', filename='js/bundle.js') }}"></script>
      </body>
</html>

This is telling the flask app to display whatever is in the bundle.js in the root div.

We want to make a webpack.config.js to keep our build configurations for react.

In the project directory run the following command.



touch webpack.config.js

And add this code to the webpack.config.js.



const path = require("path");
const { existsSync } = require("fs");

// Set the entrypoint to Index.jsx by default, but Index.tsx if using TypeScript.
let entry = "./App/js/Index.js";
if (existsSync("./App/js/Index.ts")) {
  entry = "./App/js/Index.ts";
}

module.exports = {
  mode: "development",
  entry,
  output: {
    path: path.join(__dirname, "/App/static/js/"),
    filename: "bundle.js",
  },
  devtool: "source-map",
  module: {
    rules: [
      {
        // Test for js or jsx files
        test: /\.jsx?$/,
        // Exclude external modules from loader tests
        exclude: /node_modules/,
        loader: "babel-loader",
        options: {
          presets: ["@babel/preset-env", "@babel/preset-react"],
          plugins: ["@babel/transform-runtime"],
        },
      },
      {
        // Support for TypeScript in optional .ts or .tsx files
        test: /\.tsx?$/,
        use: "ts-loader",
        exclude: /node_modules/,
      },
    ],
  },
  resolve: {
    extensions: [".js", ".jsx", ".ts", ".tsx"],
  },
};

Compile the react code in preparation for a run test.



npx webpack

In the same way that we made a index.py we want to make a root.py to display the react app.



touch App/views/root.py

Add the following code.



"""
root of react app.

URLs include:
/root/
"""
import flask
import App


@App.app.route('/root/')
def show_root():
    """Display / root for react application."""
        # Connect to database

    context = {}
    return flask.render_template("root.html", **context)

Now add this function to the __init__.py within the views folder. the whole file will look like this.

App/views/__init__.py



"""Views, one for each app's page."""
from App.views.index import show_index
from App.views.root import show_root
from App.views.images import download_file

You should not receive any errors but might get warnings.

Use the script we made earlier to run the flask app.



./bin/run

You should not get an error visit localhost:4000/root

Also ensure that our jinja index page still has our developer names.

When you want to develop and make changes to the the react app and not type npx webpack everytime, you can use the --watch flag. this will work the same way as npm start

After we are finished checking the browser. Let's shut down the flask app by clicking on the terminal and pressing control + c.



npx webpack --watch

npm packages can be installed as normal with npm i <package-name>.

The only major difference between how you interact with this react app vs. one mode with create-react-app is the front-end is essentially ran through the back-end.

We are now ready to develop a front-end with react and a back-end with flask.

We can also make jinja templates in this app when the situation fits.

This completes the react portion of this tutorial.

Part 4: Setting up an AWS account.

Its a good idea to check out the best practices for setting up AWS accounts since this is easy to forget about account security after you have had an account for a while.

Before we can deploy to AWS we first have to set up an aws account. If you already have an account set up, sign into your AWS account you can skip this section by clicking here

Go to this url to create an account.

They will send a verification code to your email.

After entering the code you will enter the will have to make a password.

Then you will be sent to this page.

You will need to add a credit card to your account even though the setup we are making is free.

Confirm your Identity.

After you confirm your phone number, choose which tier. We will use the free tier. Any tier you use will not effect the rest of this tutorial.

This will redirect you to your AWS sign-in.

Log into you account and you will be directed to this page.

Click on the settings icon in the top left corner of the page.

Now click to launch a new instance.

We are going to use an ubuntu instance. Click on Ubuntu where you see all the different virtual boxes.

Scroll down to where it says add a key pair(login) and name it.

If you start typing in the text box, a modal will popup where you can enter the name of the key.

Whether you added a key pair in the model or added directly to the textbox does not matter. As long as we choose RSA and .pem this will be fine.

Click add key pair once you named it.

This will download a aws-tutorial.pem file to you computer.

Scroll down to network settings.

Click on Allow HTTP traffic. This part is important because you will be able to do everything else in this tutorial but wont be able to ever see the website even though you did everything else right!

You can also allow HTTPS traffic but I have not tested this. you will need to buy an SSL certificate too. If someone gets this to work with allowing https traffic please let us know in the comments and how to do it.

Now click launch instance.

Scroll to the bottom and click view all instances.

You will be forwarded to a page with information about all of your instances. When you click the checkbox for the instance you just created, information about that instance will be displayed on the bottom of the page.

When you go the the address it will try and load for a while but no website will pop up.

We will add the code from the previous parts of this section to be the website in the next section.

This completes the AWS console setup portion of this tutorial.

Part 5: Deploying to AWS.

This is the most complex portion the tutorial. Be sure to read carefully because a small error in this section can cause hours of headache if you don't do this step by step. But at the end of this you will have a website ready for development, live on the internet and the ability to deploy other websites the same way.

If you stuggled with earlier parts of this tutorial, you can copy my repo to make sure that none of the errors are from any previous sections which can make debugging easier.

I recommend trying to do this on your own first.

First thing is to make a github repo.

for extra data safety make a .gitignore file and add the .pem file to it.



touch .gitignore

we want to add a lot folders to the gitignore since most of these are boilerplate files and not files that we have created.



*.pem
node_modules
env
App.egg-info
__pycache__

This will prevent any private keys from being published to github.

make an initial repo



git init && git add . && git commit -m "initialize"

create a remote origin and push it to github, which you will have to do on your own, leave it public for now, at least until you pull it into your ubuntu instance which we will do shortly. Then you can make it private if you wish.

If you completed the previous sections, you should have a aws-tutorial.pem file in your downloads folder.

Ensure you are in your project directory of your local repo, then run the following command.



mv ~/Downloads/aws-tutorial.pem .

we can also drag and drop into the project directory. As long as we have it in the root of this folder of your local repo, we are fine.

Now we give this a special 400 permission to the aws-tutorial.pem file for read only access so that we don't modify our key to the aws instance.

Run this command.



chmod 400 aws-tutorial.pem

We are now going to tunnel into our aws instance which will take use to a ubuntu terminal on aws that we control from our host machine.

To do this you want to run the following command.

Do not include the <>'s in the command but do include the ubuntu@



ssh -i aws-tutorial.pem ubuntu@<your public ipv4 DNS from the AWS cosole>

we can copy and past from our AWS console like so.



ssh -i aws-tutorial.pem ubuntu@<paste what you just copied here>

Type yes and enter to finish connecting.

You should have a proper ssh tunnel to your ubuntu instance. It should look like this.

If it didn't work. Make sure that you have your aws-tutorial.pem file in the project directory. That you ran chmod 400 aws-tutorial.pem and that the command ssh -i aws-tutorial.pem ubuntu@<public ipV4 DNS> is correct and without the angle brackets. Note that everytime you shut down and restart you instance you will have to recopy this public ipV4 DNS.

Since we are now going to be working with a linux command line interface, the commands will be a bit different.

Now we want to install nginx which will give our ubuntu server a way host a website.



sudo apt-get update



sudo apt-get install nginx

type Y to add nginx.

when you visit the site remember to make sure that it's http::/ and not https::/ when you do, you should see this page now.

We are going to have to change some files to get a configuration for our app to work correctly with njinx.

To accomplish this we will use emacs to alter files.



sudo apt-get install emacs-nox

Accept this install like last time.

Also run this command.



export EDITOR=emacs

This command will give no output.

Having emacs means we will be using nano.

Check that we have it.



which nano

output:
/usr/bin/nano

Now we will create a nginx.conf file to define how nginx will interact with our web app.



sudo nano /etc/nginx/nginx.conf

For reference, when I write control + k the plus is to indicate that you press control and k keys at the same time.

Hold control + k in order to delete all the lines in the file line by line.

after the file is completely blank, copy and paste this text using control + v



# Run nginx worker processes using user www-data, which should have been created after installing Nginx.
user www-data;

# Start as many workers as there are CPU cores.
worker_processes auto;

# Configure connection processing.
events {
    # Maximum number of simultaneous connections per worker process.
    worker_connections 1000;
}

# Configure the HTTP server.
http {
    # Directly copy data between file descriptors instead of storing it in a buffer.
    sendfile on;

    # Send the response headers and the beginning of a file in a single packet.
    tcp_nopush on;

    # Map file extensions to MIME types.
    include /etc/nginx/mime.types;

    # Specify where logs should be written.
    access_log /var/log/nginx/access.log;
    error_log /var/log/nginx/error.log;

    # Compress HTML data before sending it in responses.
    gzip on;

    # Include site-specific config files.
    include /etc/nginx/conf.d/*.conf;
}

Then press control + x then y for saving the file and enter

This will bring you back to the ubuntu terminal with the green text.

Now we want to add special configuration details specific to our App itself.

This will be a similar process as we just did but, we will be creating this file.

Type this command into the terminal.



sudo nano /etc/nginx/conf.d/App.conf

Copy and paste this file.



# Configure a virtual server. We only need one, because this machine will only host a single website.
server {
    # Configure all requests to the path /uploads.
    location /uploads {
        # Send a subrequest to /accounts/auth/. If the response is 200, proceed; if it's 403, don't
        # serve content, and just return a 403 status.
        #auth_request /accounts/auth/;

        # Serve the requested file from /var/www/uploads/<filename>.
        root /var/www;
    }

    # Configure all requests to the specific path /accounts/auth/.
    location = /accounts/auth {
        # Forward the request to the Flask app running at localhost:8000.
        proxy_pass http://localhost:8000;

        # Don't include the body of the request if any in the proxied request.
        proxy_pass_request_body off;

        # Set some headers that Nginx wants us to use for authentication subrequests.
        proxy_set_header Content-Length "";
        proxy_set_header X-Original-URI $request_uri;
    }

    # Configure all other requests to the server besides the ones that match above.
    location / {
        # Forward the request to http://localhost:8000 and return its response to the client.
        proxy_pass http://localhost:8000;

        # Make sure the proxied request's Host header is set to what the client intended.
        proxy_set_header Host $host;

        # Add a header to the proxied request indicating whether the protocol is HTTP or HTTPS.
        proxy_set_header X-Forwarded-Proto $scheme;

        # Add a header to the proxied request specifying the IP address of the original client.
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    }
}

there are some configurations in this tutorial that are based on the idea that this app will be much more heavy duty than what is given in this tutorial. Try uncommenting lines of code when you implement auth.

Press control + x then y then enter.

Now we want to restart the nginx server.



sudo systemctl restart nginx

Install our python environment.



 sudo apt-get install python3 python3-venv sqlite3

Now clone your remote repository. Alternatively you can use my repo



git clone https://github.com/TallanGroberg/aws-deploy-tutorial.git

Create a virtual environment for your project instance.



python3 --version

output:
3.10.6

This is fine if its different that our local machine but it has to still be greater that 3.10

cd into your folder.



cd aws-tutorial

We have one more file to change. Since we changed the default location for uploads in production. we have to alter our App's config.py to reference var/www instead of var/uploads.



nano App/config.py

You can use your mouse to scroll to a single line. And should be able to erase and type like in more common text editors.

We should change UPLOADS_FOLDER to be this now.



UPLOADS_FOLDER = pathlib.Path('/var/www/uploads')

Then press control + x then Y and finally enter

Create the virtual environment.



python3 -m venv env

And activate.



source env/bin/activate

(env) should be at the left side of the green text.

Run each of these commands one by one.



pip install --upgrade pip setuptools wheel



pip install -r requirements.txt



pip install -e .

This next command to install gunicorn will enable us to run a local server like in development but in production.



pip install gunicorn

Now we initialize the database.



./bin/db create

Copy the uploads folder to the var/www folder where nginx can use it.



 sudo cp -r var/uploads /var/www

Give ubuntu permissions for this new directory



sudo chown ubuntu:www-data /var/www/uploads

Now we install react related packages starting with npm and node.



sudo apt-get install nodejs npm

You will notice that we don't have the same node version.

When you see pink screens durning package downloads just press enter to get through them. They didn't cause me any problems when I picked the default everytime.



node --version

output:
v12.22.9

Now we want to install everything from our package.json



npm install .

just to be sure make sure that your package-lock.json is here.



npm i --package-lock-only

When this download is happening, we can see that we get alot of versioning warings. This will have to be addressed by updating node version manager, you can check out the github here (nvm).

Run and update for ubuntu.



sudo apt update

Install the version manager from its github directly.



curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.39.1/install.sh | bash

Now we have to source it to make sure ubuntu can take advantage of it.



source ~/.bashrc

This will cause us the have to reactivate our python environment this means that we have to reinstantiate the whole package.



source env/bin/activate



pip install --upgrade pip setuptools wheel



pip install -r requirements.txt



pip install -e .

Now we just install the node version that we had our react app running on in development.



nvm install v20.3.0

Make sure that the node version update.



node -v

output:
v20.3.0

This should ensure that when we run webpack without any errors.



npx webpack

You should receive a compiled successfully in the last line of the output.

We want to install javascript-obfuscator to make our javascript harder to understand for hackers.



npm install javascript-obfuscator

Now we run the next few commands to compile, obfuscate and replace our original build.



npx webpack



npx javascript-obfuscator App/static/js/bundle.js --reserved-strings '\s*'

press y to proceed.



mv App/static/js/bundle-obfuscated.js App/static/js/bundle.js

Now we are going to get the server running.

Before we try and run it make sure that no other gunicorn processes are running.



pkill -f gunicorn

When you run the next command, there should be no output.



pgrep -af gunicorn

Start the server in the background with this command.



gunicorn -b localhost:8000 -w 2 -D App:app

Now when you visit your aws site, you should be able to see the app running.

If it's not, we can trouble shoot with this command which will give the log level output of what is happening.

first.



pkill -f gunicorn

then.



gunicorn -b localhost:8000 -w 2 App:app --log-level debug

If you get an error about packages missing pertaining to python redo the virtual environment installation. You can do the same for react and that should do the trick.

Conclusion

I have not tried to develop anything beyond a single page react application with react with this configuration. I encourage comments about any react-packages that will not work with this configuration or if anyone could make the same functionality with create-react-app.

There is a way to write a script so that this deploy is way commands in the terminal. A great exercise would be to make this script on your own.

Let me know what I could add to this article in the comments.

5 gamified learning experiences to learn Web Dev

Chris Noring — Sun, 26 Jun 2022 23:11:51 +0000

Does learning have to be boring, not in my opinion. I'm sure you also agree that you learn better if you have fun. Here's a list of 5 gamified experiences that will help you learn web development.

Flexbox froggy. Learn to navigate a frog by and learn about Flexbox at the same time.
Minecraft Education You have code window that helps you affect the world in Minecraft, who doesn't want to build things faster? Blocks, JavaScript as well as Python available.
CodeCombat Imagine using code to take yourself through dungeons. This might look like it's made for 10 year olds but I had a ton of fun :)
Twilio Quest This is seriously good. Travelling the Universe while learning JavaScript, Python and Twilio API, sounds pretty good to me.
MakeCode arcade - skillmaps You learn to build various games while being introduced to coding. You may start with blocks, but you will be able to advance to JavaScript or Python prettty quickly.

There are a ton of these gamified platforms so I would love to hear your favourite in the comments and why :)

I wrote a free book on React

Chris Noring — Sun, 19 Jun 2022 19:47:56 +0000

Follow me on Twitter, happy to take your suggestions on topics or improvements /Chris

I'm writing a free book on React, it will always be free :)

Content

Hope you enjoy it :)

It covers:

Basics
Forms
Routing
Testing
Redux

A lot of my articles here is based on things I wrote in the book. Hope it can help someone :)

How YOU can build a GraphQL API with Apollo

Chris Noring — Sun, 05 Jun 2022 00:50:27 +0000

Follow me on Twitter, happy to take your suggestions on topics or improvements /Chris

This article is part of a series on GraphQL:

Building a GraphQL server using Node.js and Express
Building a GraphQL server using Node.js and the Apollo Framework, we are here
Consuming an Apollo GraphQL server using React

GraphQL is a new way we can use to build APIs. One of its prominent features is content-negotiation, a way for the client to tell the backend exactly what fields it needs and at what depth. This enables you to get a smaller payload of data back, which can really make a difference on a bad network connection.

Apollo is an entire platform for handling the backend as well as the client-side when using GraphQL and can even help you in the cloud. It’s well worth checking out here.

This article will be about how to create a GraphQL server using Apollo and thus only using a small part of what the Apollo platform can do. As you will see it’s pretty easy. I hope you enjoy the read :)

In this article we will cover:

Install and set up, we will learn how to install apollo-graphql and graphql libraries needed to created our server
Define a schema and resolvers, a Schema is needed to describe our types. Resolvers are functions that are invoked to handle queries from the user.
Debugging, when we start our server we can do so in a debug mode so we get a browser page we can interact with and that will help us send our queries to our server and it will also show us the results

Resources

Some of the below links are introductory so you can learn more after having read this article and some are more advanced once you feel you've grasped Graphql better.

Install and set up

To create a GraphQL server using Apollo Server we need to do three things:

install apollo-server and graphql
define a schema
define resolvers

Let’s install the needed dependencies:

npm install --save apollo-server graphql

Once we have done that we are able to proceed with the next steps.

Define a schema

To define a schema we need to be using a function called gql that allows us to define a schema like a multiline template string and most importantly using the GraphQL query language.

First, we import gql:

const { gql } = require("apollo-server");

Next up we can start defining the schema itself:

const typeDefs = gql`
  type Query {
    products: [Product]
    product(id: Int!): Product
  }
  type Product {
    id: Int,
    name: String,
    description: String
  }
  input ProductInput {
    name: String,
    description: String
  }
  type Mutation {
    login(email: String): String # token
    createProduct(product: ProductInput)
  }
`

Finally, we export the schema for later usage

module.exports = typeDefs;

The full file should now look like this:

const { gql } = require("apollo-server");
const typeDefs = gql`
  type Query {
    products: [Product]
    product(id: Int!): Product
  }
  type Product {
    id: Int,
    name: String,
    description: String
  }
  input ProductInput {
    name: String,
    description: String
  }
  type Mutation {
    login(email: String!): String # token
    createProduct(product: ProductInput): String
  }
`
module.exports = typeDefs;

Define resolvers

Next up we need to define what the following queries and mutations should do:

products, this should return a list of products
product(id: Int!), this should return one product
login(email: String), this should log in the user
createProduct(product: ProductInput), this should create a product

Let’s start a new file resolvers.js. At this point, we will keep things simple and have some local static data but we could improve this server by having it call a database or an API. So let’s define some static data at the top:

const products = [];

Now that Apollo server expects a certain type of object back with the following shape:

module.exports = {
  Query: {},
  Mutation: {}
};

Not the two keywords Query and Mutation above. These two are objects and we need to define each corresponding resolver function. Let’s start by adding a resolver function for products:

const getProducts = () => {
  return Promise.resolve(products);
}
module.exports = {
  Query: {
    products: async () => getProducts
  },
  Mutation: {}
};

A comment on getProducts() method is that we return a Promise. Now, we could return our products list directly but this is just a way to showcase that we can return an asynchronous response as well, which means this will fit well with us talking to a 3rd party API or doing a lookup on a database. To go along with our Promise response we mark it with async in our return object.

Define a resolver function with a parameter

We’ve learned how to define a resolver function, and now let’s look at how we can grab a parameter from the user request and query our backend based on it.

const getProductById = ({ productId }) => {
  return Promise.resolve(product.find(p => p.id === productId));
}
module.exports = {
  Query: {
    products: async () => getProducts,
    product: async (_, { id }) => getProductById({ productId: id })
  }
};

Above we added product to our Query object and as you can see we can quite easily dig out the id from the user request.

Define a resolver function for a mutation

To define resolvers for mutations we need to introduce a Mutation property on our exported object, just like we needed to introduce Query. So now our code should look like this:

module.exports = {
  Query: {
    products: async () => getProducts,
    product: async (_, { id }) => getProductById({ productId: id })
  },
  Mutation: {}
};

Note the addition of Mutation above.

The next step is to add our createProduct property and a resolver function that handles that, like so:

const createProduct = ({ product }) => {
  const newId = products.length === 0 ? 1: products[products.length-1].id + 1;
  products = [ …products, { …product, id: newId}];
  return Promise.resolve(‘success’)
}
module.exports = {
  Query: {
    products: async () => getProducts,
    product: async (_, { id }) => getProductById({ productId: id })
  },
  Mutation: {
    createProduct: async (_, { product }) => createProduct({ product      })
  }
};

Putting it all together

Now we have defined a set of resolver functions, let’s have a look at the resulting code in its entirety:

const products = [];
const getProducts = () => {
  return Promise.resolve(products);
}
const getProductById = ({ productId }) => {
  return Promise.resolve(product.find(p => p.id === productId));
}
const createProduct = ({ product }) => {
  const newId = products.length === 0 ? 1: products[products.length-1].id + 1;
  products = [ …products, { …product, id: newId}];
  return Promise.resolve(‘success’)
}
module.exports = {
  Query: {
    products: async () => getProducts,
    product: async (_, { id }) => getProductById({ productId: id })
  },
  Mutation: {
    createProduct: async (_, { product }) => createProduct({ product   })
  }
};

As you can see we have only operated on static data but we could easily switch those calls to target a database or a 3rd party API, that’s entirely up to you. As long as you remember to use the await keyword in your resolver definition you are good to go with interacting with anything return a Promise.

Start the server

We have defined a schema and a set of resolver functions. Now let’s import schema.js and resolvers.js and kick off our server:

const { ApolloServer } = require("apollo-server");
const typeDefs = require("./schema");
const resolvers = require("./resolvers");
const server = new ApolloServer({ typeDefs, resolvers });

server.listen().then(({ url }) => {
  console.log(`🚀 Server ready at ${url}`);
});

Debug

Navigate to http://localhost:4000 —

you should have something looking like this:

As you can see this looks very much like the GraphiQL environment we presented in the chapter GraphQL + Node.js Express. The idea is the same. On your left you have an area where you can enter queries and mutations and on your right is the results area. On the bottom left is avariables area that allows you to specify input variables that you can use the top left pane.

Example usage

Let’s try to create a query. We can do so in two ways:

using the no-keyword version, {}
using the keyword query

The first variant looks like this:

As you can see above we are only using {} to specify our query and then we query for a resource launches and drills down into the resource to get the columns that we need, i.e mission and under mission we grab the column name.

Let’s try to use the other query variant next by using the keyword query. The keyword query allows us to:

Name the query so we can refer to it later
Specify more than one query in the same window, so we can choose which one we want at a given point in time
Specify a parameter for our query ( we will show this one in our next example )

Above we managed to specify two different queries and have given them the name LaunchesNameOnly and LaunchesAll. While specifying the LaunchesAll query you can see how our environment helps us by indicating the type of isBooked, that we are about to select. The type is a Boolean!, a mandatory Boolean.

In our next example we show how you hit the large play button and is given the choice between different queries, as you can see it is a really great idea to define your queries with the keyword query so to create some order in your testing environment:

Below is simply the result of executing our query

Let’s try to use variables next and see how we can declare those in the bottom left area and use them in our queries and mutations.

Above we have defined OneLaunch that takes a parameter $id. Thereafter this parameter $id is passed down to our actual query expression like so:

launch(id: $id) {
  mission {
    name
  }
}

Now, we are ready to invoke our query, with a parameter and we simply hit the play button and we get this:

The reason we get the above is that it read from the bottom left are that looks like this:

Variables in this area is all made up of one object and we simply add keys to this object when we need a new parameter like so:

{
  "id": 1,
  "otherparam": "some value"
}

Deeper queries

One of the big selling points of GraphQL is the ability to query deep into a hierarchy, like so:

{
  orders {
    created,
    createdBy {
      name,
      created
    },
    items {
      product {
        name
      },
      price
    }
  }
}

Above we are querying on the resource orders and as you can see we are able to perform a lot of subqueries and query for orders->items->product. If we were to do that with SQL, it would be quite difficult with a lot of JOINS. So how are we able to construct this in GraphQL?

In short, we need to do two things:

define a resolver to the asked for subtype
store the data in a resolvable format. This means we need to store the data as an id/ids rather than the full object

Store the data as id/ids

We need to store an order like this:

const orders = [{
  id: 1,
  items: [1,2,3]
}]

Define a resolver for a subtype

How this is done syntactically depends on the GraphQL server implementation we are dealing with. In this case, we are dealing with Apollo, so let’s look at how Apollo handles it. We know that when we define a resolver object in Apollo it looks like this:

{
  Query: {
    orders: async () => getOrders(),
    order: async (_, { id }) => getOrder(id)
  },
  Mutation: {
    // add mutation resolvers here
  }
}

Now, our querying for orders means we are interested in querying for all its columns like created but also for a complex column like items and even items -> product so we can ask a query like this:

{
  orders {
    created,
    items {
      product {
        name
      },
      price
    }
  }
}

So how do we resolve items? Well let’s have a look at our schema definition of an order:

type Order {
  created: Date;
  items: [Item]
}

With this in mind let’s head back to our resolver object and now we will add a Order type to like so:

{
  Query: {
    orders: async () => getOrders(),
    order: async (_, { id }) => getOrder(id)
  },
  Mutation: {
    // add mutation resolvers here
  },
  Order {
    items: ({ items }) => getItems(items)
  }
}

Let’s zoom into our addition:

Order {
  items: ({ items }) => getItems(items)
}

Here we can see that we are resolving what happens if someone queries for items. We are missing the getItems() method though so let’s define that:

const itemsList = [{
  id: 1,
  name: ‘item’
},
{
  id: 2,
  name: ‘item2’
}]

const getItems = (items) => {
  return Promise.resolve(itemsList)
}

Ok, so that deals with items so how do we deal with items->product. Well, that one is quite simple, we just again look at our schema:

type OrderItem {
  price: Double;
  product: Product;
}

We see now that we need to add OrderItem to our resolver object, like so:

{
  Query: {
    orders: async () => getOrders(),
    order: async (_, { id }) => getOrder(id)
  },
  Mutation: {
    // add mutation resolvers here
  },
  Order {
    items: ({ items }) => getItems(items)
  },
  OrderItem {
    product: ({ product }) => getProduct(product)
  }
}

Summary

We have covered

Installing apollo-server and graphql, in order to have the necessary dependencies to create an apollo server.
Defined a schema and a set of resolvers.
Booted up our server,
Explored the visual environment that allows us to run queries or mutations

Moving on we will look at different ways to consume said server using apollo-client so stay tuned for that.

Go from the beginning - your first program

Chris Noring — Fri, 03 Jun 2022 13:55:33 +0000

this is the first part of series on Go, the programming language, I hope it's helpful

The language is called Go but is sometimes known as Golang as the first website for it was golang.org.

Go was created in 2009 by Robert Griesemer, Rob Pike and Ken Thompson. It's hard to estimate the number of Go developers but it's somewhere between 1.1 and 2.7 million, quite a sizeable amount. More than 2500 companies are using Go including, Google, Pinterest and Uber. So you see, used by a lot of folks by big companies.

Why was Go created?

As is often the case, a programming language is created to deal with the shortcomings of other languages. In this case, the creators wanted this new language to have the following capabilities:

Static typing and run-time efficiency from C.
Readability from JavaScript and Python.
High-performance networking and multi-processing.

It seems the creators agreed on disliking C++ :)

What is it used for though?

Here's some areas where you are likely to find a Go being used:

Cloud based and server-side apps
DevOp, automation
Command line tools
AI and data science

References

There's many great resources out there for learning the Go programming language like:

Features

So what features makes Go compelling? Well, there are some features really worth mentioning that I personally appreciate:

Static typing, I like my types :)
Package system. You can consume and create your own packages. Go to pkg.go.dev to read more on what packages there are.
Command line tools, there's a set of executables that are installed when you install Go. With these executables, you can run, build, install packages, run tests and much more.
Standard library. Go has a powerful standard library that will help you with most things you might need. You can read more about what's in the standard library here.
Built-in testing. Having a testing library that just works out of the box is something you shouldn't take for granted.
Concurrency. Go is great at handling concurrency. It uses concepts like goroutines and channels.
Garbage collection. You can read more about that here. I like when I don't have to deal with that myself and just focusing on solving problems.

Install Go

Ok then, hope you are intrigued at this point and just want to see some code? Of course you are :)

Make sure you've followed the instructions for installing Go on your machine.

https://go.dev/doc/install

A Go program

Here's what a first program can look like:

package main

import "fmt"

func main() {
 fmt.Println("hello")
}

The program in detail

package main, the entry point module needs to have this instruction.
import "fmt", fmt is standard package for input and output.
func main, entry point function, where your program starts.

Commands

Now that you have a program, there's two things you might want to do:

Run it, to see if it compiles and runs.
Create executable, an executable is no longer Go code but like any executable program on your machine.

Run your app

To run your app, type go run <file>.go, for example:

go run main.go

this produces an output looking like so:

hello

Build your app

To produce an executable, run go build <file>.go, for example:

go build main.go

It produces an executable, on MacOS and Linux that's a file with -X as permission, on Windows, it's an .exe file.

Congrats, you've created your first Go application.

Summary

In this article, you learned about the programming language Go, some features it has and how to write your first program.

Learn more

There are many great resources out there for learning the Go programming language like:

Instantly set up a new dev environment using containers and VS Code

Yohan Lasorsa — Thu, 30 Sep 2021 15:00:11 +0000

There are 3 things that make me waste a lot of time as a developer, and that can become a big source of frustration:

Set up a new development machine
Manage and update my development environment to work on a project
Reinstall everything again, because after many times installing and updating my dev tools everything broke :(

What if I told you that we can avoid these issues, and that it's possible to perfectly isolate each of your projects' environment? And to be able to share and update it easily, to have nothing to do when a newcomer joins your project?

It's now possible thanks to the Remote Development extension for VS Code and containers. In this post, I'll show you how to set up a complete dev environment inside a container and use it with your projects.

TL;DR key takeaways

You can quickly onboard a new developer in a project using dev containers, share it with your teammates, and update it with no hassle.
While nothing prevents you from using Docker and any IDE to do the same thing, the Remote Development extension provides seamless integration with VS Code to bootstrap and work with dev containers.
Dev containers also allow you to take your dev environment everywhere with online dev tools like GitHub Codespaces.

Why developing inside a container

In every software project, the developer story is (almost) always the same:

"Hey, welcome to the team! We're all excited to have you on board!" 🎉
"Here's the doc to set up your environment for the project. Let me know when you're ready!" (spoiler alert: sometimes there's no doc at all, and you'll have to ask your teammates to know what to install)

Most likely, a few hours/days later, the outcome is:

"Please help, I can't figure out why it's still not building/running properly" 😞

I've been that developer. A few times already.

After a while of spending time reviewing everything you installed, you'll eventually discover common issues usually are:

The environment setup doc is outdated (I bet you didn't see that one coming, right? Isn't that the newcomer's job to update it anyway?).
You installed the correct tools, but not the right version ("Sorry, We forgot to tell you'll have to use this specific version, as we have a few issues in our codebase").
You have some conflicts with the environment set up of other projects.

The thing is, we already know how to solve these problems, because that's the exact same issues we had with production environments when we ship our applications.

We solved all these problems by packaging the runtime environment of our applications into containers. Then, why not also use containers for our dev environment?

Requirements

You need to have these tools installed on your machine to get started:

Visual Studio Code
Docker (or any Docker-compatible client)
Remote Development Extension for VS Code

Getting started

Open VS Code on a new or existing project, then click on the bottom left
of the status bar, on the "opposing chevron" icon (let me know if you have a better name):

Once you do that, select Add Development Container Configuration Files..., and it will open a huge list of environments starters.

You'll find pre-made configurations for Node.js, Java, .Net, Go, Python, and more, most likely there is one that you can use as a starting point. Once you've made your choice, you'll notice that a new .devcontainer folder was created, containing 2 files:

We'll dig into these files just after, for now, let's reload the project to work inside a container.

Click again on that "opposing chevron" icon in the status bar, and select Reopen in container.

VS Code will reload the project's window and start building the container. Depending on your internet connection speed, this may take a while but don't worry, it's only needed the first time.

Once it's finished, you'll see that VS Code is connected to your dev container in the status bar. You can also open a terminal and play with your new dev environment (in my case, I've set up a new Node.js v14 environment).

Customizing the dev setup

Now that you have a working environment, you'll probably want to customize it for your project's needs. If you unfold the .devcontainer folder in the explorer, you'll see these 2 files that will allow you to do that:

Dockerfile: this file defines your container configuration, and is used to build the container image that will be used for your environment. If you need to install additional tools or configure scripts and environment variables for your dev environment, this is where you should look at.
devcontainer.json: this file allows to customize VS Code when the project is connected to a development container. In particular, you can specify VS Code settings, extensions, and port forwarding that will be enabled only in this project dev container.

Let's take a deeper look at how you can fine-tune your VS Code dev environment by editing your devcontainer.json file.

VS Code settings and overrides

You'll see something like this under the settings key:

// Set *default* container specific settings.json values
// on container create.
"settings": { 
  "terminal.integrated.shell.linux": "/bin/bash"
},

This key allows you to override any VS Code settings when it's connected to this container environment. For example, if you work on Windows and your default terminal is PowerShell, by connecting to the dev container the setting above with switch your default terminal to Bash instead, which will be run inside the Linux system of the dev container. With these settings, you can set a common code formatter for your team, or force GPG signing for your commits for example.

Another useful option is located under the extensions key:

// Add the IDs of the extension you want installed when
// the container is created.
"extensions": [
  "dbaeumeur.vscode-eslint"
],

This one allows specifying extensions that will be automatically installed within your dev container when it's run for the first time. Yes, you've read that well: these extensions won't pollute your global VS Code configuration, as they'll be available only when your dev container is started and connected to your project. No need to toggle your extensions anymore depending on your active project, no more having a useless C# extension active on your Java project!

One last essential option, forwardPorts, allows you to expose ports available from within your dev container and make them accessible by default from your local machine.

// Use 'forwardPorts' to make a list of ports inside the
// container available locally
"forwardPorts": [3000, 4200],

For example, if you run a web server within your dev container, without port forwarding you won't be able to access it in your browser. Note that it's also possible to add ports to forward on the fly, by clicking on the antenna icon in the status bar, then selecting Add Port:

Sharing and updating the environment

Once your container and VS Code configuration is finished, you only have to push these two files in your repository to make them available for your team. And it's one of the big benefits of this approach! To welcome a new developer to your project, you now only have to clone the project, reload VS Code to use the container config and that's it. Take a coffee (or many) while the container is downloading and installing tools on its first run, and your new developer will have a dev environment ready for work, without any effort.

Even better, if anyone updates the environment configuration, for example by upgrading a tool's version, everyone in the team will see this notification when pulling the changes:

You then only have to select Rebuild to update your environment to the latest version. No more difficulties building a project when coming back from vacation!

Where's the catch?

It almost seems too good to be true, so where's the catch?

I've been asking that to myself since I started using it from the first beta, and so far it's been a happy ride. If you wonder about the performance, yes doing everything inside a container is a bit slower than doing it on the host machine, but nothing to worry about. In some cases, you may face disk performance issues especially with lots of small files (looking at you, npm install) but there are tips to improve your filesystem performance with Docker.

The only issue I had at the beginning was with my git credentials as I'm using SSH keys, but once I've followed the guide to set up an SSH agent everything went smoothly.

You can have a look at the known limitations, but so far I've yet to have any issue with it.

What's next

This article is only a brief overview of what's possible with these tools, and of course, you can push it way further if needed, like creating a dev environment using multiple containers for example. You can have a look at the documentation for more advanced scenarios.

Going further, you could still argue that you have to install VS Code with the remote extension and a Docker-compatible runtime to make this work. It's also possible to get rid of that, using for example GitHub Codespace which makes use of the exact same tools and configuration to provide a cloud-based dev environment, accessible from any browser, without having to install anything (but a browser) on your machine.

Follow me on Twitter, I would be happy to discuss and take your suggestions!

Lazy Loading in Angular - A Beginner’s Guide

Arjav Dave — Wed, 28 Apr 2021 07:17:00 +0000

What is Lazy Loading?

Lazy loading is a process of loading components, modules or other assets of a website as required. Since, Angular creates a SPA (Single Page Application), all of its components are loaded, at once. Secondly, a lot of unnecessary libraries or modules might be loaded as well.

For a small application it would be okay, but as the application grows the load time will increase, if everything is loaded at once. Lazy loading allows Angular to load components and modules as and when needed.

First of all, to understand how lazy loading works in Angular, we need to understand the basic building blocks of the framework: NgModules.

In order to understand how Lazy Loading works we first need to understand the building block of Angular: NgModules.

What are NgModules?

Angular libraries like RouterModule, BrowserModule, FormsModule are NgModules. Angular Material, which is a 3rd party, is also a type of NgModule. NgModule consists of files & code related to a specific domain or having a similar set of functionalities.

A typical NgModule file declares components, directives, pipes, and services. It can also import other modules that are needed in the current module.

One of the important advantage of NgModules is they can be lazy loaded. Let's have a look at how we can configure lazy loading.

How to Create NgModules

In this tutorial, we will create two modules Module A and Module B which will be lazy loaded. On the main screen we will have two buttons for loading each module lazily.

Create a Project

Create a new Angular project lazy-load-demo by executing the below command.

ng new lazy-load-demo --routing --strict --style css
code lazy-load-demo

Here, we are creating a new project with routing. Secondly, the strict mode is enabled. Lastly, we are mentioning the stylesheet format to css. The second command opens your project in VS Code.

Root Module

By default, a root module or app module is created under /src/app. Below is the NgModule file created:

import { NgModule } from '@angular/core';
import { BrowserModule } from '@angular/platform-browser';

import { AppRoutingModule } from './app-routing.module';
import { AppComponent } from './app.component';

@NgModule({
  declarations: [
    AppComponent
  ],
  imports: [
    BrowserModule,
    AppRoutingModule
  ],
  providers: [],
  bootstrap: [AppComponent]
})
export class AppModule { }

First, we are importing all the required modules and components.

After that, @NgModule decorator states that AppModule class is a type of NgModule. The decorator accepts declarations, imports, providers, and bootstrap. Here are the descriptions for each of them:

declarations: The components in this module.
imports: The modules that are required by the current module.
providers: The service providers if any.
bootstrap: The root component that Angular creates and inserts into the index.html host web page.

Main screen

The main screen will have 2 buttons, namely, Load Module A & Load Module B. As the name suggests, clicking these buttons will lazily load each module.

For that, replace your app.component.html file with the contents below:

<button style="padding: 20px; color: white; background-color: green;" routerLink="a">Load Module A</button>
<button style="padding: 20px; color: white; background-color: blue;" routerLink="b">Load Module B</button>
<router-outlet></router-outlet>

Let's define the modules for routes a & b.

Lazy Loaded Modules

In order to create lazy loaded modules execute the below commands.

ng generate module modulea --route a --module app.module
ng generate module moduleb --route b --module app.module

The commands will generate two folders modulea & moduleb. Subsequently, each folder will contain their own module.ts, routing.ts and component files.

If you check your app-routing.module.ts you will see the below code for routes.

const routes: Routes = [
  { path: 'a', loadChildren: () => import('./modulea/modulea.module').then(m => m.ModuleaModule) },
  { path: 'b', loadChildren: () => import('./moduleb/moduleb.module').then(m => m.ModulebModule) }
];

It implies that when route a or b is visited load their respective modules lazily.

On running the project with ng serve, you will see the below screen:

Home Page

On clicking Load Module A button, you will be routed to page a. This is how your screen should look like.

Lazily loaded Module A

You should see a similar screen that says moduleb works! when clicked on Load Module B.

How to Verify that the Lazy Loading Worked

In order to verify the files loaded, open the developer tools by pressing F12. After that, visit the Network tab as you can see in the screenshot below. When you refresh the page it will show few files requested.

Network Tab

Go ahead and clear your list of requests by hitting the clear button as shown in the image on the right

When you click on the Load Module A, you will see a request for modulea-modulea-module.js as in the screenshot below. This verifies that the Module A was lazily loaded.

Module A Loaded

Similarly, when you click Load Module B, the moduleb-moduleb-module.js file is loaded. Hence, verifying that Module B was loaded lazily.

Module B Loaded

Use Cases

As we have seen, it’s very easy to create lazy loading modules. There are lots of use cases where they are useful, such as

Creating a separate module for pre-login vs post-login screens.
For an e-commerce website, vendor facing vs customer facing screens can belong to separate modules. You can also create a separate module for payment.
A separate CommonModule which contains shared components, directives, or pipelines is usually created. Directives like Copy Code button, components like up vote/down vote are usually included in a common module.

Conclusion

For smaller websites, it might not matter much that all the modules are loaded at once. But, as the site grows it's very effective to have separate modules which are loaded as needed.

Because of lazy loading, load time for the websites can be reduced drastically. This is specially helpful when you are trying to rank higher in SEO's. Even if not, less loading times means better user experience.

Are you interested in more articles? Check these out:

5 Ways to Increase Your Efficiency as a Developer

Arjav Dave — Thu, 22 Apr 2021 13:38:00 +0000

Like any other field, the efficiency of a developer varies based on several factors: work culture, management, personal life, skills, and so on. Some we can control, and some we cannot.

In this article, we will discuss how to make oneself better by working on things we control. There are some clichéd tips like Stay Focused, Avoid Distractions and Be in the FLOW.

But we are going to get a bit more technical on increasing the efficiency.

Journey of a Software Developer

The usual stages of a developer are shown below:

Coder -> Programmer -> Architect

Coder

Many people use the words “coder” and “programmer” interchangeably. But, there is a difference between the two. A coder is someone who knows how to write the code or a script. Optimisation and architecture are not their priority. Rather, they focus on writing code that just works.

Programmer

A programmer is a superset of a coder. They know how to write code, but in addition, they also write it with great optimisation and robustness. Secondly, time and memory complexities are kept in mind when writing code.

Architect

As you might have guessed, the architect or the software architect, to be precise, is a superset of a programmer. The architect is responsible for not only a piece of code but also how these pieces fit together.

Coder vs. Programmer vs. Architect: An Example

A great example would be to ask these three people to “Create tic-tac-toe.”

When asking the coders, they will start coding, assuming it's a 3x3 tic-tac-toe with two players. Secondly, they might write nested for loops as a brute force approach to figure out who won the match. There might also be no design patterns in place like MVC, MVVM, or others.

However, the programmer will first understand the requirements completely and then start on the coding part. They will create optimised algorithms for maintaining the state of the players and a scalable way to change the grid dimensions.

The architect will select which technology is best for designing the game. A few other questions come into play: Is it going to be a web-based or mobile app? Will an API be required or just played locally? In addition, they will also decide on which design patterns to implement. Once the architecture is defined, only then will the actual implementation start.

The goal of each developer should be to become a great architect along with having great technical skills.

Here are a few ways to improve your efficiency and skills:

1. Pen and Paper

Many people find it offensive to give a pen and paper coding interview. In my opinion, it is one of the best tools to get clarity. Jot down the things that you want to achieve and how you will achieve them.

In the tic-tac-toe example above, you can write down that there will be models like Player and Board. Secondly, a game engine will be required to calculate the result after each move. Also, write down what each class will contain, e.g., Player will have a name and symbol property.

You get the idea. Figure out your design patterns. What you are trying to achieve in code, achieve it with pen and paper first.

2. Become Your Own Manager

At the start of the day, create a TODO list of your tasks, e.g., you will complete the game engine today, or you will complete the board UI today, etc.

Try to break it down into smaller tasks of 3–4 hours each. Having clear goals for the day keeps you motivated. Secondly, it gives a sense of satisfaction when the tasks are completed; it boosts morale.

Essentially, it’s a variant of the Kanban management style. But, I have found it to be quite effective.

3. Documentation

Reference: https://www.reddit.com/r/ProgrammerHumor/comments/ijoxq6/why_read_documentation/

One of the reasons for taking time to get things right is a lack of knowledge.

Astonishingly, there are many developers who just assume how the system or library works. They spend an enormous amount of time figuring out by themselves how it works without reading the documentation. This leads to a loss in efficiency.

A simple example I personally encountered was writing blogs using the Markdown syntax. I know most of the basic syntax, but I was unaware of some of the advanced ones. I wasted around 5–10 minutes trying to figure it out. If I had gone to the documentation, instead, it would have saved me time.

Here is a Markdown cheat sheet in case you are wondering.

4. Don’t Reinvent the Wheel

If something is already available in the market, with great reliability, use it. Usually speaking the pros will outweigh the cons.

As an example, you should never write your own cryptographic function or library. There are pretty decent libraries available in almost all languages, and they will save you lots of time. More importantly, it will have the correct implementation, and it will be well optimised

But don’t overdo it. You shouldn’t import underscore or lodash just to loop through an array. It only increases your package size and hurts the user experience.

5. Testing

Test-Driven Development (TDD) is another methodology that is going strong. Writing tests is as important as writing code. Though manual testing is required, automated tests provide confidence that your system will not break logically.

Initially, it might feel that you are spending more time writing tests. But as the project grows, it would be worth it. Fewer bugs and more robustness.

Here is an article on how to do TDD in .NET Core.

Conclusion

I, personally, felt a lot of different emotions when following the above tips, but they have improved my efficiency a lot, especially creating a to-do list for the day.

XCode vs Android Studio: Which is better?

Arjav Dave — Tue, 20 Apr 2021 05:04:44 +0000

What makes me eligible to review!

I have been in the industry for more than 11+ years now. I started my career with BlackBerry (BB) Development. I feel old already!

Eclipse was our best friend back then for mobile development. I did my internship working on the famous Zagat app for BlackBerry. It was overall a great learning experience.

For my full-time job I switched to a start up named Spinlet which I hope is still going strong. I worked as BlackBerry developer in the beginning but switched to iOS development eventually.

My iOS experience was nothing but exciting in those initial years. Eventually I started my own firm Royale Cheese with a friend that provides mobile design & development.

After around 4 years of iOS development I found my way into Android development. I have got a fair share of experience with Android development as well, around 3 years to be precise.

We have since then been working on full stack. That's when I realised how horrible the mobile development tools are.

Enough with the chit chat. Here's an honest review of the mobile development tools and technologies.

XCode

It used to be a good tool in the past. But it has become terrible lately. Here is a list of all the issues even after 15 years:

Auto-completion: Firstly, who in the right mind would set Esc as the suggestions key. Secondly, the autocompletion doesn't work many a times or gives weird suggestions that are out of context.
Build Times: It takes a lot of time to create an archive or to run on a device for the first time. It's best to setup a CI/CD to archive and upload builds.
Signing & Deployment: With the latest version's it's getting easier. But, it's still confusing with the signing certificates and the provision profiles for someone who is a beginner.
Memory Hog: Somehow XCode keeps on hogging memory. For every new device on which the build needs to run it occupies 3 GB. Archives take a huge chunk and so as the simulators. Overall it occupies around 50GB if I don't clean up regularly.
Updates: Each update is around 10-12GB even the minor upgrades. What's worse is it requires more than 40GB of free space to get installed. Last but not the least, XCode takes around 12GB of the space.
Design: Initially to design UI there was struts & springs, then came the Autolayout and now the SwiftUI. It is worrisome that the methodology keeps on changing.
Cocoapods: is getting worse day by day because it's repo is getting so big. Secondly, it increases the build times by a lot.
Camera: Possibly due to hardware limitations camera was not supported in simulators. But, now it's been a while. If the location can be simulated why not the Camera?

Android Studio

I haven't got the chance to use it in the early days. But from what I have been seeing it wouldn't have been pretty. Here are some of the frustrating issues:

Fragmentation: I feel this might be on top of every ones list. Supporting the staggering amount of devices to support might just overwhelm anyone.
Gradle: Oh My God! Gradle takes forever to run builds. There are optimisation's which can help alleviate the problem, but it still remains the problem.
RAM Hogger: With emulators and IDE running together, the combo requires around 10-12GB of RAM. That's way more than what their official documentation says: 4GB.
Signing Keys: You lose your signing keys and you can't upload to the same app again. You will have to create a new app and get the reviews and downloads again. There is some improvement in this area recently.
IDE: I always felt Java based IDE's to be clumsy. That includes IntelliJ (on which Android Studio is based), Eclipse or NetBeans. It's responsiveness is not like other tools like XCode or VS Code.

XCode vs Android Studio Review

Even though it might feel Android has less issues it does have some serious issues. Fragmentation and Gradle might alone be enough to make Android look bad.

Personally I prefer to work in XCode compared to Android since I own a Mac and probably I am more used to it.

Conclusion

Overall both platforms have a huge user base. But, I feel that these tools still have a long way to go.

As an example I am absolutely in love how Microsoft has revamped dotnet to dotnet core and their IDE to VS Code.

Jot down your frustrations in the comments below.

Read some of my articles here:

Learn Test-Driven Development with Integration Tests in .NET 5.0

Arjav Dave — Wed, 14 Apr 2021 18:00:20 +0000

TDD (Test Driven Development) is a much debated word in the tech industry. Debates like Whether you should do TDD or not? or How advantageous is it? are quite popular. Simply said, TDD is write tests before you develop.

Now, there are a lot of school of thoughts regarding what type of test's are included and what are not in TDD. As an example, should it include Unit Test, Integration Test, System Test or even UAT?

In this article, we will go through a real-world example on how to write integration tests in .NET 5.0 with TDD methodology.

Project Requirements

TDD requires a very clear understanding of scope of work. Without clarity, all the test cases might not be covered.

Let's define the scope of work. We will be developing a patient admission system for a Hospital.

Business Requirements

A hospital has X ICU rooms, Y Premium rooms & Z General rooms.
ICU & Premium rooms can have a single patient at a time, while General rooms can have 2 patients. Each room has a room number.
On admitting, the patient has to provide name, age, gender & phone number.
It is possible to search a patient via name or phone number.
Same patient cannot be admitted to multiple beds while he is still checked in.
A patient cannot be admitted if all the rooms are occupied.

Model Validation Rules

Based on the above requirements, there are 2 models namely Patient & Room.

A patient's age is between 0 & 150. The length of name should be between 2 and 40. Gender can be male, female & other. Phone Number's length should be between 7 and 12 and it should all be digits.
Room type can be either "ICU", "Premium" or "General".

Test Cases

Now, that we have defined rules & requirements, lets start creating test cases. Since it's a basic CRUD application we mostly have integration tests.

Patient

Do all the model validation tests.
Admit the same patient twice
Check out the same patient twice.
Admit the same patient to multiple rooms at the same time.
Search a patient with phone number and name.

TDD Setup

In the above section we gathered requirements. Secondly, we defined the models. Finally, we created the list of test cases which we will implement.

Open your terminal and run the below script to create and setup a new project.

mkdir TDD
cd TDD
dotnet new sln
dotnet new webapi --name TDD
dotnet new xunit --name TDD.Tests
cd TDD
dotnet add package Microsoft.EntityFrameworkCore --version 5.0.5
cd ../TDD.Tests
dotnet add reference ../TDD/TDD.csproj
dotnet add package Microsoft.EntityFrameworkCore --version 5.0.5
dotnet add package Microsoft.AspNetCore.Hosting --version 2.2.7
dotnet add package Microsoft.AspNetCore.Mvc.Testing --version 5.0.5
dotnet add package Microsoft.EntityFrameworkCore.InMemory --version 5.0.5
cd ..
dotnet sln add TDD/TDD.csproj
dotnet sln add TDD.Tests/TDD.Tests.csproj
code .

The above script creates a solution file named TDD.sln. Secondly, we create 2 projects for TDD & TDD.Tests. Then we add the dependencies for each project. Lastly, we add the projects to the solution and open the project in VS Code.

Before we start testing, some more setup is required. Basically, integration tests test the a specific module without mocking. So we will be mimicking our application via TestServer.

Custom WAF

In order to mimic the TestServer there is a class called WebApplicationFactory (WAF) which bootstraps the application in memory.

In your TDD.Tests project create a file named PatientTestsDbWAF.cs with the following code.

using System.Linq;
using Microsoft.EntityFrameworkCore;
using Microsoft.AspNetCore.Hosting;
using Microsoft.AspNetCore.Mvc.Testing;
using Microsoft.Extensions.DependencyInjection;
using Microsoft.AspNetCore;


namespace TDD.Tests
{
    public class PatientTestsDbWAF<TStartup> : WebApplicationFactory<TStartup> where TStartup : class
    {

        protected override IWebHostBuilder CreateWebHostBuilder()
        {
            return WebHost.CreateDefaultBuilder()
                .UseStartup<TStartup>();
        }
        protected override void ConfigureWebHost(IWebHostBuilder builder)
        {
            builder.ConfigureServices(async services =>
           {
               // Remove the app's DbContext registration.
               var descriptor = services.SingleOrDefault(
                      d => d.ServiceType ==
                          typeof(DbContextOptions<DataContext>));

               if (descriptor != null)
               {
                   services.Remove(descriptor);
               }

               // Add DbContext using an in-memory database for testing.
               services.AddDbContext<DataContext>(options =>
                  {
                      // Use in memory db to not interfere with the original db.
                      options.UseInMemoryDatabase("PatientTestsTDD.db");
                  });
           });
        }
    }
}

We are removing the applications DbContext and adding an in memory DbContext. It is a necessary step since we don't want to interfere with the original database.

Secondly, we are initialising the database with some dummy data.

Since, DataContext is a custom class, it will give compiler error. So, we need to create it.

Data Context

Therefore, in your TDD project, create a file named DataContext.cs with the following code.

using Microsoft.EntityFrameworkCore;

namespace TDD
{
    public class DataContext : DbContext
    {
        public DataContext(DbContextOptions options) : base(options) { }

        // For storing the list of patients and their state
        public DbSet<Patient> Patient { get; set; }

        // For the storying the rooms along with their types and capacity
        public DbSet<Room> Room { get; set; }

        // For logging which patients are currently admitted to which room
        public DbSet<RoomPatient> RoomPatient { get; set; }

    }
}

Here Patient, Room & RoomPatient are Entity classes with the required properties, which we will create next.

Patient

Again, in your TDD project, create a file named Patient.cs and paste in the code below.

using System;
using System.ComponentModel.DataAnnotations;
using System.ComponentModel.DataAnnotations.Schema;

namespace TDD
{
    public class Patient
    {
        [Key]
        [DatabaseGenerated(DatabaseGeneratedOption.Identity)]
        public int Id { get; set; }

        public String Name { get; set; }

        public String PhoneNumber { get; set; }

        public int Age { get; set; }

        public String Gender { get; set; }
    }
}

Room

Create another file named Room.cs with the following code.

using System;
using System.ComponentModel.DataAnnotations;
using System.ComponentModel.DataAnnotations.Schema;

namespace TDD
{
    public class Room
    {
        [Key]
        [DatabaseGenerated(DatabaseGeneratedOption.Identity)]
        public int Id { get; set; }

        public String RoomType { get; set; }

        public int CurrentCapacity { get; set; }

        public int MaxCapacity { get; set; }
    }
}

RoomPatient

Create the last model file RoomPatient.cs with the following code.

using System.ComponentModel.DataAnnotations;
using System.ComponentModel.DataAnnotations.Schema;

namespace TDD
{
    public class RoomPatient
    {
        [Key]
        [DatabaseGenerated(DatabaseGeneratedOption.Identity)]
        public int Id { get; set; }

        [Required]
        public int RoomId { get; set; }

        [ForeignKey("RoomId")]
        public Room Room { get; set; }

        [Required]
        public int PatientId { get; set; }

        [ForeignKey("PatientId")]
        public Patient Patient { get; set; }
    }
}

Now you shouldn't be getting any compiler error.

Lastly, remove the WeatherForecast.cs and WeatherForecastController.cs files.

Go to your terminal in VS Code and run the below command.

cd TDD.Tests
dotnet test

You will see a nice green result which says 1 test passed.

Test Success

Patient Controller

Unfortunately dotnet doesn't provide a way to directly test the model's in itself. So, we will have to create a controller to test it.

Go ahead and create PatientController.cs in the Controllers folder in TDD project with the below code.

using Microsoft.AspNetCore.Mvc;

namespace TDD.Controllers
{
    [Route("api/[controller]")]
    [ApiController]
    public class PatientController : Controller
    {
        [HttpPost]
        public IActionResult AddPatient([FromBody] Patient Patient)
        {
            // TODO: Insert the patient into db
            return Created("/patient/1", Patient);
        }
    }
}

We created an api to add a patient. In order to test our model we will call this api.

That is all the things required to start testing.

Model Validation Tests

Since, we have setup the basic code for testing, let's write a test that fails. We will start our testing with the model validation tests.

Failing (Red) State

Let's create a new file named PatientTests.cs in your TDD.Tests project and delete the file named UnitTest1.cs. Copy the below code in your file.

using System.Net;
using System.Net.Http;
using System.Threading.Tasks;
using Xunit;
using System.Text;
using System.Text.Json;
using Microsoft.Extensions.DependencyInjection;
using Microsoft.AspNetCore.Mvc.Testing;
using System;
using System.Collections.Generic;
using Microsoft.EntityFrameworkCore;

namespace TDD.Tests
{
    public class PatientTests : IClassFixture<PatientTestsDbWAF<Startup>>
    {
        // HttpClient to call our api's
        private readonly HttpClient httpClient;
        public WebApplicationFactory<Startup> _factory;

        public PatientTests(PatientTestsDbWAF<Startup> factory)
        {
            _factory = factory;

            // Initiate the HttpClient
            httpClient = _factory.CreateClient();
        }

        [Theory]
        [InlineData("Test Name 2", "1234567891", 20, "Male", HttpStatusCode.Created)]
        [InlineData("T", "1234567891", 20, "Male", HttpStatusCode.BadRequest)]
        [InlineData("A very very very very very very loooooooooong name", "1234567891", 20, "Male", HttpStatusCode.BadRequest)]
        [InlineData(null, "1234567890", 20, "Invalid Gender", HttpStatusCode.BadRequest)]
        [InlineData("Test Name", "InvalidNumber", 20, "Male", HttpStatusCode.BadRequest)]
        [InlineData("Test Name", "1234567890", -10, "Male", HttpStatusCode.BadRequest)]
        [InlineData("Test Name", "1234567890", 20, "Invalid Gender", HttpStatusCode.BadRequest)]
        [InlineData("Test Name", "12345678901234444", 20, "Invalid Gender", HttpStatusCode.BadRequest)]
        public async Task PatientTestsAsync(String Name, String PhoneNumber, int Age, String Gender, HttpStatusCode ResponseCode)
        {
            var scopeFactory = _factory.Services;
            using (var scope = scopeFactory.CreateScope())
            {
                var context = scope.ServiceProvider.GetService<DataContext>();

                // Initialize the database, so that 
                // changes made by other tests are reset. 
                await DBUtilities.InitializeDbForTestsAsync(context);

                // Arrange
                var request = new HttpRequestMessage(HttpMethod.Post, "api/patient");

                request.Content = new StringContent(JsonSerializer.Serialize(new Patient
                {
                    Name = Name,
                    PhoneNumber = PhoneNumber,
                    Age = Age,
                    Gender = Gender
                }), Encoding.UTF8, "application/json");

                // Act
                var response = await httpClient.SendAsync(request);

                // Assert
                var StatusCode = response.StatusCode;
                Assert.Equal(ResponseCode, StatusCode);
            }
        }
    }
}

[Theory] attribute allows us to mention different parameters for our tests. Consequently, we don't have to write different tests for all the combinations.

Also, DBUtilities is a utility class to reinitialise the database to it's initial state. This might seem trivial when we have 1 or 2 tests but, gets critical as we add more tests.

DBUtilities

The DBUtilities class will initialise your database with 1 patient and 3 different type of rooms.

Create a file named DBUtilities.cs in your TDD.Tests project with the below code.

using System.Threading.Tasks;

namespace TDD.Tests
{
    // Helps to initialise the database either from the WAF for the first time
    // Or before running each test.
    public class DBUtilities
    {

        // Clears the database and then,
        //Adds 1 Patient and 3 different types of rooms to the database
        public static async Task InitializeDbForTestsAsync(DataContext context)
        {
            context.RoomPatient.RemoveRange(context.RoomPatient);
            context.Patient.RemoveRange(context.Patient);
            context.Room.RemoveRange(context.Room);

            // Arrange
            var Patient = new Patient
            {
                Name = "Test Patient",
                PhoneNumber = "1234567890",
                Age = 20,
                Gender = "Male"
            };
            context.Patient.Add(Patient);

            var ICURoom = new Room
            {
                RoomType = "ICU",
                MaxCapacity = 1,
                CurrentCapacity = 1
            };
            context.Room.Add(ICURoom);

            var GeneralRoom = new Room
            {
                RoomType = "General",
                MaxCapacity = 2,
                CurrentCapacity = 2
            };
            context.Room.Add(GeneralRoom);

            var PremiumRoom = new Room
            {
                RoomType = "Premium",
                MaxCapacity = 1,
                CurrentCapacity = 1
            };
            context.Room.Add(PremiumRoom);

            await context.SaveChangesAsync();
        }
    }
}

Go ahead and run the dotnet test command again and you will see 1 passed and 4 failed tests. This is because the 4 tests were expecting BadRequest but getting a Created result.

Failing (Red) State

Let's fix it!

Success (Green) State

In order to fix these we need to add attributes to our Patient.cs class.

Update the Patient.cs file as below.

using System;
using System.Collections.Generic;
using System.ComponentModel.DataAnnotations;
using System.ComponentModel.DataAnnotations.Schema;

namespace TDD
{
    public class Patient : IValidatableObject
    {
        [Key]
        [DatabaseGenerated(DatabaseGeneratedOption.Identity)]
        public int Id { get; set; }

        [Required]
        [StringLength(40, MinimumLength = 2, ErrorMessage = "The name should be between 2 & 40 characters.")]
        public String Name { get; set; }

        [Required]
        [DataType(DataType.PhoneNumber)]
        [RegularExpression(@"^(\d{7,12})$", ErrorMessage = "Not a valid phone number")]
        public String PhoneNumber { get; set; }

        [Required]
        [Range(1, 150)]
        public int Age { get; set; }

        [Required]
        public String Gender { get; set; }

        public Boolean IsAdmitted { get; set; }

        public IEnumerable<ValidationResult> Validate(ValidationContext validationContext)
        {
            // Only Male, Female or Other gender are allowed
            if (Gender.Equals("Male", System.StringComparison.CurrentCultureIgnoreCase) == false &&
                Gender.Equals("Female", System.StringComparison.CurrentCultureIgnoreCase) == false &&
                Gender.Equals("Other", System.StringComparison.CurrentCultureIgnoreCase) == false)
            {
                yield return new ValidationResult("The gender can either be Male, Female or Other");
            }

            yield return ValidationResult.Success;
        }
    }
}

Here, we have added the required attributes. We have also implemented the IValidatableObject interface so that we can verify the Gender.

Time to run the dotnet test command. You will see a nice green line saying 5 tests passed.

You can add more edge case scenarios in the InlineData to test the Patient model validation tests thoroughly.

Duplicate Patient Test

We shall now create a test which fails when we try to add a duplicate patient.

Failing (Red) Test

Create another test in your class PatientTests. Add the below code.

[Fact]
public async Task PatientDuplicationTestsAsync()
{
    var scopeFactory = _factory.Services;
    using (var scope = scopeFactory.CreateScope())
    {
        var context = scope.ServiceProvider.GetService<DataContext>();
        await DBUtilities.InitializeDbForTestsAsync(context);

        // Arrange
        var Patient = await context.Patient.FirstOrDefaultAsync();

        var Request = new HttpRequestMessage(HttpMethod.Post, "api/patient");
        Request.Content = new StringContent(JsonSerializer.Serialize(Patient), Encoding.UTF8, "application/json");

        // Act
        var Response = await httpClient.SendAsync(Request);

        // Assert
        var StatusCode = Response.StatusCode;
        Assert.Equal(HttpStatusCode.BadRequest, StatusCode);
    }
}

We have used a [Fact] attribute instead of [Theory] attribute here since we don't want to test the same method with different parameters. Instead, we want to make the same request twice.

Run dotnet test to run our newly created test. The test will fail with message Assert.Equal() Failure. Time to fix it.

Success (Green) Test

To fix the failing test we need to add the implementation for the AddPatient method in PatientController.cs. Update the file's code as below.

using System.Threading.Tasks;
using Microsoft.AspNetCore.Mvc;
using Microsoft.EntityFrameworkCore;

namespace TDD.Controllers
{
    [Route("api/[controller]")]
    [ApiController]
    public class PatientController : Controller
    {
        private readonly DataContext _context;

        public PatientController(DataContext context)
        {
            _context = context;
        }
        [HttpPost]
        public async Task<IActionResult> AddPatientAsync([FromBody] Patient Patient)
        {
            var FetchedPatient = await _context.Patient.FirstOrDefaultAsync(x => x.PhoneNumber == Patient.PhoneNumber);
            // If the patient doesn't exist create a new one
            if (FetchedPatient == null)
            {
                _context.Patient.Add(Patient);
                await _context.SaveChangesAsync();
                return Created($"/patient/{Patient.Id}", Patient);
            }
            // Else throw a bad request
            else
            {
                return BadRequest();
            }
        }
    }
}

Run the dotnet test again and you will see that the test has passed.

You can run all the tests by calling dotnet test.

Important Notes

As you add more models/domains like Doctors, Staff, Instruments etc. You will have to create more tests. Make sure to have a different WAF, utility wrappers and different Test files for each of them.

Secondly, the tests in the same file do not run in parallel. But, the tests from different files do run in parallel. Therefore, each WAF should have a different database name so that data is not misconfigured.

Lastly, the connections to the original database still needs to be setup in the main project.

Thought Process

The thought process for creating tests for all scenarios are similar.

That is, you should first identify the requirements. Then, set up a skeleton of methods and classes without implementation. Write tests to verify the implementation. Finally, refactor as needed and rerun the tests.

This tutorial didn't include authentication and authorisation for api's. You can read here on how to set it up.

Since, it is not possible to cover all the test cases, I have created a repository on Github. It covers the implementation for all the test cases and the implementation as well.

You can find the project here.

Conclusion

In order for TDD to be effective you really need to have a clear idea of what the requirements are. If the requirements keep on changing it would get very tough to maintain the tests as well as the project.

TDD mainly covers unit, integration & functional tests. You will still have to do UAT, Configuration & Production testing before you go live.

Having said that, TDD is really helpful in making your project bug free. Secondly, it boosts your confidence for the implementation. You will be able to change bits & pieces of your code as long as the tests pass. Lastly, it provides a better architecture for your project.

Hope you like the article. Let me know your thoughts or feedback.

Check more tutorials on .NET here.

Just let me login! Going Password less with .NET

Arjav Dave — Tue, 06 Apr 2021 06:40:00 +0000

Passwords are a pain in the a..

With thousand's of softwares and app's launching everyday, you would like to make your software stand out. Most importantly, it should have some unique USP, but in addition it should provide convenience and ease of use.

For instance, one of the pain points for many apps is they require a username and a password to login. I personally have to remember 10-15 passwords for apps like Gmail, Facebook, Instagram, etc. You get the idea.

In today's article we are going to create a solution for your API's that will allow your users to login without a password.

The How of Going Password less

In order to omit the password there needs to be some type of token generated for a user.

This token will then be sent to a user where only the user can access it e.g. on email or phone number. Here is an overview of the flow.

Dotnet Identity provides ways to generate tokens for email confirmation or changing email or phone. We will see more about it below.

There are mainly two token providers available

TotpSecurityStampBasedTokenProvider (Time-based One Time Password).
DataProtectionTokenProvider

TotpSecurityStampBasedTokenProvider

It generates time based tokens which are valid for around 3 minutes (Reference: Source Code). Based on the token provider the tokens are generated from the Email, PhoneNumber or user's id as well as the user's security stamp.

Dotnet Identity provides utility classes EmailTokenProvider and PhoneNumberTokenProvider that are subclasses of TotpSecurityStampBasedTokenProvider.

DataProtectorTokenProvider

If you want to generate a token that doesn't expire for a long duration DataProtectorTokenProvider is the way to go.

DataProtectorTokenProvider generates tokens using a DataProtector and cryptographic algorithms. You can check out the implementation for more details here.

In this article we are going to subclass DataProtectorTokenProvider so that our token is valid for 10 minutes.

Setting up Identity

The .NET Identity provides ways to manage users, passwords, profile data, roles, claims, tokens, email confirmation, and more.

Let's start with a scratch project. Create a new project by executing the command dotnet new webapi –-name NoPasswordProject.

dotnet add package Microsoft.EntityFrameworkCore.InMemory --version 5.0.4
dotnet add package Microsoft.AspNetCore.Identity.EntityFrameworkCore --version 5.0.4

We are going to create an in memory database for this tutorial. But you can use a database of your choice and accordingly change the package above.

Note: The in memory database will clear the users every time the server restarts.

Custom Token Provider

Let's create a custom token provider that generates token that are valid for 10 minutes.

NPTokenProvider

Create a new file called NPTokenProvider.cs. NP prefix stands for No Password.

using Microsoft.AspNetCore.DataProtection;
using Microsoft.AspNetCore.Identity;
using Microsoft.Extensions.Logging;
using Microsoft.Extensions.Options;

public class NPTokenProvider<TUser> : DataProtectorTokenProvider<TUser>
where TUser : IdentityUser
{
    public NPTokenProvider(
        IDataProtectionProvider dataProtectionProvider,
        IOptions<NPTokenProviderOptions> options, ILogger<NPTokenProvider<TUser>> logger)
        : base(dataProtectionProvider, options, logger)
    { }
}

We are here subclassing the DataProtectorTokenProvider. Nothing out of the ordinary except in the constructor we are passing NPTokenProviderOptions. The options need to be subclass of DataProtectionTokenProviderOptions.

NPTokenProviderOptions

Create a new file NPTokenProviderOptions.cs and paste in the below code.

using System;
using Microsoft.AspNetCore.Identity;

public class NPTokenProviderOptions : DataProtectionTokenProviderOptions
{
    public NPTokenProviderOptions()
    {
        Name = "NPTokenProvider";
        TokenLifespan = TimeSpan.FromMinutes(10);
    }
}

We are setting options for the tokens to be created. You can change the Name and TokenLifeSpan to your liking.

DbContext

Almost every project needs database to store it's users and other data related to the project. Dotnet EF Framework provides a nice helper DbContext to handle sessions with the database and query and save entites. So create a subclass of IdentityDbContext which is in turn a subclass of DbContext. Name the file NPDataContext.cs.

using Microsoft.AspNetCore.Identity.EntityFrameworkCore;
using Microsoft.EntityFrameworkCore;

public class NPDataContext : IdentityDbContext
{
    public NPDataContext(DbContextOptions<NPDataContext> options)
        : base(options)
    { }
}

Startup.cs

We have created the classes; now time to configure them in our Startup.cs files. In ConfigureServices add the below code at the start.

var builder = services
.AddIdentityCore<IdentityUser>()
.AddEntityFrameworkStores<NPDataContext>();

var UserType = builder.UserType;
var provider = typeof(NPTokenProvider<>).MakeGenericType(UserType);
builder.AddTokenProvider("NPTokenProvider", provider);

services.AddDbContext<NPDataContext>(options =>
    options.UseInMemoryDatabase(Guid.NewGuid().ToString()));

services.AddAuthentication(options =>
{
    options.DefaultScheme = IdentityConstants.ExternalScheme;
});

Also add app.UseAuthentication(); above app.UseAuthorization(); in Configure method.

NoPasswordController.cs

Let's create a controller for our login and verify API's. Create a NoPasswordController.cs file in your Controllers folder. Add the below content to the file.

using System;
using System.Threading.Tasks;
using Microsoft.AspNetCore.Identity;
using Microsoft.AspNetCore.Mvc;

namespace NoPasswordProject.Controllers
{
    [ApiController]
    [Route("[controller]/[action]")]
    public class NoPasswordController : ControllerBase
    {
        private readonly UserManager<IdentityUser> _userManager;

        public NoPasswordController(UserManager<IdentityUser> userManager)
        {
            _userManager = userManager;
        }
    }
}

We are injecting an instance of UserManager in our controller. UserManager is used for CRUD operations for a user as well as generating tokens and validating it.

Login API

Let's add a Login api which accepts an Email as input. The Email is the unique identifier for a user i.e. there should be a one-to-one relation ship between user and email.

Create a new function in your controller as below.

[HttpGet]
public async Task<ActionResult<String>> Login([FromQuery] string Email)
{
    // Create or Fetch your user from the database
    var User = await _userManager.FindByNameAsync(Email);
    if (User == null)
    {
        User = new IdentityUser();
        User.Email = Email;
        User.UserName = Email;
        var IdentityResult = await _userManager.CreateAsync(User);
        if (IdentityResult.Succeeded == false)
        {
            return BadRequest();
        }
    }

    var Token = await _userManager.GenerateUserTokenAsync(User, "NPTokenProvider", "nopassword-for-the-win");

    // DON'T RETURN THE TOKEN.
    // SEND IT TO THE USER VIA EMAIL.
    return NoContent();
}

Here we are fetching a User from the database. If the user doesn't exist then we create a user. Make sure to set the UserName as well or it will give runtime error.

Then based on the user, we generate a UserToken. The GenerateUserTokenAsync takes the user, token provider and the purpose for generating a token.

The token provider string should be the one you have used in NPTokenProviderOptions. The purpose can be anything you want.

Send out the token to the user via a link in a nicely designed email. When the user will click on the link in the email it will open your front-end page. Consequently this page will request the Verify api.

Verify API

Let's add another api Verify that takes the Email and Token as query parameters.

[HttpGet]
public async Task<ActionResult<String>> Verify([FromQuery] string Token, [FromQuery] string Email)
{
    // Fetch your user from the database
    var User = await _userManager.FindByNameAsync(Email);
    if (User == null)
    {
        return NotFound();
    }

    var IsValid = await _userManager.VerifyUserTokenAsync(User, "NPTokenProvider", "nopassword-for-the-win", Token);
    if (IsValid)
    {
        // TODO: Generate a bearer token
        var BearerToken = "";
        return BearerToken;
    }
    return Unauthorized();
}

We are again fetching the user based on email. As a result if we are not able to find the user return 404 Not Found.

We then continue to verify the user. VerifyUserTokenAsync takes user, token provider, purpose and token as input parameters. The purpose should be same as the one used while generating token.

If the token is not valid, return 401 Unauthorised. Otherwise return the bearer token. This is a good article on how to generate bearer token for the user.

In conclusion our article explains on how to allow users to login without a password. You can find the whole project here

Conclusion

Providing features was supposed to be the most important thing in the '90s. But today besides having great features, the convenience for the users is a priority.

We looked at one of the ways of providing convenience. That said, let us know in the comments below for more ways to do it.

Check here for more tutorials like this.

.NET 5: How to authenticate & authorise API's correctly

Arjav Dave — Thu, 01 Apr 2021 05:21:53 +0000

In over 11 years of my experience I have seen so many API's that have major security flaw. They either lack a proper setup of Authentication or Authorisation or both. The developers might feel okay since these endpoints are usually not public. But it is a huge security loop hole which anyone can easily target.

To better understand security for API's let's create a demo project for FBI. There will be an Admin who can enrol FBI Agents and change their clearance levels. Secondly FBI Agents with Clearance Level 1 will be able to access public files and agents with Clearance Level 2 will be able to access pubic & classified files.

First some theory! Not Interested? Take me to the code.

Authentication

Our Agent has successfully cleared all his exams; time to enrol him. In order to do that he will provide his documents and in return will get his badge.

In the above scenario providing documents is like login where once verified he will be provided with a token (badge). This process is called Authentication. It determines whether agents are who they claim to be.

We are going to use Json Web Tokens (JWT) Bearer tokens for authentication. Bearer tokens are a type of tokens generated by servers which contain details of the claims/roles of a user trying to login. Bearer tokens are mostly structured tokens like JWT. Read here to know more about JWT.

Authorisation

Now since the FBI Agent has got his badge he can enter the FBI building. He is also able to access public files, but when trying to access classified files he gets 401 error.

This is because FBI Agent is not authorised to access classified files. Authorisation determines what agents can and cannot access.

As mentioned above the JWT Bearer token contains claims/roles. Based on it, our server decides whether to give access to a private resource or not.

Access Flow

As you can see in the above diagram on successful login the server returns a Bearer token. The client uses the bearer token in subsequent calls to access a private resource.

These are the two main concepts that we are going to implement in our article.

Enough with the theory, show me some code!

Project Setup

Create a new project by executing the command dotnet new webapi --name FBI from your cli. It will create a project with a sample WeatherForecast api.

Why work on WeatherForecast when we can work on FBI. Go ahead and delete WeatherForecast.cs file.

Add dependencies by executing the commands

dotnet add package Microsoft.IdentityModel.Tokens --version 6.9.0
dotnet add package Microsoft.AspNetCore.Authentication.JwtBearer --version 5.0.4

In ConfigureServices function in your Startup.cs file add the below code.

var TokenValidationParameters = new TokenValidationParameters
{
    ValidIssuer = "https://fbi-demo.com",
    ValidAudience = "https://fbi-demo.com",
    IssuerSigningKey = new SymmetricSecurityKey(Encoding.UTF8.GetBytes("SXkSqsKyNUyvGbnHs7ke2NCq8zQzNLW7mPmHbnZZ")),
    ClockSkew = TimeSpan.Zero // remove delay of token when expire
};

We are defining the parameters for validating a token. Make sure that the length of the string for generating SymmetricSecurityKey is 32.

Next, setup the services to add authentication for API's.

services
    .AddAuthentication(options =>
    {
        options.DefaultScheme = JwtBearerDefaults.AuthenticationScheme;
    })
    .AddJwtBearer(cfg =>
    {
        cfg.TokenValidationParameters = TokenValidationParameters;
    });

The AddAuthentication method registers services required by authentication services. It also configures JWT Bearer Authentication as the default scheme.

The AddJwtBearer enables JWT-bearer authentication and setting the TokenValidationParameters defined above.

Now let's add some Authorisation claims for our Agent & Admin.

services.AddAuthorization(cfg =>
    {
        cfg.AddPolicy("Admin", policy => policy.RequireClaim("type", "Admin"));
        cfg.AddPolicy("Agent", policy => policy.RequireClaim("type", "Agent"));
        cfg.AddPolicy("ClearanceLevel1", policy => policy.RequireClaim("ClearanceLevel", "1", "2"));
        cfg.AddPolicy("ClearanceLevel2", policy => policy.RequireClaim("ClearanceLevel", "2"));
    });

The AddAuthorization method registers services required for authorisation. We are also adding claims for Admin, Agent, ClearanceLevel1 and ClearanceLevel2 by calling AddPolicy. A claim is a name value pair that represents what the subject is. Since clearance level 2 can also access clearance level 1 we have put "1", "2" in ClearanceLevel1. You can read more about claims here.

Lastly in the Configure method add the below line just above app.UseAuthorization();

app.UseAuthentication();

Admin Controller

Rename your file WeatherForecastController.cs to AdminController.cs. Do change the class name and constructor names as well. Finally, remove everything except the constructor.

using Microsoft.AspNetCore.Mvc;

namespace FBI.Controllers
{
    [ApiController]
    [Route("[controller]")]
    public class AdminController : ControllerBase
    {
        public AdminController() { }
    }
}

Login API

Let's create a login API for Admin so that she can get a token to perform other tasks.

[HttpPost]
[Route("[action]")]
public IActionResult Login([FromBody] User User)
{
    // TODO: Authenticate Admin with Database
    // If not authenticate return 401 Unauthorized
    // Else continue with below flow

    var Claims = new List<Claim>
            {
                new Claim("type", "Admin"),
            };

    var Key = new SymmetricSecurityKey(Encoding.UTF8.GetBytes("SXkSqsKyNUyvGbnHs7ke2NCq8zQzNLW7mPmHbnZZ"));

    var Token = new JwtSecurityToken(
        "https://fbi-demo.com",
        "https://fbi-demo.com",
        Claims,
        expires: DateTime.Now.AddDays(30.0),
        signingCredentials: new SigningCredentials(Key, SecurityAlgorithms.HmacSha256)
    );

    return new OkObjectResult(new JwtSecurityTokenHandler().WriteToken(Token));
}

In the above code User is a model with properties Username & Password. We are also creating an object of JwtSecurityToken using configurations that we have used in Startup.cs file. The token is then converted to string and returned in an OkObjectResult.

You can now open Swagger and execute the API to see a bearer token. A bearer token will be returned as you can see below.

Keep the token handy since we are going to use it in the next section. You can also visit https://jwt.io to analyse your token.

Generate Badge API

Generating badge for an Agent is a sensitive task and should only be Authorised by an Admin. We are going to add an Authorize attribute for the GenerateBadge api.

[HttpPost]
[Route("[action]")]
[Authorize(Policy = "Admin")]
public IActionResult GenerateBadge([FromBody] Agent Agent)
{
    var Claims = new List<Claim>
    {
        new Claim("type", "Agent"),
        new Claim("ClearanceLevel", Agent.ClearanceLevel.ToString()),
    };

    var Key = new SymmetricSecurityKey(Encoding.UTF8.GetBytes("SXkSqsKyNUyvGbnHs7ke2NCq8zQzNLW7mPmHbnZZ"));

    var Token = new JwtSecurityToken(
        "https://fbi-demo.com",
        "https://fbi-demo.com",
        Claims,
        expires: DateTime.Now.AddDays(30.0),
        signingCredentials: new SigningCredentials(Key, SecurityAlgorithms.HmacSha256)
    );

    return new OkObjectResult(new JwtSecurityTokenHandler().WriteToken(Token));
}

Here Agent is a model with properties Name as string and ClearanceLevel as int.

Now when you go back to swagger and try to execute GenerateBadge api it will give you 401 Unauthorised response. Since we have not passed the bearer token we are getting this error.

To be able to add the Authorize header in Swagger change the services.AddSwaggerGen as below:

services.AddSwaggerGen(c =>
{
    c.SwaggerDoc("v1", new OpenApiInfo { Title = "FBI", Version = "v1" });
    c.AddSecurityDefinition("Bearer", new OpenApiSecurityScheme
    {
        In = ParameterLocation.Header,
        Description = "Please enter JWT with Bearer into field",
        Name = "Authorization",
        Type = SecuritySchemeType.ApiKey
    });
    c.AddSecurityRequirement(new OpenApiSecurityRequirement {
    { new OpenApiSecurityScheme
            {
                Reference = new OpenApiReference { Type = ReferenceType.SecurityScheme, Id = "Bearer"}
            },
        new string[] {}
    }
    });
});

When you refresh Swagger in your browser you will notice an Authorize button on the right side above the list of apis.

Click on the newly added Authorize button in Swagger which will open up a dialog. We need to mention what type of token it is. So first enter Bearer in the field then a space and then the token generated from the /Admin/Login api from previous section.

Click on the header to lock in the token. Now you are all set. When you execute the GenerateBadge api again you will get a token (analogous to badge). Keep this token handy, since we require in next section. Also make sure to pass ClearanceLevel as 1 for now.

Agent Controller

Create a new file AgentController.cs with below content.

using Microsoft.AspNetCore.Mvc;

namespace FBI.Controllers
{
    [ApiController]
    [Route("[controller]")]
    [Authorize(Policy = "Agent")]
    public class AgentController : ControllerBase
    {
        public AgentController() { }
    }
}

As you can see above we are authorising the whole controller for Agent's access only. So even Admin won't be able to access the API's we are going to create.

Access Records API's

Let's add the api's to access both public and classified files.

[HttpGet]
[Route("[action]")]
[Authorize(Policy = "ClearanceLevel1")]
public ActionResult<String> AccessPublicFiles()
{
    return new OkObjectResult("Public Files Accessed");
}

[HttpGet]
[Route("[action]")]
[Authorize(Policy = "ClearanceLevel2")]
public ActionResult<String> AccessClassifiedFiles()
{
    return new OkObjectResult("Classified Files Accessed");
}

We have added Authorize attribute's for both API's such that public files can be accessed by ClearanceLevel1 and classified files can be accessed by ClearanceLevel2.

If you try to access these API's with the Admin token you will get 403 Forbidden error. So go ahead and click on the Authorize button again and click on logout. Then, get the token from the above step and paste in the field with Bearer as a prefix i.e. Bearer .

Now when you access /Agent/AccessPublicFiles api you will see response 200 with message Public Files Accessed. But when you try the classified api you get 403 Forbidden error.

Changing Clearance Level

Fast forward 3 years and our Agent's performance has been mind bogglingly good. Management has now decided to promote him to ClearanceLevel2.

The Agent goes to the Admin and asks her to provide a token/badge with Clearance Level 2.

The Admin calls the /Admin/Login api to generate his own token first. She then enters it in the Authorize dialog.

/Admin/GenerageBadge api is then called by Admin with value 2 in the ClearanceLevel. This generates a new token/badge which she then hands over to Agent.

The Agent enters this token/badge in the Authorize dialog and when he now calls /Agent/AccessClassifiedFiles he is pleased to see the result Classified Files Accessed.

Conclusion

You can find the whole project here on github.

API security is extremely important and shouldn't be taken lightly even if it's for internal use only. Setup Authentication and Authorisation and you are halfway there.

There are other other security measures you can security against DDoS attacks, accepting API's from a particular IP or domain only etc.

How did you like the article? What are the other security measures do you usually take? Any feedbacks or comments?

You can checkout out more tutorials here.

DEV Community: ITNEXT

Deploy on AWS using React, Flask, and SQLite.

What you will get out of this

Possible limitations

Important:

Table Of Contents.

Introduction

Why this is useful

What you will need.

prerequisite skills.

Disclaimer

Part 1: Making the database.

Part 2: Making the flask App.

Part 3: Making the react app.

Part 4: Setting up an AWS account.

Part 5: Deploying to AWS.

Conclusion

5 gamified learning experiences to learn Web Dev

I wrote a free book on React

Links

Video course

Content

How YOU can build a GraphQL API with Apollo

Resources

Install and set up

Define a schema

Define resolvers

Define a resolver function with a parameter

Define a resolver function for a mutation

Putting it all together

Start the server

Debug

Example usage

Deeper queries

Store the data as id/ids

Define a resolver for a subtype

Summary

Go from the beginning - your first program

What is it used for though?

References

Features

Install Go

A Go program

The program in detail

Commands

Run your app

Build your app

Summary

Learn more

Instantly set up a new dev environment using containers and VS Code

TL;DR key takeaways

Why developing inside a container

Requirements

Getting started

Customizing the dev setup

VS Code settings and overrides

Sharing and updating the environment

Where's the catch?

What's next

Lazy Loading in Angular - A Beginner’s Guide

What is Lazy Loading?

What are NgModules?

How to Create NgModules

Create a Project

Root Module

Main screen

Lazy Loaded Modules

How to Verify that the Lazy Loading Worked

Use Cases

Conclusion

5 Ways to Increase Your Efficiency as a Developer

Journey of a Software Developer

Coder vs. Programmer vs. Architect: An Example

1. Pen and Paper

2. Become Your Own Manager

3. Documentation

4. Don’t Reinvent the Wheel

5. Testing

Conclusion

XCode vs Android Studio: Which is better?