DEV Community: Osiroski

Stage 1: The Mayor

Osiroski — Thu, 17 Mar 2022 16:42:04 +0000

Tide alert system

In Cloud Quest, there will be various people who will be asking for your help to solve their business problems.
The first person you meet is The Mayor, he has a problem that you should solve. You will understand those problems, gather requirements, learn cloud computing concepts and suggest them solutions.

The Challenge

The website that posts the predicted wave tides at the city's shore is offline. Surfers who desire high waves plus beach-goers who desire low waves don't know when to go to the beach.

This leads to low traffic at the beaches == low tourist income.

The Solution

Through further conversation the solution is to deploy a static website with AWS by uploading your website content into S3 bucket, configuring your bucket for website hosting and speeding up content delivery using AWS CloudFront.

Hosting a static website with AWS

Aws S3

Storage is the first AWS offering that Amazon offered. Storage therefore holds a significant place in the AWS ecosystem, including some
extremely innovative uses of its storage services by AWS customers
over the year
Simple Storage Service (fondly known as S3) is one of the richest, most flexible, and, certainly, most widely used AWS offerings. It’s no exaggeration to call S3 “the filing cabinet of the Internet.” Its object storage is used in an enormous variety of applications by individuals and businesses, such as:
✓ Dropbox
✓ Netflix
✓ Medcommons

Aws CloudFront

AWS CloudFront is a globally-distributed network offered by Amazon Web Services, which securely transfers content such as software, SDKs, videos, etc., to the clients, with high transfer speed.

Step 1 — Create an S3 bucket

Login into your AWS management console and click on Services on the top navbar.
Select S3 from the Storage section. This should display the S3 dashboard.
From the S3 dashboard, click on Create bucket. Give the bucket a unique name, the name you choose must be globally unique.
Next, choose your preferred AWS Region from the drop-down.
Under Block Public Access settings , uncheck the Block all public access checkbox and accept the acknowledgement and make the bucket accessible to the public because you are going to host a website in it.
Click on disable for Bucket Versioning.
You can also Add tag to the bucket for easy identification.
Under Default encryption section, click on disable for Server-side encryption.
Click on Create bucket.

Step 2 — Upload web files to S3 bucket

After creating the bucket, you need to upload the website’s files and folders into it. The files (index.html, main.js, style.css, data.csv) are provided in the lab section on the game.

From the S3 dashboard, click on the name of the bucket you just created.
On the Objects tab, you can see that the bucket is currently empty, click on the Upload button.
Click Add files to add the website files.
After the necessary files have been added, scroll down and click on Upload.

Step-3 — Security

Add some policies to secure your bucket.

From the S3 dashboard, click on the name of the bucket, then click on Permissions tab.
Scroll down to the Bucket policy section and click on its Edit button.
To grant public read access for your website, copy the following bucket policy, and paste it in the Bucket policy editor.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "PublicReadGetObject",
            "Effect": "Allow",
            "Principal": "*",
            "Action": [
                "s3:GetObject"
            ],
            "Resource": [
                "arn:aws:s3:::Bucket-Name/*"
            ]
        }
    ]
}

Update the Resource to your bucket name. In the preceding example bucket policy, Bucket-Name is a placeholder for the bucket name.
Choose Save changes. A message appears indicating that the bucket policy has been successfully added.

Step 4 — Configure S3 bucket

Specify the default page for your website.

From the S3 dashboard, click on the name of the bucket, then click on the Properties tab.
Scroll down to the Static website hosting section and click on its Edit button.
Select Enable for Static website hosting. Also, select Host a static website for the Hosting type.
Enter the file for your index.html document.
Scroll down and click on Save Changes.

Step 5 — Serve content from S3 bucket with CloudFront

Networking & Content Delivery

Click on CloudFront.
Click on Create Distribution.
On Select a delivery method for your content page,
Click on Get Started under the Web section.
Origin Settings section, click on the Origin Domain Name field and select the S3 bucket you created earlier. In the Origin Path field, enter / to indicate root level.
Restrict Bucket Access, select Yes.
Origin Access Identity, select Create a New Identity.
Grant Read Permissions on Bucket, select Yes
Update Bucket Policy.
Default Cache Behavior Settings section, Viewer Protocol Policy, select Redirect HTTP to HTTPS.
Distribution Settings section, Default Root Object field, enter the filename at the root level, which should be your landing page (index.html).
Leave the rest of the options as default and click on Create Distribution.

After the CloudFront distribution has been deployed, copy the URL from the Domain Name column and paste it into your browser.
You should now know how to host a static website with Amazon S3 and speed up the content delivery using AWS CloudFront.

Extra steps

Amazon Code Quest: Series walkthrough

Osiroski — Thu, 17 Mar 2022 15:12:53 +0000

Code Quest: Aws training 'metaverse'

AWS Code Quest

Cloud Practitioner is an all-new 3D role-playing game, designed by AWS Training and Certification, to help adult learners gain practical AWS experience.
You will be Cloud practitioner who helps citizens of the city by solving their problems through building cloud solutions.

Code Quest strategy

You are the master of your own pace and, and your technical training will be reinforced using various hands on labs, DIY challenges, rewards, puzzles and simulations.

The game helps you to think in cloud computing terms through following 4 basic steps:

Learn

Each challenge covers a specific knowledge domain in Aws. The game has resources and videos covering that section. You will learn more about the necessary procedures required to solve the challenge

Plan

Each challenge also requires a plan or architecture. In some solutions, the plan is already completed for you and requires optimization or restructuring.

Practice

A sandbox Aws console is provided to practice and implement the solution. This is awesome as it totally eliminates the possibility of accidentally incurring costs when using your own account. Sometimes we forget to turn off the EC2 instance!!
Also a step by step instructions are available with concept explanations.

DIY

This part requires us to implement what we have practiced without the guiding hand ✋ and steps. This stage has a validation submission to verify the results.

The game will enable individuals to step into the shoes of cloud professionals and cater highly available, scalable, cost effective, efficient cloud solutions; it is perfect learning tool for someone who has no experience with cloud computing.
Keep updated as I explore the game.

Server API Endpoints

Osiroski — Sat, 12 Mar 2022 15:01:23 +0000

Models

Before you jump in and start coding the API endpoints, it's worth taking a few minutes to think about what data we need to store and the relationships between the different objects.
Our User Model will have the usual fields

Username, password, email,
Profile object containing { 1st, 2nd names, avatar url, bio, phone,...}
passwordresetToken fields
jwt Token

Defining and creating models

Models are defined using the Schema interface. The Schema allows you to define the fields stored in each document along with their validation requirements and default values.

backend/models/User.ts

import mongoose from "mongoose";
import bycrypt from "bcryptjs";
import jwt from "jsonwebtoken";
import crypto from 'crypto';
import { model, Schema, Model, Document } from 'mongoose';

//declare point type
export interface IPoint extends Document {
    type:string;
    coordinates:string;
}
//generate point schema
const Point:Schema= new Schema({
    type: {
        type: String,
        enum: ['Point'],
        required: true
    },
    coordinates: {
        type: [Number],
        required: true
    }
});
//declare user type
export interface IUser extends Document {
    getResetPasswordToken():string;
    getSignedToken():string;
    resetPasswordToken: string|undefined;
    resetPasswordExpire: string|undefined;
    matchPassword(password: string): boolean | PromiseLike<boolean>;
    username:string;
    password:string;
    email:string;
    profile: {
        firstName: String,
        lastName: String,
        avatar: String,
        bio: String,
        phone: String,
        gender: String,
        address: {
            street1: String,
            street2: String,
            city: String,
            state: String,
            country: String,
            zip: String,
            location: {
                type: IPoint,
                required: false
            }
        },
        active:true
    }

}
// define user schema
const UserSchema: Schema = new Schema({
    username: {
        type: String,
        lowercase: true,
        unique: true,
        required: [true, "Can't be blank"],
        index: true
    },
    password: {
        type: String,
        required: true,
        select: false,
        minlength:  [8, "Please use minimum of 8 characters"],
    },
    email: {
        type: String,
        lowercase: true,
        required: [true, "Can't be blank"],
        match: [/^[\w-\.]+@([\w-]+\.)+[\w-]{2,4}$/, 'Please use a valid address'],
        unique:true,
        index:true
    },
    profile: {
        firstName: String,
        lastName: String,
        avatar: String,
        bio: String,
        phone: String,
        gender: String,
        address: {
            street1: String,
            street2: String,
            city: String,
            state: String,
            country: String,
            zip: String,
            location: {
                type: Point,
                required: false
            }
        },
        required:false
    },
    resetPasswordToken: String,
    resetPasswordExpire: String,

    active: { type: Boolean, default: true }
});

UserSchema.pre<IUser>("save", async function (next: any) {
    if (!this.isModified('password')) {
        return next();
    }
    const salt = await bycrypt.genSalt(10);
    this.password = bycrypt.hashSync(this.password, 10);
    next();
});

UserSchema.methods.matchPassword= async function (password:string) {
    return await bycrypt.compare(password,this.password)   
}
UserSchema.methods.getSignedToken= function (password:string) {
    return jwt.sign({id:this._id},process.env.JWT_SECRET!,{
        expiresIn:process.env.JWT_EXPIRE
    })   
}
UserSchema.methods.getResetPasswordToken= function () {
    const resetToken= crypto.randomBytes(20).toString('hex');
    this.resetPasswordToken= crypto.
    createHash('sha256')
    .update(resetToken)
    .digest('hex');  
    this.resetPasswordExpire = Date.now() + 10*(60*1000) 
    return resetToken

}

export const User:Model<IUser> = model("User", UserSchema);

Schema methods help in preforming functions on the data fields of a document. Packages such as 'bycrypt' are used to hash the passwords before storing them in the data base. NEVER STORE RAW PASSWORDS IN THE DATABASE INCASE OF DATA BREACHES WHICH HAPPENS MORE OFTEN THAN YOU THINK

Routes

Routing refers to how an application’s endpoints (URIs) respond to client requests. These routing methods specify a callback function (sometimes called “handler functions”) called when the application receives a request to the specified route (endpoint) and HTTP method. In other words, the application “listens” for requests that match the specified route(s) and method(s), and when it detects a match, it calls the specified callback function.

backend/routes/auth.ts

import express from 'express';
const router= express.Router();
//import controllers
const {register,login,forgotPassword,resetPassword}=require('../controllers/auth');

//routes
router.route('/register').post(register);
router.route('/login').post(login);
router.route('/forgotpassword').post(forgotPassword);
router.route('/resetpassword/:resetToken').post(resetPassword);

module.exports =router;

The routes are used in backend/server.ts file.

app.use("/api/auth", require("./routes/auth"));

The full path is appended onto "api/auth" availing 4 paths:
> api/auth/register
> api/auth/login
> api/auth/forgotpassword
> api/auth/resetpassword (takes a reset token as the parameter)

Controllers

Controller functions to get the requested data from the models, create an HTTP response and return it to the user.

backend/controllers/auth.ts

We create 4 controllers used in backend/routes/auth.ts

register

Create a new User using the model.create() function and pass the required parameters from request body

import { Response, Request } from 'express';
import {IUser, User} from '../models/User'; 

exports.register= async(req:Request,res:Response,next:any)=>{
    const {username,email,password}=req.body;
    try {
        const user:IUser= await User.create({
            username
            ,email,
            password
        });
        sendToken(user,201,res)
    } catch (error:any) {
        next(error);
    }
};

login

import { Response, Request } from 'express';
import {ErrorResponse} from '../utils/errorResponse';
import {IUser, User} from '../models/User';
exports.login = async(req:Request,res:Response,next:any)=>{
    const {email,password}=req.body;
    if (!email || !password){
        return next(new ErrorResponse("Please provide a valid email and Password",400))
    };
    try {
        const user:IUser | null = await User.findOne({email}).select("+password");
        if (!user){
            return next(new ErrorResponse("Invalid Credentials",401))
        }
        const isMatch:boolean= await user.matchPassword(password);
        if (!isMatch){
            return next(new ErrorResponse("Invalid Credentials",401))
        }

        sendToken(user,200,res)
    } catch (error:any) {
        return next(new ErrorResponse(error.message,500))
    }

forgotPassword and resetPassword

Here the user methods in our User model are used to generate, verify, and change the resetTokens

exports.forgotPassword=async(req:Request,res:Response,next:any)=>{
    const {email}=req.body;

    try {
        const user:IUser | null= await User.findOne({user:email});
        if (!user){
            return next(new ErrorResponse("Email could not be sent",404));
        }
        const resetToken=user.getResetPasswordToken();
        await user.save();

        const resetUrl = `http://localhost:3000/passwordreset/${resetToken}`;
        const message = `
        <h1> You have requested a password reset </h1>
        <p> Please go to this link to reset your password </p>
        <a href=${resetUrl} clicktracking=off>${resetUrl}</a> 
        `
        try {
           await sendEmail({
               to: user.email,
               text:message,
               subject:message

           }); 
           res.status(200)
           .json({
               success: true,
               data:"Email Sent"
           })
        } catch (error) {
            user.resetPasswordToken=undefined;
            user.resetPasswordExpire=undefined;
            await user.save();

            return next(new ErrorResponse("Email could not be sent", 500))

        }
    } catch (error) {
        next(error);
    }
};
exports.resetPassword=async(req:Request,res:Response,next:any)=>{
    const {password} = req.body
    const resetPasswordToken = crypto.createHash("sha256")
    .update(req.params.resetToken)
    .digest('hex');
    try {
        const user: IUser | null   = await User.findOne({
            resetPasswordToken,
            resetPasswordExpire: {$gt: Date.now(),
            }
        })

        if (!user){
            return next(new ErrorResponse("Invalid Reset Token", 400));
        }
        user.password = password;
        user.resetPasswordToken=undefined;
        user.resetPasswordExpire= undefined; 
        await user.save();
        res.status(201)
        .json({
            success: true,
            data:"Password Reset successful"
        });

    } catch (error) {
        next(error);
    }
};

backend/utils

Contains the helper functions used in our controllers to avoid repetition

errorResponse.ts

export class ErrorResponse extends Error{
    statusCode: number;
    constructor(message:any,statusCode:number){
        super(message);
        this.statusCode= statusCode;
    }
}

emailSender.ts

Sends emails upon registration and password reset requests

import nodemailer from 'nodemailer';

interface Options {
    to: string,
    subject: string,
    text: string,

}


const sendEmail = (options: Options) => {

    const transporter = nodemailer.createTransport({
        host:process.env.EMAIL_HOST,
        port:Number(process.env.EMAIL_PORT),
        auth: {
            user:process.env.EMAIL_USER,
            pass: process.env.EMAIL_PASS
        },   
    });
    const mailOptions = {
        from: process.env.EMAIL_FROM,
        to: options.to,
        subject: options.subject,
        html: options.text

    }
    transporter.sendMail(mailOptions, function (err, info) {
        if (err) {
            console.log(err);
        } else {
            console.log(info);
        }
    })
}
module.exports = sendEmail;

Backend: Nodejs, Mongodb, Express Typescript

Osiroski — Sat, 12 Mar 2022 14:04:38 +0000

Back-end with Node.js & MongoDB

Overview
Express and Node make up the middle (application) tier. Express.js is a server-side web framework, and Node.js the popular and powerful JavaScript server platform.

Express.js and Node.js Server Tier
Express.js bills itself as a “fast, unopinionated, minimalist web framework for Node.js,” and that is indeed exactly what it is. Express.js has powerful models for URL routing (matching an incoming URL with a server function), and handling HTTP requests and responses.

By making XML HTTP Requests (XHRs) or GETs or POSTs from your React.js front-end, you can connect to Express.js functions that power your application. Those functions in turn use MongoDB’s Node.js drivers, either via callbacks for using Promises, to access and update data in your MongoDB database.

Overview

Via Express routes, HTTP request that matches a route will be checked by CORS Middleware before coming to Security layer.
Security layer includes:
1. JWT Authentication Middleware: verify SignUp, verify token
2. Authorization Middleware: check User’s roles with record in MongoDB database
3. An error message will be sent as HTTP response to Client when the middleware throw any error
Controllers interact with MongoDB Database via Mongoose library and send HTTP response to Client.

Packages

Express
bcryptjs
jsonwebtoken
mongoose
MongoDB
cors

Project

First create a 'backend' folder that we created previously and create the server. Then, we will initialize package.json using npm init.

 mkdir backend
 cd backend
 npm init- y

Project Structure

Install the required packages using npm or yarn. Since I am using Typescript the types are installed as dev dependencies. The package.json should look like this.

packages
"dependencies": {
    "bcryptjs": "^2.4.3",
    "body-parser": "^1.19.2",
    "cookie-parser": "^1.4.6",
    "cors": "^2.8.5",
    "dotenv": "^16.0.0",
    "express": "^4.17.3",
    "express-jwt": "^6.1.1",
    "jsonwebtoken": "^8.5.1",
    "mongodb": "^4.4.1",
    "mongoose": "^6.2.4",
    "morgan": "^1.10.0"
  },
  "devDependencies": {
    "@types/bcryptjs": "^2.4.2",
    "@types/cookie-parser": "^1.4.2",
    "@types/cors": "^2.8.12",
    "@types/express": "^4.17.13",
    "@types/express-jwt": "^6.0.4",
    "@types/jsonwebtoken": "^8.5.8",
    "@types/mongoose": "^5.11.97",
    "@types/morgan": "^1.9.3",
    "@types/node": "^17.0.21",
    "nodemon": "^2.0.15",
    "ts-node": "^10.6.0",
    "typescript": "^4.6.2"
  }

Folders Create the following folders

mkdir controllers routes middleware utils models config
touch server.ts config.env

– controllers
auth.ts: handle signup & signin actions
user.ts: return public & protected content
– routes
auth.ts: POST register & login
user.ts: GET public & protected resources
– middlewares
auth.ts: Authentication middleware, check protected routes
error.ts: error handling middleware
– models for Mongoose Models
user.ts: User schema and model in Db
– config
configure MongoDB database connection
configure Auth Key
– server.ts:
import and initialize necessary modules and routes, listen for connections.
– config.env:
store environment variables

backend/server.ts

Here, we are requiring express and cors to be used. const port process.env.port will access the port variable from the config.env we required.

require('dotenv').config({path:'./config.env'});

import express from "express";
import cors from "cors";
import cookieParser from 'cookie-parser';
import morgan from 'morgan';
import {json} from 'body-parser'
import mongoose from 'mongoose'
import { connectDB } from "./config/db";



const app= express();
const PORT= process.env.PORT || 5000;
const errorHandler = require('./middleware/error')

//connect to db
connectDB()

app.use(express.json());
app.use("/api/auth", require("./routes/auth"));
app.use("/api/private", require("./routes/private"));

//ErrorHandler (Should be last piece of middleware)
app.use(errorHandler);

const server=app.listen(
    PORT,()=>{
        console.log(`Server is running on port ${PORT}`)
    }
)
process.on("unhandledRejection",(error,promise)=>{
    console.log(`Logged Error: ${error}`);
    server.close(()=>process.exit(1))

})

backend/config.env

PORT=5000
MONGO_URI=mongodb://localhost:27017/<your db name>
JWT_SECRET=<JWT secret key>
JWT_EXPIRE=10min
EMAIL_HOST=<Email config>
EMAIL_PORT= <Email config>
EMAIL_USER=<Email config>
EMAIL_PASS=<Email config>
EMAIL_FROM=<Email config>

backend/config/db.ts

Here we can add the following code to connect to our database. For this to work you should have mongodb installed and Mongodb Service started.
This function is used in backend/server.ts to connect to Mongodb

import { ConnectOptions } from 'mongodb';
import mongoose from 'mongoose';
const connectDB = async ()=> {
    await mongoose.connect(process.env.MONGO_URI!);
    console.log('MongoDb Connected');   
}
module.exports=connectDB;

Upto now, we have a basic server that can connect to Mongodb. In the 3rd part of this series we will work on Server API endpoints and Error Handling

Guide to develop MERN application, step by step. Part 1

Osiroski — Fri, 04 Mar 2022 18:38:30 +0000

Comprehensive guide to build a Full-stack Nodejs Application Part 1 (Stack and Architecture)

What is the MERN Stack
Why we use MERN
Pros n Cons of MERN
MERN Application Architecture/plan
What we are building

What is the MERN Stack?

MERN stands for MongoDB, Express, React, Node, after the four key technologies that make up the stack.

MongoDB - document database
Express - Node.js web framework
React - a client-side JavaScript framework
Node - the premier JavaScript web server

Express and Node make up the middle (application) tier,. Express.js is a server-side web framework, and Node.js the popular and powerful JavaScript server platform.

Why choose MERN Stack?

The MERN stack’s main advantage in web development is that each line of code is written in JavaScript/Typescript. By using a single programming language, the MERN stack eliminates the need for context switching and greatly simplifies the entire development process, giving web developers the tools to create efficient web applications with far less effort.

Advantages of MERN Stack!

UI rendering and performance React JS is the best when it is about UI layer abstraction it provides you the freedom to build the application and organize the code however you want.
Cost-Effective As MERN Stack uses one language throughout that is JavaScript/Typescript so it will be beneficial for a company to hire JavaScript experts only rather than hiring different specialists for different technology. This move will save a lot of time and money.
Open Source All technologies that are involved in MERN are open-source. This feature allows a developer to get solutions to queries that may evolve during development, from the available open portals. As a result, it will be beneficial for a developer.
Easy to switch between client and server As everything is written in one language this is why MERN is simple and fast. And also it is easy to switch between client and server.

Disadvantages of MERN Stack!

Productivity Since React is just a library it uses many third-party libraries which provides lower developer productivity. And due to this upgrading, the React code requires more effort.
Large-Scale Applications It becomes difficult with MERN to make a large project in which many developers are involved. MERN stack is best suited for single-page applications.
Error prevention: If you want a technology stack that prevents common coding errors by its very design, then the MEAN stack is a better choice. As Angular uses Typescript, so prevents common coding errors at the coding stage itself. However, React lags behind here.

MERN App Architecture

The MERN architecture allows you to easily construct a 3-tier architecture (frontend, backend, database) entirely using JavaScript and JSON.

What we are building

Front to back or back to front?? Doesn't matter as long as u have a plan
A Full Stack project requires working on many technologies. But, without proper planning, nothing goes well during the project. You’ll face challenges every now and then if you don’t have a plan. Here’s the perfect roadmap for you to follow to yield great results.

Frontend scope
We are developing Recyski, a digital waste trading platform that allows users and waste collectors to trade/recycle wastes. This is an alternative to the e-commerce example. These libraries are optional can be substituted with your preference:

Redux for state management
Bootstrap for responsive UI
Fontawesome icons
Google fonts
Backend scope
Nodejs and Express will be useful in handling server side. After the entire MERN stack app is complete, you will learn how to refactor the app to make the backend completely serverless. The new backend will use MongoDB Realm instead of Express and Node.

NodeJS and ExpressJS build the server-side/backend of the application.
REST APIs
Database management
User authentication and authorization is controlled here
Database Server
MongoDB is a NoSQL database where each record is a document comprising of key-value pairs that are similar to JSON (JavaScript Object Notation) objects. MongoDB is flexible and allows its users to create schema, databases, tables, etc. Documents that are identifiable by a primary key make up the basic unit of MongoDB.