DEV Community: basal.

DeathMark: Programmatically scan videos for points of interest

Alex King — Wed, 13 Jan 2021 03:19:49 +0000

Problem

In 2020 like most people our company transition to more remote work and I started to invest in my home office to prepare for the coming onslaught of zoom meetings. Once I had a comfortable setup, I had everything I needed to start streaming outside of work as well because why not right? I started playing around with twitch and cutting my videos down for youtube.

Most of my streams were 2+ hours and while playing 2+ hours of games with your friends is an investment in relaxation and friendship. Watching back 2+ hours wasn't a time investment I wanted to make.

Conception

My game of choice is Valorant at the moment but this is true of most games; In the game there is a visual queue to show you have scored a point.

Because they are UI elements they are usually visually bright, consistently placed, and distinct from the background view of the game.

The score indicator is kind of like a flashing light, if I had something like a Light-Dependent Resistor, I could record when it flashed.

Application

I have a concept "check for increases in white in the video"
MDN has a great example of how to check each frame of a video using HTML canvas. The example is set up like this:

ctx1 = Canvas 1 context
ctx2 = Canvas 2 context

We are going to borrow their function and focusing on the computeFrame section. You can see below the RGB values for each pixel in each frame.

 processor.computeFrame = function computeFrame() {
//drawing the full frame to canvas
    this.ctx1.drawImage(this.video, 0, 0, this.width, this.height);
//get the frame from canvas at 0 x and 0 y 
    let frame = this.ctx1.getImageData(0, 0, this.width, this.height);

    let l = frame.data.length / 4;

    for (let i = 0; i < l; i++) {
      let r = frame.data[i * 4 + 0];
      let g = frame.data[i * 4 + 1];
      let b = frame.data[i * 4 + 2];
      if (g > 100 && r > 100 && b < 43)
        frame.data[i * 4 + 3] = 0;
    }
    this.ctx2.putImageData(frame, 0, 0);
    return;
  }

In the example, above it's checking this range of color and if it falls within the threshold it will make it alpha instead thus creating a green screen or in this case yellow screen.

Simple enough, i will just check for white pixel in the area.

//255,255,255 is white so 240 -> 255 is mostly white
 if (g > 240 && r > 240 && b < 240)
        //is white pixel
    }

But the game has complex visuals and many elements would trigger just "white"

Every picture is made up of an almost unique amount of colors and shades so all I needed to do is get as close as possible to that unique number.

let skullFound = []
let white = []
let green = []
let red = []
for (let i = 0; i < l; i++) {
      let r = frame.data[i * 4 + 0];
      let g = frame.data[i * 4 + 1];
      let b = frame.data[i * 4 + 2];
       if (isWhite(r,g,b))
         white.push({r,g,b})
       }
      if (isGreen(r,g,b))
        green.push({r,g,b})
      }
      if (isRed(r,g,b))
        red.push({r,g,b})
      }
}

if(whiteThreshold(white.length) && greenThreshold(green.length) && redThreshold(red.length)) {
 skullFound.push(video.currentTime)
 white = []
 green = []
 red = []
}

After 30ish minutes of trial and error, I was able to get about a 99% accuracy rate at 2x speed with the videos tested with 1 main exception being, if the Character Sage is within the cropped section when she is shot with a sniper rifle... which is kinda rare.

Conclusion

While the current system is not perfect, It's a simplistic solution to the problem I was facing that can be built upon later.

I believe methods like the one above can be applied to many game videos. I look forward to finding more fun techniques in this area in the future.

Free Download

https://ko-fi.com/post/DeathMark-W7W8381IO

Data-Oriented Frontend Development

Adam Munoz — Thu, 13 Feb 2020 02:50:07 +0000

In this post I'm going to be talking about a data oriented approach to frontend development.

I will present a conceptual framework first (the philosophy) and then go onto some code examples to illustrate it. The examples are not necessarily code that you would write in the real world. Rather they are meant to illustrate the foundations underlying real world code.

Again, while it is by no means a comprehensive treatment of the topic, I hope to offer a simple foundational understanding of some concepts and techniques present in frontend technologies such as React or Vue.

Conceptual framework

Let met start by trying to layout the conceptual framework and giving some definitions.

We will be looking at the following model made up of three steps:

An Action takes place (a user clicks a button, a network response arrives, etc ...)
A new State is produced as a result
A representation of that state is produced by a View

Let's look at an example:

Action ➡ User presses key 'a'
State ➡ Current state will be: text typed so far is a
View ➡ Paint character 'a'

Example 1: A user types some text into a text box

This model is not limited to software or computers, rather it's a way of thinking about processes in general.

But what are processes?

Processes

According to Merriam Webster a process is "a series of actions or operations conducing to an end" (a final state).

Example of processes are cooking recipes, lacing up your shoes and computer programs.

This definition of a process as a series of sequential steps leading to a final state stands out as quite similar to the model described above.

We say that by repeatedly performing the sequence Action ➡ State update we can achieve some useful result or results.

Of course, this definition misses the view part.

But, in fact, a view is not an essential part of a process, rather it's a window for humans to see and understand its result or progress.

We'll talk more above views in the following sections, but a very common example of views is of course user interfaces.

Actions

Actions are events that take place in the world and that affect its state.

For example, a user typed some text, a chef put an egg inside a bowl, etc ...

State

What we mean by state is a truth about the world that we care about.

Note that we are not concerned with its data representation yet. Rather we are just talking about information.

Of course, in the case of computer processes, at some point this information will be encoded in data.

For example, "So far, the user has typed the text ab" could be encoded like:

currentText = 'ab'

Example 2: Encoding state as data

View

Finally, a view will take the current state and produce a representation (typically visual) of it.

Views do not normally hold any state themselves, rather they receive the state of the world through their inputs and use it to produce the UI.

state ➡ view ➡ UI

This sequence is repeated throughout the life of the application, and as the state changes as a result of actions, the views will produce different versions of the UI (which themselves might offer new opportunities for users to trigger new actions which will again change the state and so on...)

Fig1: Action, State, View cycle

Data-Oriented frontend development

In data oriented frontend development, we focus mostly on describing how actions update the state of the program rather than on mutating the UI.

That is, we focus mainly on this part: Action ➡ State

And we delegate the work of creating and managing user interfaces to the views to whom we pass the state in order for them to produce a new version of the UI.

Since the views do not normally keep local state, it means that the the UI of our program can be deducted exclusively by knowing the inputs of the views at each point in the program.

This becomes quite handy especially as applications and user interfaces grow in complexity and richness as it frees the developer from having to keep track of UI mutations.

Functional programming and Frontend Development

In this section, I want to look at the model described so far through a different lens. Namely, the lens of functional programming.

If we consider views to be functions then we can think of our applications' user interface as a function of state.

Fig2: User interface is a function of state

As the variable state changes, so will do the user interface accordingly.

We can represent a program as a curve:

Fig3: Representing a program as curve

By the way, the shape of the curve is not particularly relevant for this example, as neither is the way we choose to numerically encode the y-axis (e.g. hashing the html string).

Of course, this abstraction will only work for as long as the views are pure functions. That is that they respect the contract that for the same input the same output will always be produced.

Or, in other words, the views must be able to produce the user interface without access to anything but their inputs.

React is an ideal tool for writing such views, and therefore very well suited to data oriented frontend development.

Let's look at some code samples in React:

// state
currentText = ''

// view
function view({text}) {
  return 
    <div>
      <p>You have typed {text.length} characters</p>
      <textarea value={text} />
    </div>
}

// user interface 
ui = <view text={currentText} />

// paint
ReactDOM.render(ui, document.getElementById('root'))

The output of this would be something like this:

Fig4: The state of our UI at the start

Now we need to create some code for action handling.

As we have stated, we will just care about producing the right data and leave it to the view to take care of producing the user interface for us.

// state
currentText = ''

// update state
function updateCurrentText(text) {
  currentText = text;
  render();
}

// view
function view({text, onTextChangeAction}) {
  return 
    <div>
      <p>You have typed {text.length} characters</p>
      <textarea value={text} onchange={onTextChangeAction} />
    </div>
}

// paint
function render() {
  ui = <view text={currentText} onTextChangeAction={updateCurrentText}/>
  ReactDOM.render(ui, document.getElementById('root'))
}

Every time the user generates a typing action, the state will be updated and the view will produce a new version of the user interface.

Fig4: The view will repaint with each state change

Immediate Mode vs Retained Mode

For years in the game development world, people have been using two techniques for creating 2D interfaces: immediate mode and retained mode.

In a nutshell, in retained mode programmers manage changes to the UI in the screen through mutation operations, while in immediate mode, programmers throw away the current version of UI and re-create it with each state change.

However, just as we know of DOM operations, graphic display operations are also quite expensive in terms of performance which means that re-painting the whole screen with each state change can quickly become inviable.

The solution to this problem is using a graphic buffer to render an in-memory version of the screen, and then diffing this in memory representation with the actual version in the screen in order to automatically calculate the necessary mutation operations.

Since memory operations are less expensive than graphic display operations, this solves the problem of performance in game programming with immediate mode.

The model described in this article suffers from the same issue. Every state change will also trigger a whole repaint of the whole UI, and while this is desirable from the point of view of making it easier to reason about permutations in the UI, just as with immediate mode, it has a performance cost. There is however a solution to this problem too. One that works in a very similar fashion to an in-memory graphic buffer: the Virtual DOM.

Fig4: Immediate mode and Virtual DOM

In a future post I will talk about how to handle state changes in an atomic transactional way and how that helps avoid a certain class of bugs.

I will also look at techniques for managing state changes through the use of persistent data structures for better performance.

Thank you for reading and I hope you enjoyed it!

React in a Serverless World

Tom Bowden — Mon, 18 Mar 2019 07:09:07 +0000

“A comparison between client-server and serverless full-stack app architectures”

The intended audience for this article is primarily front-end developers. This article assumes the reader is familiar with React and some basic GraphQL. In addition, the client-side app in this article uses recently introduced React hooks. Backend technologies will be explained in some detail, so no detailed understanding of topics such as Apollo Server, MongoDB, Mongoose, deployments, AWS, Amplify, and AppSync is necessary to follow along. Full repos are available with the app examples.

This article serves to illustrate the differences between client-server and serverless approaches to building full-stack apps, with React being used for the UI. As a starting point, we will build a simple client side app in React, used for both client-server (which will call “serverful” from now on, taking a lead from this illuminating technical report by UC Berkeley) and serverless approaches. Now, the TodoMVC project is a well-known initiative that uses a Todo app to compare web frameworks, so we’ll go with a Todo app based on it. Specifically, we’ll select the Todo MVC app from the Redux repository, use their CSS styling, remove some functionality to make it simpler for this article, and switch out class-based components for functional components using React hooks, to further simplify the code.

Starting Point: Client Side App

You can see the client side app that resulted from doing this in the following CodeSandBox:

This Todo app is a CRUD app, where the user can:

add todos by typing them into the text input area and hitting the enter key
update todos by double-clicking a todo in the list
delete todos by hovering over a todo and clicking the × icon
mark todos as completed by clicking in the circular icon to the left of the todo

For the purposes of this simple example app, some example todos were added in a data folder as an array of todo objects. The app state data is held in memory, and is not persisted across browser refreshes.

Now we have a simple client-side app starting point, there are basically two separate approaches to add data persistence for users:

serverful (client-server) architecture, where the developer selects and deploys a backend server and a persistent data source (database)
serverless architecture, where the backend server and data source implementations are outsourced to a BaaS cloud provider

We will create two full-stack solutions based on the client-only app — one for each architectural approach — explaining the process for each in detail, so that we can evaluate the pros and cons of each approach. In addition, we have chosen to only use packages written in JavaScript for both the client and the server, in the interest of simplicity.

Approach A: Serverful Architecture

The complete repos for this example can be found in the Github repos for the server and the client.

We will use a GraphQL API for our examples, even though a simple REST API would be more than sufficient for a simple Todo app. The reason for this is that GraphQL has many advantages over REST when additional features are incrementally added to a project, so it is an “aspirational” choice.

For our backend, we will use Apollo Server (v2), a community-maintained open-source GraphQL server, which works with many Node.js HTTP server frameworks. We will work with it as a standalone package, without integrating it with another web framework. We’ll deploy the server to Heroku later.

For our persistent data source, we will use MongoDB, a no-SQL database. Instead of using a native NodeJS driver for MondoDB, we will use Mongoose, which is an ORM for MongoDB, for simplicity. We’ll deploy the MongoDB database to MongoDB Atlas.

For handling GraphQL requests on the front-end, we will use Apollo Client. Since we are using React hooks, we will not use react-apollo React components that use the render prop pattern, for reasons that we explain in this article. We’ll deploy the client app to Heroku as well, as a separate app to the server app.

Let’s look at the backend code first.

Step 1 (Server): Create the GraphQL server and connect the database

The server app’s base level index.js file does the following:

Sets up the MongoDB database connection (via Mongoose)
Gets the parsed GraphQL schema type definitions and resolvers
Creates and runs the Apollo GraphQL server

const { ApolloServer, gql } = require('apollo-server')
const fs = require('fs')
const mongoose = require('mongoose')

// Database connection
mongoose.connect(process.env.MONGODB_URL, { useNewUrlParser: true })
const database = mongoose.connection
database.on('error', console.error.bind(console, 'connection error:'))
database.once('open', () => console.log('We are connected to MongoDB'))

// GraphQL schema types and resolvers
const typeDefs = gql(
  fs.readFileSync(`${__dirname}/graphql/schema.graphql`, { encoding: 'utf-8' })
)
const resolvers = require('./graphql/resolvers')

// Apollo GraphQL server
const server = new ApolloServer({ typeDefs, resolvers })
server
  .listen({ port: process.env.PORT || 4000 })
  .then(({ url }) => console.log(`GraphQL server ready on ${url}`))

Step 2 (Server): Create the GraphQL schema

A GraphQL schema describes the functionality available to the clients which connect to it. A core building block of a GraphQL schema are the type definitions. Types provide a wide range of functions:

Define the shape of the data and the data types used
Show the relationships between different types
Define which data-fetching (querying) and data-manipulating (mutating) operations can be executed by the client
Provide documentation for the client user (via introspection)

Two types within the schema are special: the Query and Mutation types. These are special because they define the entry point of every GraphQL query. The GraphQL schema describes how to get data from every point in the data graph by traversing it from one point (or “node”) to another, starting at the schema entry points. Our schema has one Query entry point: listTodos, an operation that yields an array of all of the todos held in the data source. There are four Mutation fields which are entry points — createTodo, deleteTodo, updateTodo, and toggleCompletion — which describe all the data manipulation operations that the client can do.

The input types describe the shape and data types required when the client wants to mutate data. In our schema, we define three input types: CreateTodoInput, UpdateTodoInput, and DeleteTodoInput.

Scalar types are simple tyes such as String, Boolean, and ID. User-defined types are made up of scalar types, such as our Todo type, which specifies a todo object requiring an id of type ID, some text of type String, and a completed flag of type Boolean. Square brackets around a type indicate an array, so [Todo] is an array of todos. The exclamation mark ! after a type indicates that it is non-null.

Here is our entire schema, in the src/graphql/schema.graphql file:

type Todo {
  id: ID!
  text: String!
  completed: Boolean!
}

input CreateTodoInput {
  id: ID
  text: String!
  completed: Boolean!
}

input UpdateTodoInput {
  id: ID!
  text: String
  completed: Boolean
}

input DeleteTodoInput {
  id: ID
}

type Query {
  listTodos: [Todo]
}

type Mutation {
  createTodo(input: CreateTodoInput!): Todo
  deleteTodo(input: DeleteTodoInput!): Todo
  updateTodo(input: UpdateTodoInput!): Todo
  toggleCompletion(input: UpdateTodoInput!): Todo
}

Step 3 (Server): Create the GraphQL resolvers

Let’s have a look at our resolvers, in the src/graphql/resolvers.js file, which are very simple:

const db = require('../dataSource/db')

const Query = {
  listTodos: () => db.list(),
}

const Mutation = {
  createTodo: (_, args) => db.create(args.input),
  deleteTodo: (_, args) => db.remove(args.input),
  updateTodo: (_, args) => db.update(args.input),
  toggleCompletion: (_, args) => db.toggle(args.input),
}

module.exports = { Query, Mutation }

You can think of each field in a GraphQL query as a function or method of the previous type which returns the next type. For example, as described in the GraqhQL schema, the createTodo field in the root Mutation type is a function that takes an input object and returns a Todo. Each field on each type is backed by a function called a resolver, which we explicitly specify. When a field is executed, the corresponding resolver is called to produce the next value. In our example, the createTodo resolver returns a value of type Todo (as specified by our GraphQL schema), by calling a function create in our data source interaction module db (managed by the Mongoose ORM, which will we describe in more detail soon).

Each resolver function receives four arguments (the previous obj, the arguments provided to the field args, the contextual information context, and field-specific information info). You can find more details here. We pass an input object via the mutation resolver to the Mongoose ORM via the args function argument, which will — as we will see later — return a new todo object of type Todo.

If a field produces a scalar value like a string or number, then the execution completes. However if a field produces an object value then the query will contain another selection of fields which apply to that object. This continues until scalar values are reached. GraphQL queries always end at scalar values.

In our example, the returned Todo is an object value, which contains only scalar fields (id, text, and completed). Resolving these is trivial, and the Apollo server lets us omit resolvers this simple. It will assume that if a resolver is not provided for a field, then a property of the same name should be read and returned.

Step 4 (Server): Interact with the data source (part 1, Mongoose model)

Let’s start off with the code, found in the src/dataSource/model.js file:

const mongoose = require('mongoose')

const todoSchema = new mongoose.Schema({
  id: {
    type: String,
    required: true,
  },
  text: {
    type: String,
    required: true,
    trim: true,
    minlength: 1,
  },
  completed: {
    type: Boolean,
    required: true,
    default: false,
  },
})

const Todo = mongoose.model('Todo', todoSchema)

module.exports = { Todo }

MongoDB databases comprise of collections, which resemble JavaScript arrays, holding our data documents, which resemble JavaScript objects. Mongoose defines schemas (not to be confused with GraphQL schemas), each of which maps to a MongoDB collection and defines the shape of documents within that collection.

Our Mongoose schema simply describes the shape of a our Todo documents, which have id, text, and completed keys, and the types and additional properties (such as required, default, and minLength) on them.

To use the Mongoose schema, we convert it into a Mongoose model we can work with: const Todo = mongoose.model('Todo', todoSchema). Instances of these Todo models are documents, that map to the MongoDB documents.

Step 5 (Server): Interact with the data source (part 2, Mongoose CRUD operations)

The operations to create, read, update, and delete documents in our MongoDB database are coded up in src/dataSource/db.js:

const mongoose = require('mongoose')

const { Todo } = require('./model')

const list = async () => {
  try {
    const todos = await Todo.find()
    if (!todos) return []
    return todos
  } catch (err) {
    console.log(`Error in list todos: ${err}`)
  }
}

const create = async input => {
  const newTodo = new Todo({
    id: input.id,
    text: input.text,
    completed: input.completed,
  })
  try {
    await newTodo.save()
    return newTodo
  } catch (err) {
    console.log(`Error in create todo with text ${input.text}: ${err}`)
  }
}

const remove = async input => {
  try {
    if (!input.id) return
    const removedTodo = await Todo.findOneAndDelete({ id: input.id })
    return removedTodo
  } catch (err) {
    console.log(`Error in remove todo with id ${input.id}: ${err}`)
  }
}

const update = async input => {
  try {
    const updatedTodo = await Todo.findOneAndUpdate(
      { id: input.id },
      { text: input.text, completed: input.completed },
      { new: true }
    )
    return updatedTodo
  } catch (err) {
    console.log(
      `Error in update todo with id ${input.id} and text ${input.text}: ${err}`
    )
  }
}

const toggle = async input => {
  try {
    // 1. Find todo that needs to be toggled in database
    const toggleTodo = await Todo.findOne({ id: input.id })
    // 2. Update the completed flag of that todo in the database
    const updatedTodo = await Todo.findOneAndUpdate(
      { id: toggleTodo.id },
      { text: toggleTodo.text, completed: !toggleTodo.completed },
      { new: true }
    )
    return updatedTodo
  } catch (err) {
    console.log(`Error in toggle with id ${input.id}: ${err}`)
  }
}

module.exports = { list, create, remove, update, toggle }

All database operations are asynchronous, so we are using the async/await syntax to deal with promises.

This file describes five asynchronous functions: list, create, remove, update, and toggle, which use several of Mongoose’s CRUD helper utilities. Let’s describe how each function works:

list: to get all documents in a collection, we use the Model.find() method.
create: requires an input object be passed to a newly created document, when is then saved to the database with the Document.prototype.save() method.
remove: takes the id field from the input object, and is used with the Model.findOneAndDelete() method.
update: is used with the Model.findOneAndUpdate() method. This method finds a matching document based on the conditions in the first argument, updates the document based on the second argument, and if {new: true} is added in the optional third argument, then the modified document is returned (rather than the original document, which is the default behavior).
toggle: also updates a document in the database. First, we need to find the todo for which we want to toggle the completed boolean. For this, we use the Model.findOne() method. Then we update it.

With that, we have completed all the code in src of the server side of our serverful app. You can find the project in the todo-server-app on Github.

Step 6 (Client): Creating the Apollo Client

Now that we have the GraphQL server app, we can work on the GraphQL client app. To make GraphQL requests from the client, we will make use of Apollo Client. The simplest way to get started with Apollo Client is by using the Apollo Boost package.

Let’s start with the client-only React app that we showed earlier in the CodeSandBox. The code can be cloned from a Github repo that we have prepared.

Preparation: Clone `todo-clientonly` App

In your terminal’s command-line:

git clone https://github.com/bowdentom/todo-clientonly.git todo-client-app
cd todo-client-app

The App component is where we will be making most code changes to adapt it for our GraphQL client app.

import React, { useState } from 'react'
import { generate } from 'shortid'

import TodoItem from './TodoItem'
import TodoTextInput from './TodoTextInput'
import { initialTodos } from '../data'

const App = () => {
  const [todos, setTodos] = useState(initialTodos)

  const addTodo = todoText => {
    const addedTodo = {
      id: generate(),
      completed: false,
      text: todoText,
    }
    const updateTodos = [...todos, addedTodo]
    setTodos(updateTodos)
  }

  const removeTodo = todoId => {
    const updateTodos = todos.filter(todo => todo.id !== todoId)
    setTodos(updateTodos)
  }

  const editTodo = (todoId, todoText) => {
    const updateTodos = [...todos]
    const index = updateTodos.findIndex(todo => todo.id === todoId)
    const editedTodo = updateTodos[index]
    editedTodo.text = todoText
    setTodos(updateTodos)
  }

  const completeTodo = todoId => {
    const updateTodos = [...todos]
    const index = updateTodos.findIndex(todo => todo.id === todoId)
    const editedTodo = updateTodos[index]
    editedTodo.completed = !editedTodo.completed
    setTodos(updateTodos)
  }

  return (
    <div>
      <header className="header">
        <h1>todos</h1>
        <TodoTextInput
          newTodo
          onSave={text => {
            if (text.length !== 0) addTodo(text)
          }}
        />
      </header>
      <section className="main">
        <ul className="todo-list">
          {todos.map(todo => (
            <TodoItem
              key={todo.id}
              todo={todo}
              completeTodo={completeTodo}
              editTodo={editTodo}
              removeTodo={removeTodo}
            />
          ))}
        </ul>
      </section>
    </div>
  )
}

export default App

Instead of getting hard-coded initialTodos, we will now get them via a GraphQL request query. We will call the query ListTodos. You will find it in the completed todo-client-app on Github in the file src/graphql/queries:

import gql from 'graphql-tag'

export const LIST_TODOS = gql`
  query ListTodos {
    listTodos {
      id
      text
      completed
    }
  }
`

The ListTodos query string is wrapped in a gql template tag to parse it into GraphQL AST (from the graphql-tag package).

Now, we will need the AST of the mutations as well. You can find them in file src/graphql/mutations:

import gql from 'graphql-tag'

export const CREATE_TODO = gql`
  mutation CreateTodo($input: CreateTodoInput!) {
    createTodo(input: $input) {
      id
      text
      completed
    }
  }
`

export const DELETE_TODO = gql`
  mutation DeleteTodo($input: DeleteTodoInput!) {
    deleteTodo(input: $input) {
      id
      text
      completed
    }
  }
`

export const UPDATE_TODO = gql`
  mutation UpdateTodo($input: UpdateTodoInput!) {
    updateTodo(input: $input) {
      id
      text
      completed
    }
  }
`

Now that we have these GraphQL ASTs, we can turn back to our App component, set up the Apollo Client, add a fetchTodos function to be run the ListTodos query when the component mounts, and implement the CreateTodo, DeleteTodo, and UpdateTodo mutations in the existing handlers for addTodo, removeTodo, editTodo, and completeTodo:

import React, { useState, useEffect } from 'react'
import { generate } from 'shortid'
import { ApolloClient, HttpLink, InMemoryCache } from 'apollo-boost'

import TodoItem from './TodoItem'
import TodoTextInput from './TodoTextInput'
import { LIST_TODOS } from '../graphql/queries'
import { CREATE_TODO, DELETE_TODO, UPDATE_TODO } from '../graphql/mutations'

const apiUrl =
  process.env.NODE_ENV === 'development'
    ? 'http://localhost:4000/'
    : process.env.REACT_APP_API_URL

const client = new ApolloClient({
  link: new HttpLink({ uri: apiUrl }),
  cache: new InMemoryCache(),
})

const App = () => {
  const [todos, setTodos] = useState([])

  const fetchTodos = async () => {
    // Get a fresh list of all todos from the server
    try {
      const response = await client.query({
        query: LIST_TODOS,
        fetchPolicy: 'no-cache',
      })
      const fetchedTodos = response.data.listTodos
      setTodos(fetchedTodos)
    } catch (err) {
      console.log(`Error in fetchTodos API call: ${err}`)
    }
  }

  // Fetch todos on mount
  useEffect(() => {
    fetchTodos()
  }, [])

  const addTodo = async todoText => {
    // Update UI immediately
    const addedTodo = {
      id: generate(),
      text: todoText,
      completed: false,
    }
    const updatedTodos = [...todos, addedTodo]
    setTodos(updatedTodos)
    // Send operation to the API
    try {
      const createTodoInput = {
        id: addedTodo.id,
        text: addedTodo.text,
        completed: addedTodo.completed,
      }
      await client.mutate({
        mutation: CREATE_TODO,
        variables: { input: createTodoInput },
      })
    } catch (err) {
      console.log(`Error in addTodo API call with text ${todoText}: ${err}`)
    }
  }

  const removeTodo = async todoId => {
    // Update UI immediately
    const updatedTodos = todos.filter(todo => todo.id !== todoId)
    setTodos(updatedTodos)
    // Send operation to the API
    try {
      const deleteTodoInput = {
        id: todoId,
      }
      await client.mutate({
        mutation: DELETE_TODO,
        variables: { input: deleteTodoInput },
      })
    } catch (err) {
      console.log(`Error in removeTodo API call with id ${todoId}: ${err}`)
    }
  }

  const editTodo = async (todoId, todoText) => {
    // Update UI immediately
    const updateTodos = [...todos]
    const index = updateTodos.findIndex(todo => todo.id === todoId)
    const editedTodo = updateTodos[index]
    editedTodo.text = todoText
    setTodos(updateTodos)
    // Send operation to the API
    try {
      const updateTodoInput = {
        id: todoId,
        text: todoText,
        completed: editedTodo.completed,
      }
      await client.mutate({
        mutation: UPDATE_TODO,
        variables: { input: updateTodoInput },
      })
    } catch (err) {
      console.log(
        `Error in editTodo API call with id ${todoId} and text ${todoText}: ${err}`
      )
    }
  }

  const completeTodo = async todoId => {
    // Update UI immediately
    const updateTodos = [...todos]
    const index = updateTodos.findIndex(todo => todo.id === todoId)
    const editedTodo = updateTodos[index]
    editedTodo.completed = !editedTodo.completed
    setTodos(updateTodos)
    // Send operation to the API
    try {
      const updateTodoInput = {
        id: todoId,
        text: editedTodo.text,
        completed: editedTodo.completed,
      }
      await client.mutate({
        mutation: UPDATE_TODO,
        variables: { input: updateTodoInput },
      })
    } catch (err) {
      console.log(`Error in completeTodo API call with id ${todoId}: ${err}`)
    }
  }

  return (
    <div>
      <header className="header">
        <h1>todos</h1>
        <TodoTextInput
          newTodo
          onSave={text => {
            if (text.length !== 0) addTodo(text)
          }}
        />
      </header>
      <section className="main">
        <ul className="todo-list">
          {todos.map(todo => (
            <TodoItem
              key={todo.id}
              todo={todo}
              completeTodo={completeTodo}
              editTodo={editTodo}
              removeTodo={removeTodo}
            />
          ))}
        </ul>
      </section>
    </div>
  )
}

export default App

Note that we follow the pattern of updating the UI immediately in the handlers, and then making the asynchronous GraphQL requests to the server.

With that, we have completed all the code in src of the client side of our serverful app. You can find the project in the todo-client-app on Github.

We can now move on to deploying the database, the server, and the client.

Step 7: Deploying the database

We will use Cloud Atlas as the cloud-hosting service of our MongoDB database. We will be using their free tier.

Create an Atlas user account. Click the Start free button and create an account.
Create a new cluster. First, select a Cloud Provider & Region. For example, AWS N. Virginia (us-east-1). Then, select Cluster Tier. SelectM0. The specify a Cluster Name. For example: todo-db. Then click the Create Cluster button at the bottom. It will take a few minutes for the cluster to be created.
Add database user. From the Security tab, click MongoDB Users tab, then click Add New User button. An Add New User dialog will appear. Under User Privileges, select Atlas admin. Add a username and password for the database you will deploy to Atlas. Once you have done so, click on the Add User button.
Whitelist client connection to database. From the Security tab, click IP Whitelist tab, then click Add IP Address button. An Add Whitelist Entry dialog will appear. Click the Add Current IP Address button, then click Confirm and wait for Atlas to update the firewall.
Get the URI connection string. From the Overview tab, inside the Sandbox panel for your cluster, click on the Connect button. A Connect to <YOUR CLUSTER NAME> dialog will appear. Select Connect Your Application. Then select Short SRV connection string. In the Copy the SRV address field, copy the string. It will look something like this: mongodb+srv://<Username>:<PASSWORD>@<Cluster>.mongodb.net/todo-db?retryWrites=true. We will use this URI connection string to connect to the cluster from the server app in the next step.

Step 8: Deploying the server app

We will use Heroku to deploy our server for free.

Create a Heroku user account. Sign up for a free user account on Heroku.
Download and install the Heroku CLI. Follow these instructions to download and install the Heroku CLI.

Now that we have the tools necessary to interact with Heroku from the command-line, we will deploy our NodeJS server.

Log in to Heroku from the command-line. Use the command heroku login from your terminal’s command-line to log in to Heroku.
Create the server app on Heroku. Type heroku create <YOUR_SERVER_APP_NAME> on your command-line, where <YOUR_SERVER_APP_NAME> is a unique name on Heroku for your server app.
Set configuration for the server app. With the Cloud Atlas cluster connection URI, which we will say is <YOUR_CONNECTION_URI> from the previous step, type heroku config:set MONGODB_URL=<YOUR_CONNECTION_URI> --app <YOUR_SERVER_APP_NAME>.
Deploy to Heroku. Assuming you are deploying from your Git master branch, use this command to push to Heroku, where it will be built and deployed: git push heroku master.

If successful, your server app API will be accessible on the URL: https://<YOUR_SERVER_APP_NAME>.herokuapp.com/. You will need this URL in the next step.

Step 9: Deploying the client app

Now that we have the database and server app deployed, we can deploy our client app, also using Heroku.

Set the environment. In the client app’s .env file, set the server API endpoint which will be used by Apollo Client: REACT_APP_API_URL=https://<YOUR_SERVER_APP_NAME>.herokuapp.com/.
Create the client app on Heroku. Type heroku create <YOUR_CLIENT_APP_NAME> on your command-line, where <YOUR_CLIENT_APP_NAME> is a unique name on Heroku for your client app.
Deploy to Heroku. Assuming you are deploying from your Git master branch, use this command to push to Heroku, where it will be built and deployed: git push heroku master.

If successful, your client Todo app will be accessible on the URL: https://<YOUR_CLIENT_APP_NAME>.herokuapp.com/. Your app is now ready!

Approach B: Serverless Architecture

Since we are now covering a “serverless” approach, we won’t be making a server, or even have to worry about connecting a database manually. All we need to do is code up the client side and let AWS do the rest of the heavy lifting.

Let’s get started with the todo-clientonly app, same as we did for the client-side of Approach A.

Preparation: Clone `todo-clientonly` App

In your terminal’s command-line:

git clone https://github.com/bowdentom/todo-clientonly.git todo-serverless
cd todo-serverless

Step 1: Install AWS Amplify CLI

In the command-line, do a global install on your machine:

npm install -g @aws-amplify/cli

Step 2: Configure the Amplify CLI

amplify configure

Then the following configuration steps are required:

Login. This should open up the AWS Management Console in your browser. Log in to your AWS account, then return to your command-line. Hit Enter to continue.
Specify Region. Specify the AWS Region, using your arrow keys. For example us-east-1.
Specify an IAM User. Specify the username of a new IAM user. For example: todo-serverless-cli-user.
Add IAM User. The Add User stepper in AWS Management Console should open in your browser. In your browser, the IAM user has some preconfigured settings that we can accept by clicking Next: Permissions, Next: Tags, Next: Review, and finally Create User. Once the IAM user has been created, we are given an Access key ID and a Secret access key. Make sure you copy these to a secure location, because we will need them next. Now you can return to your command-line. Hit Enter to continue.
Enter User Access Key. Enter the access key id of the newly created user, which you got previously.
Enter User Secret Access Key. Enter the secrete access key of the newly created user, which you got previously.
Specify a Profile Name. Specify the profile name. For example: todo-serverless-cli-user-profile. Now the CLI has been configured and we’re ready to begin initializing new AWS Amplify projects.

Step 3: Initialize Amplify Project

In the command-line, from the root of the todo-serverless app folder that you cloned during preparation:

amplify init

Then complete the following steps to configure Amplify for your project:

Specify Project Name. Enter a name for the project. For example, choose the default name todo-serverless by pressing Enter.
Specify Environment Name. Enter a name for the environment. For example: dev.
Choose Default Editor. Choose your default editor, using the arrow keys. For example: Visual Studio Code.
Choose Programming Language of App. Choose the type of app that you’re building. Select: javascript.
Choose Framework. “What javascript framework are you using?” Select: react.
Specify Source Directory Path. “Source Directory Path:” Enter: src, which should be the default.
Specify Distribution Directory Path. “Distribution Directory Path:” Enter: build, which should be the default.
Specify the Build Command. “Build Command:” Enter: npm run-script build, which should be the default.
Specify the Start Command. “Start Command:” Enter: npm run-script start, which should be the default.
Use AWS Profile? “Do you want to use an AWS profile?” Enter: Y.
Select AWS Profile. “Please choose the profile you want to use”. Select: todo-serverless-cli-user-profile.

The CLI will start initializing the project in the cloud. This will take a few seconds.

If successful, you should see:

Your project has been successfully initialized and connected to the cloud!

You may have noticed that Amplify CLI added a folder amplify to your todo-serverless project at the root level, and updated your .gitignore file.

Step 4: Add API using Amplify

Now we can get started creating the GraphQL API for our app. AWS provides a managed GraphQL backend service called AppSync, which we will be using. In the command-line:

amplify add api

Then complete the following steps to add AppSync:

Select GraphQL API. “Please select from one of the below mentioned services”. Select: GraphQL.
Specify API Name. Provide an API name. For example: TodoServerlessAppSyncApi.
Choose API Authorization Type. Choose an authorization type for the API. Select: API key. For our simple example app, we will not add authorization and authentication. If we wanted to sign in users, we would choose the Amazon Cognito User Pool option.
Existing GraphQL Schema? “Do you have an annotated GraphQL schema?” Select: N.
Guided Schema Creation? “Do you want a guided schema creation?” Select: Y.
Complexity of Data Relationships. “What best describes your project:” Select: Single object with fields (e.g., “Todo” with ID, name, description).
Edit Schema Now? “Do you want to edit the schema now?” Select: Y.
Edit Amplify-Generated Schema. Your code editor should open on the file on the Amplify-generated schema.graphql file at todo-serverless/amplify/backend/api/TodoServerlessAppSyncApi/schema.graphql.

You should see the following Amplify-generated schema code:

type Todo @model {
  id: ID!
  name: String!
  description: String
}

Please update the Todo type to look like this:

type Todo @model {
  id: ID!
  text: String!
  completed: Boolean!
}

Save your changes, and return to your command-line.

Continue GraphQL Autogeneration. Hit Enter to continue.

If all went successfully, you should see the following message:

GraphQL schema compiled successfully.
Edit your schema at <...>/todo-serverless/amplify/backend/api/TodoServerlessAppSyncApi/schema.graphql or place .graphql files in a directory at <...>/todo-serverless/amplify/backend/api/TodoServerlessAppSyncApi/schema
Successfully added resource TodoServerlessAppSyncApi locally.

Amplify has added the local folder amplify/backend/api/ which contains your modified schema. You can now push these changes (and your configuration choices) up to AWS to create the resources for your backend in the cloud.

Step 5: Push API Update to AWS

In the command-line, in order to build all your local backend resources and provision it in the cloud:

amplify push

Then complete the following steps to push the API to AWS:

Continue. “Are you sure you want to continue?” Select Y.
Generate Code for New GraphQL API? “Do you want to generate code for your newly created GraphQL API?” Select Y.
Choose Code Generation Language Target. “Choose the code generation language target.” For example: javascript.
Specify GraphQL. “Enter the file name pattern of graphql queries, mutations and subscriptions.” Enter src/graphql/**/*.js, which should be the default.
Generate All GraphQL Operations? “Do you want to generate/update all possible GraphQL operations - queries, mutations and subscriptions.” Select Y.
Specify Maximum Statement Depth. “Enter maximum statement depth [increase from default if your schema is deeply nested].” Enter 2, which should be the default.

You should now see the following message:
“Updating resources in the cloud. This may take a few minutes...”

If successful, you should see the message:

✔ Generated GraphQL operations successfully and saved at src/graphql
✔ All resources are updated in the cloud

Amplify has added the folder src/graphql to your project. Inside, you will see folders for queries, mutations, and subscriptions, and a schema.json file.

If you look inside the queries and mutations folders, you will see similar GraphQL queries to the ones we wrote by hand in the Approach A: Serverful Architecture section of this article, when creating the server-side app.

Step 6: Configuring Amplify in the Serverless (Client-Only) App

Now that we have set up the “serverless” resources in AWS AppSync, we can add GraphQL request code in the client-side app todo-serverless.

The base level index.js component is where we will start. You may have noticed that Amplify automatically added a configuration file todo-serverless/src/aws-exports.js in a previous step, while setting up AWS AppSync. We will use this file when configuring Amplify in our client-only app.

Update the index.js file so that it looks like this:

import React from 'react'
import ReactDOM from 'react-dom'
import Amplify from 'aws-amplify'

import App from './components/App'
import 'todomvc-app-css/index.css'
import aws_exports from './aws-exports'

Amplify.configure(aws_exports)

ReactDOM.render(<App />, document.getElementById('root'))

We will need to install aws-amplify to our project so that we can import Amplify. In your command-line, please either yarn add aws-amplify or npm install aws-amplify, to add it to our project.

Step 7: Update the Serverless (Client-Only) App with GraphQL API Requests

Let’s move over to the App component file in the src/components folder. Update this file so that it looks like this:

import React, { useState, useEffect } from 'react'
import { generate } from 'shortid'
import { API, graphqlOperation } from 'aws-amplify'

import TodoItem from './TodoItem'
import TodoTextInput from './TodoTextInput'
import { listTodos } from '../graphql/queries'
import { createTodo, deleteTodo, updateTodo } from '../graphql/mutations'

const App = () => {
  const [todos, setTodos] = useState([])

  const fetchTodos = async () => {
    try {
      const response = await API.graphql(graphqlOperation(listTodos))
      // Note: todos are under listTodos.items in this schema...
      const fetchedTodos = response.data.listTodos.items
      setTodos(fetchedTodos)
    } catch (err) {
      console.log('Error in fetchTodos:', err)
    }
  }

  // Fetch todos on mount
  useEffect(() => {
    fetchTodos()
  }, [])

  const addTodo = async todoText => {
    // Update UI immediately
    const addedTodo = {
      id: generate(),
      text: todoText,
      completed: false,
    }
    const updatedTodos = [...todos, addedTodo]
    setTodos(updatedTodos)
    // Send operation to the API
    try {
      const createTodoInput = {
        id: addedTodo.id,
        text: addedTodo.text,
        completed: addedTodo.completed,
      }
      await API.graphql(
        graphqlOperation(createTodo, { input: createTodoInput })
      )
    } catch (err) {
      console.log(`Error in addTodo API call with text ${todoText}: ${err}`)
    }
  }

  const removeTodo = async todoId => {
    // Update UI immediately
    const updatedTodos = todos.filter(todo => todo.id !== todoId)
    setTodos(updatedTodos)
    // Send operation to the API
    try {
      const deleteTodoInput = {
        id: todoId,
      }
      await API.graphql(
        graphqlOperation(deleteTodo, { input: deleteTodoInput })
      )
    } catch (err) {
      console.log(`Error in removeTodo API call with id ${todoId}: ${err}`)
    }
  }

  const editTodo = async (todoId, todoText) => {
    // Update UI immediately
    const updateTodos = [...todos]
    const index = updateTodos.findIndex(todo => todo.id === todoId)
    const editedTodo = updateTodos[index]
    editedTodo.text = todoText
    setTodos(updateTodos)
    // Send operation to the API
    try {
      const updateTodoInput = {
        id: todoId,
        text: todoText,
        completed: editedTodo.completed,
      }
      await API.graphql(
        graphqlOperation(updateTodo, { input: updateTodoInput })
      )
    } catch (err) {
      console.log(
        `Error in editTodo API call with id ${todoId} and text ${todoText}: ${err}`
      )
    }
  }

  const completeTodo = async todoId => {
    // Update UI immediately
    const updateTodos = [...todos]
    const index = updateTodos.findIndex(todo => todo.id === todoId)
    const editedTodo = updateTodos[index]
    editedTodo.completed = !editedTodo.completed
    setTodos(updateTodos)
    // Send operation to the API
    try {
      const updateTodoInput = {
        id: todoId,
        text: editedTodo.text,
        completed: editedTodo.completed,
      }
      await API.graphql(
        graphqlOperation(updateTodo, { input: updateTodoInput })
      )
    } catch (err) {
      console.log(`Error in completeTodo API call with id ${todoId}: ${err}`)
    }
  }

  return (
    <div>
      <header className="header">
        <h1>todos</h1>
        <TodoTextInput
          newTodo
          onSave={text => {
            if (text.length !== 0) addTodo(text)
          }}
        />
      </header>
      <section className="main">
        <ul className="todo-list">
          {todos.map(todo => (
            <TodoItem
              key={todo.id}
              todo={todo}
              completeTodo={completeTodo}
              editTodo={editTodo}
              removeTodo={removeTodo}
            />
          ))}
        </ul>
      </section>
    </div>
  )
}

export default App

You can now delete the src/data directory with the hardcoded todos.

You will notice how similar the API calls are to the ones we wrote in todo-client-app. This is all we have to do for this app.

You can now test your app locally, interacting with AWS AppSync’s DynamoDB in the cloud. From your command-line, either yarn start or npm start to start the development server on localhost:3000 in the browser. Add a few todos in the app, and then check that the added todos appear in DynamoDB in the AWS Management Console.

Step 8: Host the Serverless (Client-Only) App on S3

We can either choose DEV for S3 with HTTP or PROD for S3 with HTTPS with CloudFront distribution. We will go with S3 with HTTP for the purposes of this article.

From the command-line, we can add S3 hosting for our app:

amplify add hosting

Then complete the following steps to host the app:

Select Environment Setup. “Select the environment setup:” Select: DEV (S3 only with HTTP).
Specify Hosting Bucket Name. “hosting bucket name”: Select the default name.
Specify Website’s index Document. “index doc for the website”: Enter index.html, which should be the default.
Specify Website’s error Document. “error doc for the website”: Enter index.html, which should be the default.

We can now deploy to S3 using the following command in the command-line:

amplify publish

Hit Y when prompted: “Are you sure you want to continue?”.

You will then see a message saying: “Updating resources in the cloud. This may take a few minutes...”

If everything was successful, you will receive the message:

✔ Uploaded files successfully. Your app is published successfully. http://-dev.s3-website-us-east-1.amazonaws.com

Your browser should automatically open with a tab pointing to url http://<your hosting bucket name>-dev.s3-website-us-east-1.amazonaws.com.

Your app is now live on the web for you to share! Test it out, refresh the browser, and see all of your persisted todos! Congratulations on creating a serverless app!

Comparing the Serverful and Serverless Approaches (A versus B)

After going through this detailed article, you will be able to appreciate the differences between the serverful approach, in which you had to make two apps — for client and server — as well as separately deploy both in addition to a database, and the serverless approach, where you only needed to make and deploy a single client-side app with some very simple configuration for AWS using Amplify.

When writing the backend API of a serverful GraphQL app, you need to create:

GraphQL server
GraphQL schema
GraphQL resolvers
Data source CRUD functions

Then, you need to deploy both the server and the database. These are non-trivial tasks, introducing potential errors and requiring ongoing maintenance. Even for this simple todo app, it took a significant amount of time and effort to complete these tasks.

Manually creating a backend API involves a lot of code and complexity that you can pass on to a managed GraphQL BaaS, such as AWS AppSync. This will enable you to iterate on your project more quickly, which is more lean and agile.

The authors of the UC Berkeley Technical Report entitled “Cloud Programming Simplified: A Berkeley View on
Serverless Computing” predict that:

Serverless computing will become the default computing paradigm of the Cloud Era, largely replacing serverful computing and thereby bringing closure to the Client-Server Era.

We believe that startups should strive to be ‘Exponential Organizations’, leveraging cloud computing assets and technologies, and the serverless approach to making webapps is a step in the right direction.

Serverless is the future of making webapps. Welcome to the future!

Immutable data management in single state applications

Shinsuke Marty Yamada — Thu, 14 Feb 2019 09:37:17 +0000

In this post, I'm going to explore application state management with immutable data and related JavaScript data types. I will be focusing on the "Reducer" of the popular Redux library which is known for application state management.

What state management is

Recently, the concept of state management has been taking a significant and essential role in single page application development. Some would say that Redux was the library which sparked the concept in our industry. Many developers would agree, it allows them to create a complicated application in a simple way and makes it easier to see the current status of the application. As a result, our code is easier to debug. Even though it is so powerful; Redux itself is built with really simple concepts.

What simplicity does Redux bring to Javascript applications? I'd like to focus on "Reducer" which is one of the important elements included in Redux. Reducer is the only interface using a pure function to update the state. To update the state, Reducer takes the current state and difference data called Action and generates a new state.

What I specifically mention here is that Reducer doesn't update the state directly, but generates a new state. This state generated, is independent of the state before updating. This means that the previous state won't be changed after generating a new state. With that in mind, it enables us to easily trace how the state changes the application has been gone within any given time sequence. Generally, such data that cannot be changed after generation is called "immutable".

Immutable data in JavaScript

Before moving to the actual topic, I'd like to mention immutable data in JavaScript briefly. Primitive type data such as String, Number and Boolean cannot be changed after creation. On the other hand, Reference type data such as Object and Array can be updated after creation. Those are called "mutable".


    
var a = 1;
var b = { k: 1 };
var c = [1];

a = 2; // this is not changing 1 but creating 2 newly and re-assigning to a
b.k = 2; // can change a property
c[0] = 2; // can add new value
console.log(b);
console.log(c);

Looking at the example above, we can see that the object { k: 1 } and the array [1] can be changed. You might see that the number assigned to the variable "a", is also changeable, however, this is not changing "1" itself, but creating "2" and re-assigning it to "a" (The value of the variable "a" has overridden by "2").


    
var b = { k: 1 };
var c = [1];

Object.freeze(b);
Object.freeze(c);

b.k = 2; // this doesn't work
c[0] = 2; // this doesn't work
console.log(b);
console.log(c);

There is a method to make objects and arrays immutable. That's to use Object.freeze(). In the above example, the value of the object and the array are made unchangeable by applying for Object.freeze().

Immutable variable

New variable definition statements, "const" and "let" are newly added in ES6. By using const to define a variable, we can define a variable that cannot be reassigned. In the example below, you will get an error if you try to reassign a value to the variable b defined by using const. With that, we can prevent unintentional variable reassignment before it happens.


    
let a = 1;
a = 2; // You can assign 2 to a

const b = 1;
b = 2; // You cannot assign a value (Throw an error)

What we should take care with when defining an object with const is that the properties of the object are still mutable. Const only disables reassignment of a variable and it doesn't make the object immutable. To make an object immutable, we need to apply Object.freeze().


    
const c = { k: 1 };
c.k = 2; // You can change the property
c.l = 3; // You also can add a new property
console.log(c);

const d = { k: 1 };
Object.freeze(d);
d.k = 2; // You cannot change the property
d.l = 3; // You also cannot add a new property
console.log(d);

In a nested structure

I have introduced Object.freeze() as a way to make objects or array immutable. Actually, I need to tell you one thing that you need to care with when dealing with nested structures. Let's see the example below. We can see we are still able to change the value of a nested object even though we apply Object.freeze(), it is only abled to the object, not to the nested structure. This is because Object.freeze() works only for the top level object passed to it. If you want to make the whole values of the nested structure immutable, you have to apply Object.freeze() for each object.


    
const a = { k: { l: 1 } };
Object.freeze(a);
a.k = { l: 2 }; // You cannot change it but...
console.log(a.k);
a.k.l = 2; // You can change the child value 
console.log(a.k);

const b = { k: { l: 1 } };
Object.freeze(b);
Object.freeze(b.k);
a.k.l = 1; // You cannot change the value! (It's still 1)
console.log(a.k);

Immutable application state management with Reducer

Now, let's move onto the main topic, state management. I'm going to dig into the implementation of Reducer which is the important element of Redux mentioned earlier.

As expressed in the above expression, Reducer updates the state by receiving the current state and Action which is the actual changes. In that process, Reducer generates a new independent state not changing the current state. Let's say we have the current state as an object { a: 1, b: 1, c: 1 } and Action as an object { b: 2 }. Reducer generates a new state and set 2 given by Action to 'b' of the new one. Since 'a' and 'c' aren't changed, those values are copied and set to the new one as they are. As a result, object { a: 1, b: 2, c: 1 } is returned as the new one. The update completes by swapping the new one with the current one. Therefore, Reducer enables us to manage our application state in an immutable way by generating a new state every time the state is updated.

Reducer implementation example

The example code below is written based on the diagram above. As you can see, Reducer has a very simple structure only generating and returning a new state made with the current state and Action. On the 3rd line, it returns the value to the caller by overriding the new object passed to the first argument of Object.assign() with the order of the current state and the action. Since the state value generated there is stored in a different memory space from where the current state is, we can confirm the difference between the new one and the previous one when we compared them on the 17th line. As you might have already realized on the 15th line, we can still mutate the generated state like state.a = 2 directly unintentionally, however, those changes will be completely missing when swapping with the next state. Therefore, state updates always have to be done by Reducer.


    
function reducer(state, action) {
  // Generate new state
  return Object.assign({}, state, action);
  // or use spread operator of es6 like this
  // return { ...state, action };
}

// Initialize state
let state = { a: 1, b: 1, c: 1 };
// Create action
const action = { b: 2 };
// Cache previous state
const prevState = state;
// Update state
state = reducer(state, action);

console.log(state === prevState); // false
console.log(state.b); // 2

Conclusion

I introduced application state management with immutable data and related JavaScript data types while focusing on the state update function, "Reducer" of Redux. In the beginning, I mentioned about the simplicity brought by immutable data. Immutable data is unchangeable data after generated such as Number and String type. Also, it was mentioned that Object and Array type are called mutable. Apart from that, I mentioned that it is possible to make those mutable data unchangeable by applying for Object.freeze(). Finally, I introduced how Reducer actually generates a new state in an immutable with an example.
This time, I intentionally used very simple state data as an example easy to explain, however, actual state would look more complicated like using nesting. I'd like to introduce how to manage such a state by keeping in an immutable in my another post.

React Hooks: Making it easier to compose, reuse, and share React code

Tom Bowden — Wed, 06 Feb 2019 03:55:36 +0000

Hooks are an upcoming feature in React that enable you to use state and many other React features without writing a class. This has some important ramifications for the future of React code, especially with regard to how components will be composed.

The motivation for hooks, as provided by the official documentation from the Facebook React team, is that hooks solve some problems that they have encountered over five years of writing and maintaining React components. These problems are:

It’s hard to reuse stateful logic between components
Complex components become hard to understand
Classes confuse both people and machines

In this short article, we will focus on how React hooks solve the first problem — the difficulty of reusing stateful logic between components — because it has the most wide-ranging consequences.

Reusing Stateful Logic

For the past few years, the preferred ways of sharing stateful logic in React are higher-order components (HOCs) and render props. Both HOCs and render props require an additional component in the application component tree, and arguably they also make it somewhat more difficult to reason about the shared logic within the code. Now we can add React hooks as a way of sharing logic.

Let’s compare the options for dealing with cross-cutting concerns in React using a very simple example to highlight the differences between them.

Higher-Order Component

A higher-order component (HOC) is a widely-used pattern in React to reuse component logic, by wrapping the component around a target component and passing data to it via its props. In other words, a higher-order component is a function that takes your target component as an argument, and returns a the target component with additional data and functionality.

The following simple example shows a higher-order component that tracks the mouse position in a web app.

function withMousePosition(WrappedComponent) {
  return class extends Component {
    constructor(props) {
      super(props);
      this.state = { x: 0, y: 0 };
    }

    componentDidMount() {
      window.addEventListener("mousemove", this.handleMouseMove);
    }

    componentWillUnmount() {
      window.removeEventListener("mousemove", this.handleMouseMove);
    }

    handleMouseMove = event => {
      this.setState({
        x: event.clientX,
        y: event.clientY
      });
    };

    render() {
      return (
        <WrappedComponent
          {...this.props}
          mousePosition={this.state}
        />
      );
    }
  };
}

In the wrapped class component above, the mouse position is obtained via the mousemove event API provided by browser windows. We set up an event listener and update the state which holds the mouse position coordinates. The class encapsulates the functionality, and now we can share it with other components.

So, using the higher-order component pattern, the function withMousePosition takes any target component as an argument, and returns it with all its existing props plus one additional prop: the mousePosition coordinates.

    function App(props) {
      const { x, y } = props.mousePosition;

      return (
        <div className="App">
          <h1>Higher-Order Component Method</h1>
          <h2>Move the mouse around!</h2>
          <p style={{ background: "orange" }}>
            The current mouse position is ({x}, {y})
          </p>
        </div>
      );
    }

    const AppWithMousePosition = withMousePosition(App);

In this example we have shared the mousePosition coordinate data with a presentational App component. The dynamic mouse position is displayed in an orange paragraph:

    <p style={{ background: "orange" }}>
        The current mouse position is ({x}, {y})
    </p>

The wrapped AppWithMousePosition component can then be rendered to the DOM:

    ReactDOM.render(<AppWithMousePosition />, document.getElementById("root"));

Try the HOC approach for yourself in the following CodeSandbox:

https://codesandbox.io/s/43z216n6y9

Render Props

A render prop is a way of sharing code between React components using a prop whose value is a function. The prop is often called render, thus the terminology “render prop”.

Let’s see how our mouse position example introduced earlier looks when implemented using a render prop:

    class MousePosition extends Component {
      constructor(props) {
        super(props);
        this.state = { x: 0, y: 0 };
      }

      componentDidMount() {
        window.addEventListener("mousemove", this.handleMouseMove);
      }

      componentWillUnmount() {
        window.removeEventListener("mousemove", this.handleMouseMove);
      }

      handleMouseMove = event => {
        this.setState({
          x: event.clientX,
          y: event.clientY
        });
      };

      render() {
        return (
          <div
            style={{ height: "100%", width: "100%" }}
            onMouseMove={this.handleMouseMove}
          >
            {this.props.render(this.state)}
          </div>
        );
      }
    }

The stateful logic for the mouse position is the same as we used in the higher-order component earlier.

The difference between the HOC method and this render props method is that we now specify a function prop called render within the render method of the class component, which takes the state of the component as an argument, and renders it as a child of the class component:

    render() {
        return (
            <div
            style={{ height: "100%", width: "100%" }}
          onMouseMove={this.handleMouseMove}
        >
          {this.props.render(this.state)}
        </div>
      );
    }

Note that the terminology “function as child” is also used when referring to this pattern.

Now, we can wrap any target component with this MousePosition component, and dynamically render the mouse position by passing it in via the render prop. This is a dynamic way of sharing stateful logic, compared with the statically defined higher-order component.

As an aside, for more details regarding the pros and cons of higher-order components and render props, please see this excellent article by Michael Jackson, the co-author of React Router.

Returning to our example, we can now render a presentational App component by composing the MousePosition component within it. We render the dynamic mouse position within a sky-blue <p> element, which is passed via a function in the render prop:

    function App() {
      return (
        <div className="App">
          <h1>Render Props Method</h1>
          <h2>Move the mouse around!</h2>
          <MousePosition
            render={mousePosition => (
              <p style={{ background: "skyblue" }}>
                The current mouse position is ({mousePosition.x}, {mousePosition.y})
              </p>
            )}
          />
        </div>
      );
    }

To summarize, the behavior associated with listening for mousemove events and storing the mouse position coordinates has been encapsulated in the MousePosition component, and can be used flexibly in any other component, via this “render props” pattern. This is an example of a composable component that has reusable, shareable state logic.

Try the render props approach for yourself in the following CodeSandbox:

https://codesandbox.io/s/rjprzkj29p

React Hooks

Now, let’s look at how “hooks” could be used to achieve the goal of reusing stateful logic within your apps, using the very same mouse position example:

    function useMousePosition() {
      const [mousePosition, setMousePosition] = useState({ x: 0, y: 0 });

      function handleMouseMove(event) {
        setMousePosition({
          x: event.clientX,
          y: event.clientY
        });
      }

      useEffect(() => {
        window.addEventListener("mousemove", handleMouseMove);

        return () => {
          window.removeEventListener("mousemove", handleMouseMove);
        };
      }, []);

      return mousePosition;
    }

Note that we have created a “custom hook” here called useMousePosition. It is a function component, not a class component, but it does encapsulate state!

For our mouse position example, we are using two different React hooks within the body of our custom hook function:

State hook: useState
Effect hook: useEffect

The useState hook lets us add React state to function components, without having to convert them into class components. The useState function hook takes the initial value of state as an argument, and returns a two-element array containing the state value (mousePosition), and a function to update that value (setMousePosition). You can see at the bottom of the function that we are returning the mousePosition state value from the function.

The useEffect hook lets you perform side effects in function components. Examples of side effects are getting data from an API, listening for browser events, and manually changing the DOM. The useEffect hook carries out the same tasks as the lifecycle methods componentDidMount, componentDidUpdate, and componentWillUnmount combined do in class components.

useEffect takes a callback function (called the “effect”) as its first argument, and runs it after each render of the component. In our example, the effect is to set up the mousemove event listener after the first render when the component is mounted. The returned callback from the effect, if specified, serves to “clean up” before the component is unmounted. In our example, we are removing the event listener when we unmount.

    useEffect(() => {
      window.addEventListener("mousemove", handleMouseMove);

      return () => {
        window.removeEventListener("mousemove", handleMouseMove);
      };
    }, []);

Within the effect callback, we are setting up a mousemove event listener called handleMouseMove, which itself calls setMousePosition with the updated mouse coordinates whenever the user moves the mouse.

The second argument to the useEffect function hook, if specified, is an array of specific state values that the effect will run on whenever the value updates. That is, the effect will run on each re-render of the component triggered by updates to those specific state values. If no array is specified, then the default behavior is to re-render the component and fire the effect whenever any of the state values updates.

In our example, we are passing an empty array [], which means that the effect does not depend on any state value updating in our component, i.e. our effect only runs on mount and it will clean up on unmount, but it won’t run on any mousePosition updates. The event listener already updates the mousePosition, so it is unnecessary to re-render the component when that happens.

Our useMousePosition custom hook completely replicates the behavior of the class components used in the HOC and render-props patterns earlier. It fully encapsulates the behavior we need in a very compact, easy-to-understand, and reusable way.

Now, we can share this dynamic mouse position functionality in any other component. Let’s call our custom hook useMousePosition in our presentational App component:

    function App() {
      const { x, y } = useMousePosition();

      return (
        <div className="App">
          <h1>React Hook Method</h1>
          <h2>Move the mouse around!</h2>
          <p style={{ background: "palegreen" }}>
            The current mouse position is ({x}, {y})
          </p>
        </div>
      );
    }

Here, we are rendering the dynamic mouse coordinates in a pale green <p> tag.

Try the hooks approach for yourself in a CodeSandbox:

Summary

Now you’ve seen the same mouse position example implemented in three different ways: higher-order components, render props, and hooks.

It is clear that by far and away the most elegant and easy to follow code is found in the React hook approach. In addition, less code is needed to achieve the same results.

Hooks make it easier than ever to separate out stateful component logic, data, and functionality into an encapsulated structure, making it convenient to reuse and share. The implications of this should not be underestimated. This is a huge and exciting development for React and everyone who uses it!

Web Components: Understanding Attributes and Properties

Alex King — Wed, 06 Feb 2019 02:40:44 +0000

When working with component libraries there is a blending of Javascript Properties and HTML Attributes that can make it confusing to understand what they really are and how they work. This can be a problem later on so here is a breakdown.

HTML elements have attributes. Theses are strings that can be passed into an HTML Tag.

    <img src="https://picsum.photos/200/300" />

This is because a HTML Document is only a string until it is coupled with a DOM.

The Document Object Model (DOM) represents that same document ... The DOM is an object-oriented representation of the web page, which can be modified with a scripting language such as JavaScript. - MDN

When you request a tag using Javascript, it will return an Element Object from the DOM; with this Javascript object, you can change the attributes of the HTML element but you can also set properties of the object itself which don't have visible representation in the document.

Properties can be complex data like arrays, functions or other objects; This is especially useful when storing data related to a node.

Try it yourself in your browser console.

$0 //will return the selected element in chrome/firefox/safari
$0.coolBeans = ['baked', 'black', 'refried'];
$0.coolBeans; // return a real array, not a string ['baked', 'black', 'refried']

Now lets do the same with attributes

$0 //will return the selected element in chrome/firefox/safari
$0.setAttribute('coolBeans', ['baked', 'black', 'refried']);
$0.coolBeans; // return undefined
$0.getAttribute('coolBeans'); // return a string "baked,black,refried"

Things get a little more complex when you start using a property that is already defined as an attribute for that tag.

const imgTag = document.createElement('img'); //non rendered img tag
imgTag.srcset = ['https://picsum.photos/320/320 320w',
                                 'https://picsum.photos/480/320 480w'
                                 'https://picsum.photos/640/320 640w']
// this is going to return a real array right?
// nope
imgTag.srcset; //"https://picsum.photos/320/320 320w,https://picsum.photos/480/320 480w,https://picsum.photos/640/320 640w"

This Image Element's Class has a property handlers for srcset which reflect it has attributes and set the property as well.

For the most part this isn't a problem but it's helpful to understand this relationship because it will help you build better more reusable Web Components.

We will be using the library Lit Element from Google in this Article. Lit Element includes a nice base structure and lets you create HTML structure for your component using template literals.

Below is a basic example.

class BeanList extends LitElement {
  constructor() {
    super();
  }
  render() {
    return html`
       <ul>
            <li>Baked<li>
            <li>Black</li>
          <li>Refried</li>
         </ul>
    `
  }
}
customElements.define('cool-beans', BeanList);
// You can now add <cool-beans></cool-beans> to your html

This will now render a beautiful list of cool beans, but it's a static string so let's move them to an array.

class BeanList extends LitElement {
  constructor() {
    super();
    this.list = [
            'Baked',
            'Black',
            'Refried'
        ]
  }
  render() {
    return html`
      <ul>
        ${this.list.map((beanName)=> { return html`<li>${beanName}</li>`})}
      </ul>
        `
  }
}

Now we have set the .list of the object, we can access this data using Javascript.

$0 //will return the selected element in chrome/firefox/safari
$0.list // its return a real array ['Baked','Black','Refried']
$0.list = ['Baked','Black','Refried','Pino'];
$0.list // ['Baked','Black','Refried','Pino']

const x = document.createElement('cool-beans');
x.list = ['Baked','Black','Refried','Pino'];
document.body.appendChild(x);
// This WILL render the <li>Pino</li> because the list was updated before
// it rendered the tag

But you will notice the HTML doesn't update. The Object property is not connected to the Render cycle and it won't re-render if this property is changed. So let's make it listen to our new list.

class BeanList extends LitElement {
  constructor() {
    super();
    this.list = [
            'Baked',
            'Black',
            'Refried'
        ]
  }
    static get properties() {
    return {
      list
    };
  }
  render() {
    return html`
      <ul>
        ${this.list.map((beanName)=> { return html`<li>${beanName}</li>`})}
      </ul>
        `
  }
}

Now we have added a property listener, when this property is updated, it will re-render.

$0 //will return the selected element in chrome/firefox/safari
$0.list // its return a real array ['Baked','Black','Refried']
$0.list = ['Baked','Black','Refried','Pino'];
$0.list // It will re-render and you still have
              // a real ['Baked','Black','Refried','Pino']

So now our example from before works perfectly. Now the final piece of the puzzle is connecting out properties to our attributes.

class BeanList extends LitElement {
    constructor() {
    super();
    this.list = [
            'Baked',
            'Black',
            'Refried'
        ]
  }
  static get properties() {
   return {
      list: {
      reflect: true,
      type: {
        fromAttribute: x => x.split(','),
        toAttribute: x => x.join(),
      }
    },
   }
  }
  render() {
    return html`
      <ul>
        ${this.list.map((beanName)=> { return html`<li>${beanName}</li>`})}
      </ul>
        `
  }
}

Hooray with this above example, you can now do both!

$0 //will return the selected element in chrome/firefox/safari
$0.setAttribute('list', ['Baked','Black','Refried','Pino'];);
$0.list = ['Baked','Black','Refried','Pino'];

Both statements will update the Attributes, Property and HTML of the component.

fromAttribute and toAttribute are very powerful interfaces to have but lit-element comes with some pre-made converters like Array the full list can be found here.

If you want to use more complex structures, you are unlikely to reflect those as attributes so you don't need the converters.

class BeanList extends LitElement {
  static get properties() {
   return {
      list: { attribute: false },
   }
  }
  constructor() {
    super();
    this.list = [];
  }
  render() {
    return html`
      <ul>
        ${this.list.map((bean)=> { return html`<li><caption><img src="${bean.img}" />${bean.name}</caption></li>`})}
      </ul>
        `
  }
}
customElements.define('cool-beans', BeanList);

In the above example you, no attribute is created on the element only properties and you a set them using a range of framework including native HTML. Have a play.

One Caveat is that some frameworks don't pass properties correctly to Web Components yet. A list of compatible frameworks can be found here

DEV Community: basal.

DeathMark: Programmatically scan videos for points of interest

Problem

Conception

Application

Conclusion

Free Download

Data-Oriented Frontend Development

Conceptual framework

Processes

Actions

State

View

Data-Oriented frontend development

Functional programming and Frontend Development

Immediate Mode vs Retained Mode

React in a Serverless World

“A comparison between client-server and serverless full-stack app architectures”

Starting Point: Client Side App

Approach A: Serverful Architecture

Step 1 (Server): Create the GraphQL server and connect the database

Step 2 (Server): Create the GraphQL schema

Step 3 (Server): Create the GraphQL resolvers

Step 4 (Server): Interact with the data source (part 1, Mongoose model)

Step 5 (Server): Interact with the data source (part 2, Mongoose CRUD operations)

Step 6 (Client): Creating the Apollo Client

Preparation: Clone todo-clientonly App

Step 7: Deploying the database

Step 8: Deploying the server app

Step 9: Deploying the client app

Approach B: Serverless Architecture

Preparation: Clone todo-clientonly App

Step 1: Install AWS Amplify CLI

Step 2: Configure the Amplify CLI

Step 3: Initialize Amplify Project

Step 4: Add API using Amplify

Step 5: Push API Update to AWS

Step 6: Configuring Amplify in the Serverless (Client-Only) App

Step 7: Update the Serverless (Client-Only) App with GraphQL API Requests

Step 8: Host the Serverless (Client-Only) App on S3

Comparing the Serverful and Serverless Approaches (A versus B)

Immutable data management in single state applications

What state management is

Immutable data in JavaScript

Immutable variable

In a nested structure

Immutable application state management with Reducer

Reducer implementation example

Conclusion

React Hooks: Making it easier to compose, reuse, and share React code

Reusing Stateful Logic

Higher-Order Component

Render Props

React Hooks

Summary

Web Components: Understanding Attributes and Properties

Preparation: Clone `todo-clientonly` App

Preparation: Clone `todo-clientonly` App