DEV Community: Oreoluwa Ogundipe

Building a Screen Recorder for the Web With Cloudinary in a jiffy!

Oreoluwa Ogundipe — Fri, 16 Nov 2018 15:43:23 +0000

Have you ever wondered how Google Hangouts and other robust video-call web apps can not only access your screen but also enable screen sharing during calls? Wonder no more. Just read on.

This tutorial delves into how it all works and shows you how to integrate a screen recorder into a web app. The best part is that you can also send video feeds directly to Cloudinary's image and video platform for easy access and on-the-fly retrieval with Cloudinary APIs.

At the end of this tutorial, you'll have built a demo app that looks like this:

Note: This is where the demo GIFs reside.

Putting in Place the Prerequisites

Currently, some of the code in this tutorial requires a few experimental features that will be integrated into the Chrome browser soon. To turn on those features, do the following:

On Chrome, go to chrome://flags for your current Chrome flags.
Type Experimental Web Platform features in the search field at the top and press Enter.

Chrome displays the Experiments page with the section Experimental Web Platform features under Available.

Click the downward arrow in the Disabled button to choose Enabled and then click RELAUNCH NOW.

Building the Interface

To create a screen recorder for the web, you need only to code in regular HTML and JavaScript. As a start, follow the steps below to build the interface.

Create a folder called cloudy-snap. Type:

mkdir cloudy-snap
cd cloudy-snap

Create an index.html file in cloudy-snap to house the code in the demo app. Type:

touch index.html

Edit the index.html file to read like this:

<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <meta http-equiv="X-UA-Compatible" content="ie=edge">
    <title>Cloudy Snap</title>
    <style>
        body{
          display: flex;
          flex-direction: column;
          align-items: center;
        }
        .container{
          display: flex;
          flex-direction: column;
          align-items: center;
        }

        #videoElement{
          width: 50%;
          padding: 30px;
        }
    </style>
</head>
<body>
    <img src="https://res.cloudinary.com/cloudinary/image/upload/c_scale,w_200/v1/logo/for_white_bg/cloudinary_vertical_logo_for_white_bg.png">
    <h1>CloudySnap</h1>
    <p>Take a screen recording on the web and upload it to Cloudinary</p>
    <div class="container">
      <video id="videoElement" autoplay></video>
      <button id="recorder">
        Capture screen for 2 seconds and Upload to Cloudinary
      </button>
    </div>
</body>
</html>

Run the index.html file in Chrome.

Chrome then displays the interface for the demo app.

At this point, interacting with the interface results in no response from the recorder. The next section shows you how to add capabilities to the recorder and make it work.

Adding the Pertinent Features

To add features to the screen recorder, put the following script in your index.html file:

[...]

<script>
    let recorder = document.getElementById('recorder')

    let videoElement = document.getElementById('videoElement')

    recorder.onclick = async () => {
        let mStream = await navigator.getDisplayMedia({ 'video': true })

        videoElement.srcObject = mStream

        let opts = { mimeType: 'video/webm; codecs=vp9' };
        let rec = new MediaRecorder(videoElement.srcObject, opts);
        let blobs = [];

        rec.ondataavailable = (e) => (e.data && e.data.size > 0) ? blobs.push(e.data) : null;
        rec.onstop = () => {
            //  get the image blob
            let finalBlob = new Blob(blobs, { type: 'video/mp4' });
            // create form data for submission         
            let formData = new FormData();
            formData.append('upload_preset', 'CLOUDINARY_UPLOAD_PRESET');
            formData.append('api_key', "CLOUDINARY_API_KEY");
            formData.append('file', finalBlob);
            var xhr = new XMLHttpRequest();
            xhr.open("POST", 'https://api.cloudinary.com/v1_1/CLOUDINARY_CLOUD_NAME/auto/upload');

            xhr.onreadystatechange = function () {
                if (this.readyState == XMLHttpRequest.DONE && this.status == 200) {
                    console.log(this.status);
                    alert("Video uploaded to your cloudinary media library");
                }
            }

            xhr.send(formData);
        }

        rec.start(100);
        setTimeout(() => rec.stop(), 2000)
    };
</script>

[...]

Be sure to replace the variables CLOUDINARY_UPLOAD_PRESET, CLOUDINARY_API_KEY, and CLOUDINARY_CLOUD_NAME with their values for your Cloudinary account. To learn how to obtain those values, see the next section.

Note the following in the script:

In the first couple of lines, you obtain the button and video elements you created previously in the interface.
Then we assign a function to be executed when you click the recorder button. This function accesses the user's display and then assigns the stream from there to the srcObject property of the videoElement with this code snippet:

let mStream = await navigator.getDisplayMedia({ 'video': true })

videoElement.srcObject = mStream

What is being recorded on your screen then becomes visible. When you run the demo app, you'll see the display, as in this example:

The next section details how to connect a recording with Cloudinary. Afterwards, you can store recordings in the Cloudinary Media Library, gaining access to a wide array of capabilities for manipulating videos.

Connecting With Cloudinary

Understand the Connection Process

Note this code snippet in the index.html file:

[...]        
    let opts = { mimeType: 'video/webm; codecs=vp9' };
    let rec = new MediaRecorder(videoElement.srcObject, opts);
    let blobs = [];

    rec.ondataavailable = (e) => (e.data && e.data.size > 0) ? blobs.push(e.data) : null;
    rec.onstop = () => {
        //  get the image blob
        let finalBlob = new Blob(blobs, { type: 'video/mp4' });
        // create form data for submission         
        let formData = new FormData();
        formData.append('upload_preset', 'CLOUDINARY_UPLOAD_PRESET');
        formData.append('api_key', "CLOUDINARY_API_KEY");
        formData.append('file', finalBlob);
        var xhr = new XMLHttpRequest();
        xhr.open("POST", 'https://api.cloudinary.com/v1_1/CLOUDINARY_CLOUD_NAME/auto/upload');
        xhr.onreadystatechange = function () {
            if (this.readyState == XMLHttpRequest.DONE && this.status == 200) {
                console.log(this.status);
                alert("Video uploaded to your cloudinary media library");
            }
        }
        xhr.send(formData);
    }
    rec.start(100);
    setTimeout(() => rec.stop(), 2000)
[...]

We implement the core upload capability by first creating a media recording with videoElement.srcObject. Below that are the definitions of the properties for the rec variable, which instructs the recorder on how to act in various situations.

The rec.onstop property is of particular interest. When a recording is complete, the recorder obtains the recorded data and sends them as a single blob to Cloudinary with Cloudinary's Upload Presets.

To further handle the videos you have uploaded, leverage Cloudinary. First, create an account there.

Find Out Your Cloud Name

Cloudinary then takes you to your Dashboard (media console), in which your cloud name is displayed under Account Details (see the screenshot below). Replace the CLOUDINARY_CLOUD_NAME and CLOUDINARY_API_KEY variable in the index.html file in the previous code segments with that name.

Create a Cloudinary Upload Preset

Cloudinary Upload Presets enable you to set up the default behavior of your image uploads. That means that, instead of having to add parameters to apply to your images every time you upload one, you can define tags, transformations, and other analysis presets from your Cloudinary console. Simply specify the preset name in your code and you’re good to go!

To create a preset, go to the Upload Settings screen and click the Add upload preset link:

The Add upload preset screen is then displayed:

Type a name of your choice under Preset name, set Mode to Unsigned, and then specify the other details, as appropriate.

Now go back to the index.html file and replace CLOUDINARY_UPLOAD_PRESET with the name of your preset.

Test Uploads to Cloudinary

Now that you've added all your Cloudinary details to the index.html file, go to your Chrome browser and run your demo app. Afterwards, you'll see this display:

Accessing Uploaded Videos

To access your uploaded images, go to your Cloudinary Media Library:

You can now easily and efficiently transform your videos any time in numerous ways: formatting, resizing, cropping, and so forth. For details, see the Cloudinary Documentation

Moving On

You have now learned how to build a simple screen recorder for the web with Cloudinary and JavaScript. For the complete code, see the Github repository. This demo is also available on CodePen.

You can also access this article as a Gitbook tutorial.

Feel free to use the concepts explained in this tutorial to build other apps. Cloudinary offers a wide array of excellent features that make image and video management intuitive, seamless, and fast in both web and mobile apps. Do check them out.

Building an Offline PWA Camera App with React and Cloudinary

Oreoluwa Ogundipe — Tue, 11 Sep 2018 22:18:13 +0000

Note: This project was built as a thought starter for all those participating in the upcoming React Riot 48-hr hackathon. For details of the code, see the GitHub repository. You can view and edit live code on CodeSandbox or checkout out the demo. Hope you enjoy!

This tutorial steps you through the process of building an offline Progressive Web App (PWA) camera app with React and Cloudinary—one that takes pictures with your camera and then uploads it to your Cloudinary media library. A marvelous feature in the app is that when you are offline, the app stores your images in the browser’s local storage (localStorage). Once an Internet connection becomes available, the app automatically uploads all the saved pictures to your media library on Cloudinary.

Installing the Prerequisites

First, install the following software on your machine:

Node .js (version 6 or higher)
Yarn and NPM

Getting Started

Next, create a React app.

Note: If you manage dependencies on your machine with Yarn, download the package runner npx. However, if you use NPM for dependency management, you can skip that step because npx is already bundled with NPM (version 5.2 or higher).

To add npx to Yarn, run this command on your terminal:

    yarn global add npx

Afterwards, create a starter React project, which you will tweak as you proceed with this tutorial:

    npx create-react-app cloudy-cam-pwa

To ensure that the project is in place, go to the application directory and start the development server:

    cd cloudy-cam-pwa
    yarn start # or npm start

The above command starts a development server on http://localhost:3000. Navigating to that URL on your browser displays the React app:

Creating a `Webcam` Class

To grant the app access to your camera, build a Webcam class for the camera’s main capabilities by creating a webcam.js file in the src directory:

    // src/webcam.js
    export class Webcam {
      constructor(webcamElement, canvasElement) {
        this.webcamElement = webcamElement;
        this.canvasElement = canvasElement;
      }

      adjustVideoSize(width, height) {
        const aspectRatio = width / height;
        if (width >= height) {
            this.webcamElement.width = aspectRatio * this.webcamElement.height;
        } else  {
            this.webcamElement.height = this.webcamElement.width / aspectRatio;
        }
      }
    [...]

The Webcam constructor accepts two elements: WebcamElement (videoElement) and CanvasElement. The adjustVideoSize() method adjusts the video element to be proportionate to the size you specified when creating videoElement .

Now add the other methods to the Webcam class, as follows:

    // src/webcam.js
    [...]
      async setup() {
        return new Promise((resolve, reject) => {
          if (navigator.mediaDevices.getUserMedia !== undefined) {
            navigator.mediaDevices.getUserMedia({
                audio: false, video: { facingMode: 'user' }
                })
                .then((mediaStream) => {
                    if ("srcObject" in this.webcamElement) {
                        this.webcamElement.srcObject = mediaStream;
                    } else {
                        // For older browsers without the srcObject.
                        this.webcamElement.src = window.URL.createObjectURL(mediaStream);
                    }
                    this.webcamElement.addEventListener(
                        'loadeddata',
                        async () => {
                            this.adjustVideoSize(
                                this.webcamElement.videoWidth,
                                this.webcamElement.videoHeight
                            );
                            resolve();
                        },
                        false
                    );
                });
          } else {
              reject();
          }
      });
      }

    [...]

The setup() function initializes the camera from the browser and assigns the video stream to your VideoElement in the component. That means granting access to the camera and returning the videoStream function to you.

Here are the methods for capturing images:

    // src/webcam.js
    [...]
      _drawImage() {
        const imageWidth = this.webcamElement.videoWidth;
        const imageHeight = this.webcamElement.videoHeight;

        const context = this.canvasElement.getContext('2d');
        this.canvasElement.width = imageWidth;
        this.canvasElement.height = imageHeight;

        context.drawImage(this.webcamElement, 0, 0, imageWidth, imageHeight);
        return { imageHeight, imageWidth };
      }

      takeBlobPhoto() {
        const { imageWidth, imageHeight } = this._drawImage();
        return new Promise((resolve, reject) => {
            this.canvasElement.toBlob((blob) => {
                resolve({ blob, imageHeight, imageWidth });
            });
        });
      }

      takeBase64Photo({ type, quality } = { type: 'png', quality: 1 }) {
        const { imageHeight, imageWidth } = this._drawImage();
        const base64 = this.canvasElement.toDataURL('image/' + type, quality);
        return { base64, imageHeight, imageWidth };
      }
    }

The _drawImage() method takes the existing frame in videoElement when that function is called and displays the image on canvasElement. The _drawImage() method is then called in the takeBlobPhoto() and takeBase64Photo() methods to handle binary large object (blob) images or Base64 images, respectively.

Creating a `Notifier` Component

Create a components folder in the src directory to hold the components for the app:

    mkdir components

To support offline use and access, you need a Notifier component that identifies the mode that is interacting with the app.

First, create a Notifier folder in your src/components directory:

    mkdir Notifier
    cd Notifier
    touch index.js Notifier.css # on Windows, run the following instead
    # copy NUL index.js
    # copy NUL Notifier.css

Next, install a package called classnames for displaying different colors for the various modes, that is, dynamically rendering different classes:

    yarn add classnames # or npm install classnames

Afterwards, edit your Notifier/index.js file to read like this:

    // src/components/Notifier/index.js
    import React, { Component } from "react";
    import "./Notifier.css";
    import classnames from 'classnames';

    class Notifier extends Component {
      render() {
        const notifyclass = classnames('notify', {
          danger: this.props.offline
        });
        const message = this.props.offline ?
      `CloudyCam is offline! Your images will be saved now and then uploaded to your Cloudinary Media Library once your Internet connection is back up.`
      :
      `Take a picture and it will be uploaded to your Cloudinary Media Library.`;
        return (
            <div className={notifyclass}>
                <p>
                    <em>{message}</em>
                </p>
            </div>
        );
      }
    }

    export default Notifier;

Here, check the value of the offline property that is passed when Notifier is called. If offline is true, the app is in offline mode and the class and message are displayed accordingly.

Edit your Notifier/Notifier.css file to read like this:

    /* src/components/Notifier/Notifier.css */

    .notify{
        background-color: #0066B2;
        padding: 20px;
        text-align: center;
        color: white;
        margin-bottom: 20px;
    }

    .danger{
        background-color: #D77623;
    }

To use the Notifier component, edit the src/App.js file to read like this:

    // src/App.js

    import React, { Component } from 'react';
    import logo from './logo.png';
    import './App.css';
    import Notifier from './components/Notifier';

    class App extends Component {
      constructor() {
        super();
        this.state = {
          offline: false
        }
      }

      componentDidMount() {
        window.addEventListener('online', () => {
          this.setState({ offline: false });
        });

        window.addEventListener('offline', () => {
          this.setState({ offline: true });
        });
      }

      componentDidUpdate() {
        let offlineStatus = !navigator.onLine;
        if (this.state.offline !== offlineStatus) {
          this.setState({ offline: offlineStatus });
        }
      }

      render() {
        return (
          <div className="App">
            <Notifier offline={this.state.offline} />
            <header className="App-header">
              <img src={logo} className="App-logo" alt="Cloudinary Logo" />
              <h1 className="App-title">CloudyCam</h1>
            </header>
          </div>
        );
      }
    }

    export default App;

The App.js component has one state, offline, which specifies whether or not the app is in offline mode. By default, the state is false. When App.js is mounted, the componentDidMount function, which is executed when the app is loaded, listens for the online/offline event and updates the App.js state accordingly.

The render function defines the layout of the app and the Notifier component, passing the offline state as a property to Notifier for display.

Fetch the Cloudinary logo from here and save it in your src directory as logo.png.

Now you might wonder how all that is displayed in the app. In the src/index.js file, the App component is rendered on a <div> tag with the ID root, as follows:

    // src/index.js
    import React from 'react';
    import ReactDOM from 'react-dom';
    import './index.css';
    import App from './App';
    import registerServiceWorker from './registerServiceWorker';

    ReactDOM.render(<App />, document.getElementById('root'));
    registerServiceWorker();

To view your app, first run this command on your development server:

    yarn start

Afterwards, go to http://localhost:3000 on your browser to display the app. Toggle your Internet connection and you will see one of the two versions on display, depending on whether you are online or offline (see below).

Creating a `ClCamera` Component

To put Webcam to use, create a Cloudinary Camera component called ClCamera. First, create a new ClCamera folder in your src/components folder:

    mkdir ClCamera
    cd ClCamera
    touch index.js ClCamera.css # on Windows, run the command
    # copy NUL index.js
    # copy NUL ClCamera.css

Install axios, which enables you to make HTTP requests in the app:

    yarn add axios # or npm install axios

Afterwards, edit the ClCamera/index.js file to read like this:

    // src/components/ClCamera.js

    import React, { Component } from 'react';
    import { Webcam } from '../../webcam';
    import './ClCamera.css';
    import axios from 'axios';

    class ClCamera extends Component {
      constructor() {
        super();
        this.webcam = null;
        this.state = {
          capturedImage: null,
          captured: false,
          uploading: false
        }
      }

      componentDidMount() {
        // initialize the camera
        this.canvasElement = document.createElement('canvas');
        this.webcam = new Webcam(
            document.getElementById('webcam'),
            this.canvasElement
        );
        this.webcam.setup().catch(() => {
            alert('Error getting access to your camera');
        });
      }

      componentDidUpdate(prevProps) {
        if (!this.props.offline && (prevProps.offline === true)) {
          // if its online
          this.batchUploads();
        }
      }

      render() {
            const imageDisplay = this.state.capturedImage ?
                <img src={this.state.capturedImage} alt="captured" width="350" />
                :
                <span />;

            const buttons = this.state.captured ?
                <div>
                    <button className="deleteButton" onClick={this.discardImage} > Delete Photo </button>
                    <button className="captureButton" onClick={this.uploadImage} > Upload Photo </button>
                </div> :
                <button className="captureButton" onClick={this.captureImage} > Take Picture </button>

            const uploading = this.state.uploading ?
                <div><p> Uploading Image, please wait ... </p></div>
                :
                <span />

            return (
                <div>
                    {uploading}
                    <video autoPlay playsInline muted id="webcam" width="100%" height="200" />
                    <br />
                    <div className="imageCanvas">
                        {imageDisplay}
                    </div>
                    {buttons}
                </div>
            )
        }

    [...]

The ClCamera component contains three states:

The capturedImage state, which holds a Base64 version of an image.
A boolean captured state, which specifies whether an image has been captured.
An uploading state, which specifies if an image is being uploaded to Cloudinary.

When the ClCamera component is mounted, the componentDidMount() function creates a canvas element and a Webcam object, passing the videoElement and canvasElement elements as parameters. Afterwards, you initialize the camera feed.

When the app goes from offline to online mode, the componentDidUpdate method calls the batchUpload() method for uploading the images that were saved in the browser’s cache while the app was offline.

Here are the other methods that perform tasks in your app:

When the captureImage() function is clicked, the takeBase64Photo() method is called to capture the image.
The Base64 image is stored in the capturedImage state of ClCamera. And the captured state of the component is set to true.
Two buttons are displayed, which trigger the discardImage method and the uploadImage method, prompting you to either discard or upload the image, respectively. The discardImage() method discards the image from the state of ClCamera and then sets the captured state to false.

    // src/components/ClCamera/index.js
    [...]
        captureImage = async () => {
            const capturedData = this.webcam.takeBase64Photo({ type: 'jpeg', quality: 0.8 });
            this.setState({
                captured: true,
                capturedImage: capturedData.base64
            });
        }

        discardImage = () => {
            this.setState({
                captured: false,
                capturedImage: null
            })
        }

    [...]

The uploadImage function first checks your connection status and does the following:

If the connection is offline, uploadImage creates a new unique string with the prefix cloudy_pwa_ and then stores your Base64 image in the component’s this.state.capturedImage state in the browser’s localStorage. Finally, uploadImage calls the discardImage() method.
If the connection is online, uploadImage makes a POST request to upload your Base64 image along with a Cloudinary Preset as a parameter.

Note: Cloudinary Upload Presets are described in depth later in this tutorial.

    // src/components/ClCamera/index.js
    [...]

        uploadImage = () => {
            if (this.props.offline) {
                console.log("you're using in offline mode sha");
                // create a random string with a prefix
                const prefix = 'cloudy_pwa_';
                // create random string
                const rs = Math.random().toString(36).substr(2, 5);
                localStorage.setItem(`${prefix}${rs}`, this.state.capturedImage);
                alert('Image saved locally, it will be uploaded to your Cloudinary media library once internet connection is detected');
                this.discardImage();
                // save image to local storage
            } else {
                this.setState({ 'uploading': true });
                axios.post(
                    `https://api.cloudinary.com/v1_1/CLOUDINARY_CLOUD_NAME/image/upload`,
                    {
                        file: this.state.capturedImage,
                        upload_preset: 'CLOUDINARY_CLOUD_PRESET'
                    }
                ).then((data) => this.checkUploadStatus(data)).catch((error) => {
                    alert('Sorry, we encountered an error uploading your image');
                    this.setState({ 'uploading': false });
                });
            }
        }

    [...]

Note: Be sure to replace CLOUDINARY_CLOUD_NAME and CLOUDINARY_UPLOAD_PRESET with the actual values in your setup. See the next section for how to obtain those values.

**When ClCamera detects that your Internet connection has been restored, the batchUploads method is called, which searches localStorage for any previously stored images with the findLocalItems method. If no images are found, the function exits. Otherwise, the images are uploaded to the Cloudinary media library through a POST request to the upload endpoint with the image and preset as parameters. The checkUploadStatus method accepts the data response from Cloudinary’s API and then checks if the upload succeeded. In case of an error, checkUploadStatus displays a message to the effect that the image remains in localStorage for the next batch upload.

        findLocalItems = (query) => {
            let i;
            let results = [];
            for (i in localStorage) {
                if (localStorage.hasOwnProperty(i)) {
                    if (i.match(query) || (!query && typeof i === 'string')) {
                        const value = localStorage.getItem(i);
                        results.push({ key: i, val: value });
                    }
                }
            }
            return results;
        }

        checkUploadStatus = (data) => {
            this.setState({ 'uploading': false });
            if (data.status === 200) {
                alert('Image Uploaded to Cloudinary Media Library');
                this.discardImage();
            } else {
                alert('Sorry, we encountered an error uploading your image');
            }
        }

        batchUploads = () => {
            // this is where all the images saved can be uploaded as batch uploads
            const images = this.findLocalItems(/^cloudy_pwa_/);
            let error = false;
            if (images.length > 0) {
                this.setState({ 'uploading': true });
                for (let i = 0; i < images.length; i++) {
                    // upload
                    axios.post(
                        `https://api.cloudinary.com/v1_1/CLOUDINARY_CLOUD_NAME/image/upload`,
                        {
                            file: images[i].val,
                            upload_preset: 'CLOUDINARY_CLOUD_PRESET'
                        }

                    ).then(
                      (data) => this.checkUploadStatus(data)
                    ).catch((error) => {
                        error = true;
                    })
                }
                this.setState({ 'uploading': false });
                if (!error) {
                    alert("All saved images have been uploaded to your Cloudinary Media Library");
                }
            }
        }
    }

    export default ClCamera;

Note: Again, replace CLOUDINARY_CLOUD_NAME and CLOUDINARY_UPLOAD_PRESET with the actual values in your setup. See the next section for how to obtain those values.

The ClCamera component contains these style properties:

    /* src/components/ClCamera/ClCamera.css */

    .captureButton{
      margin-top: 20px;
      padding: 10px;
      padding-left: 20px;
      padding-right: 20px;
      background-color: #0066B2;
      color: white;
      border-radius: 5px;
    }

    .deleteButton{
      margin-top: 20px;
      padding: 10px;
      padding-left: 20px;
      padding-right: 20px;
      background-color: #D77623;
      color: white;
      border-radius: 5px;
    }

    .imageCanvas{
      margin-top: 20px;
      width: 100%;
      height: 200px;
      display: flex;
      justify-content: center;
    }

Setting up a Cloudinary Account

To handle image uploads in this app, leverage Cloudinary. First, create an account there.

Finding Out Your Cloud Name
Cloudinary then takes you to your Dashboard (media console), in which your cloud name is specified under Account Details (see below). Replace the CLOUDINARY_CLOUD_NAME variable in the ClCamera component in the previous code segments with that name.

Creating a Cloudinary Upload Preset
Cloudinary Upload Presets enable you to set up the default behavior of your image uploads. That means that, instead of having to add parameters to apply to your images every time you upload one, you can define tags, transformations, and other analysis presets from your Cloudinary console. Simply specify the the preset name in your code and you’re good to go!

To create a preset, go to the Upload Settings screen and click the Add upload preset link:

The Add upload preset screen is then displayed.

Enter a name under Preset name, set Mode to Unsigned, and then specify the other details, as appropriate.

When the ClCamera component uploads an image from your app, Cloudinary returns a data element that contains the information relevant to the image. That way, if you set up an Upload Preset to perform such tasks as face detection, image-color analysis, and object detection, Cloudinary returns the results to you for use as you deem appropriate. By default, Cloudinary returns the URL of your uploaded image.

Testing and Auditing CloudyCam

ClCamera is now ready for use. Update your App.js file to render the component, as follows:

    // src/App.js

    // other imports
    [...]
    import ClCamera from "./components/ClCamera";

    class App extends Component {

      // other component methods
      [...]
      render() {
        return (
          <div className="App">
            <Notifier offline={this.state.offline} />
            <header className="App-header">
              <img src={logo} className="App-logo" alt="Cloudinary Logo" />
              <h1 className="App-title">CloudyCam</h1>
            </header>
            <ClCamera offline={this.state.offline} />
          </div>
        );
      }
    }

    export default App;

Next, ensure that your development server is running on http://localhost:3000. Navigate to that URL on your browser and verify that the various versions of your app are displayed:

Accessing Uploaded Images
To access all the uploaded images, go to your Cloudinary Media Library:

Feel free to use the images for your app as you desire. The Cloudinary Documentation on the existing usages of Cloudinary is a handy reference.

Creating a Production Build
To serve your app to users, first edit the CloudyCam manifest to read like this:

    # public/manifest.json
    {
        "short_name": "CloudyCam",
        "name": "Clodinary Offline PWA Camera",
        "icons": [
            {
                "src": "favicon.ico",
                "sizes": "512x512 192x192 64x64 32x32 24x24 16x16",
                "type": "image/x-icon"
            }
        ],
        "start_url": "./index.html",
        "display": "standalone",
        "theme_color": "#000000",
        "background_color": "#ffffff"
    }

Recall that the index.js file contains this line of code:

    registerServiceWorker();

It creates a service worker that caches the various assets and sections of your app so that even when your users are offline or have a poor Internet connection, they can still interact with and use CloudyCam.

Create a production build by running this command:

    yarn build # or npm run build

Yarn then creates an optimized production build of your app and places it in the build directory, ready for your users.

Serve the production build with the serve JavaScript package by running these two commands:

    yarn global add serve # or npm install -g serve
    serve -s build

Afterwards, Yarn creates a simple static server on http://localhost:5000. Navigate to that URL for the production version of your app.

Note that a panel on Google Chrome’s Developer Console, powered by Lighthouse, enables you to validate the quality of your web pages. Click the Audits tab of the Developer Console and run an audit on the production build. The results are then displayed:

Here, CloudyCam is shown as a 100-percent PWA app even though the score reads 92. The remaining 8 percent will be achieved once your production server is running with HTTPS for all the app traffic.

Moving On

You have now learned how to build a simple PWA Camera app with React and Cloudinary. For details of the code, see the GitHub repository.

Feel free to use the concepts explained here to build other apps. Cloudinary offers a wide array of excellent features to make image and video management in web and mobile apps intuitive, seamless, and fast. Do check them out. Happy hacking!

Detecting speed of a car from two images

Oreoluwa Ogundipe — Thu, 10 Aug 2017 22:02:36 +0000

Introduction

This article is intended for both beginners and experts in Computer Vision. I hope this post does CV justice ðŸ™ðŸ¾ .

Problem

We already have motion sensors in play that does this sort of thing easily, but if we want to use computer vision, things get tricky. How to get the speed from just pictures may seem impossible at first but after reading this, hopefully things get better.

Approach

To do this we need to follow the following steps

Feed the two images to our application - obviously ðŸ˜‚
Get the difference between the two images
Calculate the ðŸš—ðŸ’¨ from the image difference

Sounds pretty easy right? Lemme take you on a ðŸŽ¢

Feed The Images To Our Application

We are using the default Buffered Image class provided by JAVA to read in the files with their paths specified.

Implementation

BufferedImage image1 = ImageIO.read(new File(img1));
BufferedImage image2 = ImageIO.read(new File(img2));

First Image

Second Image

Image Subtraction To Get The Difference

Now that we have our two images, we then perform Image Subtraction. You may be wondering what image subtraction is.
Images are pretty much an array of many pixel values that have been combined together.
Image Subtraction in the simplest of terms is Finding the difference between every pixel in an image and the corresponding pixel in another image. To perform image subtraction, both images need to have the same resolution (since we are subtracting every pixel in one from the other).
When we subtract the image, we set a threshold and set any pixel value higher than the threshold to BLACK, and pixel values lower than the threshold to WHITE ( We are doing this because we want to see the cars in black after subtracting the images )

Implementation

public BufferedImage ImageSubtract(BufferedImage img1 , BufferedImage img2){

    int imageheight = img1.getHeight();
    int imagewidth = img1.getWidth();   

    WritableRaster image1 = img1.getRaster();
    WritableRaster image2 = img2.getRaster();

    // pixel values have their, red , green and blue components

    int diffred;    
    int diffblue;
    int diffgreen;

    // set threshold for subtraction

    Color treshold = new Color(30,30,30);

    // Create an empty image
    BufferedImage DiffImage = new BufferedImage(imagewidth,imageheight,BufferedImage.TYPE_INT_RGB);

    for(int y = 0 ; y < imageheight ; y++){    
        for( int x = 0 ; x < imagewidth ; x++){
        diffred = Math.abs( image1.getSample(x, y, 0) - image2.getSample(x,y,0));
        diffgreen = Math.abs(image2.getSample(x,y,1) - image1.getSample(x, y, 1));
        diffblue = Math.abs(image2.getSample(x,y,2) - image1.getSample(x, y, 2)); 

            // combine individual rgb components to give one color

            Color diff = new Color(diffred,diffgreen,diffblue);
        DiffImage.setRGB(x, y, diff.getRGB());

            // to visualize the differences in the image

            if(DiffImage.getRGB(x, y) < treshold.getRGB()){
            diff= Color.WHITE;
        DiffImage.setRGB(x, y,diff.getRGB());
            }
        else{
            diff = Color.BLACK;
            DiffImage.setRGB(x,y,diff.getRGB());
        }
    }
    }
    try {
        // write the result image to a file
        ImageIO.write(DiffImage,"jpg",new File("MotionDetected.jpg"));
    } catch (IOException e) { e.printStackTrace(); }        
        return DiffImage;
    }

Result Image

Speed Calculation From Image Difference

Now that we have the difference visualized, its time for the "fun" part ðŸ•ºðŸ¾. Getting the speed from the image. To this, we applied some fun logic
Disclaimer : There may be a better way to implement this, but I was gunning for the method most easy to explain

We run through the "difference" image and then pick the first BLACK pixel you find and then save it. Then we run through the "difference" image again and then pick the last BLACK pixel we find.

Now some assumptions will have to be made

Time passed between the first and second picture say 5s
Assumed Magnification of the camera used to take the pictures
Length of the car in millimeters (say 6000mm)

Once these assumptions have been made, we then covert the length of the car to pixels to make life easier and then get the distance moved,
distance_moved = Math.abs(x2 - x1) - car_length_in_px;
We subtracted the car length in pixel because the distance moved is from the front of the car.
We then solve for speed
speed = (double) distance_moved / timepassed;

More details below

Implementation

public double SpeedCalc(BufferedImage img){
        double speed = 0.0;
        int x1 = 0;
        int x2 = 0;
        WritableRaster newImg = img.getRaster();
        for(int y = 0; y< img.getHeight(); y++){
            for(int x = 0; x < img.getWidth(); x++){
                if( newImg.getSample(x, y, 0)== 0){
                    x1 = x; // thats the the first x black value
                    break;
                }
                else{
                    continue;
                }
            }
            if(x1 > 0){
                break;
            }
        }
        for(int y=0; y< img.getHeight(); y++){
            for(int x = 0; x<img.getWidth(); x++){
                if( newImg.getSample(x, y, 0)== 0){
                    x2 = x; // thats the last x black value
                    break;
                }
            }
        }
        // we only really need the x values 
        int timepassed = 10; // in seconds
        double mag = 0.09; // assumed magnification more details in report
        int car_length = 6000; // in millimeters
        int car_px = (int)(car_length * mag);
        System.out.printf("Final Position of Car:%d, Initial Position of Car:%d \n" , x1,x2);
        int distance_moved = Math.abs(x2 - x1) - car_px;
        speed = (double) distance_moved / timepassed;
        return speed;
    }

Results

$ java MotionDetection images/car_moving_final_1.PNG images/car_moving_final_2.PNG
The resulting difference of the image is stored in MotionDetected.jpg
Time Passed between 2 images 7 seconds 
Final Position of Car:370, Initial Position of Car:1311 
Speed of car is 0.63 metres/second

Here's a link to the github repo if you want to see the full source code
https://github.com/oreHGA/Speed-Detector

Thanks for riding with me! ðŸ‘¨ðŸ¾”ðŸ’»

Hi, I'm Oreoluwa Ogundipe

Oreoluwa Ogundipe — Wed, 12 Jul 2017 08:28:47 +0000

I have been coding for 6 years.

You can find me on GitHub as oreHGA

I live in Lagos.

I work for Hotels.ng as a Software Engineering Intern

I mostly program in these languages: Python, JAVA, PHP

I am currently learning more about Machine Learning and Computer Vision.

Nice to meet you.

DEV Community: Oreoluwa Ogundipe

Building a Screen Recorder for the Web With Cloudinary in a jiffy!

Putting in Place the Prerequisites

Building the Interface

Adding the Pertinent Features

Connecting With Cloudinary

Understand the Connection Process

Find Out Your Cloud Name

Create a Cloudinary Upload Preset

Test Uploads to Cloudinary

Accessing Uploaded Videos

Moving On

Building an Offline PWA Camera App with React and Cloudinary

Installing the Prerequisites

Getting Started

Creating a Webcam Class

Creating a Notifier Component

Creating a ClCamera Component

Setting up a Cloudinary Account

Testing and Auditing CloudyCam

Moving On

Detecting speed of a car from two images

Introduction

Problem

Approach

Feed The Images To Our Application

Image Subtraction To Get The Difference

Speed Calculation From Image Difference

Hi, I'm Oreoluwa Ogundipe

Creating a `Webcam` Class

Creating a `Notifier` Component

Creating a `ClCamera` Component