DEV Community: Alen Duda

Bypassing CORS via custom proxy backend

Alen Duda — Mon, 13 Dec 2021 07:53:07 +0000

As a frontend developer, sooner or later you will encounter a CORS error, something like this:


 to XMLHttpRequest at 'https://...' from origin 'https://...' has been blocked by CORS policy: No 'Access-Control-Allow-Origin' header is present on the requested resource.

CORS intro

Plainly, Cross-Origin Resource Sharing is a security mechanism which enables web browsers to access data from domain2.com while browsing domain1.com. It can also be used to restrict access only to predefined domains. Basically, it requires the backend and frontend to be on the same server or to specifically set allowed origins which can access the backend.

CORS is disabled by default and, if you have access to the server-side code, there are ways to enable it. If you are in a school group project with a backend dev, be sure to remind him/her to enable CORS or you might be stuck with mock data (speaking from experience).

Inspiration

I first encountered the red CORS error in the browser console on a university project one saturday night when I tried to connect to our Java Spring backend and couldn't get it to work, even though it worked from Postman. Since Java (and specifically Spring) was (and is) almost as Ancient Greek to me, I wanted to try a way to bypass this error. Since CORS is on the browser level, an idea popped up: why not build a simple(r) JS backend which does the same API request, but has CORS enabled so I could connect to it instead of the original Java backend.

Express backend

Express.js is the first node.js web framework I encountered and is well-suited for this task. We will create a minimal node/express backend application which uses axios as the http library and the cors package to enable CORS on our server (otherwise this whole ordeal would be pointless).

Project setup and package installation

After creating a folder for our project, open a terminal and navigate to it. We initialize the most basic package.json file with

npm init -y

Once done, we install the required packages:

npm i express cors axios

Before starting to code, we need a file which will be run. Common names are server.js or app.js. Since this project will have all the code in a single file (not the best practice, but for demonstration purposes), we can simply use index.js. Create that file and modify the package.json file so the scripts key looks like this:



 "scripts": {
    "start": "node index"
  },

Coding time

Finally, time to code! Open index.js (or whatever you called it in the previous step) so we can create our server. I will copy all the code required here, along with the comments for (almost) each line.



// packages import
const express = require("express");
const app = express();
const cors = require("cors");
const axios = require("axios");
// enable CORS
app.use(cors());
// set the port on which our app wil run
// important to read from environment variable if deploying
const port = process.env.PORT || 5000;

// basic string route to prevent Glitch error
app.get("/", (req, res) => {
    res.send("Hello World!");
});

// the route we're working with
app.get("/users", (req, res) => {
    // replace with a custom URL as required
    const backendUrl = "https://jsonplaceholder.typicode.com/users";
    // return the data without modification
    axios.get(backendUrl).then(response => res.send(response.data));
});

// console text when app is running
app.listen(port, () => {
    console.log(`Server listening at http://localhost:${port}`);
});

And that is it! You can use the above code and upload it to Glitch, for example, so it can be hosted and accessed if you deploy your frontend app. That's why we require reading the PORT from environment variable (if available) and set a root route to return a simple string, otherwise Glitch would belive the app has an error since nothing is returned.

The "/users" route contains the main code we need to connect to the backend which doesn't have CORS access enabled and returns the same, unmodified data.

Additional bonus: data modification

While you can return the data as-is, nothing stops you from modifying the original response to be more adapted to your frontend app's needs. If there is a lot of data and modifications required, that could improve the frontend app's performance on lower-end devices and slower connections, since less noise data will be received and less modifications are required client-side.

Example response from original backend API:

The code snippet for modifying this is pretty straightforward (assuming the response has the same data structure as above):



    axios.get(backendUrl).then(response => {
        const lastEpisodes = response.data.data.lastAvailableEpisodes;
        const shows = lastEpisodes.map(episode => ({
            id: episode.contentItemId,
            title: episode.caption,
            audioFile: episode.audio.metadata[0].path
        }));
        res.send(shows);
    });

Example custom API response after modification:

I believe you agree that the second response is much cleaner and easier to follow.

Conclusion

This was a very basic example of using a custom, bare-bones backend as a proxy to bypass CORS-restricted content you would generally have access to. It also follows a so-called happy path, meaning there is no error handling, but that would detract from the topic. The whole process from creating the project, modifying the response and deployment to Glitch can take less than 10 minutes, which is much quicker than waiting for your backend-dev colleague to wake up the next morning when the inspiration is gone.

Deploying and building React projects to GitHub Pages with GH Actions

Alen Duda — Mon, 02 Aug 2021 08:30:45 +0000

For my private mini-projects, I prefer to use Parcel bundler over Create React App just because there is less overhead and clutter to clean up. However, by using CRA and the appropriate package, deploying can be very simple. This post will tell you how to set up build and deploy when using a custom project structure.

Creating a React/Parcel project

By following steps from this post we can be up and running in a few minutes. As always with React, you are free to organize the project structure as needed.

Build for GitHub Pages

Enable GH Pages inside repo settings (use docs folder, as detailed in the documentation )
Specify a build script - Parcel should create built files inside /docs folder. Example build script inside package.json:

"build-github": "rm -rf docs/* && parcel build index.html --public-url ./ -d docs"

These two commands:

delete the existing docs folder and all its contents (to make sure we don't have previously built files lying around)
run the Parcel build command, as always using index.html as entry point, but specifying the built files to go inside /docs folder, while setting the output files to read from that folder (./)

To test this, we can try running npm run build-github command and committing the changes, pushing them to master. A green checkmark should appear on your GH repository near the latest commit, indicating all went well. The default URL template for GH Pages is https://<your-github-username>.github.io/<your-github-repository>

Automating the build with every push to master

To avoid having to run that command manually, we should automate this step by using GitHub Actions, another free service from GitHub.

In your repository's project root, create a folder named .github and inside it another folder called workflows. There, you should create a file named build.yml (this name is optional).

Example build.yml file:

name: Build gh-pages
on:
  push:
    branches: [ master ]
jobs:
  build-gh-pages:
    runs-on: ubuntu-latest      
    steps:
      - uses: actions/checkout@v2
      - uses: actions/setup-node@v1
        with:
          node-version: 12
      - run: npm ci
      - run: npm run build-github
      - name: Commit files
        id: auto-commit-action
        uses: stefanzweifel/git-auto-commit-action@v4
        with:
          commit_message: build project for github pages
      - name: Push changes
        if: steps.auto-commit-action.outputs.changes_detected == 'true'
        uses: ad-m/github-push-action@master
        with:
          github_token: ${{ secrets.GITHUB_TOKEN }}
      - name: No changes detected
        if: steps.auto-commit-action.outputs.changes_detected == 'false'
        run: echo "No changes!"

After pushing this file to your remote repository, GitHub will read it and follow steps inside it: on every push to the branch master, it will run the steps detailed in build-gh-pages job (name optional) - do a clean npm install (npm ci) and run the npm run build-github command we created earlier. Make sure you specify the correct node version for your project, as well as replacing the build-github npm command with your custom naming, if required. Feel free to modify the commit_message variable as well!

To test this, simply make some changes to your project and push them to the remote repository. An orange circle should appear near the latest commit message on GH repo page, where you can track the progress of the build process. Once ready, a green checkbox will replace the circle and your changes should be visible.

For me, this mini-automation process was a great introduction to GH Actions. I borrowed some code and inspiration from my colleagues Davor and Maroje with only minor customization required. I hope it will help someone and spark an interest into the wonderful world of automation!

Installing fonts on a remote server

Alen Duda — Mon, 17 May 2021 18:33:12 +0000

Recently we encountered an interesting situation after migrating our web application to a new server - all the generated PDFs started to look slightly different than before. This was unexpected as no changes were made to fonts, yet after a short research the conclusion was that the fonts were different.

We used Adobe Acrobat to check for font names and found three different results for old server, new server and locally generated PDFs. This pointed to the fact that the generic sans-serif font family was declared and the tool we use for PDF generation (wkhtmltopdf) used the default sans-serif font available on the operating system the application was running on (Helvetica on local Mac OS and DejaVu Sans on new CentOS Linux 7 server). Since the documents were required to look the same and wkhtmltopdf has issues with Webfonts, the agreed solution was to install the same old font and set it as the default sans-serif on the new remote server.

Gathering information and preparing directories

After SSH-ing to the remote server, we checked which operating system was present:



cat /etc/os-release

We also used the following commands to check



fc-match -a  # print which fonts are used
fc-list # print where font files are located

This gave us enough information for further steps.

On CentOS, custom font files for each user need to be placed inside ~/.fonts directory. Since that directory didn't exist, we created it with



cd # make sure we are in home directory
mkdir .fonts

Uploading and installing the font

After the .fonts directory is created, we needed to upload the font files in that directory. This can be done in multiple ways, but here we will mention the two simplest (if you don't have sudo access).

Option 1: `wget` from a third-party server

This is a great option if the font in question is available on a publicly available third-party server like TTFonts. Inside the .fonts directory we run:



wget --no-check-certificate "http://ttfonts.net/sfonts/2/27260_NimbusSanL.ttf"
wget --no-check-certificate "https://ttfonts.net/sfonts/2/27258_NimbusSanLBold.ttf"

Notice we used both regular and bold font files.

Option 2: `scp` from local machine

In case you have the needed font files available on your machine, you need to upload them to the remote. Our colleague Maroje recently wrote a great article about how to do just that and it was very useful.

For this option, open a terminal on your local machine and navigate to the directory which contains your font files.
Since we use SSH on our server and login without a password, we used the following command to copy all .ttf files in current directory to remote server:



scp -i /Users/username/.ssh/public_key.pub *.ttf remote-user@remote-host-ip:~/.fonts

For more information please see Maroje's post as he explained it in more detail.

Setting up the alias

With the font files present on the server, they are ready to be used... but our situation required us to specify that font as the default sans-serif.

This was done by navigating to ~/.config directory and creating a new fontconfig directory to hold our font alias file, which we create using the ubiquitous nano editor.



cd ~/.config
mkdir fontconfig
cd fontconfig
nano fonts.conf

The content of fonts.conf file:



<?xml version='1.0'?>
<!DOCTYPE fontconfig SYSTEM 'fonts.dtd'>
<fontconfig>
<alias>
  <family>sans-serif</family>
  <prefer><family>NimbusSanL</family></prefer>
</alias>
</fontconfig>

And that's it! The new font started being used immediately, even though several references advised to use:



 bash

fc-cache -f -v

Conclusion

This was a short tutorial-like post on font installation on a remote Linux machine without GUI. As a mostly front-end dev, this was somewhat out of my comfort zone but very interesting and I was happy with the result. Hopefully this helps somebody and I will put the references for more info.

References

Cover image source

Using Google Sheets as a simple database with Papa Parse

Alen Duda — Mon, 15 Mar 2021 09:23:18 +0000

Inspiration

On a recent meeting, a fellow dev commented on a piece of software he saw as being "just like Windows 95". This brought up some nostalgia inside me and I quickly remembered there are several libraries available for recreating that classic look and feel.

One of the first results was React95 and the first link inside the showcase was a recipe app from one of the library's contributors. The readme stated it used Google Sheets as a database, which I found interesting and decided to investigate further.

Tabletop

The promising library user in the aforementioned example was Tabletop. I was excited while reading their npm page, but the Github readme stated how the package is getting deprecated by Google's changes. It's a shame since that package seemed to offer many utility features for listing data from spreadsheets out of the box (including reading data from multiple sheets). Luckily, they provided an alternative (albeit much more general-purpose, but still usable for simple examples).
The alternative in question is...

Papa Parse

Upon first look, Papa Parse seems to be a general-purpose CSV parser with a no-nonsense quick-start documentation that is readable at a glance. The parser will return JSON data which can be used for any purpose required, e.g.:

creating tables
visualization via charts
iterating through data to create blocks (by using a templating engine)

Creating a data source

The premise is simple: create a new Google Spreadsheet, enter the headers (column names) and fill the table with data (rows).

Next, we need to publish the worksheet: go to File -> Publish to the Web.

Make sure to click the Start publishing button. We need to parse CSV files, so we need to pass the link to CSV option to our code. If we only have a single sheet, this should suffice. However, if there are multiple sheets, we need to get links to each one by selecting them from the dropdown (and making sure CSV is selected).

Example

To demonstrate one way of using Google Sheets as a data source, we will use 2 sheets and a Codepen repo. The data is simple: I listed some movies and shows I've watched recently. Data will be shown inside two tables with minimal styling. To simplify HTML generation, Vue.js will be used.

The Sheet is available here and consists of two sheets: Movies and Shows. They have almost the same structure, with the only difference being Shows' Last watched season replacing Movies' Year column.

I followed the instructions above to get separate URLs for each sheet's CSV file and saved them to variables.

const moviesUrl =
  "https://docs.google.com/spreadsheets/d/e/2PACX-1vTtKbv_S8Fdo3HLhm64Tc94WZ6FuqtzqePIjuejNFJxKkUvAE8JF8V2KgKoz1n5jQUDfL8A3F-QoDWk/pub?gid=0&single=true&output=csv";

const showsUrl =
  "https://docs.google.com/spreadsheets/d/e/2PACX-1vTtKbv_S8Fdo3HLhm64Tc94WZ6FuqtzqePIjuejNFJxKkUvAE8JF8V2KgKoz1n5jQUDfL8A3F-QoDWk/pub?gid=1364847678&single=true&output=csv";

Next, a Vue instance is created and its data is initialized to empty arrays for movies and shows.

const app = new Vue({
  el: "#app",
  data: function () {
    return {
      movies: [],
      shows: []
    };
  },
...

Upon creation, Vue is told to use Papa Parse to get each CSV file and set it to its state. As a side note in this example, I used an arrow function as a callback for movies and a regular anonymous function for shows. Arrow function retained the lexical scope of this, while the regular function required this to be saved (this should refer to the Vue instance in order to correctly set data)

created: function () {
    this.fetchMovies();
    this.fetchShows();
  },
  methods: {
    fetchMovies() {
      Papa.parse(moviesUrl, {
        download: true,
        header: true,
        complete: (results) => this.movies = results.data
      });
    },
    fetchShows() {
      const _this = this;
      Papa.parse(showsUrl, {
        download: true,
        header: true,
        complete: function(results) {_this.shows = results.data;}
      });
    }
  }

The presentation part is very simple - a single <div> container for Vue instance and two tables with a template which iterates over the fetched data.

<div id="app">
  <table v-if="movies.length">
    <thead>
      <tr>
        <th>Title</th>
        <th>Year</th>
        <th>URL</th>
      </tr>
    </thead>
    <tbody>
      <tr v-for="movie in movies">
        <td>{{movie.Title}}</td>
        <td>{{movie.Year}}</td>
        <td><a :href="movie.Url" target="_blank">{{movie.Url}}</a></td>
      </tr>
    </tbody>
  </table>

  <table v-if="shows.length">
    <thead>
      <tr>
        <th>Title</th>
        <th>Last watched season</th>
        <th>URL</th>
      </tr>
    </thead>
    <tbody>
      <tr v-for="show in shows">
        <td>{{show.Title}}</td>
        <td>{{show["Last watched season"]}}</td>
        <td><a :href="show.Url" target="_blank">{{show.Url}}</a></td>
      </tr>
    </tbody>
  </table>
</div>

Notice how the shows' Last watched season column retains spaces just like we defined inside Google Sheets.

Conclusion

This article is purposefully simplistic and is only used as an intro to what might be possible by using Google Sheets as a data source. Its collaborative nature makes it easy to make changes which make the frontend update (similar to a very basic CMS) while retaining the ability to limit read/write access as usual.

However, there are limitations - speed, possible rate limiting and the dependency on Google to keep the CSV option available in the future. As such, this will never replace a proper database and backend combination, but can still be useful for quick prototyping and fun mini-projects, especially when the data is already available.

DEV Community: Alen Duda

Bypassing CORS via custom proxy backend

CORS intro

Inspiration

Express backend

Project setup and package installation

Coding time

Additional bonus: data modification

Conclusion

Deploying and building React projects to GitHub Pages with GH Actions

Creating a React/Parcel project

Build for GitHub Pages

Automating the build with every push to master

Installing fonts on a remote server

Gathering information and preparing directories

Uploading and installing the font

Option 1: wget from a third-party server

Option 2: scp from local machine

Setting up the alias

Conclusion

References

Using Google Sheets as a simple database with Papa Parse

Inspiration

Tabletop

Papa Parse

Creating a data source

Example

Conclusion

Option 1: `wget` from a third-party server

Option 2: `scp` from local machine