DEV Community: Diego Sánchez

Do you use UML to model complex web applications?

Diego Sánchez — Fri, 25 Sep 2020 02:24:17 +0000

A few weeks ago I was learning from a course object oriented design in which they taught me class, state machine and sequence diagrams using the The Unified Modeling Language.

I like web development so I asked myself if these diagrams, specifically class diagrams, could be used to model web applications, preferably complex ones.

At first I though this wouldn't be possible since JavaScript is a prototype based language even tho having a class keyword.

Doing a little bit of research I found and read two papers that addressed this.

The second paper gives good reasons on why use UML to model web apps in chapter 1.4.

What they do to use UML is to extended it with stereotypes, constraints and tagged values, this new extension is called Web Application Extension (WAE).

As you can see these are old papers, the examples that they show are for what today are called server side rendered web apps.

I want to know if any of you have used UML to model web pages or if you like to consider UML to do so.

mp3split: a simple shell script for splitting mp3 audio files

Diego Sánchez — Sun, 13 Sep 2020 20:54:45 +0000

Introduction

Downloading mp3 files these days or converting a music video to an mp3 file for later offline playback hasn't been easier these days with programs like youtube-dl and other alike.

For example if you liked a music mix in youtube, lets say, a 'syntwave mix' for your programming sessions and you have downloaded it with one the programs mentioned above, you now have a single mp3 file. But what happens if you didn't like a song in the mix ? Well at first you might think you can't do anything about it, because how you do delete that one annoying song and keep the rest ?

The solution to that problem is to split the single mp3 file into many more mp3 files and of course delete the one you don't want to keep.

To begin splitting the mp3 file we would need the time where each song starts i.e timestamps. Many of those music mixes found in youtube give in the video description a 'tracklist' which is primarily compose of a timestamp and the name of the song.

For example, the one found in the syntwave mix

0:00 Xtract - Audiotool Day 2016
3:55 Alison - space echo
7:25 Volt Age - Volt's Theme 
13:12 Lucy in Disguise - Southbound 
18:05 Lucy in Disguise - Echoes In Time
23:15 HOME - Flood
26:53 De Lorra/Augustus Wright - Let Us 
31:09 bl00dwave - Encounters
33:51 Emil Rottmayer - T.I.M.E ( Part 2 )
40:12 oDDling - Early Bird
43:22 hello meteor - at last light

We could do the splitting manually with ffmpeg, but it would take a lot of time. Today I present to you mp3split a simple shell script to aid you in the process of splitting large mp3 files with the help of tracklist and a single dependency ffmpeg.

Usage example

To get started, install the script running the following commands.

wget https://raw.githubusercontent.com/diegosanchezp/mp3split/master/mp3split.sh -O ~/.local/bin/mp3split && chmod 755 ~/.local/bin/mp3split

This will put the shell script in the folder ~/.local/bin/ and can be accessed via terminal as a binary file.

Run mp3split -h to test if the program is installed.

The script takes as input an mp3 file and a tracklist like the one above.

To walk you trough the usage I have downloaded the syntwave mix with youtube-dl, rename it to testsong.mp3 and put the tracklist in a text file named tracklist.txt.

If you want to split testsong.mp3 automatically, run

mp3split testsong.mp3 tracklist.txt

It will output information about the created split files to stdout

=== Begin to create mp3 split files ===
Processed 0:00 to 3:55; Xtract - Audiotool Day 2016.mp3
Processed 3:55 to 7:25; Alison - space echo.mp3
Processed 7:25 to 13:12; Volt Age - Volt's Theme.mp3
Processed 13:12 to 18:05; Lucy in Disguise - Southbound.mp3
Processed 18:05 to 23:15; Lucy in Disguise - Echoes In Time.mp3
Processed 23:15 to 26:53; HOME - Flood.mp3
De Lorra/Augustus Wright - Let Us.mp3: No such file or directory
Processed 26:53 to 31:09; De Lorra/Augustus Wright - Let Us.mp3
Processed 31:09 to 33:51; bl00dwave - Encounters.mp3
Processed 33:51 to 40:12; Emil Rottmayer - T.I.M.E ( Part 2 ).mp3
Processed 40:12 to 43:22; oDDling - Early Bird.mp3
Processed 43:22 to 0:45:55.680000; hello meteor - at last light.mp3

Let's list what files have created it.

3,1M    Alison - space echo.mp3
2,4M    bl00dwave - Encounters.mp3
5,8M    Emil Rottmayer - T.I.M.E ( Part 2 ).mp3
1,5M    hello meteor - at last light.mp3
3,4M    HOME - Flood.mp3
4,9M    Lucy in Disguise - Echoes In Time.mp3
5,2M    Lucy in Disguise - Southbound.mp3
3,1M    oDDling - Early Bird.mp3
5,3M    Volt Age - Volt's Theme.mp3
3,5M    Xtract - Audiotool Day 2016.mp3

As you can see a file is missing De Lorra/Augustus Wright

ffmpeg throws the following error.

De Lorra/Augustus Wright - Let Us.mp3: No such file or directory

What happened here? ffmpeg the program that helps to create the splits, interpreted the filename as directory followed by a name, you might want to fix it by substituting the / by a another character.

To end the article, I'd have to say that you could also apply this idea of splitting with audio books.

Thanks for reading.

Useful links

diegosanchezp / mp3split

Shell script for splitting or slicing large mp3 audio files

MP3split

A Shell script for splitting or slicing large mp3 audio files.

The idea of this program is to take an mp3 file e.g a music 'mix' downloaded from youtube or another site and split it with a timestamps file found on youtube or another site, you might want to adjust it to the format.

Required dependencies

FFmpeg

Install these dependencies with your package manager.

For example in Ubuntu and the rest of Debian based systems.

sudo apt install ffmpeg

Installation instructions

Download the script

wget https://raw.githubusercontent.com/diegosanchezp/mp3split/master/mp3split.sh -O ~/.local/bin/mp3split

This will put the shell script in the folder ~/.local/bin/ and can be accessed via terminal as a binary file.

Also the command above can be executed to update the script.

You must have the folder ~/.local/bin/ added to your $PATH variable, if you don't want…

View on GitHub

I've made a web component to easily implement and switch two themes in a webpage.

Diego Sánchez — Sat, 05 Sep 2020 21:23:04 +0000

A lot of websites nowadays have this feature of having two color schemes for the user to choose, namely "dark" and "light", implementing this is a peace of cake with the help of CSS custom properties (variables) and a bit of ECMAscript. Today I present you a web component that abstracts the ECMAscript part and lets you switch between the themes dynamically.

Using the component is straightforward. The component adds and updates a data attribute "data-theme" to the body element which you can handle in CSS via attribute selectors, so depending on the theme you have you can update your theme variables.

See a live demo before using it.

Usage example

To install it use your preferred package manager.

npm install theme-switcher-component

yarn add theme-switcher-component

In a .html file include and declare the component.

<html lang="en">
  <head>
    <meta charset="utf-8">
    <title>Theme switcher component demo</title> 
    <link rel="stylesheet" href="./styles.css">
    <!-- Include the component -->
    <script type="module" src="./node_modules/theme-switcher-component/switch.js" defer></script>
  </head>

  <body>

    <main>
      <section>
        <p>Components are not applications, they are just a part of the whole.</p>
        <!-- Declare the component -->
        <!-- The first element of the array themes defines the default theme -->
        <theme-switcher themes='["light", "dark"]' verbose></theme-switcher>
      </section>
    </main>

  </body>
</html>

Note: other names in the themes array rather than the typical "light" and "dark" can be defined, for example, "solarized" and "tango".

In a .css file declare in the root selector the css variables to use and change their values accordingly to the two themes defined selecting the body.

/* Define a theme with variables */
:root {
  /* Light theme the default*/
  --border-color: black;
  --bg-color: white;
}

/* Change variables values accordingly*/

body[data-theme="dark"]{
  --border-color: white;
  --bg-color: black;
}

/* ... More CSS rules below ... */

After a theme is changed, it gets stored to localStorage, if you want to see values of the local store and the current theme set the verbose attribute to the custom element, like the example.

That's it ! Hope this helps you.

If you want more in depth documentation check the github repository

diegosanchezp / theme-switcher-component

A web component that helps you implement and change two themes on a web page.

And the live demo code.

Using nodejs packages in the browser with import maps

Diego Sánchez — Mon, 17 Aug 2020 17:44:12 +0000

Import maps is a browser proposal that allow web pages to control the behavior of JavaScript imports, in this article will cover the usage of this feature with nodejs packages, the related problems, and posible solutions.

What are import maps
Using import maps
Generating import maps
The problem of NodeJs packages
Heritage a package manager for modern web development
Conclusion
Additional notes
References

What are import maps

Import maps is a browser proposal that allow web pages to control the behavior of JavaScript imports, in other words control over what URLs get fetched by JavaScript import statements and import() expressions.

A NodeJs developer is used to import packages this way

import { html } from 'lit-element';

If we tried to do that directly in the browser it will fail, throwing an error similar to

  Uncaught TypeError: Failed to resolve module specifier "lit-element". Relative references must start with either "/", "./", or "../".

This is happens because the specified path ('lit-element') is 'bare', meaning that is not the full path to the file where the source code of that object is.

There is also the problem that if we gave the full path of the file, that file could also be importing another package in the same way, with a bare path.

The typical workaround for this problem is to use a bundler like Rollup or Webpack that resolves all of the paths. And puts all of the required code into one single file.

There is the exception of packages that have zero dependencies (meaning that they simply do not import from another packages), they do things like relative imports, for example,import { defaultTemplateProcessor } from './lib/default-template-processor.js'; or have the so called "dist" files which have all of the required code in one single file.

In those cases we could do something like import { foo } from './node_modules/lib-foo/file.js' there will be no problems.

But what if we wanted to still use the bare paths syntax, since is better and more comfortable to work with and have something that tells that a bare path points to a full path. That's exactly what import maps does!

Quoting from the specification on chromestatus

Until import maps, the web has had no native ability to do this kind of mapping (although the specification has included a carveout for it since modules were first introduced). Import maps bring this ability to the web's native module system, putting it on parity with web developer expectations.

Using import maps

Currently import maps are a experimental feature, the only browser that has implemented it is google-chrome, so, to start using them we have two methods

Enable import maps on google-chrome by putting in the address bar the url chrome://flags/, then search for import maps in the search bar and enable the flag "Experimental Productivity Features".

Use a polyfill, in this moment there are two,es-module-shims and SystemJS for the following example I'll be using the first one.

Let's begin setting up the example by installing the required dependencies

es-dev-server we are going to need a server because the polyfill uses the fetch api.
es-module-shims the polyfill that's going to do the import magic
lit-element the dependency that uses the npm system

Install the dependencies with yarn only, we'll need the generated yarn.lock file for later.

yarn add es-dev-server -D

yarn add es-module-shims lit-element

Then we are going to craft the html file which will have the import map and the javascript code that uses imports with the bare syntax.

Make an html file that requires the lit-html dependency

<!-- index.html -->
<html>
  <head>
    <title>ES Import Maps Shim</title>
    <!-- Include the polyfill -->
    <script defer src="./node_modules/es-module-shims/dist/es-module-shims.min.js"></script>

    <!-- Declare the import map wich is a json object that starts with the key "imports". 
    This is an object that contains the mapping of the import URLs to the node_modules folder -->
<!--Line 11 below-->
    <script type="importmap-shim">
      {
        "imports": {
          "lit-html": "./node_modules/lit-html/lit-html.js",
          "lit-element": "./node_modules/lit-element/lit-element.js",
          "lit-html/lit-html.js": "./node_modules/lit-html/lit-html.js",
          "lit-html/lib/shady-render.js": "./node_modules/lit-html/lib/shady-render.js"
        }
      }
    </script>

    <!-- JavaScript code that uses bare module syntax -->
    <!-- Line 23 below -->
    <script type="module-shim">

      import { LitElement, html, css } from 'lit-element';

      class CustomBtn extends LitElement {
        static get styles() {
          return css`
            button {
              color: blue;
            }
          `;
        }

        render() {
          return html`
            <button>Custom Button</button>
          `;
        }
      }
      customElements.define('custom-button', CustomBtn);
    </script>
  </head>
  <body>
    <custom-button></custom-button>
  </body>
</html>

Next serve the html file

npx es-dev-server --app-index index.html --watch --open

Let's explain some things about the example.

We are making a simple web component with LitElement base class.
The import map makes the bare path in line 25 to be resolved to import { LitElement, html, css } from './node_modules/lit-element/lit-element.js';. It does the same for the internal imports of lit-element.
If you are using chrome with the enabled flag “Experimental Productivity Features”. You don't need line 6, change line 11 to <script type="importmap"></script> and line 23 to <script type="module"></script>.

There is still one thing you might be asking. How did you know how to list those other imports ? If you are only importing lit-element. Well that's one thing we have to take care about, the internal imports of lit-element, doing this by hand might be a huge pain, the import map you see above took me a few minutes to setup, I've had to analyze the package.json of lit-element and lit-html since the first one is dependent on the last one. Even the import map that I've provided could fail for others imports of lit-html.

So the ideal is to have something more robust, a tool that automatically generates the import map for ourselves.

Generating import maps

We'll use the package @import-maps/generate to generate the import map. The caveats are that it only supports yarn.lock file ( this is why the dependencies must have be installed only with yarn at the beginning ) and that Windows paths are not supported.

Install @import-maps/generate as a development dependency

yarn add @import-maps/generate -D

To generate the import map run the command

npx @import-map/generate

This will output the importmap in a file named import-map.json, below is an example

{
  "imports": {
    "es-module-shims/": "/node_modules/es-module-shims/dist/",
    "lit-element": "/node_modules/lit-element/lit-element.js",
    "lit-element/": "/node_modules/lit-element/",
    "lit-html": "/node_modules/lit-html/lit-html.js",
    "lit-html/": "/node_modules/lit-html/"
  }
}

Ideally you'll want to generate the import map after installing any additional dependency, to do so add to package.json a post install script

"scripts": {
  "postinstall": "generate-import-map"
}

Lastly include the import map by replacing line 11 on the index.html file used before.

<!-- index.html -->
<html>
  <head>
    <title>ES Import Maps Shim</title>
    <script defer src="./node_modules/es-module-shims/dist/es-module-shims.min.js"></script>

    <!-- Generated import map -->
    <script type="importmap-shim" src="import-map.json"></script>
    <!-- ... -->

  </head>
  <body>
    <custom-button></custom-button>
  </body>
</html>

The problem of NodeJs packages

Some NodeJs packages might not be compatible with the browser, back before the import/export (ESM modules) syntax, require() (CommonJS) was used instead. There is a chance that some package uses a dependency that uses the old syntax making it unusable in the browser since it only understands import/export. The solution to the problem would be to simply convert the old syntax to the new one, there is a tool that does that automatically Snowpack, the next example is about it.

The reason why I choose the lit-element package was because it's ESM ready, thus avoiding the problem described above.

For the example that covers this topic will continue using web components, instead of making one we will download the paper-button from @polymer. The characteristic of this is that it's not declared as an ESM module. If you inspect the node_modules/@polymer/paper-button/package.json it has no "module" field, even do the file main file uses import syntax.

Running the generator from the last example, will trow an exception

(node:10621) UnhandledPromiseRejectionWarning: Error: Cannot find module '@polymer/iron-behaviors'

I've inspect the package.json of 'iron-behaviors' and didn't find the "main" field. Which is probably what's causing problems.

Back to Snowpack, it acts as a middleware or intermediary between the dependencies stored in the node_modules folder and the web browser making these fully compatible. Takes as input a .js file declared as main in package.json, analyzes the declared imports and outputs a web_modules folder with bundled javascript files and the respective import map.

To start the example install snowpack

yarn add snowpack -D

And the component

yarn add @polymer/paper-button

Create a new snowpack.html file

<!-- snowpack.html -->
<html>
  <head>
    <title>Import Maps with Snowpack</title>

    <script defer src="./node_modules/es-module-shims/dist/es-module-shims.min.js"></script>

    <!-- Import map is located in the web_modules folder -->
    <script type="importmap-shim" src="./web_modules/import-map.json"></script>

    <script type="module-shim" src="main.js"></script>
  </head>

  <body>
    <paper-button raised>Another Button</paper-button>
  </body>

Create main.js file, inside this file import the paper-button

import '@polymer/paper-button/paper-button.js';

Update your package.json main field

{
  "main": "main.js"
}

Run the command to generate the web_modules folder

npx snowpack

If you inspect the web_modules folder there is one single js file per dependency

web_modules/
├── import-map.json
├── lit-element.js
└── @polymer
    └── paper-button
        └── paper-button.js

If some of the dependencies share a common dependency they will be put in a common folder.

Like the last generator you'll want to update the import-map.json every time a dependency is added, to do so, add to package.json a postinstall script.

"scripts": {
  "postinstall": "snowpack"
}

Run the server
npx es-dev-server --app-index snowpack.html --watch --open

You should see a blank page with a button

Heritage a package manager for modern web development

We have seen some of the problems related to node packages and how to fix them, the proposed solution required to use snowpack to act as a middleware and convert those packages to ESM modules to be used in the browser, the conversion outputted a web_modules folder which is a shallow copy of node_modules, you might find this a problem when the number of dependencies grow, since there will be code duplication.

But hey, what happens if you are a person that lives on the edge an only installs ESM ready dependencies and even don't like to much the node_modules folder because is heavy on disk usage, then it wouldn't make sense to use snowpack. You could rely on a content delivery network like unpkg to import packages and then craft an import map that points to unpkg url's, for example

{
  "imports": {
    "lit-html": "https://unpkg.com/lit-html@latest/lit-html.js?module",
    "lit-element": "https://unpkg.com/lit-element@latest/lit-element.js?module",
    "lit-html/lit-html.js": "https://unpkg.com/lit-html@latest/lit-html.js?module",
    "lit-html/lib/shady-render.js": "https://unpkg.com/lit-html@latest/lib/shady-render.js?module"
  }
}

This approach looks OK, but there is something that downgrades the development experience, every time when the web page is initially served by a static server, for example, the one we been using es-dev-server it will have to make a request to the unpkg servers to get those js files, this is something that I don't personally like since my Internet is slow, what I would like is to make the request once and get those files stored in the disk, something similar of what Deno does to "cache" it's dependencies.

Heritage is a package manager for the web based on the WICG/import-maps specification, it only have two dependencies acorn and node-fetch. Heritage installs node packages is a special way, it downloads them by default from Pika and Unpkg registries and puts them in a folder named web_modules similar to snowpack. The obvious difference here is that it doesn't need to have a node_modules folder, and for me it's a big plus.

Let's see an example similar to the others where we install two ESM ready packages lit-element and @lit-element-bootstrap/button with heritage. Since heritage generates a web_modules folder this will conflict with the previous setup.

Create a new folder

mkdir heritage
cd heritage

Install heritage

yarn add -D @eliaspourquoi/heritage

Installing dependencies with heritage is similar to yarn

npx heritage add lit-element @lit-element-bootstrap/button

What heritage will do is to use the package.json to register required packages under the webDependencies field, exactly like Snowpack before version 2. The lock file is the generated import-map.json used to manage imports by the browser.

Check the package.json

"webDependencies": {
  "lit-element": "2.3.1",
  "@lit-element-bootstrap/button": "1.0.0"
}

One minor problem here, installing es-module-shims with heritage trowed an exception, so I'll have to install it with yarn.

yarn add es-module-shims

Install the server

yarn add es-dev-server -D

Use installed packages

<!-- heritage.html -->
<html>
  <head>
    <title>Heritage</title>
    <!-- Include the polyfill -->
    <script defer src="./node_modules/es-module-shims/dist/es-module-shims.min.js"></script>

    <!-- Include the import map generated by heritage -->
    <script type="importmap-shim" src="web_modules/import-map.json"></script>

    <!-- Make a custom button, and use the downloaded web component -->
    <script type="module-shim">

      import { BsButton, BsButtonDarkCss } from '@lit-element-bootstrap/button';
      import { LitElement, html, css } from 'lit-element';

      class CustomBtn extends LitElement {
        static get styles() {
          return [
            BsButtonDarkCss 
          ];
        }

        render() {
          return html`
            <bs-button context="dark">Dark custom bootstrap button</bs-button>
          `;
        }
      }
      customElements.define('custom-button', CustomBtn);
    </script>
  </head>
  <body>
    <custom-button></custom-button>
  </body>
</html>

Run the server

npx es-dev-server --app-index heritage.html --watch --open

Conclusion

Import maps is a great proposal that enhances the development workflow and makes a bridge between the npm ecosystem and the browser, sadly is not all pretty, there is compatibility problems between the import system that uses the browser ESM and the one that nodejs used in the past commonJS, snowpack has provided a way to breach into this problem.

The one thing that I personally don't Like about snowpack is that is that there will be duplication of code, the node_modules and web_modules folder are similar. I guess that's the price to pay for a package manager that was not designed to work with the browser.

Heritage provides a morden way to install packages, it's still a new technology so I dont expect it to be as robust and have the same features as the yarn package manager for example.

Additional notes

Check out the github source code repository with all the examples included!

diegosanchezp / import-maps-examples

Examples of import-maps

Import Maps Examples

Simple examples of import maps using the polyfill es-module-shims. These examples come from the blog post, visit for better explanation.

The purpose of these examples is try to use lit-element and @polymer/paper-button with bare module specifiers in the browser without a bundler.

Get started

Dependencies can only be installed with yarn yarn install

The import-map.json file should be auto generated after installed dependencies, if not yarn run postinstall

Generate the web_modules folder npx run snowpack

To see the examples in the browser, run the scripts in package.json

Notes

@polymer /paper-button had to be installed as a development dependency, since it would have conflicts with @import-maps/generate
If you run the script "serve-chrome", you'll need to have the flag "Experimental Productivity Features" enabled on chrome. Read the blog post to find how to.

Enjoy!

View on GitHub

Snowpack and the polyfill es-module-shim can do much more than the examples provided here. I encourage you to checkout their documentation.
You can check the progress of the import maps on chromestatus.com
Import maps can also be used with unpkg content delivery network. The bare syntax can be mapped to an unpkg url,Deno uses import maps to do something similar. You can even use unpkg urls as a fallback in case a module is not available in the node_modules folder.

References

Snowpack

Import maps - chromestatus.com

Import maps - Editor's draft

Import Maps Draft Community Group Report

DEV Community: Diego Sánchez

Do you use UML to model complex web applications?

mp3split: a simple shell script for splitting mp3 audio files

Introduction

Usage example

Useful links

diegosanchezp / mp3split

Shell script for splitting or slicing large mp3 audio files

MP3split

Table of Contents

Required dependencies

Installation instructions

I've made a web component to easily implement and switch two themes in a webpage.

Usage example

diegosanchezp / theme-switcher-component

A web component that helps you implement and change two themes on a web page.

Using nodejs packages in the browser with import maps

Table of contents

What are import maps

Using import maps

Generating import maps

The problem of NodeJs packages

Heritage a package manager for modern web development

Conclusion

Additional notes

diegosanchezp / import-maps-examples

Examples of import-maps

Import Maps Examples

Get started

Notes

References