Automatic music creation without artificial intelligence

Srgy Surkv — Tue, 02 Apr 2024 07:03:37 +0000

Automatic creation of music according to specified parameters.

Open in browser - https://surikov.github.io/rockdice/main.html.
Sources - https://github.com/surikov/rockdice
Android version - https://play.google.com/store/apps/details?id=rockdice.chord.progression

Uses Web Audio API, TypeScript, Android WebView, wavetable synthesis, and basic music theory.

Description of application

In the main window, you can use the slider to select the progression. The sequence of chords determines the mood of the melody (minor on the left, major on the right, jazz seventh chord in the middle).

Round switches allow you to select riffs for the standard 4 layers of melody.

Drums - set the rhythm
Bass – harmonic basis and rhythm
The leader creates a melodic pattern.
Pad – counterpoint, long notes that show the function of a chord.

The blue button with a die selects random values.

The application modulates selected fragments to the specified chords. If you turn the switches and listen to a few melodies, you can be convinced that with any combination of chords and instruments, the sound of the result is quite “human”.

Similar projects

The first synthesizers had built-in functions for auto-generating arrangements. An example of performance on a Cassio MT-100 from the 1980s - https://www.youtube.com/watch?v=Aepm6V4yvhw

Modern Captain Plugins Epic app, suitable only for professionals -
https://www.youtube.com/watch?v=ZuB_t1DBwp8

The BandLab SongStarter web application does not support any settings, it simply generates melodies in one style -
https://www.youtube.com/watch?v=EDRPy8KtY0c

Settings

When you click on the gear, the settings window opens. You can edit volume by layers, speed, change chords, transpose:

For each track you can choose a riff from the list:

You can also choose a chord progression:

History/Undo

The ability to undo a previous action and return to a previous state is a typical requirement for any application. When you click on the button with the Cancel icon (top right), a window opens with the history of selecting melodies:

Publishing and exporting

Like any music application, RockDice allows you to export the created melody to a MIDI file or Wav file (buttons on the bottom panel). The resulting files can be edited in other music applications. Example of tracks imported into BandLab MixEditor (any track can be forked and opened in the editor):

https://www.bandlab.com/sss1024/tracks

Collaboration

The link to the melody opens in the application exactly as the author sent it.

Clicking the button with the standard Share icon opens a preview window. In it you can send a link to the melody by email, instant messenger or social networks:

Most social networks correctly recognize a link with a description and image of the selected tools:

Implementation Description

How it's done.

Playing sound

The WebAudioFont library is used to play sound:

https://github.com/surikov/webaudiofont

The library contains more than 2000 digitized instruments and basic filters for sound processing:

10-band equalizer to customize tone
echo for surround sound
dynamic compressor to highlight voices

Documentation:

https://surikov.github.io/webaudiofont/npm/src/docs/index.html

Modulation of fragments

For selected (or manually entered) chords, extractMode calculates the root note and scale, see below.

https://github.com/surikov/rockdice/blob/main/ts/code/zvoogharmonizer.ts#L1156

For music playing, about two hundred riffs with different instruments were added to the project; you can see them in the folder

https://github.com/surikov/rockdice/tree/main/patterns

Each pattern, in addition to notes, also contains a description of the mode and tonic.

The selected riffs in the morphSchedule function are transposed to the root of the selected chords and modulated according to the scale of the chord progression, see below.

https://github.com/surikov/rockdice/blob/main/ts/code/zvoogharmonizer.ts#L377

Transposing a melody involves moving all notes the same number of semitones. Simply put, make the sound higher or lower.
Modulation of a melody is a shift of the scale by certain steps. In a simple way - from minor to major, etc.

Music instrument's Features

When playing musical fragments, the specifics of performance on specific instruments are taken into account. For example:

When playing guitar with distortion, you typically use both open string strumming and palm muted strumming. Those. There is only one instrument, but for a realistic sound you need two separate sets of samples for open and muted strings
on an acoustic guitar, hitting the strings down and up is noticeably different, see example
sets of notes of similar chords (for example G and Gm) can be pressed on completely different frets, this must be taken into account when modulating fragments
and so on.

As a result, the generated music sounds less monotonous.

Application State's storage

Selected chords, fragments and other settings are saved in the browser's localStorage, see readObjectFromlocalStorage. As a result, when you reopen the application, its state is restored to the same state as in the last session.

When creating a link for publication, all data is encoded into a long, very long URL, example

https://mzxbox.ru/RockDice/share.php?seed=%7B%22drumsSeed%22%3A21%2C%22bassSeed%22%3A12%2C%22leadSeed%22%3A6%2C%22padSeed%22%3A12%2C%22drumsVolume%22%3A111%2C%22bassVolume%22%3A99%2C%22leadVolume%22%3A66%2C%22padVolume%22%3A77%2C%22chords%22%3A%5B%22Cm%22%2C%222%2F1%22%2C%22Ebm%22%2C%222%2F1%22%5D%2C%22tempo%22%3A130%2C%22mode%22%3A%22Ionian%22%2C%22tone%22%3A%22D%23%22%2C%22version%22%3A%22v2.83%22%2C%22comment%22%3A%22%22%2C%22ui%22%3A%22web%22%7D

Link markup

Published links comply with the Open Graph and Twitter Cards protocols.

These protocols are supported by most social networking systems. networks. Essentially, this is a requirement that the page being linked to include a description of the page and a preview image.

The image and page for publication are generated dynamically by a regular PHP script, see.

https://github.com/surikov/rockdice/blob/main/server/share.php

All information for the preview is in the tags

<meta name="twitter:card" content="summary" /> <meta property="og:title" content="<?php echo $line; ?>" /> <meta property="og:url" content="https://mzxbox.ru/RockDice/share.php?seed=<?php echo $encoded; ?>" /> <meta property="og:image" content="https://mzxbox.ru/RockDice/picture.php?drums=<?php echo $drums; ?>&prog=<?php echo urlencode($line); ?>&bass=<?php echo $bass; ?>&lead=<?php echo $lead; ?>&pad=<?php echo $pad; ?>" />

Scripts are hosted on regular hosting at a price of approximately 200 rubles per month.

Android

To create a mobile version, you can use the WebView component.

https://developer.android.com/reference/android/webkit/WebView

Essentially, it's regular Chrome built into an Activity. Web pages are opened locally from application resources, files and digitized tools are also downloaded locally.

In terms of functionality, the mobile version is no different from the web version. Here is the compiled application (not updated for a long time):

https://play.google.com/store/apps/details?id=rockdice.chord.progression

WebView has some minor limitations that should be taken into account.

For example, access to the file system for saving melodies in MIDI and Wav files. In a large browser, it is enough to create a blob with binary data of the correct MIME type and open a link to it, for example

let blob: Blob = new Blob([dataview], { type: 'audio/wav' }); let ourl = URL.createObjectURL(blob); let a = document.createElement("a"); document.body.appendChild(a); a.href = ourl; a.download = 'rockdice'; a.click();

The browser correctly recognizes the file type and displays the normal save dialog with the correct extension.

For an embedded WebView, you will need to create a save function using Android, embed it in the application's JavaScript via addJavascriptInterface, and access it from the web page.

It is also necessary that published links to melodies open not in the phone’s browser, but in the mobile version of the application. To do this, you need to configure a filter in the AndroidManifest.xml of the application:

<intent-filter> <action android:name="android.intent.action.VIEW" /> <category android:name="android.intent.category.DEFAULT" /> <category android:name="android.intent.category.BROWSABLE" /> <data android:host="surikov.github.io" android:pathPrefix="/rockdice/main.html" android:scheme="https" /> </intent-filter>

You can do without the mobile version, because Chrome on phones is no different from Chrome on a computer.

How to use MIDI for web in short

Srgy Surkv — Wed, 28 Aug 2019 10:12:16 +0000

MIDI player

Use MIDI parser to read notes from MIDI file
Use WebAudioFont to play musical instruments in a browser
See result

MIDI keyboard

Use React to draw piano keyboard
Use Chrome to read MIDI events from hardware keyboard
Use midi-sounds-react to play sound
See result

MIDI editor

Use three.js to create catchy UI
Use WebAudioFont to play music
See result

DEV Community: Srgy Surkv