DEV Community: Michael Rosario

Music Maker: Using NodeJS to Create Songs

Michael Rosario — Fri, 29 Jul 2022 22:32:36 +0000

In my graduate school career, I had the opportunity with our evolutionary complexity lab to study creating music using neural networks and interactive genetic algorithms. It’s fun to study these two topics together since I enjoy crafting code and music. If you enjoy this too, you might enjoy Music maker, a library I created to explore generating music with code. Sonic Pi by Sam Aaron, a popular tool to teach music theory and Ruby code, inspired me to build this tool. My team from InspiredToEducate.NET enjoyed teaching a coding workshop on music using Sonic Pi. We, however, encountered a challenge of installing Sonic-Pi on a lab of computers. The workshop targets 5th-grade to 8th-grade students who have good typing skills. It would be cool if something like Sonic-Pi supported features like Blockly coding too.

In terms of musical motivations, I wanted to provide features like the following:

Motivations

Like Sonic-Pi, the tool should make it easy to generate chords and scales.
I wanted the tool to have a concept of players who can generate music over a chord progression. I believe it would be cool to grow an ecosystem of players for various time signatures and musical types.
I wanted to support the MIDI file format for output making it possible to blend music from this tool in sequencers broadly available on the market. This also enables us to print out sheet music using the MIDI files.
Building drum patterns can be tedious at times, I wanted to create a way to express rhythm simply.
We desired to have a browser-based interface that a teacher could install on a Raspberry Pi or some other computer. This was a key idea from one of my teachers. I’m focusing on building a tool that works on a local area network. (not the broad internet) From a coding perspective, we need to build a tool that could interface with Blockly coding someday. JavaScript became a logical choice.

About NG-Music-Maker

Learn more at this GIThub repo

Features

Express server to connect MIDI file services to APIs
Angular project to demo features
Angular project includes a drum pattern maker
TypeScript API for expressing ideas like chords, chord progressions, scales, and players
Players iterate over chord progressions using a style

Code Tour

To explore how the MIDI services work.. please inspect the examples here. (see examples project)


let midiServices = new MidiServices();
let scaleServices = new ScaleService(midiServices);
let chordServices = new ChordServices(midiServices);

function makeScale() {
    var file = new Midi.File();

    // Build a track
    var track = new Midi.Track();
    track.setTempo(80);
    file.addTrack(track);

    // Make a scale
    var scale = scaleServices.MakeScale("c4", ScaleType.MajorPentatonic, 2);
    for (var i = 0; i < scale.length; i++) {
        track.addNote(0, scale[i], 50);
    }

    // Write a MIDI file
    fs.writeFileSync('test.mid', file.toBytes(), 'binary');
}

function threeNotes() {
    var file = new Midi.File();

    // Build a track
    var track = new Midi.Track();
    track.setTempo(80);
    file.addTrack(track);

    let mm = midiServices;
    let beat = 50;
    track.addNote(0, mm.GetNoteNumber("c4"), beat);
    track.addNote(0, mm.GetNoteNumber("d4"), beat);
    track.addNote(0, mm.GetNoteNumber("e4"), beat);

    // Write a MIDI file
    fs.writeFileSync('test2.mid', file.toBytes(), 'binary');
}

function drumTest()
{
    var file = new Midi.File();

    // Build a track
    var track = new Midi.Track();
    track.setTempo(80);
    file.addTrack(track);

    let mm = midiServices;

    var addRhythmPattern = mm.AddRhythmPattern;
    addRhythmPattern(track, "x-x-|x-x-|xxx-|x-xx",DrumNotes.ClosedHighHat);
    fs.writeFileSync('drumTest.mid', file.toBytes(), 'binary');
}

function chordProgressions(){
    var file = new Midi.File();

    // Build a track
    var track = new Midi.Track();
    track.setTempo(80);
    file.addTrack(track);

    var chordList = new Array();
    chordList.push(new ChordChange(chordServices.MakeChord("e4", ChordType.Minor),4));
    chordList.push(new ChordChange(chordServices.MakeChord("c4", ChordType.Major),4));
    chordList.push(new ChordChange(chordServices.MakeChord("d4", ChordType.Major),4));
    chordList.push(new ChordChange(chordServices.MakeChord("c4", ChordType.Major),4));

    var chordPlayer = new SimplePlayer()
    chordPlayer.PlayFromChordChanges(track, chordList, 0);

    chordPlayer = new Arpeggio1()
    chordPlayer.PlayFromChordChanges(track, chordList, 0);

    chordPlayer = new RandomPlayer()
    chordPlayer.PlayFromChordChanges(track, chordList, 0);

    chordPlayer = new BassPlayer1()
    chordPlayer.PlayFromChordChanges(track, chordList, 0);

    chordPlayer = new BassPlayer2()
    chordPlayer.PlayFromChordChanges(track, chordList, 0);

    chordPlayer = new BassPlayer3()
    chordPlayer.PlayFromChordChanges(track, chordList, 0);

    chordPlayer = new OffBeatPlayer()
    chordPlayer.PlayFromChordChanges(track, chordList, 0);

    fs.writeFileSync('chordProgressions.mid', file.toBytes(), 'binary');
}


makeScale();
threeNotes();
drumTest();
chordProgressions();

How Pair Programming Helps The Team

Michael Rosario — Sat, 14 May 2022 19:36:08 +0000

For developer teams, I have enjoyed seeing ways that pair programming can to grow team morale, make better code and help share knowledge. In general, pair programming encourages developers to gang up on strategic or challenging problems. As I reflect upon some of my favorite times from my teams, I loved the times that we used pair programming to kill the road block or flow problems.

Here are some of the general benefits of pair programming:

Two minds make better code: Doug Shimp, one of my early Scrum trainers, used to promote the phrase that no mind should work alone. Pair programming gives team members a concrete practice and pattern for collaborating on problems. As I pair, I want to avoid my partner from feeling like I am micro-managing them. It is ok to have the coder make mistakes to enable them to figure out a problem on their own. Compilers, linters, and test automation should catch large classes of errors.

Better code: In formal software engineering studies, peer reviews have become one of the top practices to help find construction bugs early. If your team practices pair programming from time to time, it has similar properties to a peer review. Code is not everything too. It is important for agile teams to take time to design together. In the initial stages of a feature, pairs can sketch out a design (UML, component model, entity relationship diagram, etc.) and talk through strategy at a higher level.

Blending strengths: Teams might find it helpful to pair together to combine team strengths. Let us say that you have a feature that really requires front-end and back-end work. Could the feature go faster if a back-end engineer and a front-end engineer took a couple of hours pair programming together to hammer out essential details? I have enjoyed pair programming with our data science team. It has been fun to trade ideas. (ML stuff and agile engineering concepts)

We are social: As humans, we have an inherit need to connect to each other. Even if you are introverted, there is still a deep human need to connect to groups and peers. When I am pair programming, I am less likely to check email and social media since I want to make sure I am appropriately engaging with my partner. There is a special kind of flow or fun factor that happens when you are ganging up on a problem.

Sharing knowledge and discipline: If you are introducing team members into a new code base, it can be helpful to give them a tour of the code base while getting some work done. Let us say that you are trying to encourage a team to do more unit test development in your server code. It can be helpful to pair program to talk through the construction ideas, ways to make useful tests, and other design ideas. In general, pair programming can be a wonderful teaching tool. I love learning new IDE tricks and tools from other developers.

So... How do you do pair programming well?

Like many, I am still trying to become more effective at pair programming with my team. I am also encouraging my team members to pair program with each other. I drafted this blog post to collect a few practices to help me and my team to grow.

Start with 2 to 4 hours per week: Through our Google DevFest conference, I had learned that Pluralsight executes a great deal of pair programming in their mobile engineering team. It became very tempting to recommend day long pair programming sessions. A sizable number of blog posts recommend starting small. (2 to 4 hours per week) After the team experiments with these small experiences, teams can decide organically about using the practice more.

Define a good schedule, tools, and scope: Pair programming sessions should have a clear schedule and goal. During the pair programming session, block out email/social communication to deeply focus on the experience and the needs of your partner. If you commit to pair programming for one hour, make sure to stick to your time box to respect everyone’s time. As a team, consider talking about the common tools that will make pair programming more productive. Encourage experimentation.

Talk while you make stuff: It is often helpful to talk through your thought process while you are coding. The act of asking questions or talking helps increase the clarity of problems before you. During the pair programming session, the person who is not coding (supporter) can help you reflect and follow your mental model. I really like the concept that the supporter actively complements those activities of the coder. The supporter might search for useful blog posts, sketch out design ideas, and help the coder avoid errors in real time.

Every 30 minutes, pass the keyboard: To encourage equal coding time, I like to encourage the practice of passing the keyboard at regular intervals. You and your partner can decide on what works for you. As you pair, make sure to schedule breaks.

References:

Photo attribution: https://flickr.com/photos/rafaelmob/

Looking for a role where your app makes a difference? Are you passionate about empowering people and growing the next generation of VR/AR training? If you're an experienced full-stack web developer looking to grow your mastery, consider learning more about DI's high impact career opportunities at https://bit.ly/36Um4J0

Impressions of Blazor Server from an Angular Dev

Michael Rosario — Sat, 29 Jan 2022 22:31:32 +0000

In the next 3 years, I can see many enterprises considering Blazor server for their web application needs. I have enjoyed my journey of building Angular 2+ clients for .NET core applications. From an architecture perspective, I do feel like modern web application development has become more complex in the past 5 years. In this post, I wanted to share my raw impressions of test driving Blazor server application development. Please keep in mind that modern Angular has become my favorite front-end JavaScript SPA tech. In my brief test drive, I enjoy the potential to drive simplicity back into modern web applications. If you're trying to hire a full-stack Angular/DotNetCore application, your search becomes more challenging. Since a Blazor dev focuses on 80% C# for code-behind, your enterprise hiring complexities can be simplified. From a quick analysis of Blazor server, Microsoft has matched most of the major benefits of React's component model. As an experienced .NET guy, I love the simple component model syntax as compared to React. While Blazor does not have the experience and depth of open source community as React or Angular, I'm impressed with the COTS and open source trends happening around Blazor. I believe that Blazor designers have tried to match the best ideas from Angular/React while trying to keep things simple. In general, I feel like .NET core has helped make C# development fun again. I feel like Blazor has just added some gas to that fire.

Why I like Angular for SPA development?

Writing clean and testable code: In Agile environments, we have to embrace change of requirements and business conditions. On my teams, we strive to make sure that key business logic services in client and server code has good unit tests or integration tests. Given Miško Hevery served as one of the founders of Angular, Google has engineered Angular for many levels of test automation.
Component model: I really love the component model of Angular. Since I really enjoy C#, I have also liked exploring TypeScript for component code. As you get started with Angular, you can accomplish a lot of UX complexity using inputs and outputs to components. The component system enables you to break down complex UX into smaller "Lego" blocks.
Scaling JavaScript: Since Angular embraces TypeScript, it does not really block you from JavaScript, the powerful ecosystem of JavaScript libraries, and browser API's . For an enterprise team, we enjoy the benefits of type safety for refactoring, code completion and tooling support. This will continue to be strong benefit for Angular.
Angular has an amazing community. To be honest, Angular has a higher concept count as compared with React. I, however, feel that the Angular team does a good job promoting great documentation. While great complexity is possible in Angular, the community actively and critically talks about ways to keep Angular sustainable, clean, and maintainable. If you want new toys for Angular, you'll find plenty on GitHub.

Tear down of a Blazor component

In order to learn anything, I believe you got to build something. I started exploring Blazor by building a small question and answer portal(i.e. like StackOverFlow ) for my team at work. In general, users will be able to post questions using Markdown syntax. Users will have the ability to post answers to questions and up vote the best information. In the following code tear down, we'll explore the add question component of the application.

If you'd like to see the whole application in context, you can check out my repo here:
https://github.com/michaelprosario/q-and-a

@page "/add-question"
@inherits AddQuestionComponentBase

<h1>Ask a question</h1>
<br>
<h3>Title</h3>
<HxInputText @bind-Value="@Record.Name" Placeholder="Enter question title" />
<br>
<h3>Question Details</h3>

<MarkdownEditor @bind-Value="@Record.Content" ValueHTMLChanged="@OnMarkdownValueHTMLChanged" />

<h3>Tags</h3>
<HxInputText @bind-Value="@Record.Tags" Placeholder="Enter tags" />
<br>
@if (ValidationFailures.Count > 0)
{
    <ul>
        @foreach (var validationFailure in ValidationFailures)
        {
            <li>@validationFailure</li>
        }
    </ul>
}
<HxButton OnClick="OnSaveQuestion" Color="ThemeColor.Primary">Save Question</HxButton>

As you skim over the code, it feels very similar to the component code you'd author in Angular or React. At the very top of the page, we have added routing to the "add-question" url. We have also expressed the idea that the component code or Razor code has a C# class to encapsulate behavior.

@page "/add-question"
@inherits AddQuestionComponentBase

In this brief code snippet, you can see that I've integrated 2 Blazor UI libraries

https://github.com/havit/Havit.Blazor: This project includes a Bootstrap 5 component bundle for common UX elements, grids, and patterns.
PSC.Blazor.Components.MarkdownEditor: This component provides a simple mark down editor. In the code sample below, you can see that we can bind the value to our record content. As the user changes data in the editor, those changes are written into the data bound variable. (Input concept). Further, the editor throws events when data changes in the editor. In my application, I am able to capture the HTML representation of post using this event.

<MarkdownEditor 
@bind-Value="@Record.Content" 
ValueHTMLChanged="@OnMarkdownValueHTMLChanged" 
/>

CSharp code behind

using DocumentStore.Core.Requests;
using DocumentStore.Core.Services.DocumentStore.Core.Services;
using FluentValidation.Results;
using Microsoft.AspNetCore.Components;
using QA.Core.Entities;
using QA.Core.Validators;
using System;
using System.Collections.Generic;
using System.Linq;
using System.Threading.Tasks;

namespace QA.Server
{
    public partial class AddQuestionComponentBase : ComponentBase
    {

        [Inject] NavigationManager NavigationManager { get; set; }
        [Inject] private IDocumentsService<Question> DocumentsService { get; set; }
        public Question Record{ get; set; }
        private string markdownHtml = "";

        public IList<ValidationFailure> ValidationFailures = new List<ValidationFailure>();

        private void OnNewRecord()
        {            
            Record = new Question
            {
                Id = Guid.NewGuid().ToString()
            };            
        }

        private async Task OnSave()
        {
            ValidationFailures = new List<ValidationFailure>();

            Record.CreatedBy = "system";
            Record.PermaLink = Record.Name;
            Record.Abstract = markdownHtml;
            Record.HtmlContent = markdownHtml;

            var questionValidator = new QuestionValidator();
            var validationResults = questionValidator.Validate(this.Record);
            if(validationResults.Errors.Count() > 0)
            {
                ValidationFailures = validationResults.Errors;
                return;
            }

            var command = new StoreDocumentCommand<Question>
            {
                Document = Record,
                UserId = "system"
            };
            var response = await this.DocumentsService.StoreDocument(command);
            if (!response.Ok())
            {
                ValidationFailures = response.ValidationErrors;
            }
            else
            {
                NavigationManager.NavigateTo($"view-question/{Record.Id}", true);
            }
        }

        protected Task OnMarkdownValueHTMLChanged(string value)
        {
            markdownHtml = value;
            return Task.CompletedTask;
        }

        protected override async Task OnInitializedAsync()
        {
            OnNewRecord();
        }

        protected async Task OnSaveQuestion()
        {
            await OnSave();
        }
    }
}

In the component base class, you have the opportunity to describe the behavior of the Blazor component and describe dependencies. In Angular, we inject service dependencies into a component class through the constructor. In Blazor, you execute the same concept through property injection and "inject" attributes. Like Angular, properties of the component base class become available to the Razor markup file. (i.e. Record) For some of my Unity 3D amgios, this feels a lot like a "GameObject" script.

namespace QA.Server
{
    public partial class AddQuestionComponentBase : ComponentBase
    {

        [Inject] NavigationManager NavigationManager { get; set; }
        [Inject] private IDocumentsService<Question> DocumentsService { get; set; }
        public Question Record{ get; set; }
        private string markdownHtml = "";

When we boot the component, we execute the new record setup method.

protected override async Task OnInitializedAsync()
{
    OnNewRecord();
}

When we save data from the form into the database, I love the simplicity. In the Angular and DotNetCore server architecture, we have a strong separation of backend services from the front-end work. On the DotNetCore side of the house, we will expose business logic services through a set of controller classes and related security guards. On the Angular/TypeScript client, the client developer will need to create a proxy services to connect to each business logic service. While this is not hard, it's a tedious chore for someone to do. Some teams have used OpenAPI/Swagger to code generate their proxy classes. It's great to delegate to robots! Check out NSwagStudio if you're interested.
https://github.com/RicoSuter/NSwag/wiki/NSwagStudio

In the world of Blazor Server, all of this complexity melts away. The client component model can the business services without the junk of JSON, proxies and controllers. Since I'm using Steve Smith's Clean Architecture pattern (https://github.com/ardalis/CleanArchitecture), I was able to adapt my project setup quickly to abandon the former "controller" centered architecture. If I get a postive response from the service, I can navigate the user to another page.

private async Task OnSave()
{
    // mapping and validation code goes here ...........
    var command = new StoreDocumentCommand<Question>
    {
        Document = Record,
        UserId = "system"
    };
    var response = await this.DocumentsService.StoreDocument(command);
    if (!response.Ok())
    {
        ValidationFailures = response.ValidationErrors;
    }
    else
    {
        NavigationManager.NavigateTo($"view-question/{Record.Id}", true);
    }
}

In a future blog post, I'll try to outline a few other benefits to Blazor. For now, I wanted to close with a final consideration if you're thinking about leaving Angular for Blazor. In the Blazor community, the technology shifts toward favoring C# over calling raw JavaScript. In the .NET community, there's a significant number of enterprise developers who love this story. If you're building a form over data application, Blazor will do just fine. Most of the major component vendors have built C# API layers to integrate with their components.

Check out this post to review the open-source and COTS solutions connected to Blazor.

https://github.com/AdrienTorris/awesome-blazor

If your web application client needs to interface with a JavaScript library or an obscure browser feature, you need to consider the cost of building proxy classes(JavaScript interop classes) to connect your C# client code to the browser.

My initial test drives of Blazor and their open source community feel productive so far. I look forward to seeing this technology grow.

Power Long-Running Python Jobs using RabbitMQ

Michael Rosario — Sat, 22 Jan 2022 14:32:35 +0000

In any distributed service solution, you have many options for enabling services to communicate with each other. In the python world, you could use HTTP requests and the Flask framework. Other options include gRPC, XML web services, or raw sockets. In some of my personal Python hacking, I wanted to experiment with longer running Python jobs. This post provides some motivations for communication in distributed systems using message queues. I'm hoping this pattern will help machine learning jobs, IoT data captures, or data transformation processes.

https://blog.iron.io/introduction-to-message-queue-architecture/

Install Python tools

In your working folder, let's start by installing tools.
In this tutorial, we're using natural language toolkit(nltk) to execute part of speech tagging of sentences. We're using the pandas data library to manage simple tables of data. For this recipe, we will use RabbitMQ and the related pika library to provide inter-process communication.

pip install nltk
python -m nltk.downloader popular
pip install pandas
pip install pika

Install RabbitMQ using docker-compose

Using the following docker-compose file, we can start a test instance of RabbitMQ
Make sure that you have installed Docker compose tools properly.

version: "3.2"
services:
  rabbitmq:
    image: rabbitmq:3-management-alpine
    container_name: 'rabbitmq'
    ports:
        - 5672:5672
        - 15672:15672
    volumes:
        - ~/.docker-conf/rabbitmq/data/:/someDataFolderOnHostComputer/
        - ~/.docker-conf/rabbitmq/log/:/someDataFolderOnHostComputer
    networks:
        - rabbitmq_go_net

networks:
  rabbitmq_go_net:
    driver: bridge

Let's make test data.

For this exercise, let's make a test data file called "data.json."

[
    {"sentence": "Cheese burgers and ham are yummy!", "foo": 42},
    {"sentence": "We don't talk about bruno", "foo": 42},
    {"sentence": "I see you!", "foo": 42}
]

How to send data for processing

#!/usr/bin/env python
import pika

connection = pika.BlockingConnection(pika.ConnectionParameters(host='localhost'))
channel = connection.channel()
queue = "input_queue"

file = open('data.json','r')
json_text = file.read()

channel.queue_declare(queue=queue)
channel.basic_publish(exchange='', routing_key=queue, body=json_text)
print(" [x] Sent json")
connection.close()

Using the pika library, we start a connection to the RabbitMQ message on localhost. After reading the input data (data.json) into a variable, we send a message to RabbitMQ with the data. In the send operation, we needed to specify the queue name, an empty exchange value, and the message body.

Receiving data and processing it

#!/usr/bin/env python
import pandas as pd
import json
import nltk
import pika, sys, os

input_queue = "input_queue"
output_queue = "output_queue"

def processSentencesJson(json_text):
    df = pd.read_json(json_text)
    results = []
    for index, row in df.iterrows():
        sentence = row[0]     
        tokens = nltk.word_tokenize(sentence)
        tagged = nltk.pos_tag(tokens)
        results.append(tagged)

    return results

def main():
    connection = pika.BlockingConnection(pika.ConnectionParameters(host='localhost'))
    channel = connection.channel()

    channel.queue_declare(queue=input_queue)

    def callback(ch, method, properties, body):
        print(" [x] Received json")
        results = processSentencesJson(body)
        channel.basic_publish(exchange='', routing_key=output_queue, body=json.dumps(results))

    channel.basic_consume(queue=input_queue, on_message_callback=callback, auto_ack=True)

    print(' [*] Waiting for messages. To exit press CTRL+C')
    channel.start_consuming()

if __name__ == '__main__':
    try:
        main()
    except KeyboardInterrupt:
        print('Interrupted')
        try:
            sys.exit(0)
        except SystemExit:
            os._exit(0)

In the receiving python process, we start things off by naming the input and output message queue names.

input_queue = "input_queue"
output_queue = "output_queue"

In the main function, we establish a connection to the RabbitMQ. We also define a call back function to execute when we receive a message on the input queue. In the call back function, we execute the "processSentencesJson" function and return results to the output message queue.

I do admire the brief and concise nature of this solution.

def main():
    connection = pika.BlockingConnection(pika.ConnectionParameters(host='localhost'))
    channel = connection.channel()

    channel.queue_declare(queue=input_queue)

    def callback(ch, method, properties, body):
        print(" [x] Received json")
        results = processSentencesJson(body)
        channel.basic_publish(exchange='', routing_key=output_queue, body=json.dumps(results))

    channel.basic_consume(queue=input_queue, on_message_callback=callback, auto_ack=True)

    print(' [*] Waiting for messages. To exit press CTRL+C')
    channel.start_consuming()

In the following function, we accept a JSON string of sentences. Using the pandas library, we convert the JSON string into a data frame structure. We loop over the data frame rows and execute part of speech tagging using NLTK. For each iteration of the loop, we contribute tagged sentences into the results array.


def processSentencesJson(json_text):
    df = pd.read_json(json_text)
    results = []
    for index, row in df.iterrows():
        sentence = row[0]     
        tokens = nltk.word_tokenize(sentence)
        tagged = nltk.pos_tag(tokens)
        print(tagged)
        results.append(tagged)

    return results

The system will execute part of speech tagging for each sentence using the following format.

[('We', 'PRP'), ('do', 'VBP'), ("n't", 'RB'), ('talk', 'VB'), ('about', 'IN'), ('bruno', 'NN')]

Photo credit to Rgourley on Flickr