DEV Community: Emilia G.

A Sneak Peek at our Haskell Training Course

Emilia G. — Wed, 20 Mar 2024 19:55:43 +0000

Our Haskell training course is a unique part of working as an engineer in Stack Builders. It helps to build our community and helps engineers see problems in new ways, regardless of whether they’re working on Haskell projects, or in languages. We hope that this high-level overview helps you to find the motivation to learn Haskell.

Given the interest in our last post “The Uncommon Advantage from Training in Unpopular Programming“, we wanted to provide some more details on the Haskell Language Training Course used for full-time staff at Stack Builders. For the past 9 years, we’ve offered Haskell training to our developers. During this time, we’ve taught approximately a hundred developers and invested thousands of hours in learning and sharing Haskell concepts. In this article, we highlight the curriculum and experiences of some of our developers - we hope that it’s useful in your own Haskell journey or if you decide to implement your own in-house Haskell training program.

Our industry-relevant Haskell curriculum

Stack Builders' 10-week Haskell course is crafted based on our extensive experience in the software industry. The curriculum is designed to meet the demands of real-world projects, ensuring that participants gain practical skills applicable to commercial software development.
These are the modules of our Haskell course. Each module offers a variety of exercises that help implement learning hands-on:

Module 1: Introduction (Getting started and Types and classes)
Module 2: Functions (Defining functions and Recursive and higher-order functions)
Module 3: Types and classes and IO
Module 4: Monads (Functors, Applicatives, and Monads)
Module 5: Lazy evaluation

We use Programming in Haskell by Graham Hutton to support our course, but we encourage developers to refer to resources including the following to supplement “Programming in Haskell”:

Learn you a Haskell for great good! by Miran Lipovača
Get Programming with Haskell by Will Kurt
CIS 194: Introduction to Haskell (Spring 2013) by Brent Yorgey
Learn4Haskell by Kowainik

Hands-on mentorship and code reviews

Unlike many online courses that rely solely on pre-recorded lectures, our course offers close guidance, mentorship, and code reviews in smaller teams. Each module has programming exercises that we’ve selected from several resources to ensure our developers are clear on concepts and can apply them later. This hands-on approach provides the opportunity to apply theoretical knowledge to actual coding scenarios. This prepares our developers to work on real-world Haskell projects.

Focus on functional programming principles

Our course goes beyond teaching Haskell syntax. It emphasizes implementing functional programming principles in other languages, promoting cleaner code, improved maintainability, and enhanced software reliability. This approach sets it apart from courses that focus solely on language features without a broader perspective on software development.
This is what our developers say about the Haskell Training Course at Stack Builders and how it has helped them grow:

Jorge Galarza - Software Developer - 1.5 years at Stack Builders
The Haskell training course not only deepened my comprehension of functional programming but also instilled a commitment to fundamental functional programming principles. Whether working in Haskell or other languages, this experience influenced my approach to coding and also caused a paradigm shift in my conventional programming mindset. Consistently applying principles such as pure functions, immutability, and strong type systems has fostered code reliability and clarity leading to more resilient and forward-thinking software development in the projects that I've worked on.

Diego Balseiro - Senior Software Developer - 6 years at Stack Builders
The Haskell Training Course has been a part of our company for several years. It's a unique aspect of our business that sets us apart from others. Haskell is taught to all our employees, including our interns, which is something not many companies across the globe can say.
Haskell is much more than just a programming language. It's a method of viewing computation that can offer a fresh perspective even when you're programming in more traditional languages like JavaScript or Python.
Functional Programming and Haskell are at the core of Stack Builders. We believe that it's this focus that has contributed significantly to our success. Haskell offers a unique way to think and reason about code.
Our Haskell course could provide you with a new set of tools for your developer toolkit to innovate and solve complex problems.

Alex Mejía - Senior Software Developer - 3 years at Stack Builders
Learning Haskell has been a great experience that has allowed me to be a better developer. It gave me the ability to be proficient in a new language and allowed me to be more effective in my current projects. It also gave me the ability to effectively use functional programming concepts in other languages like Python, JavaScript, and more.

Franz Guzmán - Software Developer - 2.5 years at Stack Builders
I began the course with some prior experience in Haskell. However, a certain curiosity persisted about what I could pick up from it, as most concepts were still unfamiliar to me. During the course, I not only demystified the language but also revealed its elegance and power. What stood out for me was the ability to effortlessly build upon existing software through a simple analysis of the function's input and output types. The course also highlighted functional thinking, encouraging us to view inputs as data streams and enabling the seamless chaining of data transformations. This paradigm has proven to be a valuable skill applicable to any programming language, especially as modern languages increasingly embrace the functional approach.

The Haskell training course is a unique part of working as an engineer in Stack Builders. It helps to build our community and helps engineers see problems in new ways, regardless of whether they’re working on Haskell projects, or in languages like Python, TypeScript, or Kotlin. While we aren’t able to make the course available externally at this time, we hope that this high-level overview helps you to find the motivation to learn Haskell on your own or start an in-house Haskell initiative at the company where you work.

Feel free to contact us if you have questions or would like information on joining our team.

Combining Nix with Terraform for better DevOps

Emilia G. — Mon, 18 Mar 2024 19:07:30 +0000

An investment in creating the best software in the world will still go to waste if the infrastructure it runs on isn’t well-managed. At Stack Builders, our mission is to push the boundaries of the software industry through quality, pragmatic solutions that bring to life our client's vision, and one way we are doing this is to leverage our training and staff experience with functional programming to make DevOps better.

In that light, we’re thrilled to announce the release of nixpkgs-terraform! This Nix project significantly enhances the flexibility and control of your Terraform environment, making its installation and management declarative and reproducible. This is largely thanks to Nix and its unique capabilities for precisely pinning packages to a version, ensuring its consistency across different environments. Forget about writing documentation on environment setup - just use one of the provided templates to bootstrap your Terraform installation and get the whole team on the same version with minimal manual intervention.

No need for deep knowledge of Nix either - just keep adding tools to your template of choice to craft a reproducible, declarative, and portable development environment for everyone on the project so that you can focus on what matters.

At Stack Builders, our DevOps/SRE team uses nix-shells to install all of the necessary tooling to work on various Infrastructure-as-Code projects across the board. However, while nix-shells are useful for creating portable development environments, more fine-grained control over the different Terraform versions was lacking. We've taken inspiration from nixpkgs-python and nixpkgs-ruby, and we're grateful for the paths they've paved.

We’ve noticed that some users have been asking about how to use older versions of Terraform in their Nix setups [1, 2]. This is an example of the diverse needs of people and the importance of maintaining backward compatibility. We hope that nixpkgs-terraform will be a useful tool for these users.

The design of nixpkgs-terraform follows a bottom-up approach, making it more approachable for newcomers. We hope this will empower everyone to take their Nix experience to the next level!

Looking forward to your feedback and contributions. Happy Nixing!

Don't forget to subscribe to our newsletter.

Video Generation with Python

Emilia G. — Mon, 11 Mar 2024 21:21:21 +0000

Python has become a popular programming language for different applications, including data science, artificial intelligence, and web development. But, did you know creating and rendering fully customized videos with Python is also possible? At Stack Builders, we have successfully used Python libraries such as MoviePy, SciPy, and ImageMagick to generate videos with animations, text, and images. In this article, we will look closer at how Python can be used for video generation and explore some of the powerful libraries and tools that make it possible.

The Challenge

Creating a customized video might sound easy, especially if tools for video editing are an option. Creating one or maybe a couple of them might even sound fun, but if you need to generate hundreds or even thousands of customized videos daily, this does not sound like a good idea. Imagine
Facebook manually creating millions of videos for friendship anniversaries, birthdays and marketing campaigns for all their users. That would require an extreme amount of time to do, create delays in the delivery of the videos, and take away from an ideal user experience.

Producing a video programmatically might not be a task developers come across daily. Having a firm understanding of what the final video should look like is vital for accomplishing this task.

Facing the Challenge

Here is where software development comes in handy and helps automate tasks to make our lives easier once again. Having a toolbelt filled with tools like Python gives us the upper hand in this challenge. This tutorial will explain how to work with multiple video segments using MoviePy tools together.

What is MoviePy?

You might have heard of FFMPEG or ImageMagick for image and video edition in a programmatic way. MoviePy is a Python module for video editing (Python wrapper for FFMPEG and ImageMagick). It provides
functions for cutting, concatenations, title insertions, video compositing, video processing, and the creation of custom effects. It can read and write common video and audio formats and be run on any platform with Python 2.7 or 3+.

What is ImageMagick?

ImageMagick is an open-source software suite used for editing and manipulating digital images. It includes a command-line interface for executing complex image processing tasks, as well as APIs for integrating its features into software applications. MoviePy TextClip requires ImageMagick.

What is SciPy?

SciPy is a library used for scientific computing and technical computing. SciPy contains modules for image processing. MoviePy video tools plugins like Segmenting benefit from this to locate objects and separate them.

What Do We Need?

For this tutorial, ImageMagick is required and Python 3 with pipenv to manage its dependencies. All the modules can be installed from PyPI (Python Package Index) using pip. For further instructions check the readme file for this tutorial.

Getting Started

This sample code will help to understand some basic concepts about Python, MoviePy and image processing libraries.

Importing the libraries

Go the root directory of your project and create a new file named python_video.py and start by adding the required libraries at the top:

import moviepy.editor as mpy from moviepy.video.tools.segmenting import findObjects

In this piece of code we are telling Python to add the libraries that are going to be used in this tutorial. To keep it simple, we are adding all of the MoviePy tools inside its editor module and renaming it to mpy to keep it short.

In the next line, the MoviePy plugin find_objects is imported to locate and separate objects in a clip. We are adding it to locate five stars in an PNG file for later animating them in the final clip.

Creating Constants

First of all, we are going to create some constants to keep the code easy to understand. The first one is WHITE. It is
going to be the background color. MoviePy uses the traditional
RGB scale (0 - 255).

The second one is SCREEN_SIZE. We will be using a standard of 640x480.

The rest of them, VERTICAL_SPACE and HORiZONTAL_SPACE, are used for adding homogenous horizontal and vertical space across elements in the video:

WHITE = (255, 255, 255)
SCREEN_SIZE = (640, 480)
VERTICAL_SPACE=30
HORIZONTAL_SPACE=100

Adding an Image File

It's time to use MoviePy! We can add some images to the video to make it more attractive to the viewers. For this we will be using the following code:

SB_LOGO_PATH = "./static/StackBuildersLogo.jpg"

sb_logo = mpy.ImageClip(SB_LOGO_PATH).\
    set_position(('center', 0)).\
    resize(width=200)

The mpy.ImageClip function receives as a parameter the image to render. In this case the image path is set in the constant SB_LOGO_PATH. Using ImageClip attributes, it's possible to position the image using x, y coordinates. In this case, it's possible to mix keywords like center, bottom and top with specific coordinates.

Finally, the resize attribute reduces the size of the image keeping its ratio aspect using the key arguments width or height like in the snippet.
It is possible to set a custom size explicitly using a tuple with the new dimensions (width, height).

Writing Text

Now that we have the image in place, using TextClip we are able to create an instance of a video with centered text below the logo image.

    txt_clip = mpy.TextClip(
        "Let's build together",
        font="Charter-bold",
        color="RoyalBlue4",
        kerning=4,
        fontsize=30,
    ).\
    set_position(("center", sb_logo.size[1] + VERTICAL_SPACE]))

The variable txt_clip is created for use in video composition later; it is a TextClip instance that receives the text to display as the first parameter. The other parameters are used to style the text. Setting font, color, kerning and fontsize is enough.

Finally, it sets the position it will have when included in compositions, with the function set_position, it receives a tuple that contains the horizontal and vertical values. The positions are set here directly to use the sb_logo attribute size that provides a tuple of the clip dimensions (width, height). This helps to calculate in an easier way
the y-position of each clip. In this case the text is horizontally centered using the center key and has a calculated vertical space value based on the logo clip vertical size plus a constant to put some extra space.

Let’s also add an additional text to the video. As the txt_clip, create a new variable named txt_watermarkr and add it after the txt_clip code. This will work like a watermark to indicate that this video is made with Python so it will have a light red color and will be located at the bottom of the screen.

   txt_watermark = mpy.TextClip(
       "This video was generated with Python",
       color="#ff8c82",
       kerning=4,
       fontsize=22,
       stroke_color="#ff8c82",
       stroke_width=0.4,
   ).set_position(("center", sb_logo.size[1] + txt_clip.size[1] + VERTICAL_SPACE * 6))

Adding Animation/FX

With the text and image ready there is just one thing left to implement: Rotating stars (just because this video deserves 5 stars).

MoviePy is capable of locating and separating objects like letters or shapes in a clip. In this case, we are going to use an image with stars to apply a rotate effect in each of them.

STARS_PATH = "./static/stars-5.png"

stars_clip = mpy.CompositeVideoClip(
    [mpy.ImageClip(STARS_PATH).set_position("center")], size=SCREEN_SIZE
stars = findObjects(stars_clip)
)

As for the image file, we first start creating an ImageClip with the image containing the stars by passing the path set in the constant 'STARS_PATH'.

We can add it to a composite video by using mpy.CompositeVideoClip we can generate a VideoClip made up of other clips displayed together. For this sample the first parameter is a list with a single element and the second argument is the screen size 640x680.

Next, let’s assign it to the stars_clip var to pass it as an argument in the function findObjects to get each star separated in a list:

CLOCKWISE_ANGLE = -90

def rotate(stars):
    return [
        star.rotate(lambda t: t * CLOCKWISE_ANGLE, expand=False)
        .fx(mpy.vfx.mask_color)
        .set_position(((i + 1) * HORIZONTAL_SPACE, sb_logo.size[1] + txt_clip.size[1] + VERTICAL_SPACE * 2))
        for i, star in enumerate(stars)
    ]

This is where the fun begins! As we said before, this video deserves 5 stars,
and since configuring them to rotate one-by-one could be very repetitive, we are going to create a function that receives a list of stars and use a for loop in a list comprehension to return the list of stars inside clips (Like we
did with the txt_clip and sb_logo clips) and rotating them in a clockwise direction.

The rotate function receives as a first parameter the stars ImageClips to rotate. It uses the video effects function rotate with a lambda function as first parameter to rotate the star on its own axis in clockwise direction for each frame t by multiplying the time in the video by (-90) degrees. The other parameter, expand, is set to False to maintain the same size of the clip.

If the angle of rotation is not one of 90, 180, -90, -180 (degrees) there will be black borders. We can make them transparent with fx(mpy.vfx.mask_color), which sets a transparent mask color by default.

Finally, the position of each star is calculated so that they are centered below the image/text and horizontally separated equally between each other.

Generating the Video Clip

final_clip = (
    mpy.CompositeVideoClip([sb_logo, txt_clip] + rotate(stars), size=SCREEN_SIZE)
    .on_color(color=WHITE, col_opacity=1)
    .set_duration(10)
)

This piece of code is using mpy.CompositeVideoClip to compose a video clip with the clips already created sb_logo and txt_clip. Additionally it concatenates to that all the rotating stars provided by our rotate method.

The second parameter in CompositeVideoClip is the size of the final clip, in this case (640x480).

The on_color attribute helps to set the background color of the VideoClipto color=WHITE. The on_color attribute has some extra parameters to change size, position and color opacity, but for now we don't need them.

And last but not least, the video duration is set to 10 seconds using set_duration attribute.

final_clip.write_videofile("video_with_python.mp4", fps=10)

Finally, the write_videofile attribute writes in the disk the VideoClip
result of the CompositeVideoClip stored in memory. Its first parameter is
the video file name, then we define the fps rate. The write_videofile
has multiple parameters but we just need to set these two for now. You can also generate a GIF file by using write_gif instead.

Now if we run the code using python python_video.py, we will
have a 10 seconds video like this:

Figure 1 - Video generated by our code

This might seem simple, but it's our first video generated by code! If you want to dig deeper into the code, you can find the base code of the final version of the project

Conclusions

In summary, using Python for bulk video generation is a valuable asset for engineers working alongside marketing and content creation teams. This tutorial is meant to be a helpful resource to kickstart your journey.

In the upcoming weeks, we will cover the implementation of golden testing for this type of video production, providing further insights into its features and benefits. Ensure you subscribe to our blog to stay updated!