DEV Community: Gustavo Tavares

About Makefiles – Part I

Gustavo Tavares — Fri, 22 Apr 2022 17:16:50 +0000

Hello,

Today I will briefly dissert about Makefiles and make, an efficient automation tool to compiles many programs at once.

I am writing this because when doing my SPO600 Project, I encountered many Makefiles with different functions and I was curious on how they work.

What I noticed first is that not all the Makefiles compiles things, most of them just have flags and configurations like this:
A common use of Makefiles is to set variables to be included in other Makefiles:

But first, lets understand how hey work from the beginning:

Whats the purpose of `Makefile`?

A Makefile is a set of instructions that will be read when the make command is called.

Make, then will read the Makefile follow its instructions and compile and link the program it wants to do.

A function of the Makefile is to create several make scripts like:

make: will run the Makefile
make clean: will erase the binaries compiled by the Makefile and others
make install will create and install tar packages and others
make distclean and make realclean will remove same files as make clean but TAGS, makefiles and config.status files, make realclean also removes info files generated by .texinfo

The makefile can also tell make how to run miscellaneous commands when explicitly asked (for example, to remove certain files as a clean-up operation).

Source: GNU

How `Make` works?

The command make will read the directory and search for a Makefile. Then it will start processing the first rule. A rule is a instruction with a target, a prerequisite and a recipe.

A target is usually the name of a file that is generated by a program; examples of targets are executable or object files. A target can also be the name of an action to carry out, such as ‘clean’ (see Phony Targets).
A prerequisite is a file that is used as input to create the target. A target often depends on several files.
A recipe is an action that make carries out. A recipe may have more than one command, either on the same line or each on its own line. Please note: you need to put a tab character at the beginning of every recipe line! This is an obscurity that catches the unwary. If you prefer to prefix your recipes with a character other than tab, you can set the .RECIPEPREFIX variable to an alternate character (see Special Variables).

Source: GNU

Looking easy right?
Lets take a look at this next screenshot and change our minds:

As we can see, it is really hard to understand what’s going on because there is a lot of variables being used.

Can we use Variables to make our life easier?

Yes we can, to avoid duplication and reduce redundancy we can set a variable.
If we take a look at the screenshot above, we can see that the second line is declaring a variable and assigning a value to it.

SKIPHEADERS = compat/s32pthreads.h

This way, everytime we want to use it we just need to do $(SKIPHEADERS).

For example:

TESTOOLS is being declared and right away its being used in HOSTPROGS.

I will continue explaining about Makefiles and make in the next post!
See you there and Thank you for Reading!

SPO 600 – Project Step 3 – Analysis

Gustavo Tavares — Wed, 20 Apr 2022 23:05:56 +0000

Hello!

Is time for my final blog about SPO600.
The analysis for what I have done on Step 2.
A quick recap on what happened on step 2 was that I was able to implement auto-vectorization on FFmepg package.

So lets start our analysis.

A snippet of the disassembly (using `objdump -d`)

Because I used auto-vectorization, there was many places on the code that got optimized to sve2.
Lets take a look on the first whilelo we have on the screenshot and see if we can understand what's going on, lets use the arm64 documentation for that:

“WHILELO
While incrementing unsigned scalar lower than scalar

Generate a predicate that starting from the lowest numbered element is true while the incrementing value of the first, unsigned scalar operand is lower than the second scalar operand and false thereafter up to the highest numbered element.

The full width of the scalar operands is significant for the purposes of comparison, and the full width first operand is incremented by one for each destination predicate element, irrespective of the predicate result element size. The first general-purpose source register is not itself updated.”

From: https://developer.arm.com/documentation/ddi0596/2020-12/SVE-Instructions/WHILELO--While-incrementing-unsigned-scalar-lower-than-scalar-

Basically whilelo is a loop. And its taking Scalable predicate registers P0, WZR register and register w1.
Next instruction is mov, which is moving #0x0 to the register x0.
Again another mov moving #0 to register z1.

Next instruction is LD1D (vector plus immediate),

“Gather load doublewords to vector (immediate index)
Gather load of doublewords to active elements of a vector register from memory addresses generated by a vector base plus immediate index. The index is a multiple of 8 in the range 0 to 248. Inactive elements will not cause a read from Device memory or signal faults, and are set to zero in the destination vector.
“

From: https://developer.arm.com/documentation/ddi0596/2021-12/SVE-Instructions/LD1D--vector-plus-immediate---Gather-load-doublewords-to-vector--immediate-index--
This one is complex; This will load doubleword to a vector as it said, it is preparing for the next instruction.
Storing the doublewords into z0

Next one is:

“ST1D (scalar plus immediate)
Contiguous store doublewords from vector (immediate index)

Contiguous store of doublewords from elements of a vector register to the memory address generated by a 64-bit scalar base and immediate index in the range -8 to 7 which is multiplied by the vector's in-memory size, irrespective of predication, and added to the base address. Inactive elements are not written to memory.”

From: https://developer.arm.com/documentation/ddi0596/2020-12/SVE-Instructions/ST1D--scalar-plus-immediate---Contiguous-store-doublewords-from-vector--immediate-index--

This one will get what was on z0 (used on the previous instruction), multiply by the predicator and add to the base address (z1).

The next one:

INCB, INCD, INCH, INCW (scalar)
Increment scalar by multiple of predicate constraint element count
Determines the number of active elements implied by the named predicate constraint, multiplies that by an immediate in the range 1 to 16 inclusive, and then uses the result to increment the scalar destination.
The named predicate constraint limits the number of active elements in a single predicate to:
* A fixed number (VL1 to VL256)
* The largest power of two (POW2)
* The largest multiple of three or four (MUL3 or MUL4)
* All available, implicitly a multiple of two (ALL).
Unspecified or out of range constraint encodings generate an empty predicate or zero element count rather than Undefined Instruction exception.
It has encodings from 4 classes: Byte , Doubleword , Halfword and Word

From: https://developer.arm.com/documentation/ddi0596/2020-12/SVE-Instructions/INCB--INCD--INCH--INCW--scalar---Increment-scalar-by-multiple-of-predicate-constraint-element-count-

The last instruction in the 'whilelo' is incrementing x0 by multiple of predicate constraint element count.

As looking at the 'objdump', I can see that many pieces of it were optimized for sve2. Its instructions are using vectors and those vectors are using many registers at once as seem above. This means that it will execute faster than the previous implementation without vectors.

As I showed on Step 2, I did already some tests on it to see if it was working. I processed a video using the new compiled sv2 FFmpeg and the output was working as intended.

I have 2 directories, FFmpeg with sve2 implementation and FFmpeg0 without it:

I made the same test on both, I used the same input file “Flame.avi”.
Here are some screenshots:
On FFmpeg0 :

I used this command to compile my input file
Here is the result of the command:

And here are two screenshots side by side of flami.avi and output.avi played on vlc on the terminal:
Flame.avi:

Output.avi:

On FFmpeg (with sve2):

Flame.avi:

Output.avi (made with sve2 instructions):

Both output.avi files worked the exactly same way.

Other Analysis:

For the future, when we have armv9 hardware available:
Soon the auto vectorization will come to -O2 flag for gcc, this means that it will be considered a safe optimization.

The developers will have to change the Configure script so that they accept vectorization, and after that moment all the FFmepg compiled with that new flag will have sve2 instructions.

A check if the hardware is armv9 or armv8 will be needed, if it armv9 then use the ‘configure’ file for armv9 with the sve2 optimizations, If armv8 hardware is present, then use sve optimizations only as it is right now.

This is my analysis for Step 3 of our project.
I hope you enjoyed reading this.

Thank you

SPO600 Project – Step 2 – SVE2 Implementation

Gustavo Tavares — Sun, 10 Apr 2022 22:15:16 +0000

Hey There

Its time for Step 2 of our SPO600 Project.
Before we start, lets do a quick review on what we need to do in this project

Step 1: Research a library level package to be a candidate for sve2 implementation.

Step 2: Implement sve2 to the chosen package.

Step 3: Upstream your changes or prepare it for future implementation.

Continuing with FFmpeg

After choosing this package, I had to follow some steps to make sure it was able to receive sve2 through auto vectorization.

Check if there was previous implementation of sve (There was as said in Step 1)
Check if the compiler could apply the auto vectorization on this package.
Check the correct Makefile to change in order to apply the auto vectorization to all files in the package.

My Approach

After taking a look at the .S and .c files with neon optimization on them:

I realized that this package was able to receive the auto vectorization from the compiler, so I decided to give it a try.

Then I started looking for a Makefile, but for my surprise there was quite a few:

Where should I start searching, many of these files have some configuration that I cannot even understand properly.

So I decided to start from the beginning: ./Makefile
It looked like this:

And kept going and going, but no sign of compiler and optimizations.

But there was something there that caught my attention:

At the very top of the file there was an include that could help me, and then there I went to see if I could find the gcc instructions.

The congif.mak is generated by the ./configure script and it enables the neon optimization and others:

But when I checked the config.mak file looking for the gcc optimizations I found that they were disabling the vectorization:

So I decide to change it to enable the vectorization:

And then I run the make command.

After built, it was time to try it:

For my surprise, the first run I got core dumped error, which this time was a very welcome error message.

It meant that the program was built in a way that it could not be run by the current system.

So I tried to run it using the qemu-aarch64 emulator and for my surprise the program worked fine!

I tried to test it with a sample file few times to see if it worked and here is the result:

It converted my sample.avi to output.avi with 24 framerates as I requested.

It was time to check if there was indeed sv2 optimizations inside the binary file.

So I used Objdump -d and I found that there was really sve2 in there:

Here are some examples:

We can see z and p registers being used together with the whilelo instruction.

To sum up

Step 2 was an adventure. At first, I thought that going for auto-vectorization would be an easy task, I though that a Makefile would be waiting for me just to change the arguments of the compiler but in the end I had dozens of Makefiles, each with different configurations and it required a loot of reading and research to make it work.

I had to learn that there was configure scripts needed to make the configurations appears and the file I was looking for was not even a Makefile, it was a .mak one.

I pretend to write a little more about Makefiles as it seems to me a powerful tool and way more complicated and deeper than I have imagined.

Thank you for reading!

SPO600 – Final Project - Step 1

Gustavo Tavares — Mon, 28 Mar 2022 20:38:10 +0000

Hello!

We are getting closer to an end here and we are finally starting our final Project (We will be working in the Open!).

Step 1

For the first step of the project, we were supposed to choose some packages that would beneficiate from sve2 instructions.

The ideal package is one that process massive amounts of data. This way the sve2 can be used at its maximum capabilities to improve performance.

After some research, I found two candidates that could benefit from sve2:
Gstreamer1 and FFmpeg.

Gstream1

According to them:

 GStreamer is a streaming media framework, based on graphs of filters which operate on media data. 

Applications using this library can do anything from real-time sound processing to playing videos, and just about anything else media-related.  

Its plugin-based architecture means that new data types or processing capabilities can be added simply by installing new plugins.

Gstream1 use inline assembler code for many functions, as we can see here:

static inline void
inner_product_gint16_full_1_neon (gint16 * o, const gint16 * a,
    const gint16 * b, gint len, const gint16 * icoeff, gint bstride)
{
    uint32_t remainder = len % 16;
    len = len - remainder;

    asm volatile ("      vmov.s32 q0, #0\n"
                  "      cmp %[len], #0\n"
                  "      beq 2f\n"
                  "      vmov.s32 q1, #0\n"
                  "1:"
                  "      vld1.16 {d16, d17, d18, d19}, [%[b]]!\n"
                  "      vld1.16 {d20, d21, d22, d23}, [%[a]]!\n"
                  "      subs %[len], %[len], #16\n"
                  "      vmlal.s16 q0, d16, d20\n"
                  "      vmlal.s16 q1, d17, d21\n"
                  "      vmlal.s16 q0, d18, d22\n"
                  "      vmlal.s16 q1, d19, d23\n"
                  "      bne 1b\n"
                  "      vadd.s32 q0, q0, q1\n"
                  "2:"
                  "      cmp %[remainder], #0\n"
                  "      beq 4f\n"
                  "3:"
                  "      vld1.16 {d16}, [%[b]]!\n"
                  "      vld1.16 {d20}, [%[a]]!\n"
                  "      subs %[remainder], %[remainder], #4\n"
                  "      vmlal.s16 q0, d16, d20\n"
                  "      bgt 3b\n"
                  "4:"
                  "      vadd.s32 d0, d0, d1\n"
                  "      vpadd.s32 d0, d0, d0\n"
                  "      vqrshrn.s32 d0, q0, #15\n"
                  "      vst1.16 d0[0], [%[o]]\n"
                  : [a] "+r" (a), [b] "+r" (b),
                    [len] "+r" (len), [remainder] "+r" (remainder)
                  : [o] "r" (o)
                  : "cc", "q0", "q1",
                    "d16", "d17", "d18", "d19",
                    "d20", "d21", "d22", "d23");
}

FFmpeg

According to them:

FFmpeg is a complete and free Internet live audio and video broadcasting solution for Linux/Unix. It also includes a digital  VCR. It can encode in real time in many formats including MPEG1 audio and video, MPEG4, h263, ac3, asf, avi, real, mjpeg, and flash.

The Second option has many files that use neon and use gcc to compile.

Planning my approach

Auto-vectorization:
My plan for implementing sve2 in this project is to use the auto-vectorization, this means I will change the Makefile and include options that will make the compiler applies the optimizations for me.
Here is a exemple of Makefile from gstreamer1:

As we can see they are using -O0 for optimizations which means no optimizations at all.

So I will include the options -O3 -march=armv8-a+sve2 and test it to see if the improvements were made.

About Makefiles

Makefiles can be complicated, I thought this would be an easy approach first but there is so many Makefiles in a project and they are linked to each other.
Take a look at this example from FFmpeg:

MAIN_MAKEFILE=1
include ffbuild/config.mak

vpath %.c    $(SRC_PATH)
vpath %.cpp  $(SRC_PATH)
vpath %.h    $(SRC_PATH)
vpath %.inc  $(SRC_PATH)
vpath %.m    $(SRC_PATH)
vpath %.S    $(SRC_PATH)
vpath %.asm  $(SRC_PATH)
vpath %.rc   $(SRC_PATH)
vpath %.v    $(SRC_PATH)
vpath %.texi $(SRC_PATH)
vpath %.cu   $(SRC_PATH)
vpath %.ptx  $(SRC_PATH)
vpath %.metal $(SRC_PATH)
vpath %/fate_config.sh.template $(SRC_PATH)

TESTTOOLS   = audiogen videogen rotozoom tiny_psnr tiny_ssim base64 audiomatch
HOSTPROGS  := $(TESTTOOLS:%=tests/%) doc/print_options

# $(FFLIBS-yes) needs to be in linking order
FFLIBS-$(CONFIG_AVDEVICE)   += avdevice
FFLIBS-$(CONFIG_AVFILTER)   += avfilter
FFLIBS-$(CONFIG_AVFORMAT)   += avformat
FFLIBS-$(CONFIG_AVCODEC)    += avcodec
FFLIBS-$(CONFIG_POSTPROC)   += postproc
FFLIBS-$(CONFIG_SWRESAMPLE) += swresample
FFLIBS-$(CONFIG_SWSCALE)    += swscale

FFLIBS := avutil

DATA_FILES := $(wildcard $(SRC_PATH)/presets/*.ffpreset) $(SRC_PATH)/doc/ffprobe.xsd

SKIPHEADERS = compat/w32pthreads.h

# first so "all" becomes default target
all: all-yes

include $(SRC_PATH)/tools/Makefile
include $(SRC_PATH)/ffbuild/common.mak

FF_EXTRALIBS := $(FFEXTRALIBS)
FF_DEP_LIBS  := $(DEP_LIBS)
FF_STATIC_DEP_LIBS := $(STATIC_DEP_LIBS)

$(TOOLS): %$(EXESUF): %.o
        $(LD) $(LDFLAGS) $(LDEXEFLAGS) $(LD_O) $^ $(EXTRALIBS-$(*F)) $(EXTRALIBS) $(ELIBS)

target_dec_%_fuzzer$(EXESUF): target_dec_%_fuzzer.o $(FF_DEP_LIBS)
target_dec_%_fuzzer$(EXESUF): target_dec_%_fuzzer.o $(FF_DEP_LIBS)
        $(LD) $(LDFLAGS) $(LDEXEFLAGS) $(LD_O) $^ $(ELIBS) $(FF_EXTRALIBS) $(LIBFUZZER_PATH)

tools/target_bsf_%_fuzzer$(EXESUF): tools/target_bsf_%_fuzzer.o $(FF_DEP_LIBS)
        $(LD) $(LDFLAGS) $(LDEXEFLAGS) $(LD_O) $^ $(ELIBS) $(FF_EXTRALIBS) $(LIBFUZZER_PATH)

target_dem_%_fuzzer$(EXESUF): target_dem_%_fuzzer.o $(FF_DEP_LIBS)
        $(LD) $(LDFLAGS) $(LDEXEFLAGS) $(LD_O) $^ $(ELIBS) $(FF_EXTRALIBS) $(LIBFUZZER_PATH)

tools/target_dem_fuzzer$(EXESUF): tools/target_dem_fuzzer.o $(FF_DEP_LIBS)
        $(LD) $(LDFLAGS) $(LDEXEFLAGS) $(LD_O) $^ $(ELIBS) $(FF_EXTRALIBS) $(LIBFUZZER_PATH)

tools/target_io_dem_fuzzer$(EXESUF): tools/target_io_dem_fuzzer.o $(FF_DEP_LIBS)
        $(LD) $(LDFLAGS) $(LDEXEFLAGS) $(LD_O) $^ $(ELIBS) $(FF_EXTRALIBS) $(LIBFUZZER_PATH)

… 
(it keeps going and going…)

Looks like some kind of Martian language to me.

Finally

After some research I decided to go with ffmpeg.
Because gstream1 don't use make and Makefiles to compile, it uses meson and ninja which makes life difficult for me as I have no knowledge at all in those technologies.

To change the ffmpeg Makefile, I will have to change the 'config.mak' file inside the ffbuild directory as it send configurations to the Makefile which builds the project.

Thats it for now!
Thank you for reading!

SPO600 Lab06 - sve2

Gustavo Tavares — Mon, 21 Mar 2022 02:14:44 +0000

Hi there,

This lab build up on top of the past lab, this time we need to make a new volume function that will use SVE2 syntax.

Even though it looks simple, it is really hard to understand inline assembler and intrinsic code.

For this lab, I decided to change one of the previous functions to use sve2.

My Approach

The one function I decided to change was vol4(remembering that vol4.c was made by the professor, not by me):

First things I had to change was adding the sve header file:

After that, instead of using the v registers, I used z as instructed in the sve2 instructions:

(Image from: https://developer.arm.com/documentation/102340/0001/SVE2-architecture-fundamentals?lang=en)

The change was not huge, but without the sve2 instructions on the Makefile it could not compile:

And to run it we need to to use qemu-aarch64:

Without it we have a illegal instruction error:
Core dumped:

On object dump, we can see that sqrdmulh is using the z registers.
Objdump -d

Thank you for your reading!

SPO600 Lab 05 - Algorithm Selection

Gustavo Tavares — Wed, 09 Mar 2022 02:50:49 +0000

Hello

In this lab we are supposed to run different versions of the same program and analyze the performance difference.

This program basically will scale a number of samples that will be decided later but that must be enough for the program to run for 20 seconds or more.

Each version of the program has some variance in in the algorithm.
Here are they:
From our lab page

1.  vol0.c is the basic or naive algorithm. This approach multiplies each sound sample by the volume scaling factor, casting from signed 16-bit integer to floating point and back again. Casting between integer and floating point can be expensive operations.

2.  vol1.c does the math using fixed-point calculations. This avoids the overhead of casting between integer and floating point and back again.


3.  vol2.c pre-calculates all 65536 different results, and then looks up the answer for each input value.

4.  vol3.c is a dummy program - it doesn't scale the volume at all. It can be used to determine some of the overhead of the rest of the processing (besides scaling the volume) done by the other programs.


5.  vol4.c uses Single Instruction, Multiple Data (SIMD) instructions accessed through inline assembley (assembly language code inserted into a C program). This program is specific to the AArch64 architecture and will not build for x86_64.

6.  vol5.c uses SIMD instructions accessed through Complier Intrinsics. This program is also specific to AArch64.

In this lab we will use the AArch64 and x86_64 machines provided by the professor.

AArch64

First, as said by the professor, we need to calculate the overhead of a dummy program.
This way we will know how much overhead we have and this way we can decrease it from our other algorithms runs.

So lets set the number of samples so our program can run for over 20 seconds:

Then we compile our programs using the make command:

And now lets run vol3 (Dummy program) for 5 times and see how long it takes in average.

Lets discard the highest and the lowest values and get the average: 26.946 seconds
This is the value of our overhead that we will subtract from the other programs.

Now lets do the same with others:

vol0:

Again, lets discard the highest and lowest ones and get the average, then we subtract from the dummy overhead.
Our Average is 27.684 seconds.
27.684 - 26.946 (overhead) = 0.738s.
We had 0.738s scaling time.

vol1:

Repeating the process, we have 27.501s average
Subtracting the overhead we have: 0.555s scaling time

vol2:

Average: 33.748s
Subtracting the overhead we have: 6.802s scaling time.

vol4:

Average: 26.763s.
Subtracting the overhead we have: -0.183s scaling time.

vol5:

Average: 26.729s.
Subtracting the overhead we have: -0.216s scaling time.

I tried to change the Makefile to try to make it run faster by using the -Ofast and enable the vectorization

And had a 3 second average difference from the -O2.

Now lets go to x86_64 machine

x86_65

The dummy (vol3) ovearhead value is 23.650s.

Vol0 is the naïve implementation:
The average was 24.158 s.
Minus the overhead, the time scaling is: 0.508s

Vol1 is the fixed point implementation:
The average was 23.736s.
Minus the overhead, the time scaling is: 0.086s

Vol2 is the precalculated implementation:
The average was 24.676sz.
Minus the overhead, the time scaling is: 1.026s

Finally

Lets answer the questions on the c code:

The dummy program will help us measure the overhead, this is the process that every other vol program will go through so we need to know how much it is in order to know how much is the scaling process time.

Those parts are needed because if you don’t use the result of your calculations, the compiler will see it and avoid it as you are not using it anyway, so In the end your program will end up doing nothing at all.

That’s a good question. I would say to make sure it’s a positive number, but to be honest Im not sure. ☹

I could notice that different algorithms produce different outputs. But running the same program multiple times will produce the same result even with different performance times. I would say that the difference results between algorithms, in terms of audio is not big enough for us to perceive.

Thank you for reading!

Spo600 Lab 4

Gustavo Tavares — Mon, 07 Mar 2022 00:42:25 +0000

Lab 04 is here

In this lab we were supposed to:

Login into Portugal and Israel servers.
Unpack a tarball, which for a windows user like me can be a new experience.
Explore the files inside the spo600 folder and use the makefiles and make command to build the code.
Use objdump -d to inspect what was inside the binary file.
Modify the Hello World! In AArch64 and x82_64 to make it loop.
Modify the loop to run until 30 iterations (which took hours for me)

I started by the AArch64 as described in the lab, the professors lecture gave us the bread and the butter for this lab, which without it probably it was going to be impossible for me.

AArch64

Here is my final AArch64 code looping from 00 to 29:

Aarch64 Final code:

.text
.globl _start
min = 0                          /* starting value for the loop index; note that this is a symbol (constant), not a variable */
max = 30
                        /* loop exits when the index hits this number (loop condition is i<max) */


_start:

        mov     x15, 10 // for division
        mov     x19, min // value of the counter
        add     x18, x19, '0' //counter
        adr     x17, msg+6
        adr     x20, msg+7

loop:
        cmp x19, x15

        b.lt inside //start from smallest decimals

        udiv x12, x19, x15 // divide x19/x15 and store in x12 // udiv r0,r1,r2     // unsigned - divide r1 by r2, places quotient into r0 - remainder is not calculated (use msub)
        add x14, x12, '0'
        strb w14,[x17]

inside:

        msub x10,x15,x12,x19 //msub r0,r1,r2,r3  // load r0 with r3-(r1*r2) (useful for calculating remainders)
        add x10, x10, '0'
        strb w10,[x20]

        //Printing msg

        mov     x0, 1           /* file descriptor: 1 is stdout */
        adr     x1, msg         /* message location (memory address) */
        mov     x2, len         /* message length (bytes) */


        // call syscall

        mov     x8, 64          /* write is syscall #64 */
        svc     0               /* invoke syscall */


        add x19, x19, 1 //counter +1
        cmp x19, max
        b.ne loop


        //return

        mov     x0, 0           /* status -> 0 */
        mov     x8, 93          /* exit is syscall #93 */
        svc     0               /* invoke syscall */

.data
msg:    .ascii      "Loop: #\n"
len=    . - msg

As mentioned by the professor in the lab requirements, we had to use udiv and calculate the reminder in order to get 2 decimal cases.
Problems I have encountered:

Assembly is hard, all these registers can get messy, so I started taking notes on what register means what. This way is easier for me to control how many I used and for what I have used them for.
The cheat sheet is essential, and even with it still confusing to understand sometimes how it works.
Infinite loops and characters, these two were my most common results when trying to make it work. My code was going over the ACII : many times and sometimes the loop never ended.

x86_64

The x86_64 proved harder than the AArch64.
As the professor said, we need more steps to make the division work which makes the code longer and harder to make it work.
I could not make the code work, it compiles and run but there is no output printed.

.text
.globl  _start

min = 0                         /* starting value for the loop index; note that this is a symbol (constant), not a variable */
max = 30                       /* loop exits when the index hits this number (loop condition is i<max) */
_start:

        mov     $min,%r15           /* loop index */
        mov     $10, %r14
    mov     $10, %r13
        mov     %r15,%r13       /* move 15 to 13 */
        mov     %r15,%r14       /* move 15 to 14 */

        mov     $10,%r8
        movq    $len,%rdx                       /* message length */


loop:

        mov     $0, %r11
            mov     %r9, %r12

            div     %r8    

        add     $'0',%12

        movb    %r14b,msg+6      /* Put r14 into msg+6 first #*/


inner: 

        add     $'0', %11
        movb    %r13b,msg+7 



             movq    $msg,%rsi                       /* message location */
            movq    $1,%rdi                         /* file descriptor stdout */
        movq    $1,%rax                         /* syscall sys_write */
        syscall


        inc     %r15                /* increment index */
        cmp     $max,%r15           /* see if we're done */
        jne     loop                /* loop if we're not */



        movq    $0,%rdi                         /* exit status */
        movq    $60,%rax                        /* syscall sys_exit */
        syscall

.section .data

msg:    .ascii      "Loop: ##\n"
        len = . - msg

Finally

It is hard. To learn assembly language is hard, and to learn two different systems at the same time makes it even harder.
Its clear that my preference is the AArch64, I think its easier for me and the registers are easier to remember.
Im pretty sure im close to make the x86-64 version of the loop to work, just need to figure out how to understand and work with the registers a little more.
I will certainly go back to it.

Thank you for reading.

SPO600 - W5

Gustavo Tavares — Tue, 01 Mar 2022 22:04:18 +0000

Hello there,

This time we will talk about what we learned in Week 5 in SPO600 classes. Make, Makefiles, and different computer architectures (x86_64 and AARCH64).

Makes and Makefiles

In week 5, we talked about Makes and Makefiles, both used when building software.
Let’s take a look on both and see how they work:

Make

Make is a specialized scripting language used to build software according to or WIKI.
The interesting about make is that its commands are not executed in a linear logic as we are used to, instead they follow a input and output order and make will automatically sequence the order for us.

When we run the make command, it will execute the makefile, but what is it?

Makefile

The makefile is a script, a set of commands with variables and targets to create a file.
A makefile looks like this (example from our WIKI):

CC=cc
CFLAGS=-O3

all:         half double

half:        half.o sauce.o
             ${CC} ${CFLAGS} -o half half.o sauce.o

double:      double.o sauce.o
             ${CC} ${CFLAGS} -o double double.o sauce.o

half.o:      half.c number.h
             ${CC} ${CFLAGS} -c half.c

double.o:    double.c number.h
             ${CC} ${CFLAGS} -c double.c

sauce.o:     sauce.c
             ${CC} ${CFLAGS} -c sauce.c

So when the make is exectured, it will use this makefile as guide and 5 compilations will be performed:

$ make
cc         -O3 -c half.c
cc         -O3 -c sauce.c
cc         -O3 -o half half.o sauce.o
cc         -O3 -c double.c
cc         -O3 -o double double.o sauce.o

I like to imagine that make is the cook and makefile is the recipe for a software.

x86_64 vs AArch64

Let’s talk about the register differences between x86_64 and AArch64:

x86_64

x86 - The Intel/AMD architecture which debuted with the Intel 8086 processor (16-bit), gained desktop and server dominance as the 386/486/x86 32-bit architecture, and was extended by AMD to the 64-bit x86_64 architecture. Intel and AMD vigorously compete with x86_64 CPUs, which continue as the preeminent server architecture and most popular desktop architecture.

source: WIKI

The general-purpose registers:

• rax - register a extended
• rbx - register b extended
• rcx - register c extended
• rdx - register d extended
• rbp - register base pointer (start of stack)
• rsp - register stack pointer (current location in stack, growing downwards)
• rsi - register source index (source for data copies)
• rdi - register destination index (destination for data copies)
• r8 - register 8
• r9 - register 9
• r10 - register 10
• r11 - register 11
• r12 - register 12
• r13 - register 13
• r14 - register 14
• r15 - register 15

AArch64

ARM - An architecture which started with the Acorn computer company, became the dominant mobile and embedded architecture in its 32-bit incarnations, and was extended to 64-bit in version 8 (ARMv8) with the AArch64 mode. 64-bit ARM processors are dominant in smartphone applications and starting to be compete in server and high-performance computing systems.

source: WIKI

• r0 through r30 – general registers
• x0 through x30 - for 64-bit-wide access (same registers)
• w0 through w30 - for 32-bit-wide access (same registers - upper 32 bits are either cleared on load or sign-extended (set to the value of the most significant bit of the loaded value)).

SPO600 - Week 4

Gustavo Tavares — Fri, 18 Feb 2022 19:43:20 +0000

Hey there,

In week 4 we looked on how to compiler optimize our code. It was made clear to us that the way we write code actually will be rewritten by the compiler, so our code gets optimized.

The optimization that the compiler does, most of the time is to make the code achieve the same result but with improved performance, in order to do that it changes our code in certain ways that we will take a look right now.

It is important to say that we, as programmers, should not worry about doing this optimization ourselves, the compiler will automatically do it for us.

About Code Rewriting Optimizations

There is a series of techniques of Code Rewriting Optimizations that the compiler does like: Strength Reduction, Hoisting, Hoisting II - Loop-Invariant Expression, Pre-calculation of Constants and others… More can be seen on our wiki page here.

Strength Reduction

We all know that some operations are more expansive than others, for example a multiplication is more expansive than an addition.
The Strength Reduction is nothing less than replacing a more expansive operation for a cheaper one but that will achieve the same result.

Take a look on the example extracted from https://wiki.cdot.senecacollege.ca/wiki/Compiler_Optimizations#GCC_Optimization_Options

//BEFORE
int x;
for (x=0; x < 10; x++) {
    printf("%d\n", x*6);
}

//AFTER

int x;
for (x=0; x < 60; x+=6) {
    printf("%d", x);
}

Hoisting

Hoisting involves moving an operation outside of the loop.
If that operation inside the loop is resulting in the same result as if it was outside of the loop, the compiler will just move it outside.

//For example
for(int i = 0; i  < 5; i++){

var1=0; 
b=b+5;

}

//Will become: 

var1=0;
for(int i= 0; i < 5; i++){

b=b+5;

}

Hoisting II - Loop-Invariant Expression

Hoisting II - Loop-Invariant Expression will get an expansive calculation inside a loop and bring it outside so it can be made just once, then the result will be placed back on its former place.

For example,

Before:

for(int i = 0; i  < 5; i++){
foo(a, 5*5)
}

//After:
int b = 5*5;
for(int i = 0; i  < 5; i++){
foo(a, b)
}

Pre-calculation of Constants

Pre-calculation of Constants is basically getting a constant expression that will never change and replace it for its result.

See the example from https://wiki.cdot.senecacollege.ca/wiki/Compiler_Optimizations#GCC_Optimization_Options

Before:
ff = (212-32)/100; /* factor for celcius-farenheit conversion */
conv = c * ff + 32;

After:
conv = c * 1.800 + 32;

There is many that you can read on our wiki.

What I learned

The code that I write as a programmer, and the code that will be transformed in machine language are actually different. The compiler will get everything I have done and will optimize it.

The compiler is smarter than me, that’s not hard to see, but its good to know that it will take care of these kind of optimizations for me.

SPO600 - Lab03

Gustavo Tavares — Mon, 07 Feb 2022 19:20:36 +0000

For lab 03, we were supposed to create a basic game or calculator using 6502 assembly language.

My idea

We the professor noticed us, Assembly language is hard, and we should keep it simple as much as we can.
My idea was to create a simple calculator that would divide any number from 0 to 9 by 2 using LSR(Logical Shift Right).

The way it works

First I display a message to the user:
Type a number from 0 to 9:

I did it using dcb and CHROUT, this way:

LDY #$00

nextchar:  ;DISPLAY INITIAL MESSAGE

    LDA msg,Y
    BEQ getnumber
    JSR CHROUT
    INY
    BNE nextchar    

msg:
dcb "T","y","p","e",32,"a",32,"n","u","m","b","e","r",32,"f","r","o","m",32,"0",32,"t","o",32,"9",":",0

Then I get the input from the user using CHRIN:

getnumber:   ; ACCEPT ONLY NUMBERS

    LDY #$00
    JSR CHRIN

    CMP #$00
    BEQ getnumber

    CMP #$30
    BMI getnumber

    CMP #$39
    BPL getnumber

    LDX A ;Load x register with A 

    JSR CHROUT

This code means accept only numbers from 0 to 9, if user enters something different than that then it will be ignored.

After that I display the following message:
Your number dividev by to is:

dividemsg: ;DISPLAY DIVIDE MESSAGE

    LDA msg2,y
    BEQ result
    JSR CHROUT
    INY
    BNE dividemsg   

msg2:
dcb $0d,"Y","o","u","r",32,"n","u","m","b","e","r",32,"d","i","v","i","d","e","d",32,"b","y",32,"2",32,"i","s",":",0

After the message is displayed the result should be outputs in the result label:

result:  ; GET THE RESULT - LSR DIVIDE BY 2

    LDY #$00
    CPX #$00 ;Compare X with 0
    BEQ zero ;Display divide by 0 message

    LSR X   ;Divide by 2
    LDA X   ; load result in A
    JSR CHROUT ;Display the result

It is important to say that I could not make this part work properly, the result is not displaying to the user.

And if the user is trying to divide by 0, then a message should be displayed:

zero:   ;MESSAGE IN CASE DIVIDING BY 0

    LDA msg3,y
    BEQ done
    JSR CHROUT
    INY
    BNE zero


done: BRK

The entire code can be found here:


define      SCINIT      $ff81 ; initialize/clear screen
define      CHRIN       $ffcf ; input character from keyboard
define      CHROUT      $ffd2 ; output character to screen
define      SCREEN      $ffed ; get screen size
define      PLOT        $fff0 ; get/set cursor coordinates


    LDY #$00

nextchar:  ;DISPLAY INITIAL MESSAGE

    LDA msg,Y
    BEQ getnumber
    JSR CHROUT
    INY
    BNE nextchar    

getnumber:   ; ACCEPT ONLY NUMBERS

    LDY #$00
    JSR CHRIN

    CMP #$00
    BEQ getnumber

    CMP #$30
    BMI getnumber

    CMP #$39
    BPL getnumber

    LDX A ;Load x register with A 

    JSR CHROUT



LDY #$00

dividemsg: ;DISPLAY DIVIDE MESSAGE

    LDA msg2,y
    BEQ result
    JSR CHROUT
    INY
    BNE dividemsg   


result:  ; GET THE RESULT - LSR DIVIDE BY 2

    LDY #$00
    CPX #$00 ;Compare X with 0
    BEQ zero ;Display divide by 0 message

    LSR X   ;Divide by 2
    LDA X   ; load result in A
    JSR CHROUT ;Display the result



zero:   ;MESSAGE IN CASE DIVIDING BY 0

    LDA msg3,y
    BEQ done
    JSR CHROUT
    INY
    BNE zero


done: BRK

msg:
dcb "T","y","p","e",32,"a",32,"n","u","m","b","e","r",32,"f","r","o","m",32,"0",32,"t","o",32,"9",":",0


msg2:
dcb $0d,"Y","o","u","r",32,"n","u","m","b","e","r",32,"d","i","v","i","d","e","d",32,"b","y",32,"2",32,"i","s",":",0

msg3:
dcb 32,"C","a","n","n","o","t",32,"d","i","v","i","d","e",32,"0","!",0

SPO600 - W3

Gustavo Tavares — Mon, 07 Feb 2022 17:18:59 +0000

In week 3 was time to learn about:

Math
Characters
and strings!
Branches, Jumps and Procedures

About Math

6502 Assembly language can perform calculations on binary and decimal mode.

In binary, it will perform the operations on a 8-bite value.
In decimal, the bytes are treated as 2 decimal digits.

The operations are:

ADC (Add with Carry)
This will perform the following: the value in the accumulator + the specified byte + the carry flag.
SBC (Subtract with Carry)
This will perform the following: the value in the accumulator - the specified byte - (not Carry)
LSR (Logical Shift Right)
This will perform a division by 2.
ASL (Arithmetic Shift Left)
This will perform a multiplication by 2.

About Characters and Strings

In 6502, we can use dcb to make a string. Keep in mind that if you want to use enter you need to use $0d and 32 for space, for example:

msg:
dcb "T","y","p","e",32,"a",32,"n","u","m","b","e","r",32,"f","r","o","m",32,"0",32,"t","o",32,"9",":",0

The 0 at the end is to control the end of the string.

Having this, we can use JSR CHROUT to output the values to the screen.

LDY #$00
nextchar:  ;DISPLAY INITIAL MESSAGE

    LDA msg,Y
    BEQ getnumber
    JSR CHROUT
    INY
    BNE nextchar

For Input, we can use CHRIN to get input from user.

getnumber:   ; ACCEPT ONLY NUMBERS

    LDY #$00
    JSR CHRIN

    CMP #$00
    BEQ getnumber

    CMP #$30
    BMI getnumber

    CMP #$39
    BPL getnumber

    JSR CHROUT

This code will accept only a number from 0 to 9 from the user.

Branches, Jumps and Procedures

Branches, Jumps and Procedures are how we can control the flow of our program.
Branches can be used for conditional jumps
For example:
We can use CMP #$00 (Compare to 0) and then BNE branchName (Branch if not equal) or BEQ branchName (branch if eqqual).

Jumps can be used to perform a change in the branch independently of any conditions.
For Exemple:
JMP done will bring you to the done label.

The procedures will transfer the control to a subroutine.

For example, if you want to execute a subroutine you can:
JSR myfunction

Myfunction: …
RTN

Note that in your subroutine you need to return.

The system have some already pre defined sobroutines:

define      SCINIT      $ff81 ; initialize/clear screen
define      CHRIN       $ffcf ; input character from keyboard
define      CHROUT  $ffd2 ; output character to screen
define      SCREEN      $ffed ; get screen size
define      PLOT        $fff0 ; get/set cursor coordinates

This was it for W3

Thank you so much for reading!

SPO600 - LAB 02

Gustavo Tavares — Mon, 31 Jan 2022 22:41:16 +0000

Lab 02 - 6502!

This lab goal is to calculate the performance of a 6502 program.
The program is this one and can be run HERE:

    lda #$00    ; set a pointer at $40 to point to $0200
    sta $40
    lda #$02
    sta $41

    lda #$07    ; colour number

    ldy #$00    ; set index to 0

loop:   sta ($40),y ; set pixel at the address (pointer)+Y

    iny     ; increment index
    bne loop    ; continue until done the page

    inc $41     ; increment the page
    ldx $41     ; get the current page number
    cpx #$06    ; compare with 6
    bne loop    ; continue until done all pages

In order to calculate how many seconds, the program will take to run, we need to calculate how many cycles each instruction will take and how much time each instruction will take to be completed.

To do that, we have to take a look at this.

This website provide us the clock values for each instruction, this way we are able to measure the total time that the program will take at 1MHz.

Lets Calculate Performance

Lets start with our first instruction:
lda #$00

Lda is our first instruction, so now lets take a look at the website to see the cycles values:

Here we can see that LDA takes 2 cycles, because we are using immediate addressing.

To keep track, lets make a chart with all the values:

As we can see in my chart, we are getting the number of cycles, how many times they will happen, the alternative cycles and how many times they will happen and making a sum of the total cycles.

Here we can see the final number of cycles are multiplied by the CPU speed, giving us the time needed to run the program.

I tried to optimize it and make it run faster but I am finding difficult to work with the pages, my goal was to make for each loop a pixel on each page to become yellow, this way maybe it would be faster but I could not make it work to calculate its speed.

Experimenting:

The first program will give us this screen:

When experimenting with tya instruction, we will have this result:

We can see now that we have stripes with different colors in the screen, tya means transfer Index Y to accumulator which means that every time the loop happens the accumulator (which it’s the color) will get incremented by one.

But when trying to use lsa after tya I keep getting this error:

My thoughts on this Lab

It is really cool to understand how to calculate the cycles and the time that takes for a 6502 program to run, this way we are able to make tests to make it faster.
But I feel like sometimes working with memory addresses can be complicated and confusing, there is many ways to write it and I feel like I need more time to understand it better.
I could not optimize the program the way I wanted to, which made me frustrated but I am decided to come back to this lab in the future and try it again.

Thank you for reading!

DEV Community: Gustavo Tavares

About Makefiles – Part I

Whats the purpose of Makefile?

How Make works?

Can we use Variables to make our life easier?

SPO 600 – Project Step 3 – Analysis

Hello!

A snippet of the disassembly (using objdump -d)

Other Analysis:

SPO600 Project – Step 2 – SVE2 Implementation

Hey There

Continuing with FFmpeg

My Approach

To sum up

SPO600 – Final Project - Step 1

Hello!

Step 1

Gstream1

FFmpeg

Planning my approach

About Makefiles

Finally

SPO600 Lab06 - sve2

Hi there,

My Approach

SPO600 Lab 05 - Algorithm Selection

Hello

AArch64

Now lets do the same with others:

vol0:

vol1:

vol2:

vol4:

vol5:

x86_65

Finally

Spo600 Lab 4

Lab 04 is here

AArch64

x86_64

Finally

SPO600 - W5

Hello there,

Makes and Makefiles

Make

Makefile

x86_64 vs AArch64

x86_64

AArch64

SPO600 - Week 4

Hey there,

About Code Rewriting Optimizations

Strength Reduction

Hoisting

Hoisting II - Loop-Invariant Expression

Pre-calculation of Constants

What I learned

SPO600 - Lab03

My idea

The way it works

SPO600 - W3

About Math

About Characters and Strings

Branches, Jumps and Procedures

This was it for W3

SPO600 - LAB 02

Lets Calculate Performance

Experimenting:

My thoughts on this Lab

Whats the purpose of `Makefile`?

How `Make` works?

A snippet of the disassembly (using `objdump -d`)