Challenge RE #13

Let's do another re challenge, if you've been reading my recent posts you know that I'm solving the challenges from challenges.re. This time, I'm going to analyze the short but not less interesting #13. Let's start.

Analysis

This time we don't have a binary file, instead we have a quite short assembly code. Using 128-bit registers, xmm0 and xmm1. For this program we have three arguments, rsi, rdx and rdi. The first two contains unsigned integers, given the way we are accessing them we can infer they are being passed on 2 arrays.

Here is the code

f:
    xor rax, rax
.L4:
    movdqu  xmm0, XMMWORD PTR [rsi+rax]
    movdqu  xmm1, XMMWORD PTR [rdx+rax]
    pmaxub  xmm0, xmm1 ;; returns maximum of both
    movdqu  XMMWORD PTR [rdi+rax], xmm0
    add rax, 16
    cmp rax, 1024
    jne .L4
    rep ret

Immediately we can notice that we have a loop, and that the iteration will end when rax will be 1024. On each iteration we increase rax by 16, so we will iterate 64 times. Now what's done on these iterations?

Actually quite simple, we are comparing each elements of the array, the one in rsi and rdx, and selecting the maximum value between the two to be saved in rdi. Given that rdi it's a 64 bit register and we iterate exactly 64 times, we will fill rdi with these maximum elements.

With this, we can already give our formal description of the program:

Formal description

Given two arrays, a and b, with unsigned integers, we return and array arr where for each i = 0, ..., 64, we have arr[i] = max(a[i], b[i]).

That's it!!