DEV Community

Lahari Tenneti
Lahari Tenneti

Posted on

LLVM #6 — Compiling to Object Code.

#llvm #compilers #computerscience #learning

This is a short chapter but with a very tangible and "objective" payoff.

Basically, every function we typed into the REPL got compiled and ran inside the same process, in RAM. The JIT took the IR, converted it into machine code, and executed it instantly. When the process exits, it's all gone.

Now, instead of "running" the code, we write it to the disk as a .o file, which is compiled into machine code in a format the linker understands. This can be linked with any other C++ program, enabling us to call the Kaleidoscope functions as if they were normal C functions. The code actually outlives the compiler process that produced it.

What I built: Commit f229e86

What I understood:

  • The process of emitting object code includes picking a target, describing the machine, configuring the module, and emitting.

1) The Target Triple:

  • So LLVM is designed for cross-compilation, meaning it can target an Intel Mac, an ARM Phone, a Windows PC... anything.
  • For this, it needs a complete machine profile encoded as a string called the target triple.
  • Format: <architecture>-<vendor>-<operating-system>-<ABI>
  • Example: x86_64-unknown-linux-gnu means a 64-bit Intel, unspecified vendor, Linux, and GNU calling conventions.
  • It's just that instead of hardcoding a triple, we ask LLVM for the current machine's: 
auto TargetTriple = sys::getDefaultTargetTriple();
Enter fullscreen mode Exit fullscreen mode

2) Initialising subsystems:

  • LLVM doesn't activate all its backends by default. The JIT only needed the native target. But for writing an object file to the disk, we need everything (hardware platform information, core codegen, machine-code abstractions, and assembly reader and writer). 
InitializeAllTargetInfos() //registers available hardware platforms so LLVM knows what targets exist
InitializeAllTargets() //loads the code generators; without this, lookupTarget() returns nullptr even with a valid triple
InitializeAllTargetMCs() //handles the MC layer to turn abstract instructions into actual bytes
InitializeAllAsmParsers() //enables reading assembly text as input
InitializeAllAsmPrinters() //enables writing machine instructions to a file; needed for addPassesToEmitFile()
Enter fullscreen mode Exit fullscreen mode

3) Target Machine:

  • Once the matching target is obtained from the registry, we create the Target Machine, which is a generic CPU with no special features.
  • We also mark the data layout and triple onto the module, which tells the optimiser about things like pointer sizes, alignment rules, and the target's memory layout. 
auto TM = Target->createTargetMachine(Triple(TargetTriple), "generic", "", opt, Reloc::PIC_);
TheModule->setDataLayout(TM->createDataLayout());
TheModule->setTargetTriple(Triple(TargetTriple));
Enter fullscreen mode Exit fullscreen mode

4) Emitting:

  • A legacy::PassManager with one pass runs over the module and writes the object file: 
raw_fd_ostream dest("output.o", EC, sys::fs::OF_None);
legacy::PassManager pass;
TM->addPassesToEmitFile(pass, dest, nullptr, CodeGenFileType::ObjectFile);
pass.run(*TheModule);
dest.flush();
Enter fullscreen mode Exit fullscreen mode

What I didn't uderstand:

  • More like where I faced issues: 1) Empty object file:
  • The tutorial places all the emit code at the bottom of main(), after MainLoop() returns. The idea is to parse everything, and then bake it all to disk. I expected this to work: 
./toy < test.k        #should parse celsius, emit output.o
nm output.o           #should show: T celsius
Enter fullscreen mode Exit fullscreen mode
  • But instead got: 0000000000000000 t ltmp0
  • A t instead of a T meant it was a local symbol, implying the celsius function was absent.
  • This was because inside HandleDefinition(), right after codegen, the module moves into the JIT and is reset.
  • By the time main() reached the emit code, TheModule has nothing inside it. We'd be compiling an empty module to disk, and thus the ltmp0.
  • The answer is to emit inside HandleDefinition() before the JIT move. The function is still alive in TheModule at that point, so pass.run(*TheModule) actually has something to work with. 2) Symbol name mismatch:
  • Even after fixing the empty module, the link still failed: 
Undefined symbols for architecture arm64:
  "_celsius", referenced from: _main in testing_temp.o
ld: symbol(s) not found
Enter fullscreen mode Exit fullscreen mode
  • The linker is looking for _celsius (with an underscore). On macOS, C symbols in object files get a _ prefix automatically, so celsius in the Kaleidoscope source becomes _celsius in output.o. But testing_temp.cpp was declaring it as plain celsius, so the linker couldn't match them.
  • The fix is an asm label that tells the linker the exact symbol name to look for: 
extern "C" {
    double celsius(double fahrenheit) __asm__("_celsius");
}
Enter fullscreen mode Exit fullscreen mode

Running it:

  • test.k 
def celsius(fahrenheit) var result = (fahrenheit - 32.0) * 0.555556 in result
Enter fullscreen mode Exit fullscreen mode
  • testing_temp.cpp 
#include <iostream>

extern "C" {
    double celsius(double fahrenheit) __asm__("_celsius");
}

int main() {
    double f = 68.0;
    double c = celsius(f);

    std::cout << f << " degrees Fahrenheit is " << c << " degrees Celsius!" << std::endl;
    return 0;
}
Enter fullscreen mode Exit fullscreen mode
  • We feed the celsius function through the compiler, link the object file, and run it: 
./toy < test.k
clang++ testing_temp.cpp output.o -o test_celsius
./test_celsius   #68 degrees Fahrenheit is 20 degrees Celsius!
Enter fullscreen mode Exit fullscreen mode

What's next: Debugging!

Musings:
Sometimes, I feel like I have no idea where things are heading. Like I have absolutely zero control over the outcome of things I put effort into. But Indian philosophy and even Greek (I’ve been reading Marcus Aurelius’ Meditations lately) seem to emphasise that that’s exactly the point. That we do our duty and just let go of the rest. Slightly harder than I thought.

In times like these, among the few things that grounds me is looking at the night sky (of all directions). I feel like they’ve been witness to countless tales like these. In a way, knowing that I’m a tiny, tiny, one-millionth of the pale blue dot that is our earth, amidst this vast endlessness that is our universe, helps me feel like the things I think of, may after all, not really be as final as they feel in the moment. And that all will be okay. Those stars remind me of Tennyson’s ‘For men may come and men may go, but I go on forever’ and help me ground myself into, and enjoy the now.

Top comments (0)