DEV Community: Matias

Rusty Boy - Week 1: Of Timing and Modes

Matias — Mon, 22 Jun 2020 03:37:35 +0000

Hello there!

It's been a week since my last report and, as promised, I'm back here with the news of the rusty lad. It's been an interesting week. Let's jump into it.

Monday

I started the week by looking into Dr. Mario and why it wouldn't boot. I haven't seen this one getting to the menu on Rusty-Boy since forever, so it was about time I took a look.
So I set a breakpoint after the Bootrom's execution and started stepping. I use BGB as my control emulator, since it's pretty accurate and it has a great debugger.

It's about Time

After a while stepping, interrupts got enabled. This is where debugging starts to get tricky. My timing isn't great, especially when compared to BGB's, so I start triggering interrupts often while my control emulator just keeps running fine. My solution to this was to just set a breakpoint on the return address, and let the game run the interrupt routine. Doing this showed an interesting thing: the game would very frequently enter the Timer interrupt, and it'd spend a good chunk of time on it. Eventually, I hit Resume after entering an interrupt and the game never returned from it. That's where I decided to check something - what would happen if I disabled the Timer interrupt? Well, the game got to the menu just fine!

Now that's something I haven't seen in a while!

But good things don't last for long, and a second or two later, the game goes into a white screen and never recovers. It enters a loop where it waits for a certain display mode switch to happen, but the LCD is disabled, so it won't happen (as far as I know at least). It's not the first time I've seen this issue, which makes it a bit more interesting, but I poked around for a while and couldn't find an issue, so I moved on for the time being.

The Rogue Window - Part 1

To finish the day, I took a look at the Window rendering.
As I briefly mentioned last week, the window likes to render in the middle of the background in a few games. Apparently just pushing the window out of the screen was simpler than just, you know, disabling it. I'm not sure I really get why, but ok.

I started looking at the most obvious thing - checking if the coordinates were actually correct. My test subject for this was Bomberman. It uses the window in fixed locations during its intro sequences, and it actually showed the issue perfectly.

The Window clearly is not supposed to be there!

So I loaded the game on both Rusty-Boy and BGB, paused it on the roughly the same spot, and compared both emu's window coordinates. And yeah, the values were indeed different! Progress, right? Nope. A few minutes later I remembered a kind of important detail - Rusty Boy shows coordinates in decimal values, while BGB shows all values in hexadecimal. So I changed the debugger's formatting so it'd show hex values, and now the values are the same. So much for that theory then, back to the drawing board.

Tuesday

This one is barely worth mentioning, since I spent most of the day trying to fix networking issues around the house. Still managed to work on the emu a bit.
I messed around with the Window's position a bit and came to the conclussion that the issue was definitely on the blitting stage. Blitting is basically the process of merging textures together (the very simplified explanation I work with). Since this is basically the most interesting thing that happened this day, I might as well explain a bit how I do rendering on Rusty Boy.

Rusty Rendering 101

Rendering in Rusty Boy has been quite the rollercoaster, but currently it's done mostly using Textures.

The Background and the Window both have their own Textures, which are drawn line by line and then sent to the UI thread for displaying when the Vblank period is about to end.
Sprites and Tiles are cached. This cache is invalidated based on a hash of their memory sections. If it changes, then the cache gets nuked and regenerated. Tiles have 2 banks in memory, so they get also get one cache each.

The rendering process begins on the emulation thread, on the Video chip side of things. It first checks whether or not it should regenerate caches, and acts accordingly. Then, if it's on the correct mode, it draws a line of the background and the window. The lines are drawn using the tiles in the cache, and which tile to use is indicated by the data in the background maps - these contain a value pointing to each tile that should be drawn on screen. For the background, the lines usually have an offset - the scrolling values. This allows games to move the background horizontally or vertically, but it can also be changed when reaching a specific line using interrupt to generate some interesting effects. The results are then saved as an array of color data, and sent to the UI thread.

The UI thread will receive an object that contains 2 backgrounds (one scrolled for the screen, and a pure one), the window, and all the sprites. The actual textures are then generated and upscaled to the target resolution, so they can then be blitted into a final one which is showed on the Screen window. This is done for the background, the window (if it's enabled), and all sprites (if they should be displayed). The unscrolled Background and Window textures are also put into the Video Debugger - it can be useful to debug graphical issues (like the window being on top of the background when it shouldn't).

The background and the window, as shown on the Video Debugger

I think that basically sums up the current rendering process. Let's move on.

Wednesday

Oh boy, Wednesday was quite a ride.
I started the day by polishing code a bit here and there. I started with the Timer, since I haven't touched that in centuries. Nothing was fixed, but it looks better now at least. Then I moved to the Video file... Dear God why did I do that.

The Failed Cleanup Attempt

My goal was basically to clean up interrupt requests and mode switching. This involves mostly 2 registers: LCD Status and Interrupts Flags.

This LCD Status (also known as LCD STAT or just STAT) stores the current mode of the Video chip, and when it should cause interrupts. The Interrupt flags one is the register that stores the currently requested interrupts. If, say, the timer wants to request an interrupt, that register is the place to go.

Now, when you change mode, you have to modify the first 2 bits of LCD STAT (should store 0-3 depending on the mode), and depending on other bits on the register, you may also have to request an interrupt. This whole process was a bit convoluted on Rusty-Boy, so I decided to get some fresh air and try to clean it up, maybe even fix something in the process, right? Yeah, no.

The cleanups broke everything

Games ranged from not running, to showing broken graphics, and even executing invalid opcodes! How?! I did have some typos on the initial re-implementation of some things, but even after they were fixed games were still broken.

I took the easy way out. I ran git reset --hard and kicked the cleanup to some random point in the future. I didn't want to deal with it for the time being. So I moved on to something else, which brings us to our next topic of the day.

The Input Hack

I mentioned on the previous post that I had an input problem.

Basically, button presses are sent from the UI thread to the emulation thread for the CPU to handle. The issue is that both threads run at very different speeds, so inputs tend to get lost in translation. If your ROM only checks for button presses (like an input test for example), it tends to register an input press if you try hard enough. But games do other things, not just constantly check for inputs, so you need to really try to hopefully get one press registered. It's a really ugly situation.

Tl;Dr: You press a button, and RNGesus decides whether or not it'll get through.

As you can probably imagine, this makes testing a bit difficult, since I can't get past menus in most cases. So I decided to implement a workaround (or hack) after I come up with a better fix (famous last words).
This hack basically sends 4 input events per button press. Why 4 specifically? No idea, 4 seemed like a nice number in my mind. So with this hack in place, you can actually start playing stuff. Mostly Tetris, since it's the most reliable game so far. rex-run also works pretty nicely, it's a port of the dino game from Chrome, you know the one you get when you are offline? That one. It's a fun game that I have way more playtime than I'd like to admit.

It's a bit hard to play at super speed

Let's continue.

Thursday

This was "Pokemon Red Debugging" day, which of course turned into "Hit your head against a Wall" day. Nothing is never simple here.

Investigating Pokemon Red

Ever since I moved away from SDL for rendering and into OpenGL+Dear Imgui, Pokemon Red just loops forever after the bootrom.
My first instinct was trying to look into the commit to see if anything seemed bad, but this didn't go quite as planned. The commit is kinda big, which made searching for differences a bit hard. I tried, but I couldn't spot any major differences that'd cause the game to get stuck like that. What was going on?. Well, time to get BGB running again then.

After stepping through instructions for a while, I got to a point where Rusty-Boy would acknowledge an interrupt, but never come back from it. A Vblank interrupt to be more precise. This didn't really answer many questions, but rather brought a few more to the table - it's not the first time it calls a vblank interrupt, why does it get stuck on that one specifically? I poked around some things a bit more, but couldn't find a reason so I just wrapped the day there and went to bed.

I don't really have a solid theory for that one, there might be some timing-related issues with interrupts, but I haven't stepped into the vblank routine yet. I'll probably take a look at a disassembly of the game first to have a better idea of what I'm looking at.

Friday

This day started a bit rough with me trying, again, to clean up mode switching and interrupts. As expected, it didn't go any better than the previous attempt, and ended up with the same solution. git reset --hard. Maybe next week.
So I moved on to something that I had some confidence on, Sprites.

Sprites

Sprites are pretty simple overall, they are basically tiles with flags.
Each Sprite has an entry on a special region in memory, the OAM (Object Attribute Memory). The entries are made from 4 bytes:

Y position
X position
Tile ID
Flags (Priority, Flip on X or Y, palette to use)

There can only be 40 sprites on memory at the same time.

With the tile cache in place, the implementation was pretty painless. I added a Sprite struct that contained the screen coordinates, color data, and the flip flags for each sprite. And I cached them, same idea as the tile cache - have a hash of the memory region, regenerate the cache when the hash changes. Easy, right? Yeah, but not typo-proof.

The first Sprite that got rendered

In case you missed it, it's a zero on the top left corner of the screen. Just that.
It didn't take me very long to notice what was wrong. I copy-pasted the code for hashing the tile banks and used it for sprites. But I never changed the memory region to hash! So if the tile cache wasn't invalidated, the sprites wouldn't either. Fixed that and...

It's a little confused, but it's got the spirit.

Wasn't looking too great, but it was getting there. This one took a bit longer to figure out.
The culprit for this one was the scaling. I was multiplying the size of the sprites by the scaling factor, but not the position. The X and Y coordinates were assuming, of course, a 160x144 screen - the Gameboy's screen. But after applying scaling, that's not the resolution we have. So after taking that into account, we have perfectly working sprites. Mostly.

Evolution of Sprites in Tetris

Looking pretty good so far, but I'm still missing some things. Most notably 8x16 sprites, priority, and transparency.
8x16 sprites are probably going to happen soon, the others I have to read up a bit on.

The Rogue Window - Part 2

A few hours after fixing the sprite's position, I got an idea, a spark of hope: what if all this time I just had to scale the window coordinates, just like the sprites? Just as I finished that thought my toaster yeeted my toasts, really dramatic. So I rushed to my PC to test it, scared that the moment of enlightenment would go away, and indeed, that was all that was needed. The window was no longer over things it shouldn't.

Bomberman's intro was finally fixed!

With the last window-related issue, I can probably say with a certain degree of certainty that the window is currently rendering perfectly.

So, you are all caught up now. This is where Rusty-Boy is right now. This was a bit of a frustrating week, with a lot of things not really going anywhere, but Sprites and fixing the window saved it really.

I'll probably try to figure out what's up with Dr. Mario's loop and Pokemon Red's interrupt. Hopefully it gets somewhere.

See ya in a week!

PS: I never linked the emu itself last week, so you can go check it out here.

Rusty Boy - Week 0: Where are we?

Matias — Mon, 15 Jun 2020 07:35:34 +0000

Rusty Boy, what is that?

Great question! Rusty Boy (also known as rusty-boi and rusty-boy, depending on the day), is the name of my Gameboy emulator, which I'm writing in Rust. I know, a very creative name.

A bit of History

I started this project back on August 2019, almost a year ago.
I've always been interested on emulation, it's a pretty fascinating thing, but I've never coded anything on that area. So having a lot of free time in my hands, I started with the classic CHIP8 interpreter. It was written in C#, and I got it running on a day (give or take).

Around this time I also discovered Rust, which would later become my go-to language to do stuff. But let's not get ahead of ourselves here.

I wanted to learn Rust, so I did what any reasonable human would do: I started reading The Book. A few chapters later and I did, again, what any reasonable human would do: I started writing an emulator on a language I barely knew.

Here's where a big question showed up: what platform will I emulate?. I had a few candidates - the NES, Sega Megadrive, another CHIP8 interpreter. I started with the NES, but it didn't really get anywhere, but the rusty-* naming was already there. This is when I landed on the Gameboy. This is where the juicy part of our story begins.

A Gameboy Emulator

Seemed simple enough, right? A handheld console from 1989, can't be so hard. Oh Lord, I didn't know where I was getting myself into. In retrospective, this project is probably not the best "Hello, World!" kind of project to start learning a language, but it's been a fun ride so far.

A very early build of what would become Rusty-Boy

The Gameboy is a pretty interesting console. The good thing about emulating something this old is that a lot of things are very well tested and documented. Not all of them, but you get a head start at least. You can get creative later.

So, how's Rusty-Boy running? Another great question right there!

The Tests

The go-to tests when developing an GB emulator are blargg's CPU tests. This test suite checks that your emulator executes most instructions properly, and that their flags are also correct.

On this area, Rusty-Boy is currently doing pretty well with just 4 tests failing out of 11.

Not perfect, but getting there

Running the individual tests I can see what instructions are currently failing the tests, and they range from expected (DAA on test 1), to not quite (ADC and SBC on tests 4 and 9). I'll get them fixed eventually.

Gekkio's test for mooneye-gb are also a very comprehensive test suite for different areas of the emulator - timing of instructions, unused bits of memory registers, DMA transfers, memory banks for cartridges, and a lot of others. While I haven't ran all of them yet, I can pass some of them. I'll probably fail most of the other though, since timings are not my biggest priority right now.

One of Gekkio's tests passing

More recently, I even wrote a simple test myself! I was having some issues with CPU interrupts, so I wrote a test on GB Assembly, which I checked with BGB (a very accurate emulator with a great debugger) and the various docs I've been using.

My interrupts test

While the test itself is pretty simple, it helped me pinpoint some issues with my interrupt handling.

The Games

It's a Gameboy. Part of it is running games, right? Well, yeah, but not quite here. Let me explain.

Rusty-Boy has been though a lot of rewrites. The latest rewrite resulted on a huge speedboost. While I'm very happy of how it runs, it's also a problem. Games tend to run very fast, which would render them unplayable, or at least very hard to play. That is of course, if I actually managed to get any inputs through - games mostly ignore inputs.

My best theory for this issue right now is that since it runs so fast, inputs get written to the register, but the window for them to actually get read by the game is pretty small. Limiting speed should eventually solve the issue (I hope). Games still run though, to different degrees of success.

The next biggest issue for games would be the absence of sprites. They were implemented a while back, but got temporally axed on the last rewrite (the same that brought the speed issues). So a considerable amount of graphics are missing.

Tetris:

A timeless classic. Runs pretty well actually, or so it'd seem at least. Falling pieces are missing because of sprites not being implemented.

Tetris' Menu working perfectly
Tetris ingame - Sprites are missing

The Legend of Zelda: Link's Awakenings:

The intro sequence runs pretty well, but I can't say the same for the menu. Can't progress any further because of broken inputs.

The intro sequence is mostly fine
But the menu is pretty broken

Deadeus

A recently released horror game. Currently boots fine to the menu, but the menu's background gets partially covered by the Gameboy's window.

The black window covers most of the background

What else can it do?

Well, Rusty-Boy has some nice features I've been recently adding.
This is the emulator's window with all its components enabled:

Rusty-Boy's Main Window

I've been adding debugging capabilities into the emulator itself, which includes viewing CPU Registers, settings breakpoints, viewing video registers, the background and the window before getting merged into a final frame, interrupts, and the memory disassembler.
While some things might be a bit close to the feature creep area cough disassembler cough, it's pretty nice to have all that data in one window.

Closing Up

Well, this has been my crash course for you into Rusty-Boy, featuring a trip down memory lane, and the current state of yet another Gameboy emulator.
Hope it was a good read and that you'll come back, since I plan to write a new entry each week with new changes and improvements to the emulator. There's still a lot to do, and I still want to work on it, so we'll keep moving forward. But now with cool looking reports.

See you in a week?