DEV Community: Ashton Scott Snapp

Game Market Survey

Ashton Scott Snapp — Wed, 03 Nov 2021 02:59:01 +0000

So, this isn't directly development related. But, I thought about this site and figured there would probably be some people on here willing to do a survey. This is a survey I did before, before I made a dev.to account, but I've changed it a bit and I think enough has changed from last time that it makes sense to do this again. So, here you go.

Prefer links? Click here.

Writing an Assembler in Rust, and How I'm Reworking the Lexer (again)

Ashton Scott Snapp — Wed, 27 Oct 2021 19:29:52 +0000

Hello people! It's been a bit. Two months or so, to be specific. But I'm back, and I'm still working on this project. Obligatory Github reference:

AshtonSnapp / hasm

The Official Cellia Cross-Assembler for Modern Computers

hasm

The Homebrew Assembler. Currently supporting the 16-bit Cellia architecture and the 8-bit ROCKET88 architecture.

Each architecture supported by hasm will be separated into its own module, although every architecture's assembler code will have the same general structure: you have a lexer which takes in files of assembly code and outputs streams of tokens which are fed into a parser which structures those tokens into a file syntax tree. Then the syntax trees are fed into a linker which tries to combine all of these trees into a single program tree, which is finally fed into a binary generator which does exactly what you think.

Right now I'm still trying to implement the assemblers for the two architectures I mentioned earlier, and I've only just now gotten to the parser. It's going to be a pain to write anything that's actually decently capable, but it'll be worth…

View on GitHub

And now for something you've probably heard before: I'm reworking the lexer. I'm not switching away from the logos crate, don't worry. I just realized that there's a better way to implement some of the token variants and their callbacks, and also I need to figure out how to handle in-assembly operators.

First, let's have an example - addresses. The way addresses used to work in the lexer is that there was an AddressInfo struct that contained the address type, the number base, and the value as a signed 32-bit integer. However, I realized two things: first, we don't need to remember what base the number was in, and second, we can pair the number type with the address type. So relative addresses can use an i16 while absolute addresses can use a u32 (because there's no u24 type).

This has resulted in the creation of the AddressType enum, which has each variant take an integer argument that varies in size depending on the address type. This is what it looks like in the code:

pub enum AddressType {
    Absolute(u32),
    IndirectAbsolute(u32),
    ZeroBank(u16),
    IndirectZeroBank(u16),
    DirectPage(u8),
    Port(u16),
    Relative(i16),
    StackRelative(i16)
}

Next we have the work-in-progress that is the address callback. After learning that logos only returns the text that triggered the callback, the code can be a lot simpler. Also, strip_prefix and strip_suffix are now in Rust Stable. So I don't have to use replacen. Yay!

The callback works quite simply. It takes the token slice and checks it for certain starting or ending characters. A pair of parentheses indicates an indirect address, for example. If it starts with either IP or SP, that's a relative address with the letter before the P indicating what pointer it's relative to. An ending p indicates a port address, and an ending d indicates a direct page address. Then, a $ indicates hexadecimal and a % indicates decimal. Simple.

This code is still majorly a work in progress, but I plan to commit to the GitHub repo once I get it to a certain point. Until then, y'all have an awesome day!

Writing an Assembler in Rust, and How I Got the Build to Pass

Ashton Scott Snapp — Wed, 11 Aug 2021 21:50:41 +0000

Wassup! I got the thing to compile, and I've added more stuff! Also, Github repository link go go go! (edit: apparently it's displaying the wrong GH-Actions badge here, but it displays correctly on github. Caching issue, maybe?)

AshtonSnapp / hasm

The Official Cellia Cross-Assembler for Modern Computers

hasm

The Homebrew Assembler. Currently supporting the 16-bit Cellia architecture and the 8-bit ROCKET88 architecture.

View on GitHub

So, I got the lexer to work. Finally. It compiles. I've done a few things since last post to 1) get the thing to compile, and 2) whittle down on the number of compiler warnings. First off, the error from last time was fixed by actually passing strings into the lexer. I did that by making a function which takes a file and verbose flag and makes a lines iterator over the file, which feeds each line into the lexer. Then it checks for errors. Hopefully someday this assembler will have errors that are as helpful as rustc's, but for now they're just "hey there's an error on this line and it has this byte span. what, you want the character span? Good luck, Unicode is strange." I also had to implement all of the callback functions, and then it finally compiled.

But I still got compiler warnings to deal with. First, some warnings said that certain enum variants were never constructed. Simple fix, just implement (derive) the serde::Serialize and serde::Deserialize traits on everything that isn't a function inside the lex.rs file. I probably did some other stuff to get rid of other warnings, but I'm not perfect and neither is my memory

What I do know is that I have two warnings left to get rid of, and they're confusing. Apparently, certian callback functions are apparently never used, those being instruction_large and instruction_xlarge. Only those two. I have no idea why it's just those two functions, and I have no idea why they aren't being used. It's a mystery.

Well, besides those two remaining warnings, the lexer is finished. Including the two warnings, I've still got some work to do before I can move on to the parser. Nevertheless, I hope all of you have an awesome day!

Writing an Assembler in Rust, and Why the Build Keeps Failing

Ashton Scott Snapp — Wed, 14 Jul 2021 18:27:17 +0000

First off, I feel like the second part of the title is something pretty much every developer has experienced at one point. Second, obligatory GitHub repository plug:

AshtonSnapp / hasm

The Official Cellia Cross-Assembler for Modern Computers

hasm

The Homebrew Assembler. Currently supporting the 16-bit Cellia architecture and the 8-bit ROCKET88 architecture.

View on GitHub

Did a few things since last post. First, I set up a GitHub actions workflow that will build and test the project every time someone commits. Right now, I'm the only person working on this. Sadly. Second, I've implemented a variant for instructions and an enum that contains literally every single instruction in the architecture, all 51 of them, and four different callback functions for determining what instruction a token is.

Then I had to give directives higher priority because logos was getting confused, and I also made roughly too many regexps for addresses and immediates because of how callbacks work, and also because the architecture has 7 different addressing modes and I want to allow decimal, hexadecimal, and binary numbers for both addresses and immediates. 7 addressing modes plus two sizes for immediates, multiplied by 3 bases, means 27 regexps for addresses and immediates alone. I haven't even made the callbacks yet, which all those will do is make sure it's a valid number.

And now, I have an interesting... problem whenever I try to compile the program. This issue only popped up recently, after the previous post, meaning something changed for some god dang reason, and now I'm getting E0277: the size for values of type [u8] cannot be known at compilation time, and the error is pointing out the logos derive bit, followed by a line inside logos::lexer::Lexer. Why is this happening? I have no idea what I did that caused this, but whatever it was happened some time between the last post and this one!

Honestly, at this point, I should probably go over to the logos repository and make an issue because I have no idea what caused this or how to fix it. Well, until next time, have an awesome day. Because I most certainly am not having an awesome day.

Writing an Assembler in Rust, and How I'm Redoing the Lexer

Ashton Scott Snapp — Tue, 13 Jul 2021 02:33:14 +0000

I've continued work on the assembler I've been working on. I finished the Lexer, got it to compile, ran it, tested it, and found that the lexer just did not work. Luckily, I found a crate called logos that helps you make a fast lexer, which I'm using in order to re-do the lexer.

AshtonSnapp / hasm

The Official Cellia Cross-Assembler for Modern Computers

hasm

The Homebrew Assembler. Currently supporting the 16-bit Cellia architecture and the 8-bit ROCKET88 architecture.

View on GitHub

As of right now, I am writing some callback functions. Specifically, I'm writing the one responsible for handling character immediates (immediate in the sense that the processor doesn't have to fetch an address then fetch the value from that address, it just has to fetch the value). This is being done via a giant match statement. Here's basically what the code for this function looks like right now:

fn char(lex: &mut Lexer<Token>) -> Result<u8, ()> {
    let slice: &str = lex.slice();

    let poss_char: &str = &slice[slice.len() - 2];

    // Welcome to hell.

    match poss_char {
        "\x00" => Ok(0),
        "\x01" => Ok(1),
        "\x02" => Ok(2),
        "\x03" => Ok(3),
        "\x04" => Ok(4),
        "\x05" => Ok(5),
        "\x06" => Ok(6),
        "\x07" => Ok(7),
        "\x08" => Ok(8),
        "\x09" => Ok(9),
        "\x0A" => Ok(10),
        "\x0B" => Ok(11),
        "\x0C" => Ok(12),
        "\x0D" => Ok(13),
        "\x0E" => Ok(14),
        "\x0F" => Ok(15),
        "\x10" => Ok(16),
        "\x11" => Ok(17),
        "\x12" => Ok(18),
        "\x13" => Ok(19),
        "\x14" => Ok(20),
        "\x15" => Ok(21),
        "\x16" => Ok(22),
        "\x17" => Ok(23),
        "\x18" => Ok(24),
        "\x19" => Ok(25),
        "\x1A" => Ok(26),
        "\x1B" => Ok(27),
        "\x1C" => Ok(28),
        "\x1D" => Ok(29),
        "\x1E" => Ok(30),
        "\x1F" => Ok(31),
        "\x20" => Ok(32),
        "\x21" => Ok(33),
        "\x22" => Ok(34),
        "\x23" => Ok(35),
        "\x24" => Ok(36),
        "\x25" => Ok(37),
        "\x26" => Ok(38),
        "\x27" => Ok(39),
        "\x28" => Ok(40),
        "\x29" => Ok(41),
        "\x2A" => Ok(42),
        "\x2B" => Ok(43),
        "\x2C" => Ok(44),
        "\x2D" => Ok(45),
        "\x2E" => Ok(46),
        "\x2F" => Ok(47),
        "\x30" => Ok(48),
        "\x31" => Ok(49),
        "\x32" => Ok(50),
        "\x33" => Ok(51),
        "\x34" => Ok(52),
        "\x35" => Ok(53),
        "\x36" => Ok(54),
        "\x37" => Ok(55),
        "\x38" => Ok(56),
        "\x39" => Ok(57),
        "\x3A" => Ok(58),
        "\x3B" => Ok(59),
        "\x3C" => Ok(60),
        "\x3D" => Ok(61),
        "\x3E" => Ok(62),
        "\x3F" => Ok(63),
        "\x40" => Ok(64),
        "\x41" => Ok(65),
        "\x42" => Ok(66),
        "\x43" => Ok(67),
        "\x44" => Ok(68),
        "\x45" => Ok(69),
        "\x46" => Ok(70),
        "\x47" => Ok(71),
        "\x48" => Ok(72),
        "\x49" => Ok(73),
        "\x4A" => Ok(74),
        "\x4B" => Ok(75),
        "\x4C" => Ok(76),
        "\x4D" => Ok(77),
        "\x4E" => Ok(78),
        "\x4F" => Ok(79),
        "\x50" => Ok(80),
        "\x51" => Ok(81),
        "\x52" => Ok(82),
        "\x53" => Ok(83),
        "\x54" => Ok(84),
        "\x55" => Ok(85),
        "\x56" => Ok(86),
        "\x57" => Ok(87),
        "\x58" => Ok(88),
        "\x59" => Ok(89),
        "\x5A" => Ok(90),
        "\x5B" => Ok(91),
        "\x5C" => Ok(92),
        "\x5D" => Ok(93),
        "\x5E" => Ok(94),
        "\x5F" => Ok(95),
        "\x60" => Ok(96),
        "\x61" => Ok(97),
        "\x62" => Ok(98),
        "\x63" => Ok(99),
        "\x64" => Ok(100),
        "\x65" => Ok(101),
        "\x66" => Ok(102),
        "\x67" => Ok(103),
        "\x68" => Ok(104),
        "\x69" => Ok(105),
        "\x6A" => Ok(106),
        "\x6B" => Ok(107),
        "\x6C" => Ok(108),
        "\x6D" => Ok(109),
        "\x6E" => Ok(110),
        "\x6F" => Ok(111),
        "\x70" => Ok(112),
        "\x71" => Ok(113),
        "\x72" => Ok(114),
        "\x73" => Ok(115),
        "\x74" => Ok(116),
        "\x75" => Ok(117),
        "\x76" => Ok(118),
        "\x77" => Ok(119),
        "\x78" => Ok(120),
        "\x79" => Ok(121),
        "\x7A" => Ok(122),
        "\x7B" => Ok(123),
        "\x7C" => Ok(124),
        "\x7D" => Ok(125),
        "\x7E" => Ok(126),
        "\x7F" => Ok(127),
        _ => Err(())
    }
}

Yes. I had to write all of that. Because I can't really guarantee that whoever's using the assembler has Unicode support in their program. That whole function was painful to write. At least now, all I have to write in terms of callback functions are the ones for character escape sequences, strings, addresses, immediates, identifiers (labels and symbols), and actual instruction mnemonics. (Also, need to stop trying to Ctrl+S while using a browser)

Writing an Assembler in Rust, and How I Got Stuck on the Lexer

Ashton Scott Snapp — Sat, 02 Jan 2021 16:07:28 +0000

So, I've been writing an assembler in Rust for an architecture that hasn't been implemented yet (because I made the architecture). You can find it here:

AshtonSnapp / hasm

The Official Cellia Cross-Assembler for Modern Computers

hasm

The Homebrew Assembler. Currently supporting the 16-bit Cellia architecture and the 8-bit ROCKET88 architecture.

View on GitHub

So far I have most of the Lexer complete. The lexer is the first part of a compiler, assembler, interpreter, anything. Your source code whatever goes into the lexer, which then goes through your code line by line and turns it into tokens. These tokens are then passed on to whatever is next, usually the parser, and the lexer's job is done.

Now, so far, the Lexer is pretty much the only thing in the assembler at the moment. It has a token struct, which contains a content string and a TokenInfo enum with immediate, address, identifier, register, operation, directive, and tab variants. The first four variants also have arguments (because that's a thing in Rust) - ImmediateInfo, AddressInfo, IdentifierType, and RegisterType respectively. ImmediateInfo and AddressInfo are structs while IdentifierType and RegisterType are enums. And then it's just enums all the way down.

Besides the actual tokens we have a bunch of helper functions: immediate(), address(), identifier(), and register(), one for each TokenInfo variant with an argument. Operations, directives, and tabs are handled in the main lexer functions. But I won't bore ya with all the details here - I linked the GitHub repository above.

One problem(?) with how I've implemented the lexer is that, while lex() is the function called by main(), lex() calls a function called tokenize_line() for each line of the source file. This function only has the context of the line it is working on, which makes things a bit difficult for things like identifiers. See, when an identifier is being defined there is context in the current line that tells it what kind of identifier it is (either a label or a symbol). This context does not exist when working with an identifier that is being used (i.e. as an argument to a directive other than the symbol definition directive, or as an argument to an operation).

I have been trying to implement a way for the lex() function to fix this, but so far my efforts have been in vain. As an alternative I could give tokenize_line() more context, in the form of a vector of known identifiers. However, that would not be adequate for label identifiers. Symbols have to be defined before they are first used, so this would work for symbols. But labels can be defined anywhere no matter where it is first used, which is why that would be inadequate.

I want to finish up with the lexer and work on something else, but if I can't get this working I can't finish the lexer. Which is why, as I said in the title, I'm stuck on the lexer.

P.S. While looking up stuff about lexers and Rust I came across a crate called Logos that makes really fast lexers. Considering I already have most of the lexer implemented I'm probably not going to refactor my code to work with this crate (because it would be a lot of code to refactor and it would make a lot of time spent working on this lexer meaningless).

Le Edit: Making some progress. When indexing a multi-dimensional array/vector, is the first index for the main array/vector or the sub-array/sub-vector?

Getting into Rust, any tips or recommendations?

Ashton Scott Snapp — Wed, 24 Jun 2020 20:31:00 +0000

Hello. I've been trying to learn Rust for a while but there are some bits I still don't understand. I'm most familiar with Python as it is the first programming language I have learned. Any tips for me as I dive back into Rust?