So it looks like I've forgotten about a change that needs to be made. Currently, the code won't figure out how to deal with a command before the string. In addition, if you only want to state to be 1 in a string and 0 outside a string, I would recommend changing it to be a boolean so operations such as switching it can be made easier. Changing to a boolean only requires changing the state = 0 outside the for loop to state = False. However, it requires a few more changes inside the for loop.
state = 0
state = False
In the first if statement, state == 1 can just become state, so the if statement ends up like this:
state == 1
if state and (char == "\"" or char == "'"):
state = False # if we're in a string and see a quote, set the state to not be in a string
continue # continue to the next character
Additionally, the next if statement can just become if state: instead of if state == 1:.
if state == 1:
Now that change I forgot about can be implemented. To know whether to enter a string, only the current character needs to be looked at, not the entire token. Additionally, since the end of strings was already handled in the first if statement, it can be assumed that a seeing a " or ' at this point in the loop means a string is starting. This changes that last if statement to look like this:
if char == "\"" or char == "'":
tok = tok[:-1] # get rid of the " or ' added to tok
state = True # set the state to be in a string
After doing these changes, I get the following output ($ represents the bash prompt):
$ python3 string.py
out "Hello, world!"
One last closing recommendation is that if you really want to get involved in creating programming languages, I would recommend reading the book Language Implementation Patterns by Terence Parr. It's a good book that goes through how to make a lexer and multiple types of parsers. Unfortunately, the example code is written in Java and the book uses the author's parser generator, ANTLR. That being said, it is still a good look into a variety of parsing techniques and I would recommend it.
We're a place where coders share, stay up-to-date and grow their careers.
We strive for transparency and don't collect excess data.