DEV Community

Deniz Basgoren
Deniz Basgoren

Posted on

Parsing TCL

#tcl

The following is my take on how commands are parsed in TCL (Tool Command Language). I didn't test these by myself. It's just my interpretation of the following sources:

https://www.ee.columbia.edu/~shane/projects/sensornet/part1.pdf
https://tmml.sourceforge.net/doc/tcl/Tcl.html

The rules

There are a few simple rules.

  • Escaping characters always works as in C: \n \r \t \\ \" \{ \[ \$ \xhh \ooo \uhhhh \Uhhhhhhhh. These have a special meaning " " { } [ ] $.
  • Commands are separated by either ; or newline. Words are separated by space.
  • Stuff like loops, functions, if-else are all commands. They don't have special syntax.
  • The expr command is a bit complicated. I'll explain it later, after the main algorithm.

The algorithm

  • Start from leftmost character and go rightwards to determine the words. Space means word ends and a new word begins.
  • If the first character of a word is " { [ we will go until the matching " } ] and count the whole thing as a word. These can nest. You can think of the first " { [ as +1 value, and " } ] as -1 value. The word ends when we reach 0 and then encounter a space. Examples: 
set len [string length foobar]
Enter fullscreen mode Exit fullscreen mode

Here word1:set word2:len and word3:string length foobar

set len [expr [string length foobar] + $x]
Enter fullscreen mode Exit fullscreen mode

word1:set word2:len word3:expr [string length foobar] + $x

puts stdout "The length of $s is [string length $s]."
Enter fullscreen mode Exit fullscreen mode

word1:puts word2:stdout word3:The length of $s is [string length $s].

Note: [ can also start in the middle of a word.

puts stdout $x+$y=[expr $x + $y]
Enter fullscreen mode Exit fullscreen mode

word1:puts word2:stdout word3:$x+$y=[expr $x + $y]

set concat $a$b$c
Enter fullscreen mode Exit fullscreen mode

word1:set word2:concat word3:$a$b$c

One interesting case where " doesn't end a word because there's a [ that needs to be closed first:

puts "results [format "%f %f" $x $y]"
Enter fullscreen mode Exit fullscreen mode

word1:puts word2:results [format "%f %f" $x $y]

One interesting case where " doesn't start a group because it's not the first character of a word. Same applies to { too but not [. [ can start in the middle of a word as well.

set silly a"b
Enter fullscreen mode Exit fullscreen mode

word1:set word2:silly word3:a"b

  • Ok, now that we have our words, now we go one word after the other and perform substitutions depending on what kind of a word it is.
  • For plain words and " " words we perform variable substitution:

-- $name becomes the value of the variable name
-- \$name becomes the string $name
-- ${name} becomes the value of the variable name, but in this case variable can be made up of pretty much any letter. So ${.o} becomes the value of the variable .o
-- $name(index) becomes the value at index index of array name. Not sure, but extra processing seems to be done on index too.

  • For { } words variable substitution is not done. Content is taken verbatim.
  • For plain words and " " words after $ substitution every [ ] part is processed separately as a new TCL-process and it returns a string. This string is then pasted in place of [ ].

Examples:

set len [string length foobar]
Enter fullscreen mode Exit fullscreen mode

word1:set word2:len word3:string length foobar
word3 is a []word so parse it in a new shell:
word1:string word2:length word3:foobar
This command returns 6 when executed, so we now have
word1:set word2:len word3:6

set x 7
set len [expr [string length foobar] + $x]
Enter fullscreen mode Exit fullscreen mode

After the inner command we have

set len [expr 6 + $x]
Enter fullscreen mode Exit fullscreen mode

And len = 6+7 = 13

puts stdout "The length of $s is [string length $s]."
# Result: The length of Hello is 5.
puts stdout {The length of $s is [string length $s].}
# Result: The length of $s is [string length $s].
Enter fullscreen mode Exit fullscreen mode

The quotes (") are redundant in the following:

puts stdout "[expr $x + $y]"
Enter fullscreen mode Exit fullscreen mode 
proc Diag {a b} {
  expr sqrt($a * $a + $b * $b)
}
Enter fullscreen mode Exit fullscreen mode

Due to {} the contents of the proc are not parsed but passed to proc as is. proc command then uses eval to parse them.

while {$i <= $x}
Enter fullscreen mode Exit fullscreen mode

In while blocks make sure to use { } so that the contents are not parsed, but sent to while command as is. while command then uses eval to check the condition on each iteration.

set x xvalue
set y "foo {$x} bar"
# Result: foo {xvalue} bar
Enter fullscreen mode Exit fullscreen mode

Here word3:foo {$x} bar, and it's a " " word. So we do $ substitution and treat { } as normal characters.

set x [cmd1][cmd2]
Enter fullscreen mode Exit fullscreen mode

Here word3 is the concatenation of the results of cmd1 and cmd2.

set x $
Enter fullscreen mode Exit fullscreen mode

A lone $ character is interpreted as a regular $ character.

set y [set x 0][incr x][incr x]
Enter fullscreen mode Exit fullscreen mode

Prints 012

set x 6
set y 7
puts [format "The answer: %d" [expr {$x * $y}]]
Enter fullscreen mode Exit fullscreen mode

Doesn't look scary anymore after knowing the rules above.

Expr

This command can take any number of arguments. It first concatenates them all, then evaluates the entire thing in math context. It's better to put the entire expr in a { } or " " and pass it as a single argument, instead of relying on expr's variadic nature. This makes sure we don't get rounding errors while translating back and forth between strings and floats.

Top comments (0)