This could be done in a couple different ways. My point was that you can use multiple functions with tail calls rather than one big self.state == self.STATE_TAG switch statement. In this case, the input for loop works like a trampoline, so the stack doesn't build up. It could have been done the same way in Python (where self.state IS the "handle next byte" method). That wouldn't be true for all FSMs, though.

In something that take input one piece at a time, it'd probably still be simplest to use a coroutine. Also, everything here could be made private except new, if it mattered. I tried to make a straightforword translation of the first Python sample. It looks rather like a recursive descent parser.

    local header, footer, dle = '\97', '\98', '\253'
    
    function wait_header(s, byte)
       if byte == header then s.frame = {}; return in_msg end
       return wait_header
    end
    
    function in_msg(s, byte)
       if byte == dle then
          return after_dle
       elseif byte == footer then
          return table.concat(s.frame)
       else
          s.frame[#s.frame+1] = byte    --append to frame buffer
          return in_msg
       end
    end
    
    function after_dle(s, byte)
       s.frame[#s.frame+1] = s.adf(byte)
       return in_msg
    end
    
    function new(after_dle_func)
       local state, func = { adf = after_dle_func, frame = {} }, wait_header
       return function(byte)
                 func = func(state, byte)
                 if type(func) == "string" then return func end
              end
    end
    
    -- test --
    foo = new(function(x) print "(in after_dle_func)"; return x:upper() end)
    
    -- this is like "for /./ in string do ..."
    for b in string.gmatch("foo\97frame contents\253dle\98end", ".") do
       local res = foo(b)
       if res then print("GOT FRAME: ", res); break end
    end
    

    > dofile("/tmp/lua-6043PeX")
    (in after_dle_func)
    GOT FRAME: 	frame contentsDle

Also, Shriram Krishnamurthi's "The Swine Before Perl" (http://www.cs.brown.edu/~sk/Publications/Talks/SwineBeforePe...) also has a good example of using tail calls to write FSMs in Scheme.