A TokenStream is a list of tokens, gathered during the parse of some entity (say a method). Entities populate these streams by being registered with the lexer. Any class can collect tokens by including TokenStream. From the outside, you use such an object by calling the start_collecting_tokens method, followed by calls to add_token and pop_token.
Adds tokens to the collected tokens
# File lib/rdoc/token_stream.rb, line 13 def add_tokens(*tokens) tokens.flatten.each { |token| @token_stream << token } end
Starts collecting tokens
# File lib/rdoc/token_stream.rb, line 22 def collect_tokens @token_stream = [] end
Remove the last token from the collected tokens
# File lib/rdoc/token_stream.rb, line 31 def pop_token @token_stream.pop end
Generated with the Darkfish Rdoc Generator 2.