Class TokenStream


  • public class TokenStream
    extends java.lang.Object
    • Nested Class Summary

      Nested Classes 
      Modifier and Type Class Description
      (package private) static class  TokenStream.State  
    • Constructor Summary

      Constructors 
      Constructor Description
      TokenStream​(java.io.Reader reader)  
    • Constructor Detail

      • TokenStream

        TokenStream​(java.io.Reader reader)
    • Method Detail

      • tokenize

        java.util.List<Token> tokenize()
      • tokenizeLine

        private void tokenizeLine​(java.util.List<Token> tokenList,
                                  java.lang.String line)
      • isTrailingBackSlash

        boolean isTrailingBackSlash​(java.lang.String line,
                                    int next)
      • isWhiteSpace

        boolean isWhiteSpace​(char c)
      • isNonWhiteSpaceSeparator

        boolean isNonWhiteSpaceSeparator​(char c)