Package antlr
Class CharScanner
- java.lang.Object
-
- antlr.CharScanner
-
- All Implemented Interfaces:
TokenStream
- Direct Known Subclasses:
ActionLexer
,ActionLexer
,ActionLexer
,ActionLexer
,ANTLRLexer
,ANTLRTokdefLexer
,CodeLexer
,DebuggingCharScanner
,PreprocessorLexer
public abstract class CharScanner extends java.lang.Object implements TokenStream
-
-
Field Summary
Fields Modifier and Type Field Description protected Token
_returnToken
protected boolean
caseSensitive
protected boolean
caseSensitiveLiterals
protected boolean
commitToPath
Used during filter mode to indicate that path is desired.static char
EOF_CHAR
protected ANTLRHashString
hashString
protected LexerSharedInputState
inputState
protected java.util.Hashtable
literals
(package private) static char
NO_CHAR
protected boolean
saveConsumedInput
protected int
tabsize
Tab chars are handled by tab() according to this value; override method to do anything weird with tabs.protected ANTLRStringBuffer
text
protected java.lang.Class
tokenObjectClass
protected int
traceDepth
Used to keep track of indentdepth for traceIn/Out
-
Constructor Summary
Constructors Constructor Description CharScanner()
CharScanner(InputBuffer cb)
CharScanner(LexerSharedInputState sharedState)
-
Method Summary
All Methods Instance Methods Concrete Methods Modifier and Type Method Description void
append(char c)
void
append(java.lang.String s)
void
commit()
void
consume()
void
consumeUntil(int c)
Consume chars until one matches the given charvoid
consumeUntil(BitSet set)
Consume chars until one matches the given setboolean
getCaseSensitive()
boolean
getCaseSensitiveLiterals()
int
getColumn()
boolean
getCommitToPath()
java.lang.String
getFilename()
InputBuffer
getInputBuffer()
LexerSharedInputState
getInputState()
int
getLine()
int
getTabSize()
java.lang.String
getText()
return a copy of the current text bufferToken
getTokenObject()
char
LA(int i)
protected Token
makeToken(int t)
int
mark()
void
match(char c)
void
match(BitSet b)
void
match(java.lang.String s)
void
matchNot(char c)
void
matchRange(char c1, char c2)
void
newline()
void
panic()
void
panic(java.lang.String s)
This method is executed by ANTLR internally when it detected an illegal state that cannot be recovered from.void
reportError(RecognitionException ex)
Parser error-reporting function can be overridden in subclassvoid
reportError(java.lang.String s)
Parser error-reporting function can be overridden in subclassvoid
reportWarning(java.lang.String s)
Parser warning-reporting function can be overridden in subclassvoid
resetText()
void
rewind(int pos)
void
setCaseSensitive(boolean t)
void
setColumn(int c)
void
setCommitToPath(boolean commit)
void
setFilename(java.lang.String f)
void
setInputState(LexerSharedInputState state)
void
setLine(int line)
void
setTabSize(int size)
void
setText(java.lang.String s)
void
setTokenObjectClass(java.lang.String cl)
void
tab()
advance the current column number by an appropriate amount according to tab size.int
testLiteralsTable(int ttype)
int
testLiteralsTable(java.lang.String text, int ttype)
Test the text passed in against the literals table Override this method to perform a different literals test This is used primarily when you want to test a portion of a token.char
toLower(char c)
void
traceIn(java.lang.String rname)
void
traceIndent()
void
traceOut(java.lang.String rname)
void
uponEOF()
This method is called by YourLexer.nextToken() when the lexer has hit EOF condition.-
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
-
Methods inherited from interface antlr.TokenStream
nextToken
-
-
-
-
Field Detail
-
NO_CHAR
static final char NO_CHAR
- See Also:
- Constant Field Values
-
EOF_CHAR
public static final char EOF_CHAR
- See Also:
- Constant Field Values
-
text
protected ANTLRStringBuffer text
-
saveConsumedInput
protected boolean saveConsumedInput
-
tokenObjectClass
protected java.lang.Class tokenObjectClass
-
caseSensitive
protected boolean caseSensitive
-
caseSensitiveLiterals
protected boolean caseSensitiveLiterals
-
literals
protected java.util.Hashtable literals
-
tabsize
protected int tabsize
Tab chars are handled by tab() according to this value; override method to do anything weird with tabs.
-
_returnToken
protected Token _returnToken
-
hashString
protected ANTLRHashString hashString
-
inputState
protected LexerSharedInputState inputState
-
commitToPath
protected boolean commitToPath
Used during filter mode to indicate that path is desired. A subsequent scan error will report an error as usual if acceptPath=true;
-
traceDepth
protected int traceDepth
Used to keep track of indentdepth for traceIn/Out
-
-
Constructor Detail
-
CharScanner
public CharScanner()
-
CharScanner
public CharScanner(InputBuffer cb)
-
CharScanner
public CharScanner(LexerSharedInputState sharedState)
-
-
Method Detail
-
append
public void append(char c)
-
append
public void append(java.lang.String s)
-
commit
public void commit()
-
consume
public void consume() throws CharStreamException
- Throws:
CharStreamException
-
consumeUntil
public void consumeUntil(int c) throws CharStreamException
Consume chars until one matches the given char- Throws:
CharStreamException
-
consumeUntil
public void consumeUntil(BitSet set) throws CharStreamException
Consume chars until one matches the given set- Throws:
CharStreamException
-
getCaseSensitive
public boolean getCaseSensitive()
-
getCaseSensitiveLiterals
public final boolean getCaseSensitiveLiterals()
-
getColumn
public int getColumn()
-
setColumn
public void setColumn(int c)
-
getCommitToPath
public boolean getCommitToPath()
-
getFilename
public java.lang.String getFilename()
-
getInputBuffer
public InputBuffer getInputBuffer()
-
getInputState
public LexerSharedInputState getInputState()
-
setInputState
public void setInputState(LexerSharedInputState state)
-
getLine
public int getLine()
-
getText
public java.lang.String getText()
return a copy of the current text buffer
-
getTokenObject
public Token getTokenObject()
-
LA
public char LA(int i) throws CharStreamException
- Throws:
CharStreamException
-
makeToken
protected Token makeToken(int t)
-
mark
public int mark()
-
match
public void match(char c) throws MismatchedCharException, CharStreamException
-
match
public void match(BitSet b) throws MismatchedCharException, CharStreamException
-
match
public void match(java.lang.String s) throws MismatchedCharException, CharStreamException
-
matchNot
public void matchNot(char c) throws MismatchedCharException, CharStreamException
-
matchRange
public void matchRange(char c1, char c2) throws MismatchedCharException, CharStreamException
-
newline
public void newline()
-
tab
public void tab()
advance the current column number by an appropriate amount according to tab size. This method is called from consume().
-
setTabSize
public void setTabSize(int size)
-
getTabSize
public int getTabSize()
-
panic
public void panic()
- See Also:
panic(String)
-
panic
public void panic(java.lang.String s)
This method is executed by ANTLR internally when it detected an illegal state that cannot be recovered from. The default implementation of this method calls {@link java.lang.System.exit(int)} and writes directly to {@link java.lang.System.err)} , which is usually not appropriate when a translator is embedded into a larger application. It is highly recommended that this method be overridden to handle the error in a way appropriate for your application (e.g. throw an unchecked exception).
-
reportError
public void reportError(RecognitionException ex)
Parser error-reporting function can be overridden in subclass
-
reportError
public void reportError(java.lang.String s)
Parser error-reporting function can be overridden in subclass
-
reportWarning
public void reportWarning(java.lang.String s)
Parser warning-reporting function can be overridden in subclass
-
resetText
public void resetText()
-
rewind
public void rewind(int pos)
-
setCaseSensitive
public void setCaseSensitive(boolean t)
-
setCommitToPath
public void setCommitToPath(boolean commit)
-
setFilename
public void setFilename(java.lang.String f)
-
setLine
public void setLine(int line)
-
setText
public void setText(java.lang.String s)
-
setTokenObjectClass
public void setTokenObjectClass(java.lang.String cl)
-
testLiteralsTable
public int testLiteralsTable(int ttype)
-
testLiteralsTable
public int testLiteralsTable(java.lang.String text, int ttype)
Test the text passed in against the literals table Override this method to perform a different literals test This is used primarily when you want to test a portion of a token.
-
toLower
public char toLower(char c)
-
traceIndent
public void traceIndent()
-
traceIn
public void traceIn(java.lang.String rname) throws CharStreamException
- Throws:
CharStreamException
-
traceOut
public void traceOut(java.lang.String rname) throws CharStreamException
- Throws:
CharStreamException
-
uponEOF
public void uponEOF() throws TokenStreamException, CharStreamException
This method is called by YourLexer.nextToken() when the lexer has hit EOF condition. EOF is NOT a character. This method is not called if EOF is reached during syntactic predicate evaluation or during evaluation of normal lexical rules, which presumably would be an IOException. This traps the "normal" EOF condition. uponEOF() is called after the complete evaluation of the previous token and only if your parser asks for another token beyond that last non-EOF token. You might want to throw token or char stream exceptions like: "Heh, premature eof" or a retry stream exception ("I found the end of this file, go back to referencing file").
-
-