org.antlr.runtime.Lexer

	Property	Defined by
		backtrackingLevel : int	BaseRecognizer
		charIndex : int [read-only] What is the index of the current character of lookahead?	Lexer
		charPositionInLine : int [read-only]	Lexer
		charStream : CharStream	Lexer
		grammarFileName : String For debugging and other purposes, might want the grammar name.	BaseRecognizer
		line : int [read-only]	Lexer
		numberOfSyntaxErrors : int Get number of recognition errors (lexer, parser, tree parser).	BaseRecognizer
		sourceName : String [read-only]	Lexer
		state : RecognizerSharedState State of a lexer, parser, or tree parser are collected into a state object so the state can be shared.	BaseRecognizer
		text : String Return the text matched so far for the current token or any text override.	Lexer
		tokenNames : Array Used to print out token names like ID during debugging and error reporting.	BaseRecognizer

	Property	Defined by
		input : CharStream Where is the lexer drawing characters from?	Lexer

Public Methods

	Method		Defined by
		Lexer(input:CharStream = null, state:RecognizerSharedState = null)	Lexer
		alreadyParsedRule(input:IntStream, ruleIndex:int):Boolean Has this rule already parsed input at the current index in the input stream? Return the stop token index or MEMO_RULE_UNKNOWN.	BaseRecognizer
		beginResync():void A hook to listen in on the token consumption during error recovery.	BaseRecognizer
		consumeUntil(input:IntStream, bitSet:BitSet):void Consume tokens until one matches the given token set	BaseRecognizer
		consumeUntilToken(input:IntStream, tokenType:int):void	BaseRecognizer
		displayRecognitionError(tokenNames:Array, e:RecognitionException):void	BaseRecognizer
		emit():Token The standard method called to automatically emit a token at the outermost lexical rule.	Lexer
		emitErrorMessage(msg:String):void Override this method to change where error messages go	BaseRecognizer
		emitToken(token:Token):void Currently does not support multiple emits per nextToken invocation for efficiency reasons.	Lexer
		endResync():void	BaseRecognizer
		getCharErrorDisplay(c:int):String	Lexer
		getErrorHeader(e:RecognitionException):String What is the error header, normally line/character position information?	BaseRecognizer
		getErrorMessage(e:RecognitionException, tokenNames:Array):String	Lexer
		getRuleMemoization(ruleIndex:int, ruleStartIndex:int):int Given a rule number and a start token index number, return MEMO_RULE_UNKNOWN if the rule has not parsed input starting from start index.	BaseRecognizer
		getRuleMemoizationCacheSize():int return how many rule/input-index pairs there are in total.	BaseRecognizer
		getTokenErrorDisplay(t:Token):String How should a token be displayed in an error message? The default is to display just the text, but during development you might want to have a lot of information spit out.	BaseRecognizer
		match(c:int):void	Lexer
		matchAny():void	Lexer
		matchAnyStream(input:IntStream):void Match the wildcard: in a symbol	BaseRecognizer
		matchRange(a:int, b:int):void	Lexer
		matchStream(input:IntStream, ttype:int, follow:BitSet):Object Match current input symbol against ttype.	BaseRecognizer
		matchString(s:String):void	Lexer
		memoize(input:IntStream, ruleIndex:int, ruleStartIndex:int):void Record whether or not this rule parsed the input at this position successfully.	BaseRecognizer
		mismatchIsMissingToken(input:IntStream, follow:BitSet):Boolean	BaseRecognizer
		mismatchIsUnwantedToken(input:IntStream, ttype:int):Boolean	BaseRecognizer
		mTokens():void This is the lexer entry point that sets instance var 'token'	Lexer
		nextToken():Token Return a token from this source; i.e., match a token on the char stream.	Lexer
		recover(re:RecognitionException):RecognitionException Lexers can normally match any char in it's vocabulary after matching a token, so do the easy thing and just kill a character and hope it all works out.	Lexer
		recoverFromMismatchedSet(input:IntStream, e:RecognitionException, follow:BitSet):Object Not currently used	BaseRecognizer
		recoverFromMismatchedToken(input:IntStream, ttype:int, follow:BitSet):Object Attempt to recover from a single missing or extra token.	BaseRecognizer
		recoverStream(input:IntStream, re:RecognitionException):void Recover from an error found on the input stream.	BaseRecognizer
		reportError(e:RecognitionException):void	Lexer
		reset():void	Lexer
		skip():void Instruct the lexer to skip creating a token for current lexer rule and look for another token.	Lexer
		toStrings(tokens:Array):Array A convenience method for use most often with template rewrites.	BaseRecognizer
		traceIn(ruleName:String, ruleIndex:int):void	Lexer
		traceInSymbol(ruleName:String, ruleIndex:int, inputSymbol:Object):void	BaseRecognizer
		traceOut(ruleName:String, ruleIndex:int):void	Lexer
		traceOutSymbol(ruleName:String, ruleIndex:int, inputSymbol:Object):void	BaseRecognizer

	Method	Defined by
		combineFollows(exact:Boolean):BitSet	BaseRecognizer
		computeContextSensitiveRuleFOLLOW():BitSet Compute the context-sensitive FOLLOW set for current rule.	BaseRecognizer
		computeErrorRecoverySet():BitSet	BaseRecognizer
		getCurrentInputSymbol(input:IntStream):Object Match needs to return the current input symbol, which gets put into the label for the associated token ref; e.g., x=ID.	BaseRecognizer
		getMissingSymbol(input:IntStream, e:RecognitionException, expectedTokenType:int, follow:BitSet):Object Conjure up a missing token during error recovery.	BaseRecognizer
		mismatch(input:IntStream, ttype:int, follow:BitSet):void Factor out what to do upon token mismatch so tree parsers can behave differently.	BaseRecognizer
		pushFollow(fset:BitSet):void Push a rule's follow set using our own hardcoded stack	BaseRecognizer

	Constant	Defined by
		DEFAULT_TOKEN_CHANNEL : int = 0 [static]	BaseRecognizer
		HIDDEN : int = 99 [static]	BaseRecognizer
		INITIAL_FOLLOW_STACK_SIZE : int = 100 [static]	BaseRecognizer
		MEMO_RULE_FAILED : int = -2 [static]	BaseRecognizer
		MEMO_RULE_UNKNOWN : int = -1 [static]	BaseRecognizer
		NEXT_TOKEN_RULE_NAME : String = "nextToken" [static]	BaseRecognizer

Property detail

charIndex

property

charIndex:int [read-only]

What is the index of the current character of lookahead?

Implementation
public function get charIndex():int

charPositionInLine

property

charPositionInLine:int [read-only]Implementation
public function get charPositionInLine():int

charStream

property

charStream:CharStream  [read-write]Implementation
    public function get charStream():CharStream
    public function set charStream(value:CharStream):void

input

property

protected var input:CharStream

Where is the lexer drawing characters from?

line

property

line:int [read-only]Implementation
public function get line():int

sourceName

property

sourceName:String [read-only]Implementation
public function get sourceName():String

text

property

text:String [read-write]

Return the text matched so far for the current token or any text override.

Implementation
public function get text():String
public function set text(value:String):void

Constructor detail

Lexer

()

constructor

public function Lexer(input:CharStream = null, state:RecognizerSharedState = null)Parameters

	`input:CharStream` (default = `null`)

	`state:RecognizerSharedState` (default = `null`)

Method detail

emit

()

method

public function emit():Token

The standard method called to automatically emit a token at the outermost lexical rule. The token object should point into the char buffer start..stop. If there is a text override in 'text', use that to set the token's text. Override this method to emit custom Token objects.

Returns

Token

emitToken

()

method

public function emitToken(token:Token):void

Currently does not support multiple emits per nextToken invocation for efficiency reasons. Subclass and override this method and nextToken (to push tokens into a list and pull from that list rather than a single variable as this implementation does).

Parameters

token:Token

getCharErrorDisplay

()

method

public function getCharErrorDisplay(c:int):StringParameters

c:int

Returns

String

getErrorMessage

()

method

public override function getErrorMessage(e:RecognitionException, tokenNames:Array):StringParameters

	`e:RecognitionException`

	`tokenNames:Array`

Returns

String

match

()

method

public function match(c:int):voidParameters

c:int

matchAny

()

method

public function matchAny():void

matchRange

()

method

public function matchRange(a:int, b:int):voidParameters

	`a:int`

	`b:int`

matchString

()

method

public function matchString(s:String):voidParameters

s:String

mTokens

()

method

public function mTokens():void

This is the lexer entry point that sets instance var 'token'

nextToken

()

method

public function nextToken():Token

Return a token from this source; i.e., match a token on the char stream.

Returns

Token

recover

()

method

public function recover(re:RecognitionException):RecognitionException

Lexers can normally match any char in it's vocabulary after matching a token, so do the easy thing and just kill a character and hope it all works out. You can instead use the rule invocation stack to do sophisticated error recovery if you are in a fragment rule.

Parameters

re:RecognitionException

Returns

RecognitionException — This method should return the exception it was provided as an argument. This differs from the Java runtime so that an exception variable does not need to be declared in the generated code, thus reducing a large number of compiler warnings in generated code.

reportError

()

method

public override function reportError(e:RecognitionException):voidParameters

e:RecognitionException

reset

()

method

public override function reset():void

skip

()

method

public function skip():void

Instruct the lexer to skip creating a token for current lexer rule and look for another token. nextToken() knows to keep looking when a lexer rule finishes with token set to SKIP_TOKEN. Recall that if token==null at end of any token rule, it creates one for you and emits it.

traceIn

()

method

public function traceIn(ruleName:String, ruleIndex:int):voidParameters

	`ruleName:String`

	`ruleIndex:int`

traceOut

()

method

public function traceOut(ruleName:String, ruleIndex:int):voidParameters

	`ruleName:String`

	`ruleIndex:int`

API Documentation	All Packages \| All Classes \| Index \| Frames No Frames
Class Lexer	Properties \| Methods \| Constants

Package	org.antlr.runtime
Class	public class Lexer
Inheritance	Lexer BaseRecognizer
Implements	TokenSource