Class TokenImpl
- All Implemented Interfaces:
Token
,TokenTypes
Token
.
Note: The instances of Token
returned by
RSyntaxDocument
s are pooled and should always be treated as
immutable. They should not be cast to TokenImpl
and modified.
Modifying tokens you did not create yourself can and will result in
rendering issues and/or runtime exceptions. You have been warned!
- Version:
- 0.3
-
Field Summary
FieldsModifier and TypeFieldDescriptionprivate boolean
Whether this token is a hyperlink.private int
The language this token is in,>= 0
.private Token
The next token in this linked list.private int
The offset into the document at which this token resides.char[]
The text this token represents.int
int
private int
The type of token this is; for example,TokenTypes.FUNCTION
.Fields inherited from interface org.fife.ui.rsyntaxtextarea.TokenTypes
ANNOTATION, COMMENT_DOCUMENTATION, COMMENT_EOL, COMMENT_KEYWORD, COMMENT_MARKUP, COMMENT_MULTILINE, DATA_TYPE, DEFAULT_NUM_TOKEN_TYPES, ERROR_CHAR, ERROR_IDENTIFIER, ERROR_NUMBER_FORMAT, ERROR_STRING_DOUBLE, FUNCTION, IDENTIFIER, LITERAL_BACKQUOTE, LITERAL_BOOLEAN, LITERAL_CHAR, LITERAL_NUMBER_DECIMAL_INT, LITERAL_NUMBER_FLOAT, LITERAL_NUMBER_HEXADECIMAL, LITERAL_STRING_DOUBLE_QUOTE, MARKUP_CDATA, MARKUP_CDATA_DELIMITER, MARKUP_COMMENT, MARKUP_DTD, MARKUP_ENTITY_REFERENCE, MARKUP_PROCESSING_INSTRUCTION, MARKUP_TAG_ATTRIBUTE, MARKUP_TAG_ATTRIBUTE_VALUE, MARKUP_TAG_DELIMITER, MARKUP_TAG_NAME, NULL, OPERATOR, PREPROCESSOR, REGEX, RESERVED_WORD, RESERVED_WORD_2, SEPARATOR, VARIABLE, WHITESPACE
-
Constructor Summary
ConstructorsConstructorDescriptionCreates a "null" token.TokenImpl
(char[] line, int beg, int end, int startOffset, int type, int languageIndex) Constructor.Constructor.Creates this token as a copy of the passed-in token. -
Method Summary
Modifier and TypeMethodDescriptionprivate StringBuilder
appendHtmlLexeme
(RSyntaxTextArea textArea, StringBuilder sb, boolean tabsToSpaces) Appends an HTML version of the lexeme of this token (i.e.appendHTMLRepresentation
(StringBuilder sb, RSyntaxTextArea textArea, boolean fontFamily) Appends HTML code for painting this token, using the given text area's color scheme.appendHTMLRepresentation
(StringBuilder sb, RSyntaxTextArea textArea, boolean fontFamily, boolean tabsToSpaces) Appends HTML code for painting this token, using the given text area's color scheme.char
charAt
(int index) Returns the character at the specified offset in the token.boolean
containsPosition
(int pos) Returns whether the token straddles the specified position in the document.void
Makes one token point to the same text segment, and have the same value as another token.int
documentToToken
(int pos) Returns the position in the token's internal char array corresponding to the specified document position.boolean
endsWith
(char[] ch) Returns whether this token's lexeme ends with the specified characters.boolean
int
Returns the end offset of this token in the document (exclusive).private static String
getHTMLFormatForColor
(Color color) Returns aString
of the form "#xxxxxx" good for use in HTML, representing the given color.getHTMLRepresentation
(RSyntaxTextArea textArea) Returns aString
containing HTML code for painting this token, using the given text area's color scheme.int
Returns the language index of this token.Returns the last token in this list that is not whitespace or a comment.Returns the last paintable token in this token list, ornull
if there is no paintable token.Returns the text of this token, as a string.int
getListOffset
(RSyntaxTextArea textArea, TabExpander e, float x0, float x) Determines the offset into this token list (i.e., into the document) that covers pixel locationx
if the token list starts at pixel locationx0
Returns the token after this one in the linked list.int
Returns the offset into the document at which this token resides.int
getOffsetBeforeX
(RSyntaxTextArea textArea, TabExpander e, float startX, float endBeforeX) Returns the position in the document that represents the last character in the token that will fit intoendBeforeX-startX
pixels.char[]
Returns the character array backing the lexeme of this token.int
Returns the offset into the character array of the lexeme inToken.getTextArray()
.int
getType()
Returns the type of this token.float
getWidth
(RSyntaxTextArea textArea, TabExpander e, float x0) Returns the width of this token given the specified parameters.float
getWidthUpTo
(int numChars, RSyntaxTextArea textArea, TabExpander e, float x0) Returns the width of a specified number of characters in this token.int
hashCode()
boolean
is
(char[] lexeme) Returns whether this token's lexeme matches a specific character array.boolean
is
(int type, char[] lexeme) Returns whether this token is of the specified type, with the specified lexeme.boolean
Returns whether this token is of the specified type, with the specified lexeme.boolean
Returns whether this token is a comment.boolean
Returns whether this token is a comment or whitespace.boolean
Returns whether this token is a hyperlink.boolean
Returns whether this token is an identifier.boolean
Returns whether this token is aTokenTypes.SEPARATOR
representing a single left curly brace.boolean
Returns whether this token is "paintable;" i.e., whether the type of this token is one such that it has an associated syntax style.boolean
Returns whether this token is aTokenTypes.SEPARATOR
representing a single right curly brace.boolean
isSingleChar
(char ch) Returns whether this token is the specified single character.boolean
isSingleChar
(int type, char ch) Returns whether this token is the specified single character, and of a specific type.boolean
Returns whether this token is whitespace.int
length()
Returns the length of this token.listOffsetToView
(RSyntaxTextArea textArea, TabExpander e, int pos, int x0, Rectangle rect) Returns the bounding box for the specified document location.void
makeStartAt
(int pos) Makes this token start at the specified offset into the document.void
moveOffset
(int amt) Moves the starting offset of this token.void
set
(char[] line, int beg, int end, int offset, int type) Sets the value of this token to a particular segment of a document.void
setHyperlink
(boolean hyperlink) Sets whether this token is a hyperlink.void
setLanguageIndex
(int languageIndex) Sets the language index for this token.void
setNextToken
(Token nextToken) Sets the "next token" pointer of this token to point to the specified token.void
setOffset
(int offset) Sets the offset into the document at which this token resides.void
setType
(int type) Sets the type of this token.boolean
startsWith
(char[] chars) Returns whether this token starts with the specified characters.int
tokenToDocument
(int pos) Returns the position in the document corresponding to the specified position in this token's internal char array (textOffset
-textOffset+textCount-1
).toString()
Returns this token as aString
, which is useful for debugging.
-
Field Details
-
text
public char[] textThe text this token represents. This is implemented as a segment so we can point directly to the text in the document without having to make a copy of it. -
textOffset
public int textOffset -
textCount
public int textCount -
offset
private int offsetThe offset into the document at which this token resides. -
type
private int typeThe type of token this is; for example,TokenTypes.FUNCTION
. -
hyperlink
private boolean hyperlinkWhether this token is a hyperlink. -
nextToken
The next token in this linked list. -
languageIndex
private int languageIndexThe language this token is in,>= 0
.
-
-
Constructor Details
-
TokenImpl
public TokenImpl()Creates a "null" token. The token itself is not null; rather, it signifies that it is the last token in a linked list of tokens and that it is not part of a "multi-line token." -
TokenImpl
Constructor.- Parameters:
line
- The segment from which to get the token.beg
- The first character's position inline
.end
- The last character's position inline
.startOffset
- The offset into the document at which this token begins.type
- A token type listed as "generic" above.languageIndex
- The language index for this token.
-
TokenImpl
public TokenImpl(char[] line, int beg, int end, int startOffset, int type, int languageIndex) Constructor.- Parameters:
line
- The segment from which to get the token.beg
- The first character's position inline
.end
- The last character's position inline
.startOffset
- The offset into the document at which this token begins.type
- A token type listed as "generic" above.languageIndex
- The language index for this token.
-
TokenImpl
Creates this token as a copy of the passed-in token.- Parameters:
t2
- The token from which to make a copy.
-
-
Method Details
-
appendHTMLRepresentation
public StringBuilder appendHTMLRepresentation(StringBuilder sb, RSyntaxTextArea textArea, boolean fontFamily) Description copied from interface:Token
Appends HTML code for painting this token, using the given text area's color scheme.- Specified by:
appendHTMLRepresentation
in interfaceToken
- Parameters:
sb
- The buffer to append to.textArea
- The text area whose color scheme to use.fontFamily
- Whether to include the font family in the HTML for this token. You can passfalse
for this parameter if, for example, you are making all your HTML be monospaced, and don't want any crazy fonts being used in the editor to be reflected in your HTML.- Returns:
- The buffer appended to.
- See Also:
-
appendHTMLRepresentation
public StringBuilder appendHTMLRepresentation(StringBuilder sb, RSyntaxTextArea textArea, boolean fontFamily, boolean tabsToSpaces) Description copied from interface:Token
Appends HTML code for painting this token, using the given text area's color scheme.- Specified by:
appendHTMLRepresentation
in interfaceToken
- Parameters:
sb
- The buffer to append to.textArea
- The text area whose color scheme to use.fontFamily
- Whether to include the font family in the HTML for this token. You can passfalse
for this parameter if, for example, you are making all your HTML be monospaced, and don't want any crazy fonts being used in the editor to be reflected in your HTML.tabsToSpaces
- Whether to convert tabs into spaces.- Returns:
- The buffer appended to.
- See Also:
-
appendHtmlLexeme
private StringBuilder appendHtmlLexeme(RSyntaxTextArea textArea, StringBuilder sb, boolean tabsToSpaces) Appends an HTML version of the lexeme of this token (i.e. no style HTML, but replacing chars such as\t
,<
and>
with their escapes).- Parameters:
textArea
- The text area.sb
- The buffer to append to.tabsToSpaces
- Whether to convert tabs into spaces.- Returns:
- The same buffer.
-
charAt
public char charAt(int index) Description copied from interface:Token
Returns the character at the specified offset in the token.- Specified by:
charAt
in interfaceToken
- Parameters:
index
- The index. This should be in the range0-(
.Token.length()
-1)- Returns:
- The character.
- See Also:
-
containsPosition
public boolean containsPosition(int pos) Description copied from interface:Token
Returns whether the token straddles the specified position in the document.- Specified by:
containsPosition
in interfaceToken
- Parameters:
pos
- The position in the document to check.- Returns:
- Whether the specified position is straddled by this token.
-
copyFrom
Makes one token point to the same text segment, and have the same value as another token.- Parameters:
t2
- The token from which to copy.
-
documentToToken
public int documentToToken(int pos) Description copied from interface:Token
Returns the position in the token's internal char array corresponding to the specified document position.Note that this method does NOT do any bounds checking; you can pass in a document position that does not correspond to a position in this token, and you will not receive an Exception or any other notification; it is up to the caller to ensure valid input.
- Specified by:
documentToToken
in interfaceToken
- Parameters:
pos
- A position in the document that is represented by this token.- Returns:
- The corresponding token position >=
textOffset
and <textOffset+textCount
. - See Also:
-
endsWith
public boolean endsWith(char[] ch) Description copied from interface:Token
Returns whether this token's lexeme ends with the specified characters. -
equals
-
getEndOffset
public int getEndOffset()Description copied from interface:Token
Returns the end offset of this token in the document (exclusive). In other words, the token ranges from[getOffset(), getEndOffset())
.- Specified by:
getEndOffset
in interfaceToken
- Returns:
- The end offset of this token.
- See Also:
-
getHTMLFormatForColor
Returns aString
of the form "#xxxxxx" good for use in HTML, representing the given color.- Parameters:
color
- The color to get a string for.- Returns:
- The HTML form of the color. If
color
isnull
,#000000
is returned.
-
getHTMLRepresentation
Description copied from interface:Token
Returns aString
containing HTML code for painting this token, using the given text area's color scheme.- Specified by:
getHTMLRepresentation
in interfaceToken
- Parameters:
textArea
- The text area whose color scheme to use.- Returns:
- The HTML representation of the token.
- See Also:
-
getLanguageIndex
public int getLanguageIndex()Description copied from interface:Token
Returns the language index of this token.- Specified by:
getLanguageIndex
in interfaceToken
- Returns:
- The language index. A value of
0
denotes the "main" language, any positive value denotes a specific secondary language. - See Also:
-
getLastNonCommentNonWhitespaceToken
Description copied from interface:Token
Returns the last token in this list that is not whitespace or a comment.- Specified by:
getLastNonCommentNonWhitespaceToken
in interfaceToken
- Returns:
- The last non-comment, non-whitespace token, or
null
if there isn't one.
-
getLastPaintableToken
Description copied from interface:Token
Returns the last paintable token in this token list, ornull
if there is no paintable token.- Specified by:
getLastPaintableToken
in interfaceToken
- Returns:
- The last paintable token in this token list.
-
getLexeme
Description copied from interface:Token
Returns the text of this token, as a string.Note that this method isn't used much by the
ryntaxtextarea
package internally, as it tries to limit memory allocation. -
getListOffset
Description copied from interface:Token
Determines the offset into this token list (i.e., into the document) that covers pixel locationx
if the token list starts at pixel locationx0
. This method will return the document position "closest" to the x-coordinate (i.e., if they click on the "right-half" of the
w
inawe
, the caret will be placed in between thew
ande
; similarly, clicking on the left-half places the caret between thea
andw
). This makes it useful for methods such asviewToModel
found injavax.swing.text.View
subclasses.- Specified by:
getListOffset
in interfaceToken
- Parameters:
textArea
- The text area from which the token list was derived.e
- How to expand tabs.x0
- The pixel x-location that is the beginning oftokenList
.x
- The pixel-position for which you want to get the corresponding offset.- Returns:
- The position (in the document, NOT into the token list!) that
covers the pixel location. If
tokenList
isnull
or has typeToken.NULL
, then-1
is returned; the caller should recognize this and return the actual end position of the (empty) line.
-
getNextToken
Description copied from interface:Token
Returns the token after this one in the linked list.- Specified by:
getNextToken
in interfaceToken
- Returns:
- The next token.
-
getOffset
public int getOffset()Description copied from interface:Token
Returns the offset into the document at which this token resides. -
getOffsetBeforeX
public int getOffsetBeforeX(RSyntaxTextArea textArea, TabExpander e, float startX, float endBeforeX) Description copied from interface:Token
Returns the position in the document that represents the last character in the token that will fit intoendBeforeX-startX
pixels. For example, if you're using a monospaced 8-pixel-per-character font, have the token "while" andstartX
is0
andendBeforeX
is30
, this method will return the document position of the "i" in "while", because the "i" ends at pixel24
, while the "l" ends at32
. If not even the first character fits inendBeforeX-startX
, the first character's position is still returned so calling methods don't go into infinite loops.- Specified by:
getOffsetBeforeX
in interfaceToken
- Parameters:
textArea
- The text area in which this token is being painted.e
- How to expand tabs.startX
- The x-coordinate at which the token will be painted. This is needed because of tabs.endBeforeX
- The x-coordinate for which you want to find the last character oft
which comes before it.- Returns:
- The last document position that will fit in the specified amount of pixels.
-
getTextArray
public char[] getTextArray()Description copied from interface:Token
Returns the character array backing the lexeme of this token. This value should be treated as read-only.- Specified by:
getTextArray
in interfaceToken
- Returns:
- A character array containing the lexeme of this token.
- See Also:
-
getTextOffset
public int getTextOffset()Description copied from interface:Token
Returns the offset into the character array of the lexeme inToken.getTextArray()
.- Specified by:
getTextOffset
in interfaceToken
- Returns:
- The offset of the lexeme in the character array.
- See Also:
-
getType
public int getType()Description copied from interface:Token
Returns the type of this token. -
getWidth
Description copied from interface:Token
Returns the width of this token given the specified parameters.- Specified by:
getWidth
in interfaceToken
- Parameters:
textArea
- The text area in which the token is being painted.e
- Describes how to expand tabs. This parameter cannot benull
.x0
- The pixel-location at which the token begins. This is needed because of tabs.- Returns:
- The width of the token, in pixels.
- See Also:
-
getWidthUpTo
Description copied from interface:Token
Returns the width of a specified number of characters in this token. For example, for the token "while", specifying a value of3
here returns the width of the "whi" portion of the token.- Specified by:
getWidthUpTo
in interfaceToken
- Parameters:
numChars
- The number of characters for which to get the width.textArea
- The text area in which the token is being painted.e
- How to expand tabs. This value cannot benull
.x0
- The pixel-location at which this token begins. This is needed because of tabs.- Returns:
- The width of the specified number of characters in this token.
- See Also:
-
hashCode
public int hashCode() -
is
public boolean is(char[] lexeme) Description copied from interface:Token
Returns whether this token's lexeme matches a specific character array. -
is
public boolean is(int type, char[] lexeme) Description copied from interface:Token
Returns whether this token is of the specified type, with the specified lexeme.This method is preferred over the other overload in performance-critical code where this operation may be called frequently, since it does not involve any String allocations.
-
is
Description copied from interface:Token
Returns whether this token is of the specified type, with the specified lexeme.The other overload of this method is preferred over this one in performance-critical code, as this one involves a String allocation while the other does not.
-
isComment
public boolean isComment()Description copied from interface:Token
Returns whether this token is a comment. -
isCommentOrWhitespace
public boolean isCommentOrWhitespace()Description copied from interface:Token
Returns whether this token is a comment or whitespace.- Specified by:
isCommentOrWhitespace
in interfaceToken
- Returns:
- Whether this token is a comment or whitespace.
- See Also:
-
isHyperlink
public boolean isHyperlink()Description copied from interface:Token
Returns whether this token is a hyperlink.- Specified by:
isHyperlink
in interfaceToken
- Returns:
- Whether this token is a hyperlink.
- See Also:
-
isIdentifier
public boolean isIdentifier()Description copied from interface:Token
Returns whether this token is an identifier.- Specified by:
isIdentifier
in interfaceToken
- Returns:
- Whether this token is an identifier.
-
isLeftCurly
public boolean isLeftCurly()Description copied from interface:Token
Returns whether this token is aTokenTypes.SEPARATOR
representing a single left curly brace.- Specified by:
isLeftCurly
in interfaceToken
- Returns:
- Whether this token is a left curly brace.
- See Also:
-
isRightCurly
public boolean isRightCurly()Description copied from interface:Token
Returns whether this token is aTokenTypes.SEPARATOR
representing a single right curly brace.- Specified by:
isRightCurly
in interfaceToken
- Returns:
- Whether this token is a right curly brace.
- See Also:
-
isPaintable
public boolean isPaintable()Description copied from interface:Token
Returns whether this token is "paintable;" i.e., whether the type of this token is one such that it has an associated syntax style. What this boils down to is whether the token type is greater thanToken.NULL
.- Specified by:
isPaintable
in interfaceToken
- Returns:
- Whether this token is paintable.
-
isSingleChar
public boolean isSingleChar(char ch) Description copied from interface:Token
Returns whether this token is the specified single character.- Specified by:
isSingleChar
in interfaceToken
- Parameters:
ch
- The character to check for.- Returns:
- Whether this token's lexeme is the single character specified.
- See Also:
-
isSingleChar
public boolean isSingleChar(int type, char ch) Description copied from interface:Token
Returns whether this token is the specified single character, and of a specific type.- Specified by:
isSingleChar
in interfaceToken
- Parameters:
type
- The token type.ch
- The character to check for.- Returns:
- Whether this token is of the specified type, and with a lexeme Equaling the single character specified.
- See Also:
-
isWhitespace
public boolean isWhitespace()Description copied from interface:Token
Returns whether this token is whitespace.- Specified by:
isWhitespace
in interfaceToken
- Returns:
true
iff this token is whitespace.- See Also:
-
length
public int length()Description copied from interface:Token
Returns the length of this token. -
listOffsetToView
public Rectangle listOffsetToView(RSyntaxTextArea textArea, TabExpander e, int pos, int x0, Rectangle rect) Description copied from interface:Token
Returns the bounding box for the specified document location. The location must be in the specified token list; if it isn't,null
is returned.- Specified by:
listOffsetToView
in interfaceToken
- Parameters:
textArea
- The text area from which the token list was derived.e
- How to expand tabs.pos
- The position in the document for which to get the bounding box in the view.x0
- The pixel x-location that is the beginning oftokenList
.rect
- The rectangle in which we'll be returning the results. This object is reused to keep from frequent memory allocations.- Returns:
- The bounding box for the specified position in the model.
-
makeStartAt
public void makeStartAt(int pos) Makes this token start at the specified offset into the document.Note: You should not modify
Token
instances you did not create yourself (e.g., came from anRSyntaxDocument
). If you do, rendering issues and/or runtime exceptions will likely occur. You have been warned!- Parameters:
pos
- The offset into the document this token should start at. Note that this token must already contain this position; if it doesn't, an exception is thrown.- Throws:
IllegalArgumentException
- If pos is not already contained by this token.- See Also:
-
moveOffset
public void moveOffset(int amt) Moves the starting offset of this token.Note: You should not modify
Token
instances you did not create yourself (e.g., came from anRSyntaxDocument
). If you do, rendering issues and/or runtime exceptions will likely occur. You have been warned!- Parameters:
amt
- The amount to move the starting offset. This should be between0
andtextCount
, inclusive.- Throws:
IllegalArgumentException
- Ifamt
is an invalid value.- See Also:
-
set
public void set(char[] line, int beg, int end, int offset, int type) Sets the value of this token to a particular segment of a document. The "next token" value is cleared.- Parameters:
line
- The segment from which to get the token.beg
- The first character's position inline
.end
- The last character's position inline
.offset
- The offset into the document at which this token begins.type
- A token type listed as "generic" above.
-
setHyperlink
public void setHyperlink(boolean hyperlink) Sets whether this token is a hyperlink.- Specified by:
setHyperlink
in interfaceToken
- Parameters:
hyperlink
- Whether this token is a hyperlink.- See Also:
-
setLanguageIndex
public void setLanguageIndex(int languageIndex) Sets the language index for this token. If this value is positive, it denotes a specific "secondary" language this token represents (such as JavaScript code or CSS embedded in an HTML file). If this value is0
, this token is in the "main" language being edited. Negative values are invalid and treated as0
.- Specified by:
setLanguageIndex
in interfaceToken
- Parameters:
languageIndex
- The new language index. A value of0
denotes the "main" language, any positive value denotes a specific secondary language. Negative values will be treated as0
.- See Also:
-
setNextToken
Sets the "next token" pointer of this token to point to the specified token.- Parameters:
nextToken
- The new next token.- See Also:
-
setOffset
public void setOffset(int offset) Sets the offset into the document at which this token resides.- Parameters:
offset
- The new offset into the document.- See Also:
-
setType
public void setType(int type) Description copied from interface:Token
Sets the type of this token. -
startsWith
public boolean startsWith(char[] chars) Description copied from interface:Token
Returns whether this token starts with the specified characters.- Specified by:
startsWith
in interfaceToken
- Parameters:
chars
- The characters.- Returns:
- Whether this token starts with those characters.
- See Also:
-
tokenToDocument
public int tokenToDocument(int pos) Description copied from interface:Token
Returns the position in the document corresponding to the specified position in this token's internal char array (textOffset
-textOffset+textCount-1
).Note that this method does NOT do any bounds checking; you can pass in an invalid token position, and you will not receive an Exception or any other indication that the returned document position is invalid. It is up to the user to ensure valid input.
- Specified by:
tokenToDocument
in interfaceToken
- Parameters:
pos
- A position in the token's internal char array (textOffset
-textOffset+textCount
).- Returns:
- The corresponding position in the document.
- See Also:
-
toString
Returns this token as aString
, which is useful for debugging.
-