public class TokenAnnotation extends TextPipeAnnotation implements ReversableToken
TokenAnnotation
represents a single token generated by a tokeniser.
It can be further annotated. It is aware of its off-sets in the
RawStringAnnotation that generated it.Constructor and Description |
---|
TokenAnnotation(String token,
String raw,
int start,
int stop) |
Modifier and Type | Method and Description |
---|---|
String |
getRawString() |
String |
getStringToken() |
String |
reverse(List<? extends ReversableToken> tokens) |
addAllAnnotations, addAnnotation, getAnnotationKeyList, getAnnotationsFor
public TokenAnnotation(String token, String raw, int start, int stop)
public String getStringToken()
public String getRawString()
getRawString
in interface ReversableToken
public String reverse(List<? extends ReversableToken> tokens)
reverse
in interface ReversableToken