Module io.jenetics.ext

Package io.jenetics.ext.internal.parser

Class CharSequenceTokenizer

java.lang.Object

io.jenetics.ext.internal.parser.CharSequenceTokenizer

All Implemented Interfaces:

Tokenizer<Token<String>>

public abstract class CharSequenceTokenizer
extends Object
implements Tokenizer<Token<String>>

Base class for all tokenizers.

Since:

7.1

Version:

7.1

Field Summary

Fields

Modifier and Type	Field	Description
protected char	c
protected int	pos

Constructor Summary

Constructors

Modifier	Constructor	Description
protected	CharSequenceTokenizer(CharSequence input)	Create a new tokenizer from the given character sequence.

Method Summary

Modifier and Type	Method	Description
protected void	consume()
static boolean	isAlphabetic(char c)
static boolean	isNonEof(char ch)
protected char	LA(int index)
protected void	match(char ch)
protected void	WS()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface io.jenetics.ext.internal.parser.Tokenizer

next, tokens

Field Details

pos

protected int pos

c

protected char c

Constructor Details

CharSequenceTokenizer

protected CharSequenceTokenizer(CharSequence input)

Create a new tokenizer from the given character sequence.

Parameters:

input - the input character sequence

Throws:

NullPointerException - if the given input is null

Method Details

match

protected void match(char ch)

consume

protected void consume()

LA

protected char LA(int index)

isNonEof

public static boolean isNonEof(char ch)

isAlphabetic

public static boolean isAlphabetic(char c)

WS

protected void WS()