Module io.jenetics.ext
Package io.jenetics.ext.internal.parser

Interface Tokenizer<T>

Type Parameters:
T - the token value type
All Known Implementing Classes:
CharSequenceTokenizer
Functional Interface:
This is a functional interface and can therefore be used as the assignment target for a lambda expression or method reference.
@FunctionalInterface public interface Tokenizer<T>
Interface for all tokenizers.
Since:
7.1
Version:
7.1

  • Method Summary

    Modifier and Type
    Method
    Description
    T
    next()
    Return the next available token, or null if no further tokens are available.
    default Stream<T>
    tokens()
    Return a stream of tokens, generated by this tokenizer.

  • Method Details

    • next

      T next()
      Return the next available token, or null if no further tokens are available.
      Returns:
      the next available token

    • tokens

      default Stream<T> tokens()
      Return a stream of tokens, generated by this tokenizer. The returned stream terminates, when the null-token is encountered.
      Returns:
      a new tokens stream of this tokenizer