ITokenizer (Okapi Framework SDK 1.47.0 API)

All Known Implementing Classes:

RbbiTokenizer
```
public interface ITokenizer
```

- Nested Class Summary
  
  Nested Classes
  Modifier and Type Interface Description
  
  static class ITokenizer.TokenType
- Method Summary
  
  All Methods Instance Methods Abstract Methods
  Modifier and Type Method Description
  
  boolean hasNext()
  
  void init(String text, LocaleId language)
  
  Token next()

Method Detail

hasNext
```
boolean hasNext()
```

next
```
Token next()
```

init

void init(String text,
          LocaleId language)