This step creates a wrapper around the main step and processes text units concurrently. Example of use:
pipeline.addStep(new TextUnitWorkQueueStep(new LongRunningStep(), 4));
- LongRunningStep is the step that will be processed concurrently and is wrapped by ThreadedWorkQueueStep
- 4 is the number of threads or work-queues to create. In this case 4 text units will be processed concurrently. The optimal number of threads is dependent on the wrapped step.
Takes: Filter events. Sends: Filter events. (TextUnits and DocumentParts are buffered and sent as MultiEvents)
- Does the same processing as the main step. Processes multiple text units concurrently.
See the Okapi Developer Wiki for more details.
This step has no parameters.
- Will only process text units concurrently.
- Steps must be essentially stateless: TextUnit in, TextUnit out
- DocumentParts are passed through without processing of any kind. If DocumentPart processing is needed then a new implementation must be created
or override the startConcurrentProcesing() method.