Package net.sf.okapi.steps.repetitionanalysis

Class RepetitionAnalysisStep

java.lang.Object

net.sf.okapi.common.pipeline.BasePipelineStep

net.sf.okapi.steps.repetitionanalysis.RepetitionAnalysisStep

All Implemented Interfaces:

AutoCloseable, Function<Stream<Event>,Stream<Event>>, IPipelineStep

public class RepetitionAnalysisStep
extends BasePipelineStep

The step analyzes repetitions in input documents. Either exact or configurable fuzzy search is performed.

2 types of annotations are created for found repetitive segments -- RepetitiveSegmentAnnotation and AltTranslationsAnnotation. RepetitiveSegmentAnnotation's are attached to all repetitive source segments. AltTranslationsAnnotation's are attached to target segments, corresponding to repetitive source segments. AltTranslationsAnnotation is not attached for the first repetitive segment not to be counted by counting steps twice as repetitive with itself.

Constructor Summary

Constructors

Constructor	Description
RepetitionAnalysisStep()

Method Summary

Modifier and Type	Method	Description
void	cancel()	Cancel processing on this pipeline.
static boolean	checkSegments(Segment sseg, Segment tseg)
void	close()
String	getDescription()	Gets a short localizable description of what this step does.
String	getName()	Gets the localizable name of this step.
Parameters	getParameters()	Gets the current parameters for this step.
protected Event	handleEndDocument(Event event)	Handles the EventType.END_DOCUMENT event.
protected Event	handleStartDocument(Event event)	Handles the EventType.START_DOCUMENT event.
protected Event	handleTextUnit(Event event)	Handles the EventType.TEXT_UNIT event.
void	setParameters(IParameters params)	Sets new parameters for this step.
void	setSourceLocale(LocaleId sourceLocale)	Delegate to concrete class
void	setTargetLocale(LocaleId targetLocale)

Methods inherited from class net.sf.okapi.common.pipeline.BasePipelineStep

destroy, getHelpLocation, getSourceLocale, getTargetLocale, handleCustom, handleDocumentPart, handleEndBatch, handleEndBatchItem, handleEndGroup, handleEndSubDocument, handleEndSubfilter, handleEvent, handleMultiEvent, handlePipelineParameters, handleRawDocument, handleStartBatch, handleStartBatchItem, handleStartGroup, handleStartSubDocument, handleStartSubfilter, isDone, isLastOutputStep, setLastOutputStep

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface java.util.function.Function

andThen, compose

Methods inherited from interface net.sf.okapi.common.pipeline.IPipelineStep

apply, handleStream

Constructor Detail

RepetitionAnalysisStep

public RepetitionAnalysisStep()

Method Detail

getName

public String getName()

Description copied from interface: IPipelineStep

Gets the localizable name of this step.

Returns:

the localizable name of this step.

getDescription

public String getDescription()

Description copied from interface: IPipelineStep

Gets a short localizable description of what this step does.

Returns:

the text of a short description of what this step does.

getParameters

public Parameters getParameters()

Description copied from interface: IPipelineStep

Gets the current parameters for this step.

Specified by:

getParameters in interface IPipelineStep

Overrides:

getParameters in class BasePipelineStep

Returns:

the current parameters for this step or null if there are no parameters.

cancel

public void cancel()

Description copied from interface: IPipelineStep

Cancel processing on this pipeline. Each IPipelineStep is responsible to implement a cancel method that will interrupt long running operations

Specified by:

cancel in interface IPipelineStep

Overrides:

cancel in class BasePipelineStep

setParameters

public void setParameters(IParameters params)

Description copied from interface: IPipelineStep

Sets new parameters for this step.

Specified by:

setParameters in interface IPipelineStep

Overrides:

setParameters in class BasePipelineStep

Parameters:

params - the new parameters to use.

setSourceLocale

public void setSourceLocale(LocaleId sourceLocale)

Description copied from interface: IPipelineStep

Delegate to concrete class

Specified by:

setSourceLocale in interface IPipelineStep

Overrides:

setSourceLocale in class BasePipelineStep

setTargetLocale

public void setTargetLocale(LocaleId targetLocale)

Specified by:

setTargetLocale in interface IPipelineStep

Overrides:

setTargetLocale in class BasePipelineStep

close

public void close()

handleStartDocument

protected Event handleStartDocument(Event event)

Description copied from class: BasePipelineStep

Handles the EventType.START_DOCUMENT event.

Overrides:

handleStartDocument in class BasePipelineStep

Parameters:

event - event to handle.

Returns:

the event returned.

handleEndDocument

protected Event handleEndDocument(Event event)

Description copied from class: BasePipelineStep

Handles the EventType.END_DOCUMENT event.

Overrides:

handleEndDocument in class BasePipelineStep

Parameters:

event - event to handle.

Returns:

the event returned.

checkSegments

public static boolean checkSegments(Segment sseg,
                                    Segment tseg)

handleTextUnit

protected Event handleTextUnit(Event event)

Description copied from class: BasePipelineStep

Handles the EventType.TEXT_UNIT event.

Overrides:

handleTextUnit in class BasePipelineStep

Parameters:

event - event to handle.

Returns:

the event returned.