Package net.sf.okapi.filters.html

Class HtmlFilter

java.lang.Object

net.sf.okapi.common.filters.AbstractFilter

net.sf.okapi.filters.abstractmarkup.AbstractMarkupFilter

net.sf.okapi.filters.html.HtmlFilter

All Implemented Interfaces:

AutoCloseable, Iterator<Event>, IFilter

public class HtmlFilter
extends AbstractMarkupFilter

Field Summary

Fields inherited from interface net.sf.okapi.common.filters.IFilter

SUB_FILTER

Constructor Summary

Constructors

Constructor	Description
HtmlFilter()

Method Summary

Modifier and Type	Method	Description
void	close()	Close the filter and all used resources.
protected PropertyTextUnitPlaceholder	createPropertyTextUnitPlaceholder(PropertyTextUnitPlaceholder.PlaceholderAccessType type, String name, String value, net.htmlparser.jericho.Tag tag, net.htmlparser.jericho.Attribute attribute)	Create a PropertyTextUnitPlaceholder given the supplied type, name and Jericho Tag and Attribute.
ISkeletonWriter	createSkeletonWriter()	Default case.
protected TextFragment.TagType	determineTagType(net.htmlparser.jericho.Tag tag)	Filter specific method for determining TextFragment.TagType
protected void	endFilter()	End the current filter processing and send the Ending Event
protected TaggedFilterConfiguration	getConfig()	Get the current TaggedFilterConfiguration.
Parameters	getParameters()	Gets the current parameters for this filter.
ExtractionRuleState.ExtractionRule	getRuleTypeFromStartTag(net.htmlparser.jericho.EndTag endTag, EnumSet<TaggedFilterConfiguration.RULE_TYPE> ruleTypes)
protected void	handleEndTag(net.htmlparser.jericho.EndTag endTag)	Handle end tags, including empty tags.
protected String	normalizeAttributeName(String attrName, String attrValue, net.htmlparser.jericho.Tag tag)	Some attributes names are converted to Okapi standards such as HTML charset to "encoding" and lang to "language"
void	open(RawDocument input, boolean generateSkeleton)	Start a new IFilter using the supplied RawDocument.
protected void	preProcess(net.htmlparser.jericho.Segment segment)	Do any handling needed before the current Segment is processed.
void	setParameters(IParameters params)	Sets new parameters for this filter.
void	setParametersFromFile(File config)	Initialize filter parameters from a Java File.
void	setParametersFromString(String config)	Initialize filter parameters from a String.
void	setParametersFromURL(URL config)	Initialize filter parameters from a URL.
protected void	startFilter()	Initialize rule state and parser.
protected void	updateEndTagRuleState(net.htmlparser.jericho.EndTag endTag, ExtractionRuleState.ExtractionRule rule)
protected void	updateStartTagRuleState(net.htmlparser.jericho.StartTag startTag, ExtractionRuleState.ExtractionRule rule)

Methods inherited from class net.sf.okapi.filters.abstractmarkup.AbstractMarkupFilter

addCodeToCurrentTextUnit, addCodeToCurrentTextUnit, addToDocumentPart, addToTextUnit, addToTextUnit, addToTextUnit, addToTextUnit, canStartNewTextUnit, createEventBuilder, createPropertyTextUnitPlaceholders, detectEncoding, disambiguateElementRuleTypes, endDocumentPart, endGroup, endTextUnit, getCurrentDocName, getEventBuilder, getMainAttributeRule, getMainElementRule, getParsedHeader, getRuleState, getTextUnitId, handleCdataSection, handleCharacterEntity, handleComment, handleDocTypeDeclaration, handleDocumentPart, handleNumericEntity, handleProcessingInstruction, handleServerCommon, handleServerCommonEscaped, handleStartTag, handleText, handleXmlDeclaration, hasNext, isBOM, isDocumentEncoding, isInline, isInsideTextRun, isMatchedTag, isPreserveWhitespace, isUtf8Bom, isUtf8Encoding, isWhiteSpace, next, open, peekTempEvent, postProcessTextUnit, setCurrentDocName, setDocumentPartId, setMimeType, setPreserveWhitespace, setTextUnitMimeType, setTextUnitName, setTextUnitType, startDocumentPart, startGroup, startGroup, startTextUnit, startTextUnit, startTextUnit, startTextUnit

Methods inherited from class net.sf.okapi.common.filters.AbstractFilter

addConfiguration, addConfiguration, addConfiguration, addConfigurations, cancel, createEndFilterEvent, createFilterWriter, createStartFilterEvent, findConfiguration, getConfiguration, getConfigurations, getDisplayName, getDocumentId, getDocumentName, getEncoderManager, getEncoding, getFilterConfigurationMapper, getMimeType, getName, getNewlineType, getParameters, getParametersClassName, getParentId, getSrcLoc, getTrgLoc, isCanceled, isGenerateSkeleton, isMultilingual, removeConfiguration, setDisplayName, setDocumentName, setEncoding, setFilterConfigurationMapper, setGenerateSkeleton, setMultilingual, setName, setNewlineType, setOptions, setParentId, setSrcLoc, setTrgLoc

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface net.sf.okapi.common.filters.IFilter

stream

Methods inherited from interface java.util.Iterator

forEachRemaining, remove

Constructor Detail

HtmlFilter

public HtmlFilter()

Method Detail

createSkeletonWriter

public ISkeletonWriter createSkeletonWriter()

Description copied from class: AbstractFilter

Default case. Override if needed.

Specified by:

createSkeletonWriter in interface IFilter

Overrides:

createSkeletonWriter in class AbstractFilter

Returns:

new instance of GenericSkeletonWriter

open

public void open(RawDocument input,
                 boolean generateSkeleton)

Description copied from class: AbstractMarkupFilter

Start a new IFilter using the supplied RawDocument.

Specified by:

open in interface IFilter

Overrides:

open in class AbstractMarkupFilter

Parameters:

input - - input to the IFilter (can be a CharSequence, URI or InputStream)

generateSkeleton - - true if the IFilter should store non-translatble blocks (aka skeleton), false otherwise.

close

public void close()

Description copied from class: AbstractMarkupFilter

Close the filter and all used resources.

Specified by:

close in interface AutoCloseable

Specified by:

close in interface IFilter

Overrides:

close in class AbstractMarkupFilter

startFilter

protected void startFilter()

Initialize rule state and parser. Called before processing of each input.

Overrides:

startFilter in class AbstractMarkupFilter

endFilter

protected void endFilter()

End the current filter processing and send the Ending Event

Overrides:

endFilter in class AbstractMarkupFilter

preProcess

protected void preProcess(net.htmlparser.jericho.Segment segment)

Description copied from class: AbstractMarkupFilter

Do any handling needed before the current Segment is processed. Default is to do nothing.

Overrides:

preProcess in class AbstractMarkupFilter

updateStartTagRuleState

protected void updateStartTagRuleState(net.htmlparser.jericho.StartTag startTag,
                                       ExtractionRuleState.ExtractionRule rule)

Overrides:

updateStartTagRuleState in class AbstractMarkupFilter

getRuleTypeFromStartTag

public ExtractionRuleState.ExtractionRule getRuleTypeFromStartTag(net.htmlparser.jericho.EndTag endTag,
                                                                  EnumSet<TaggedFilterConfiguration.RULE_TYPE> ruleTypes)

Overrides:

getRuleTypeFromStartTag in class AbstractMarkupFilter

updateEndTagRuleState

protected void updateEndTagRuleState(net.htmlparser.jericho.EndTag endTag,
                                     ExtractionRuleState.ExtractionRule rule)

Overrides:

updateEndTagRuleState in class AbstractMarkupFilter

handleEndTag

protected void handleEndTag(net.htmlparser.jericho.EndTag endTag)

Description copied from class: AbstractMarkupFilter

Handle end tags, including empty tags.

Overrides:

handleEndTag in class AbstractMarkupFilter

createPropertyTextUnitPlaceholder

protected PropertyTextUnitPlaceholder createPropertyTextUnitPlaceholder(PropertyTextUnitPlaceholder.PlaceholderAccessType type,
                                                                        String name,
                                                                        String value,
                                                                        net.htmlparser.jericho.Tag tag,
                                                                        net.htmlparser.jericho.Attribute attribute)

Description copied from class: AbstractMarkupFilter

Create a PropertyTextUnitPlaceholder given the supplied type, name and Jericho Tag and Attribute.

Overrides:

createPropertyTextUnitPlaceholder in class AbstractMarkupFilter

Parameters:

type - - PropertyTextUnitPlaceholder.PlaceholderAccessType is one of TRANSLATABLE, READ_ONLY_PROPERTY, WRITABLE_PROPERTY

name - - attribute name

value - - attribute value

tag - - Jericho Tag which contains the attribute

attribute - - attribute as a Jericho Attribute

Returns:

a PropertyTextUnitPlaceholder representing the attribute

normalizeAttributeName

protected String normalizeAttributeName(String attrName,
                                        String attrValue,
                                        net.htmlparser.jericho.Tag tag)

Description copied from class: AbstractMarkupFilter

Some attributes names are converted to Okapi standards such as HTML charset to "encoding" and lang to "language"

Specified by:

normalizeAttributeName in class AbstractMarkupFilter

Parameters:

attrName - - the attribute name

attrValue - - the attribute value

tag - - the Jericho Tag that contains the attribute

Returns:

the attribute name after it as passe through the normalization rules

getConfig

protected TaggedFilterConfiguration getConfig()

Description copied from class: AbstractMarkupFilter

Get the current TaggedFilterConfiguration. A TaggedFilterConfiguration is the result of reading in a YAML configuration file and converting it into Java Objects.

Specified by:

getConfig in class AbstractMarkupFilter

Returns:

a TaggedFilterConfiguration

setParameters

public void setParameters(IParameters params)

Description copied from interface: IFilter

Sets new parameters for this filter.

Specified by:

setParameters in interface IFilter

Overrides:

setParameters in class AbstractFilter

Parameters:

params - The new parameters to use.

getParameters

public Parameters getParameters()

Description copied from interface: IFilter

Gets the current parameters for this filter.

Specified by:

getParameters in interface IFilter

Overrides:

getParameters in class AbstractFilter

Returns:

The current parameters for this filter, or DefaultParameters if this filter has no parameters.

setParametersFromURL

public void setParametersFromURL(URL config)

Initialize filter parameters from a URL.

Parameters:

config -

setParametersFromFile

public void setParametersFromFile(File config)

Initialize filter parameters from a Java File.

Parameters:

config -

setParametersFromString

public void setParametersFromString(String config)

Initialize filter parameters from a String.

Parameters:

config -

determineTagType

protected TextFragment.TagType determineTagType(net.htmlparser.jericho.Tag tag)

Description copied from class: AbstractMarkupFilter

Filter specific method for determining TextFragment.TagType

Overrides:

determineTagType in class AbstractMarkupFilter

Parameters:

tag - Jericho Tag start or end tag

Returns:

PLACEHOLDER, OPEN, CLOSED TextFragment.TagType