SRX - Revision history

Translate5Support at 08:24, 10 January 2026

2026-01-10T08:24:08Z

← Older revision		Revision as of 04:24, 10 January 2026
Line 169:		Line 169:
	==Hint: Knowing when a no-break rule will match==		==Hint: Knowing when a no-break rule will match==

	Like any regular expression, a no-break rule matches a number of characters in a string. For example, the pattern "\s+" matches all whitespaces within the string <pre>" "</pre>.		Like any regular expression, a no-break rule matches a number of characters in a string. For example, the pattern "\s+" matches all whitespaces within the string: <pre>" "</pre>

	Whether a no-break rule takes effect to overwrite a break rule that follows further down in the SRX is defined by		Whether a no-break rule takes effect to overwrite a break rule that follows further down in the SRX is defined by

Translate5Support at 08:23, 10 January 2026

2026-01-10T08:23:04Z

← Older revision		Revision as of 04:23, 10 January 2026
Line 167:		Line 167:
	'''Note that the framework implements a [[SRX Extensions\|few extensions to SRX]].'''		'''Note that the framework implements a [[SRX Extensions\|few extensions to SRX]].'''

	==Hint: ~~When~~ a no-break rule will match ~~and when not~~==		==Hint: Knowing when a no-break rule will match==

	A no-break rule ~~can (as any regex)~~ matches a number of characters in a string. ~~Like~~ \s+ ~~does match~~ all whitespaces within ~~this~~ string " ".		Like any regular expression, a no-break rule matches a number of characters in a string. For example, the pattern "\s+" matches all whitespaces within the string <pre>" "</pre>.

	If a no-break rule takes effect to overwrite a break-rule that follows further down in the ~~srx~~ is defined by		Whether a no-break rule takes effect to overwrite a break rule that follows further down in the SRX is defined by

	* if it matches the part of the string, that should not break		* if it matches the part of the string, that should not break
	* AND if the position where it matches THE LAST is the same position or an earlier position than the last position, where the break-rule matches		* AND if the position where it matches THE LAST is the same position or an earlier position than the last position, where the break-rule matches

	Therefore the following ~~srx~~ will NOT ~~lead to~~ prevent ~~the~~ splitting of the sentence after "Co."		Therefore, the following SRX will NOT prevent splitting the sentence after "Co."

	<pre><?xml version="1.0" encoding="UTF-8"?>		<pre><?xml version="1.0" encoding="UTF-8"?>
Line 230:		Line 230:
	</srx></pre>		</srx></pre>

	~~But~~ this one will prevent the break:		However, this one will prevent the break:

	<pre><?xml version="1.0" encoding="UTF-8"?>		<pre><?xml version="1.0" encoding="UTF-8"?>
Line 284:		Line 284:
	</srx>		</srx>
	</pre>		</pre>
	~~while both no-break-rules match the relevant part "Co." in the sentence.~~

	[[Category:Segmentation]] [[Category:SRX]]		[[Category:Segmentation]] [[Category:SRX]]

Translate5Support at 08:17, 10 January 2026

2026-01-10T08:17:00Z

← Older revision		Revision as of 04:17, 10 January 2026
Line 166:		Line 166:

	'''Note that the framework implements a [[SRX Extensions\|few extensions to SRX]].'''		'''Note that the framework implements a [[SRX Extensions\|few extensions to SRX]].'''

			==Hint: When a no-break rule will match and when not==

			A no-break rule can (as any regex) matches a number of characters in a string. Like \s+ does match all whitespaces within this string " ".

			If a no-break rule takes effect to overwrite a break-rule that follows further down in the srx is defined by

			* if it matches the part of the string, that should not break
			* AND if the position where it matches THE LAST is the same position or an earlier position than the last position, where the break-rule matches

			Therefore the following srx will NOT lead to prevent the splitting of the sentence after "Co."

			<pre><?xml version="1.0" encoding="UTF-8"?>
			<srx xmlns="http://www.lisa.org/srx20"
			xmlns:okpsrx="http://okapi.sf.net/srx-extensions"
			version="2.0">

			<header segmentsubflows="yes" cascade="no">
			<formathandle type="start" include="no"></formathandle>
			<formathandle type="end" include="yes"></formathandle>
			<formathandle type="isolated" include="no"></formathandle>

			<okpsrx:options oneSegmentIncludesAll="no"
			trimLeadingWhitespaces="yes"
			trimTrailingWhitespaces="yes"
			useJavaRegex="yes"
			useIcu4JBreakRules="no"
			treatIsolatedCodesAsWhitespace="no">
			</okpsrx:options>

			<okpsrx:sample language="de" useMappedRules="yes">
			Die Test GmbH + Co. KG mit Sitz in Stuttgart ist cool.
			</okpsrx:sample>

			<okpsrx:rangeRule></okpsrx:rangeRule>
			</header>

			<body>
			<languagerules>
			<languagerule languagerulename="German">

			<rule break="no">
			<beforebreak>\bCo\.\s</beforebreak>
			<afterbreak></afterbreak>
			</rule>

			<!-- sentence final punctuation (incl. quotation marks) - GERMAN -->
			<rule break="yes">
			<beforebreak>\.</beforebreak>
			<afterbreak>\s</afterbreak>
			</rule>

			</languagerule>
			</languagerules>

			<maprules>
			<languagemap languagepattern="(DE\|de).*"
			languagerulename="German">
			</languagemap>
			</maprules>
			</body>

			</srx></pre>

			But this one will prevent the break:

			<pre><?xml version="1.0" encoding="UTF-8"?>
			<srx xmlns="http://www.lisa.org/srx20"
			xmlns:okpsrx="http://okapi.sf.net/srx-extensions"
			version="2.0">

			<header segmentsubflows="yes" cascade="no">
			<formathandle type="start" include="no"></formathandle>
			<formathandle type="end" include="yes"></formathandle>
			<formathandle type="isolated" include="no"></formathandle>

			<okpsrx:options oneSegmentIncludesAll="no"
			trimLeadingWhitespaces="yes"
			trimTrailingWhitespaces="yes"
			useJavaRegex="yes"
			useIcu4JBreakRules="no"
			treatIsolatedCodesAsWhitespace="no">
			</okpsrx:options>

			<okpsrx:sample language="de" useMappedRules="yes">
			Die Test GmbH + Co. KG mit Sitz in Stuttgart ist cool.
			</okpsrx:sample>

			<okpsrx:rangeRule></okpsrx:rangeRule>
			</header>

			<body>
			<languagerules>
			<languagerule languagerulename="German">

			<rule break="no">
			<beforebreak>\bCo\.</beforebreak>
			<afterbreak>\s</afterbreak>
			</rule>

			<!-- sentence final punctuation (incl. quotation marks) - GERMAN -->
			<rule break="yes">
			<beforebreak>\.</beforebreak>
			<afterbreak>\s</afterbreak>
			</rule>

			</languagerule>
			</languagerules>

			<maprules>
			<languagemap languagepattern="(DE\|de).*"
			languagerulename="German">
			</languagemap>
			</maprules>
			</body>

			</srx>
			</pre>
			while both no-break-rules match the relevant part "Co." in the sentence.

	[[Category:Segmentation]] [[Category:SRX]]		[[Category:Segmentation]] [[Category:SRX]]

Ysavourel: 1 revision imported

2016-06-04T23:19:59Z

1 revision imported

← Older revision	Revision as of 19:19, 4 June 2016
(No difference)

Ysavourel at 06:06, 20 July 2011

2011-07-20T06:06:15Z

New page

{{Standards Common Menu}}
__TOC__
==Overview==

The SRX (Segmentation Rules eXchange) format is a standard to save segmentation rules in a file so they can be used between different tools.

It originally maintained by the OSCAR special interest group of the Localisation Industry Standards Association (LISA). In March 2011 LISA was closed and its standards moved under Creative Commons license.

The version 2.0 is the latest version of the specification and can be found here: http://www.gala-global.org/oscarStandards/srx/srx20.html.

SRX rules are grouped into named sets that are activated based the code of the language of the text to process. Each rule defines the text parts before and after the inter-segment location, and specifies if the location should be a break or not. The text parts are defined using [[Regular Expressions|regular expressions]].

Example of SRX simple rules:

<pre><?xml version="1.0" encoding="UTF-8"?>
<srx xmlns="http://www.lisa.org/srx20" version="2.0">
<header segmentsubflows="yes" cascade="no">
<formathandle type="start" include="no"></formathandle>
<formathandle type="end" include="yes"></formathandle>
<formathandle type="isolated" include="no"></formathandle>
</header>
<body>
<languagerules>
<languagerule languagerulename="default">
<rule break="no">
<beforebreak>([A-Z]\.){2,}</beforebreak>
<afterbreak>\s</afterbreak>
</rule>
<rule break="yes">
<beforebreak>\.</beforebreak>
<afterbreak>\s</afterbreak>
</rule>
</languagerule>
</languagerules>
<maprules>
<languagemap languagepattern=".*" languagerulename="default"></languagemap>
</maprules>
</body>
</srx></pre>

In this example, there are two rules.

The second one specifies that when an inter-character location is preceded by a period and followed by a white space, the rule is to break at that position.

The first rule specifies that when an inter-character location is preceded by the patter <code>([A-Z]\.){2,}</code> and followed by a white paces, the rule is to not break at that position. Because the first rule is placed before the second rule it takes precedence.

So, based on those rules, the following text:

I'm in the U.K. for now. But I plan to move to Papua New Guinea.

will break down into two segments:

[I'm in the U.K. for now.]
[ But I plan to move to Papua New Guinea.]

If the first rule was not there, it would break down into three segments:

[I'm in the U.K.]
[ for now.]
[ But I plan to move to Papua New Guinea.]

==SRX Versions Issue==

There are two versions of SRX: 1.0 and 2.0.

SRX version 1.0 has been implemented by several tools that interpreted how to process the SRX rules in different ways. As a result the same SRX 1.0 document used on different tools may give you different segmentation.

To resolve this issue, an updated version 2.0 specification has been published and provides better implementation guidelines. So, in theory, the same version 2.0 document should give you the same segmentation in all tools.

You can find the specifications of SRX on the LISA web site:

* SRX 1.0: http://www.gala-global.org/oscarStandards/srx/srx10.html
* SRX 2.0: http://www.gala-global.org/oscarStandards/srx/srx20.html

===Implementation Differences for SRX 1.0===

There are two main types of implementations of SRX 1.0: the intended one, and one that use a cascading matching of the language maps.

Tools like SDLX implemented the intended SRX 1.0 behavior (non-cascading). Others, like Swordfish implemented SRX 1.0 with a cascading behavior.

In an SRX document, the segmentation rules are grouped into several <code><languagerule></code> elements. This way you can define different sets of rules that you apply for different languages. The select of which group of rules is to use for a given language is driven by a table defined in the <maprules> element. Each entry in <code><maprules></code> is a <code><languagemap></code>. This entry has two information: a regular expression pattern that corresponds to what language code should use the entry, and a pointer to the group of rules for this entry.

<pre><languagerules>
<languagerule languagerulename='default'>
</languagerule>
<languagerule languagerulename='japanese'>
</languagerule>
<languagerules>

<maprules>
<languagemap languagepattern='ja.*' languagerulename='japanese'/>
<languagemap languagepattern='.*' languagerulename='default'/>
</maprules></pre>

The difference between the SRX 1.0 implementations is how they lookup the <code><maprules></code> for a given language code.

# Some will use only the first <code><languagemap></code> that has a languagepattern matching the language code.
# Other will use all <code><languagemap></code> that have a <code>languagepattern</code> matching the language code.

The first interpretation is the correct one: In SRX 1.0 you use the only first <code><languagemap></code> that matches the given language code.

It is true that there is nothing in the SRX 1.0 specification that says explicitly it should work that way. But there is also nothing explicitly (or implicitly) that says all matching <code><languagemap></code> should be used.

The clue to the intended behavior is in the example of the SRX 1.0 specification:

<languagerules>
<languagerule languagerulename="Default">
<span class="blue"> <rule break="no">
<beforebreak>^\s*[0-9]+\.</beforebreak>
<afterbreak>\s</afterbreak>
</rule></span>
<span class="green"> <rule break="no">
<beforebreak>[Ee][Tt][Cc]\.</beforebreak>
<afterbreak>\s[a-z]</afterbreak>
</rule></span>
...
</languagerule>
<languagerule languagerulename="Japanese">
<span class="blue"> <rule break="no">
<beforebreak>^\s*[0-9]+\.</beforebreak>
<afterbreak>\s</afterbreak>
</rule></span>
<span class="green"> <rule break="no">
<beforebreak>[Ee][Tt][Cc]\.</beforebreak>
<afterbreak></afterbreak>
</rule></span>
<rule break="yes">
<beforebreak>[\xff61\x3002\xff0e\xff1f\xff01]+</beforebreak>
<afterbreak></afterbreak>
</rule>
...
</languagerule>
</languagerules>

<maprules>
<maprule maprulename="Default">
<languagemap languagepattern="JA.*" languagerulename="Japanese"/>
<languagemap languagepattern=".*" languagerulename="Default"/>
</maprule>
</maprules>

In this example, there is the same rules defined in both the Default and the Japanese groups. If SRX 1.0 intended to use all the <code><languagemap></code> elements that match the given language code, there would be no point to have duplicated rules in Japanese. The Japanese group would have only the extra Japanese-specific rules.

===How to Convert From SRX 1.0 to SRX 2.0?===

The SRX 2.0 specification resolve the cascading issue by making it an option.

When loading or importing SRX 1.0 documents into an SRX 2.0 editor, you must be careful about setting properly the cascade option depending on the provenance of the document.

* SRX 1.0 rules coming from Trados, SDLX and some other tools that implement the normal SRX 1.0 behavior (no cascading). So you should make sure that option is not set after you open the file.

* SRX 1.0 rules coming from Heartsome, Swordfish, and some other tools that are designed with cascading. So you should make sure that option is set after you have open the file.

==SRX and Java==

The SRX standard uses ICU regular expressions, however it is very difficult to implement the same set of expression using Java and some other programing languages.

See more details in the [[SRX and Java]] section.

==SRX in the Okapi Framework==

The Okapi framework uses SRX in many places. For example:

* [[Ratel]] is an application to edit SRX rules in WYSIWYG mode.
* Steps like the [[Segmentation Step]], the [[Sentence Alignment Step]] or the [[Batch Translation Step]] use SRX rules.

'''Note that the framework implements a [[SRX Extensions|few extensions to SRX]].'''

[[Category:Segmentation]] [[Category:SRX]]