Okapi Framework - User contributions [en]

JSON Filter

2018-11-07T18:44:32Z

Kuro: <note> support added

{{Filters Header}}
==Overview==

The JSON Filter is an Okapi component that implements the IFilter interface for JSON (Javascript Object Notation).

The implementation is based on the JSON specifications: http://www.json.org/

The following is an example of a very simple JSON file. The translatable text is highlighted:

{"menu": {
"value": "File",
"popup": {
"menuitem": [
{"value": "New"},
{"value": "Open"},
{"value": "Close"}
]
}
}}

==Processing Details==

===Input Encoding===

JSON files are normally in one of the Unicode encoding, but the filter supports any encoding. It decides which encoding to use for the input file using the following logic:

* If the file has a Unicode Byte-Order-Mark:
** Then, the corresponding encoding (e.g. UTF-8, UTF-16, etc.) is used.
* Else, if a header entry with a <code>charset</code> declaration exists in the first 1000 characters of the file:
** If the value of the <code>charset</code> is "<code>charset</code>" (case insensitive):
*** Then the file is likely to be a template with no encoding declared, so the current encoding (auto-detected or default) is used.
*** Else, the declared encoding is used. Note that if the encoding has been detected from a Byte-Order-Mark and the encoding declared in the header entry does not match, a warning is generated and the encoding of the Byte-Order-Mark is used.
* Otherwise, the input encoding used is the default encoding that was specified when setting the filter options.

===Output Encoding===

If the output encoding is UTF-8:

* If the input encoding was also UTF-8, a Byte-Order-Mark is used for the output document only if one was detected in the input document.
* If the input encoding was not UTF-8, no Byte-Order-Mark is used in the output document.

===Line-Breaks===

The type of line-breaks of the output is the same as the one of the original input.

===Comments===

Though not technically legal in JSON these comment types are supported:
<code>
* // comment
* # comment
* /* comment */
* 
</code>

==Parameters==

=== Options Tab===

====Stand-alone strings====

<cite>Extract strings without associated key</cite> — Set this option to extract string that are not associated directly to a key value.

====Strings with keys====

<cite>Extract all key/strings pairs</cite> — Set this option to extract all strings that have a key associated. If a regular expression for exceptions is defined, the strings that have a key matching the expression are not extracted.

<cite>Do not extract key/string pairs</cite> — Set the option to not extract any string that has an associated key. If a regular expression for exceptions is defined, the strings that have a key matching the expression are extracted.

<cite>Excepted when the key matches the following regular expression</cite> — Enter a regular expression that correspond to the keys that should have a behavior inverse to the default behavior you have selected for the key/strings pairs.

<cite>Use the key as the resname</cite> — Set this option to use the value of the key as the value of the name of the extracted item (<code>resname</code> in XLIFF).

<cite>Use the full key path</cite> — Set this option to use the full key path in the <code>resname</code>. For example: <code>/menu/value/popup/menuitem/value</code>. The use key name as resname option must be set for this option to take effect. If enabled, exception regular expressions apply to the full path.

<cite>Include leading "/" on key path</cite> — Set this option to have a leading character '/' in the full key path.

<cite>Comma-separated list of simple keys, values of which to appear as <note> in XLIFF</cite> — Specify a list of keys separated by a comma. The value of the key-value pairs matching the specified keys will be transferred to <note> elements in XLIFF. (This is available in M37 dev build after mid October, 2018.)

===Content Processing Tab===

<cite>Process text content with this sub-filter</cite> — Specify an Okapi filter ID (e.g. <code>okf_html</code>) to process the content of all translatable text with that filter. Leave this field blank for default behavior.

<cite>Find inline codes by patterns defined below</cite> — Set this option to use the specified regular expressions on the text of the extracted items. Any match will be converted to an inline code.

'''Note:''' This option cannot be used together with the sub-filtering option.

By default the expression is:

((%(([-0+#]?)[-0+#]?)((\d\$)?)(([\d\*]*)(\.[\d\*]*)?)[dioxXucsfeEgGpn])
|((\\r\\n)|\\a|\\b|\\f|\\n|\\r|\\t|\\v)
|(\{\d.*?\}))

{{CodeFinder Help}}

==Limitations==

Comments within a JSON string are parsed as part of the string content, not as comments. A configured subfilter will then process these as true comments (they will become part of the skeleton or whatever the filter is configured to do).
[[Category:Filters]]

Longhorn

2018-08-13T22:56:43Z

Kuro: Java version requirement updated

__TOC__
==Overview==

Longhorn is a server application that allows you to execute Batch Configurations remotely on any set of input files. Batch Configurations which include pre-defined pipelines and filter configurations, can be exported from [[Rainbow]].

The distribution also includes a client library to access the Longhorn Web services.

==Download and Installation==

* '''Stable release: http://bintray.com/okapi/Distribution/Longhorn

* <del>Development release (snapshot): http://okapiframework.org/snapshots</del> Development snapshots are not currently available.

To install Longhorn:

* Unzip the distribution file on your server.
* Follow the instructions provided with the <code>readme</code> file of the distribution.
* Starting with M36, Longhorn requires Java 1.8.

==Functionality==

To process files with Longhorn these steps are required:
# Create a temporary project
# Upload a Batch Configuration file into that project
# Upload the input files into that project
# Execute the project
# Download the output files
# Delete the project

==Usage==

There are three ways to access Longhorns functionality. There is
* a REST interface,
* a Java API and
* an HTML client.

They can be used as described below.

===REST-Interface===

Longhorn can be accessed directly via HTTP methods:
;POST http://{host}/okapi-longhorn/projects/new : Creates a new temporary project and returns its URI (e.g. <code>http://localhost/okapi-longhorn/projects/1</code>) in the <tt>Location</tt> header of the response.
;POST http://{host}/okapi-longhorn/projects/1/batchConfiguration : Uploads a Batch Configuration file (must be given as multipart/form-data)
;POST http://{host}/okapi-longhorn/projects/1/inputFiles.zip : Adds input files as a zip archive (the zip will be extracted and the included files will be used as input files)
;PUT http://{host}/okapi-longhorn/projects/1/inputFiles/help.html : Uploads a file that will have the name 'help.html'
;GET http://{host}/okapi-longhorn/projects/1/inputFiles/help.html: Retrieve an input file that was previously added with PUT or POST
;POST http://{host}/okapi-longhorn/projects/1/tasks/execute : Executes the Batch Configuration on the uploaded input files
;POST http://{host}/okapi-longhorn/projects/1/tasks/execute/en-US/de-DE : Executes the Batch Configuration on the uploaded input files with the source language set to 'en-US' and the target language set to 'de-DE'
;POST http://{host}/okapi-longhorn/projects/1/tasks/execute/en-US?targets=de-DE&targets=fr-FR : Executes the Batch Configuration on the uploaded input files with the source language set to 'en-US' and multiple target languages, 'de-DE' and 'fr-FR'
;GET http://{host}/okapi-longhorn/projects/1/outputFiles : Returns a list of the output files generated
;GET http://{host}/okapi-longhorn/projects/1/outputFiles/help.out.html : Accesses the output file 'help.out.html' directly
;GET http://{host}/okapi-longhorn/projects/1/outputFiles.zip : Returns all output files in a zip archive
;DEL http://{host}/okapi-longhorn/projects/1 : Deletes the project
;GET http://{host}/okapi-longhorn/projects : Returns a list of all projects on the server

===REST-Interface Sample code: Python===

This example works with the requests package - minidom is used to parse the XML project list.

import requests
from xml.dom import minidom

url = 'http://localhost:8080/okapi-longhorn/'

Code to create a new project

r = requests.post(url+'projects/new')
print r.text

Code to '''list''' existing projects (i.e.: to check if the project was created, and to get the ID of the last project)

r = requests.get(url+'projects/')

xmlstring = minidom.parseString(r.text)
itemlist = xmlstring.getElementsByTagName('e')
lastproject = len(itemlist)

Code to '''post''' a '''batch config file'''

batchfile = open('/home/user/batchconfig.bconf', 'rb')
r = requests.post(url+'projects/'+str(lastproject)+'/batchConfiguration', files=dict(batchConfiguration=batchfile))

Code to '''put''' a string as a '''file'''

payload = "hello world!"
r = requests.put(url+'projects/'+str(lastproject)+'/inputFiles/test.txt', files=dict(inputFile=payload))

Code to '''post''' a '''file'''

payload = open('/home/user/test.txt', 'rb')
r = requests.post(url+'projects/'+str(lastproject)+'/inputFiles/test.txt', files=dict(inputFile=payload))

===REST-Interface access by curl===
Longhorn API can be accessed with the curl command, as shown below. Below, '-X GET' is not necessary but used there for clarity.

$ curl -X POST -i 'http://localhost:8080/okapi-longhorn/projects/new'

This cretes a new project. The "Location:" header in the response
shows the URL for managing this project.

$ curl -X POST -F batchConfiguration=@tmp/batchconfig.bconf 'http://localhost:8080/okapi-longhorn/projects/1/batchConfiguration'

This submits the batch config file exported from Rainbow.

$ curl -X POST -F inputFile=@tmp/commonmark_original.md 'http://localhost:8080/okapi-longhorn/projects/1/inputFiles/commonmark.md'

This sends a raw file to be extracted.

$ curl -X POST 'http://localhost:8080/okapi-longhorn/projects/1/tasks/execute'

Longhorn executes the project after receiving this.

$ curl -X GET 'http://localhost:8080/okapi-longhorn/projects/1/outputFiles'

Longhorn returns the list of output files. For example:

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<l>
<e>pack1/manifest.rkm</e>
<e>pack1/original/commonmark.md</e>
<e>pack1/work/commonmark.md.xlf</e>
</l>

$ curl -X GET 'http://localhost:8080/okapi-longhorn/projects/1/outputFiles/pack1/work/commonmark.md.xlf'

You can obtain obtain one of the output files like this.

$ curl -X GET -o out.zip 'http://localhost:8080/okapi-longhorn/projects/1/outputFiles.zip'

Or get all the output files in a zip file.

$ curl -X DELETE 'http://localhost:8080/okapi-longhorn/projects/1'

This ends the project.

===Java API===

The API is distributed as a <code>.jar</code> file in the Longhorn distribution package. You can also build it from the Okapi source code via Maven from the project <code>lib-longhorn-api</code>.

====Maven====
The API is available as a maven dependency. Add this repository to your <tt>pom.xml</tt>:
<repository>
<id>okapi-longhorn-release</id>
<name>Okapi Longhorn Release</name>
<url>http://repository-opentag.forge.cloudbees.com/release/</url>
</repository>

Along with this dependency, substituting in a valid version number (e.g, <tt>0.27</tt>):
<dependency>
<groupId>net.sf.okapi.lib</groupId>
<artifactId>okapi-lib-longhorn-api</artifactId>
<version>${okapi.version}</version>
</dependency>

====Sample Code====

LonghornService ws = new RESTService(new URI("http://localhost:9095/okapi-longhorn"));

// Create project
LonghornProject proj = ws.createProject();

// Post batch configuration
File bconfFile = new File("C:\\setup.bconf");
proj.addBatchConfiguration(bconfFile);

// Send input files

// First by single upload...
File file1 = new File("C:\\help.html");
// * in the root directory
proj.addInputFile(file1, file1.getName());
// * and in a sub-directory
proj.addInputFile(file1, "samefile/" + file1.getName());

// ...then by package upload
File inputPackage = new File("C:\\more_files.zip");
proj.addInputFilesFromZip(inputPackage);

// Execute pipeline
// Languages don't matter
proj.executePipeline();
// Languages matter
proj.executePipeline("en-US", "de-DE");

// Get output files
ArrayList<LonghornFile> outputFiles = proj.getOutputFiles();

// Does the fetching of files work?
for (LonghornFile of : outputFiles) {
InputStream is = of.openStream();
//TODO save InputStream to local file
}

// Delete project
proj.delete();

===HTML-Client===

You can create projects and upload/download files via an integrated HTML client, too. Uploading input files (and downloading output files) as a zip archive is currently not implemented for the HTML client.

[[File:longhorn_html_client.png]]

===Configuration===
Since Okapi M22 Okapi Longhorn can be build to run multiple instances on one server.
You can adjust the build so that it is possible to run multiple Longhorn instances in one JBoss application server. Therefore, the build must be called with an additional parameter:

mvn clean verify -DuseUniqueContextRoot

====Configure working directory path====
Longhorn has 2 options to configure the working directory of longhorn (sort by priority):
#system parameter "LONGHORN_WORKDIR"
#configuration file in user.home "/okapi-longhorn-configuration.xml"
If nothing is defined, the working-directory is in user.home in folder "Okapi-Longhorn-Files".
Longhorn configuration file example:

<longhorn-config>
<use-unique-working-directory>True</use-unique-working-directory>
<working-directory>D:\testData\longhorn-files</working-directory>
</longhorn-config>

====Configuration Options====

{| class="wikitable"
! option
! description
! data type
|-
| working-directory
| path of the working directory
| string
|-
| use-unique-working-directory
| if set to true the version of longhorn will be added to working directory name
e.g path/to/working/directory_M0.21
| boolean(True or False)
|}

[[Category:Longhorn]]

Longhorn

2018-08-13T22:53:32Z

Kuro: Adding curl examples

__TOC__
==Overview==

Longhorn is a server application that allows you to execute Batch Configurations remotely on any set of input files. Batch Configurations which include pre-defined pipelines and filter configurations, can be exported from [[Rainbow]].

The distribution also includes a client library to access the Longhorn Web services.

==Download and Installation==

* '''Stable release: http://bintray.com/okapi/Distribution/Longhorn

* <del>Development release (snapshot): http://okapiframework.org/snapshots</del> Development snapshots are not currently available.

To install Longhorn:

* Unzip the distribution file on your server.
* Follow the instructions provided with the <code>readme</code> file of the distribution.
* Starting with m24, Longhorn requires Java 1.7.

==Functionality==

To process files with Longhorn these steps are required:
# Create a temporary project
# Upload a Batch Configuration file into that project
# Upload the input files into that project
# Execute the project
# Download the output files
# Delete the project

==Usage==

There are three ways to access Longhorns functionality. There is
* a REST interface,
* a Java API and
* an HTML client.

They can be used as described below.

===REST-Interface===

Longhorn can be accessed directly via HTTP methods:
;POST http://{host}/okapi-longhorn/projects/new : Creates a new temporary project and returns its URI (e.g. <code>http://localhost/okapi-longhorn/projects/1</code>) in the <tt>Location</tt> header of the response.
;POST http://{host}/okapi-longhorn/projects/1/batchConfiguration : Uploads a Batch Configuration file (must be given as multipart/form-data)
;POST http://{host}/okapi-longhorn/projects/1/inputFiles.zip : Adds input files as a zip archive (the zip will be extracted and the included files will be used as input files)
;PUT http://{host}/okapi-longhorn/projects/1/inputFiles/help.html : Uploads a file that will have the name 'help.html'
;GET http://{host}/okapi-longhorn/projects/1/inputFiles/help.html: Retrieve an input file that was previously added with PUT or POST
;POST http://{host}/okapi-longhorn/projects/1/tasks/execute : Executes the Batch Configuration on the uploaded input files
;POST http://{host}/okapi-longhorn/projects/1/tasks/execute/en-US/de-DE : Executes the Batch Configuration on the uploaded input files with the source language set to 'en-US' and the target language set to 'de-DE'
;POST http://{host}/okapi-longhorn/projects/1/tasks/execute/en-US?targets=de-DE&targets=fr-FR : Executes the Batch Configuration on the uploaded input files with the source language set to 'en-US' and multiple target languages, 'de-DE' and 'fr-FR'
;GET http://{host}/okapi-longhorn/projects/1/outputFiles : Returns a list of the output files generated
;GET http://{host}/okapi-longhorn/projects/1/outputFiles/help.out.html : Accesses the output file 'help.out.html' directly
;GET http://{host}/okapi-longhorn/projects/1/outputFiles.zip : Returns all output files in a zip archive
;DEL http://{host}/okapi-longhorn/projects/1 : Deletes the project
;GET http://{host}/okapi-longhorn/projects : Returns a list of all projects on the server

===REST-Interface Sample code: Python===

This example works with the requests package - minidom is used to parse the XML project list.

import requests
from xml.dom import minidom

url = 'http://localhost:8080/okapi-longhorn/'

Code to create a new project

r = requests.post(url+'projects/new')
print r.text

Code to '''list''' existing projects (i.e.: to check if the project was created, and to get the ID of the last project)

r = requests.get(url+'projects/')

xmlstring = minidom.parseString(r.text)
itemlist = xmlstring.getElementsByTagName('e')
lastproject = len(itemlist)

Code to '''post''' a '''batch config file'''

batchfile = open('/home/user/batchconfig.bconf', 'rb')
r = requests.post(url+'projects/'+str(lastproject)+'/batchConfiguration', files=dict(batchConfiguration=batchfile))

Code to '''put''' a string as a '''file'''

payload = "hello world!"
r = requests.put(url+'projects/'+str(lastproject)+'/inputFiles/test.txt', files=dict(inputFile=payload))

Code to '''post''' a '''file'''

payload = open('/home/user/test.txt', 'rb')
r = requests.post(url+'projects/'+str(lastproject)+'/inputFiles/test.txt', files=dict(inputFile=payload))

===REST-Interface access by curl===
Longhorn API can be accessed with the curl command, as shown below. Below, '-X GET' is not necessary but used there for clarity.

$ curl -X POST -i 'http://localhost:8080/okapi-longhorn/projects/new'

This cretes a new project. The "Location:" header in the response
shows the URL for managing this project.

$ curl -X POST -F batchConfiguration=@tmp/batchconfig.bconf 'http://localhost:8080/okapi-longhorn/projects/1/batchConfiguration'

This submits the batch config file exported from Rainbow.

$ curl -X POST -F inputFile=@tmp/commonmark_original.md 'http://localhost:8080/okapi-longhorn/projects/1/inputFiles/commonmark.md'

This sends a raw file to be extracted.

$ curl -X POST 'http://localhost:8080/okapi-longhorn/projects/1/tasks/execute'

Longhorn executes the project after receiving this.

$ curl -X GET 'http://localhost:8080/okapi-longhorn/projects/1/outputFiles'

Longhorn returns the list of output files. For example:

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<l>
<e>pack1/manifest.rkm</e>
<e>pack1/original/commonmark.md</e>
<e>pack1/work/commonmark.md.xlf</e>
</l>

$ curl -X GET 'http://localhost:8080/okapi-longhorn/projects/1/outputFiles/pack1/work/commonmark.md.xlf'

You can obtain obtain one of the output files like this.

$ curl -X GET -o out.zip 'http://localhost:8080/okapi-longhorn/projects/1/outputFiles.zip'

Or get all the output files in a zip file.

$ curl -X DELETE 'http://localhost:8080/okapi-longhorn/projects/1'

This ends the project.

===Java API===

The API is distributed as a <code>.jar</code> file in the Longhorn distribution package. You can also build it from the Okapi source code via Maven from the project <code>lib-longhorn-api</code>.

====Maven====
The API is available as a maven dependency. Add this repository to your <tt>pom.xml</tt>:
<repository>
<id>okapi-longhorn-release</id>
<name>Okapi Longhorn Release</name>
<url>http://repository-opentag.forge.cloudbees.com/release/</url>
</repository>

Along with this dependency, substituting in a valid version number (e.g, <tt>0.27</tt>):
<dependency>
<groupId>net.sf.okapi.lib</groupId>
<artifactId>okapi-lib-longhorn-api</artifactId>
<version>${okapi.version}</version>
</dependency>

====Sample Code====

LonghornService ws = new RESTService(new URI("http://localhost:9095/okapi-longhorn"));

// Create project
LonghornProject proj = ws.createProject();

// Post batch configuration
File bconfFile = new File("C:\\setup.bconf");
proj.addBatchConfiguration(bconfFile);

// Send input files

// First by single upload...
File file1 = new File("C:\\help.html");
// * in the root directory
proj.addInputFile(file1, file1.getName());
// * and in a sub-directory
proj.addInputFile(file1, "samefile/" + file1.getName());

// ...then by package upload
File inputPackage = new File("C:\\more_files.zip");
proj.addInputFilesFromZip(inputPackage);

// Execute pipeline
// Languages don't matter
proj.executePipeline();
// Languages matter
proj.executePipeline("en-US", "de-DE");

// Get output files
ArrayList<LonghornFile> outputFiles = proj.getOutputFiles();

// Does the fetching of files work?
for (LonghornFile of : outputFiles) {
InputStream is = of.openStream();
//TODO save InputStream to local file
}

// Delete project
proj.delete();

===HTML-Client===

You can create projects and upload/download files via an integrated HTML client, too. Uploading input files (and downloading output files) as a zip archive is currently not implemented for the HTML client.

[[File:longhorn_html_client.png]]

===Configuration===
Since Okapi M22 Okapi Longhorn can be build to run multiple instances on one server.
You can adjust the build so that it is possible to run multiple Longhorn instances in one JBoss application server. Therefore, the build must be called with an additional parameter:

mvn clean verify -DuseUniqueContextRoot

====Configure working directory path====
Longhorn has 2 options to configure the working directory of longhorn (sort by priority):
#system parameter "LONGHORN_WORKDIR"
#configuration file in user.home "/okapi-longhorn-configuration.xml"
If nothing is defined, the working-directory is in user.home in folder "Okapi-Longhorn-Files".
Longhorn configuration file example:

<longhorn-config>
<use-unique-working-directory>True</use-unique-working-directory>
<working-directory>D:\testData\longhorn-files</working-directory>
</longhorn-config>

====Configuration Options====

{| class="wikitable"
! option
! description
! data type
|-
| working-directory
| path of the working directory
| string
|-
| use-unique-working-directory
| if set to true the version of longhorn will be added to working directory name
e.g path/to/working/directory_M0.21
| boolean(True or False)
|}

[[Category:Longhorn]]

Markdown Filter

2018-05-11T22:41:00Z

Kuro: /* Processing Details */

{{Filters Header}}
==Overview==

The Markdown Filter is an Okapi component for extracting translatable text from Markdown files. See https://en.wikipedia.org/wiki/Markdown for more information about the format.
Markdown is a family of formats, not all of them mutually compatible. This filter is designed to work with markdown based on the [http://commonmark.org CommonMark] specification, with additional features to support [https://guides.github.com/features/mastering-markdown/ GitHub-flavored Markdown].

==Processing Details==

===Input Encoding===

The filter decides which encoding to use for the input file using the following logic:

If the file has a Unicode Byte-Order-Mark:
Then, the corresponding encoding (e.g. UTF-8, UTF-16, etc.) is used.
Otherwise, the input encoding used is the default encoding that was specified when setting the filter options.

===HTML Elements===
The HTML Inline Elements, i.e. the tags, and the HTML Block, a chunk of text sandwiched between a block-forming start tag and its corresponding end tag, are processed by the HTML filter. The HTML filter to use can be customized separately.

===Inline Codes===
The [[HTML_Filter#Inline_Code_Finder|Inline Code Finder]] is supported by this filter.

The subfilter applies to the translatable text within the proper part of Markdown document. It does not apply to the HTML inline tags or HTML blocks. For that, you would need to enable and specify the inline code pattern for the HTML filter separately, name the configuration as okf_html@''arbitary-name''.fprm, and specify that name for the htmlSubfilter parameter.

Note, the support of the Inline Code Finder was temporarily unavailable in some snapshot builds of version 0.36, but it has been restored.

==Parameters==

; Translate URLs (translateUrls)
: By default, URLs in link and image statements are not exposed for translation. If this option is enabled, they will be extracted. Note: URLs are currently extracted inline in their containing segment, rather than as a subflow. Default: false

; Translate Code Blocks (translateCodeBlocks)
: This option controls whether the contents of fenced code blocks are exposed for translation. Default: true

; Translate YAML Metadata Header (translateImageAltText)
: Some markdown formats support a [http://pandoc.org/MANUAL.html#extension-yaml_metadata_block YAML Metadata Header] that contains key/value data. By default, this header is not exposed for translation. When the "Translate YAML Metadata Header" option is enabled, the header will be parsed and the metadata values will be exposed for translation. Default: false

; Translate Image Alt Text (translateImageAltText)
: The alt text for a graphic image in the form of <nowiki>![alt text](https://foo.com/images/bar.jpg)</nowiki> or as the alt attribute of an img tag <nowiki><img src="https://foo.com/images/bar.jpg" alt="alt text"></nowiki> will be extracted if this parameter is true. Default: true.

; HTML Subfilter Configuration ID (htmlSubfilter)
: The custom configuration ID of the HTML filter that will be called to process HTML contents within Markdown documents. The configuration file must be saved in a known location with ''.fprm'' suffix. Specify nothing to use the default HTML filter configuration tailored for the Markdown filter. Default: (empty)

; Use Code Finder (useCodeFinder)
: Determines whether to use the Inline Code Finder or not. Default: false

; Number of Code Finder Rules (codeFinderRules.count)
: The number of rules, i.e. regular expression patterns. Default: 1

; Code Finder Rule N (codeFinderRules.rule``N``)
: N'th matching pattern for codes.

; Sample Text (codeFinderRules.sample)
: Sample text to test the rules on UI.

; Use All Rules (codeFinderRules.useAllRulesWhenTesting)
: Determines whether to apply all rules when testing on UI.

==Limitations==

=== Subflows are Not Supported ===

When there is a subflow of text in the middle of the main text, the subflow will be inter-mixed with the main flow of text. For example, for this run of Markdown text:

<nowiki>
Please click ![The Information desk logo](images/circled-i.jpg) for help.
</nowiki>

The extracted text in the XLIFF file will look like this:

<nowiki>
Please click <x id="1"/>The Information desk logo<x id="2/> for help.
</nowiki>

[[Category:Filters]]

Markdown Filter

2018-05-11T22:38:12Z

Kuro:

{{Filters Header}}
==Overview==

The Markdown Filter is an Okapi component for extracting translatable text from Markdown files. See https://en.wikipedia.org/wiki/Markdown for more information about the format.
Markdown is a family of formats, not all of them mutually compatible. This filter is designed to work with markdown based on the [http://commonmark.org CommonMark] specification, with additional features to support [https://guides.github.com/features/mastering-markdown/ GitHub-flavored Markdown].

==Processing Details==

===Input Encoding===

The filter decides which encoding to use for the input file using the following logic:

If the file has a Unicode Byte-Order-Mark:
Then, the corresponding encoding (e.g. UTF-8, UTF-16, etc.) is used.
Otherwise, the input encoding used is the default encoding that was specified when setting the filter options.

===Inline Codes===
The [[HTML_Filter#Inline_Code_Finder|Inline Code Finder]] is supported by this filter.

The subfilter applies to the translatable text within the proper part of Markdown document. It does not apply to the HTML inline tags or HTML blocks. For that, you would need to enable and specify the inline code pattern for the HTML filter separately, name the configuration as okf_html@''arbitary-name''.fprm, and specify that name for the htmlSubfilter parameter.

Note, the support of the Inline Code Finder was temporarily unavailable in some snapshot builds of version 0.36, but it has been restored.

==Parameters==

; Translate URLs (translateUrls)
: By default, URLs in link and image statements are not exposed for translation. If this option is enabled, they will be extracted. Note: URLs are currently extracted inline in their containing segment, rather than as a subflow. Default: false

; Translate Code Blocks (translateCodeBlocks)
: This option controls whether the contents of fenced code blocks are exposed for translation. Default: true

; Translate YAML Metadata Header (translateImageAltText)
: Some markdown formats support a [http://pandoc.org/MANUAL.html#extension-yaml_metadata_block YAML Metadata Header] that contains key/value data. By default, this header is not exposed for translation. When the "Translate YAML Metadata Header" option is enabled, the header will be parsed and the metadata values will be exposed for translation. Default: false

; Translate Image Alt Text (translateImageAltText)
: The alt text for a graphic image in the form of <nowiki>![alt text](https://foo.com/images/bar.jpg)</nowiki> or as the alt attribute of an img tag <nowiki><img src="https://foo.com/images/bar.jpg" alt="alt text"></nowiki> will be extracted if this parameter is true. Default: true.

; HTML Subfilter Configuration ID (htmlSubfilter)
: The custom configuration ID of the HTML filter that will be called to process HTML contents within Markdown documents. The configuration file must be saved in a known location with ''.fprm'' suffix. Specify nothing to use the default HTML filter configuration tailored for the Markdown filter. Default: (empty)

; Use Code Finder (useCodeFinder)
: Determines whether to use the Inline Code Finder or not. Default: false

; Number of Code Finder Rules (codeFinderRules.count)
: The number of rules, i.e. regular expression patterns. Default: 1

; Code Finder Rule N (codeFinderRules.rule``N``)
: N'th matching pattern for codes.

; Sample Text (codeFinderRules.sample)
: Sample text to test the rules on UI.

; Use All Rules (codeFinderRules.useAllRulesWhenTesting)
: Determines whether to apply all rules when testing on UI.

==Limitations==

=== Subflows are Not Supported ===

When there is a subflow of text in the middle of the main text, the subflow will be inter-mixed with the main flow of text. For example, for this run of Markdown text:

<nowiki>
Please click ![The Information desk logo](images/circled-i.jpg) for help.
</nowiki>

The extracted text in the XLIFF file will look like this:

<nowiki>
Please click <x id="1"/>The Information desk logo<x id="2/> for help.
</nowiki>

[[Category:Filters]]

FAQ

2018-05-11T21:48:03Z

Kuro: Fixing Yahoo! group URL

==Capabilities==

====What formats are supported?====

The framework offers filters for many file formats, including XML, XLIFF, TMX, HTML, DOCX, ODT, Properties, PO, and many more. 
For a more complete list of the supported formats, see the "[[Filters]]" page.

Note that you can also create your own filter configurations to support some formats. You can also create your own filters and use them seamlessly with the Okapi tools.

====How do I extract text for translation?====

See the article "[[How to Extract Text for Translation]]" in the [[Knowledge Base]].

====Does Okapi provide a translation editor?====

Not at this time. The Okapi tools allow you to create translation packages in various formats that can be opened in different translation editors such as OmegaT, MemoQ, Trados Workbench, Swordfish, Wordfast, etc.

For translating XLIFF files see: "[[How to Translate XLIFF Documents]]".

====Does Okapi provide a TM (Translation Memory)?====

Yes. There are currently two TM engines implemented in the framework:

* [[Pensieve TM]] is the main TM engine.
* [[SimpleTM TM]] is a limited and older engine that '''is being progressively phased out'''.

You can also use third-part TM engines through the the different [[Connectors|connectors]] that the framework provides. For example: the [[Translate Toolkit TM Connector|Translate Toolkit TM]], [[GlobalSight TM Connector|GlobalSight TM]], the [[OpenTran Translation Repository Connector|OpenTran Translation Repository]], [[MyMemory TM Connector|MyMemory]], etc. For a complete list and more details see the "[[Connectors]]" page.

====Does Okapi provide a MT (Machine Translation) system?====

Not at this time. But you can use different third-party MT system using one of the connectors distributed with the framework. For example you can work with [[Google MT v2 Connector|Google MT]], [[Apertium MT Connector|Apertium MT]], [[Microsoft Translator Connector|Microsoft Translator]], etc. For a complete list, see the [[Connectors|Connectors page]].

====Why is there several distributions, isn't Java cross-platform?====

Yes, Java is cross-platform, and most of the Okapi code runs anywhere Java runs.
However, for a better internationalization support and a more seamless integration with each platform, we have selected to use Eclipse SWT (http://www.eclipse.org/swt) as the foundation for the UI of our applications. That library requires a different distribution for each platform and architecture.

Okapi's source code has been carefully designed to separate UI-dependant code and non-UI code, so most of the components (such as the [[Filters]], the [[Steps]] and the [[Connectors]]) can be used on any platform.

====Can I change the Java VM settings when running the tools?====

Yes. See [[How to Change the Java Parameters for Rainbow]]. You can follow the same steps for all Okapi tools.

==Simple Troubleshooting==

====Is there a Getting Started guide?====

Yes. See the "[[Getting Started]]" page.

====When I try to start Rainbow/Ratel/CheckMate nothing happens. What is wrong?====

* Check that you have the proper version of Java (1.7 or above).
* Make sure you have installed the correct distribution for your platform.
* If your machine is 32-bit make sure to have installed the 32-bit distribution.
* If your machine is 64-bit make sure to have installed the 64-bit distribution.

==Licenses==

====Under what licence the Okapi Framework is developed?====

* The source code is under [https://www.apache.org/licenses/LICENSE-2.0 Apache Licence version 2.0].
* The documentation is under [http://creativecommons.org/licenses/by-sa/3.0/ Creative Commons Attribution-ShareAlike License (CC-BY-SA)].

====Can I use Okapi's components in my applications?====

Yes. The project uses the Apache license which allows open-source or commercial products to use our applications and components. See more information the license at [https://www.apache.org/licenses/LICENSE-2.0].

==Support==

====Is there a users group or a support mailing list?====

Yes. There are two main mailing lists. Both have public archives, and both require registration to post a message:

* [http://groups.yahoo.com/group/okapitools/ https://groups.yahoo.com/group/okapitools/] is the group and mailing list '''for the end users'''.
* [http://groups.google.com/group/okapi-devel https://groups.google.com/group/okapi-devel] is the group and mailing list '''for the developers''' working on the source code.

====How do I report bugs or request enhancement?====

* You can post a bug report or an enhancement request in the issues tracking page: http://code.google.com/p/okapi/issues/entry if you have a Google account.

* You can post a message to the [http://groups.yahoo.com/group/okapitools/ Okapi Tools users group] if you are part of the group.

* You can just [mailto:okapitools@opentag.com&subject=Feedback send feedback by email].

==Miscellaneous==

====What does 'Okapi' mean?====

An okapi is an African animal looking somewhat like [http://en.wikipedia.org/wiki/Okapi a cross between a zebra and a giraffe]. Okapi is pronounced [http://en.wikipedia.org/wiki/Wikipedia:IPA_for_English /oʊˈkɑːpɪ/] ([http://www.m-w.com/cgi-bin/audio.pl?okapi001.wav=okapi hear it])

The usage of this name for the framework has its roots to much older projects. At some point it was an acronym for "Open Kit API".

====What happened to the .NET Okapi?====

The older version of the Okapi Framework for .NET is no longer developed. Its distribution and source code is still available here: http://sourceforge.net/projects/okapi/. All new development is now done in the Java branch.

====Where is Olifant?====

Olifant, the TMX editor, is currently only part of the .NET Okapi. It is still available [http://sourceforge.net/projects/okapi/files/ from the SourceForge project]. Note that Olifant is for Windows only.

==For developers==

====Getting set up====

* Check out the source code from Bitbucket using git clone: https://bitbucket.org/okapiframework/okapi
* Or, if you want to submit pull requests, first create a fork of the Okapi project.
* Import into your IDE. For example, in Eclipse go to File > Import > Maven > Existing Maven project.
If you want to keep several distinct Okapi repositories in the same Eclipse workspace (for instance, your fork and the main Okapi project), you need to assign a name template under the "Advanced" section in the first step of the import wizard.
* The "master" branch contains the latest release version. The "dev" branch contains the current work (the "snapshot" in Maven terms).
* See also: https://bitbucket.org/okapiframework/okapi/wiki/How%20to%20Contribute
Happy coding!

====How to build okapi-lib locally====

The Okapi Framework consists of Maven projects. However, in order to build the apps and lib projects locally, you need to use the Ant build configurations.

For instance, to create a local version of okapi-lib.jar, go to <OKAPI_HOME>/deployment/maven/ and run ant -f build_okapi-lib.xml init okapiLib. The jar will be generated in <OKAPI_HOME>/deployment/maven/dist_common/lib/.

If you use the default build.xml by running above command without the -f option, platform-specific distributions of the apps will be created plus the platform-indipendent okapi-lib.jar.

Markdown Filter

2018-03-29T03:23:31Z

Kuro:

Markdown Filter

2018-03-29T00:10:45Z

Kuro:

Markdown Filter

2018-03-29T00:07:07Z

Kuro: Added description of translateImageAltText and htmlSubfilter. Clarified the default values.