Pensieve TM

Pensieve TM is the Okapi Framework's own TM engine. It is still under development, but can be already used for production.

The engine can be used:

• From your desktop.
• From a Web server (Note that, at this time, the Web server option is not distributed, as it is still experimental).

Local Pensieve TM

A local Pensieve TM database is stored in a directory that must contain only the TM-related files. It is recommended to use the extension .pentm for such directories.

Creating a TM

You can create such TM multiple ways, for example:

• Using the pre-defined pipeline Pensieve TM Import in Rainbow.
• Using the -imp command of Tikal
• Using the TM Import Step
• Using the Format Conversion Step
• Using the Batch Translation Step

See also the article "How to Create a Pensieve TM" in the Knowledge Base.

Using a TM

Access to the TM is provided by the Pensieve TM Connector, which can be used anywhere an Okapi TM connector can be used.

You can also read and write a Pensieve TM just like any other bilingual document by using the Pensieve TM Filter.

See also the article "How to Query a Pensieve TM" in the Knowledge Base.

Pensieve TM Sever

Note: This component is currently experimental, with very limited options and meant only for test and development.

This component is not currently part of the Okapi distribution, but it is available from the build and the Maven artifacts.

Limitations

Currently queries made of text that is shorter than 4 characters (the size of the n-grams used in the index) or equal to one of the n-grams in the stop-list (n-grams that are weeded out of the indexed text because they are very frequent) cannot be retrieved using the fuzzy search.