Comparison Profile
Enhancements
Standardize text
The GUI documentation for //Standardize text// describes the current three-stage select field in the i-net PDFC profile sidebar. The selection is located in the Enhancements group.
Three transformation modes
| Sidebar choice | Raw values | Effect |
|---|---|---|
| Off | FILTERS does not contain TEXTTRANSFORM | The TextTransform filter is not executed. An existing TRANSFORM_OPERATIONS value has no effect while the filter is disabled. |
| Simplify complex characters | FILTERS contains TEXTTRANSFORM; TRANSFORM_OPERATIONS contains REPLACE_IDENTICAL | Performs the basic normalization of complex characters. |
| Equalize common OCR mistakes | FILTERS contains TEXTTRANSFORM; TRANSFORM_OPERATIONS contains REPLACE_IDENTICAL,REPLACE_CONFUSABLES | Performs the basic normalization and additionally equalizes typical OCR mistakes. This is the strongest sidebar mode. |
For the strongest mode, the sidebar writes both operations in this order. REPLACE_CONFUSABLES alone is a valid raw value, but it does not represent the complete strongest sidebar choice.
Raw profile values
The filter name is TEXTTRANSFORM. It is enabled through the comma-separated FILTERS list. The current implementation uses TRANSFORM_OPERATIONS for a comma-separated list of enum values:
| Raw key | Value or range | Meaning and default |
|---|---|---|
FILTERS | contains TEXTTRANSFORM | Enables TextTransform processing. The default value of the property list contains INVISIBLEELEMENTS,TEXTTRANSFORM,SOLVEFALSEREPLACE. |
TRANSFORM_OPERATIONS | REPLACE_IDENTICAL and/or REPLACE_CONFUSABLES | Defines the transformations to execute. The default value is REPLACE_IDENTICAL. Operations are executed in implementation order. |
Both properties take effect only when the plugin is installed. The sidebar implementation validates the selection against off, REPLACE_IDENTICAL, and REPLACE_CONFUSABLES. The value off is not stored in TRANSFORM_OPERATIONS; it removes TEXTTRANSFORM from FILTERS.
Mapping to the current profile sidebar
The Standardize text select field is provided by PDFCFilterTextTransformStructureProvider with the UI key pdfc.filter.texttransform. The mapping is:
| UI value | Sidebar choice | Stored raw values |
|---|---|---|
off | Off | FILTERS without TEXTTRANSFORM |
REPLACE_IDENTICAL | Simplify complex characters | FILTERS with TEXTTRANSFORM and TRANSFORM_OPERATIONS=REPLACE_IDENTICAL |
REPLACE_CONFUSABLES | Equalize common OCR mistakes | FILTERS with TEXTTRANSFORM and TRANSFORM_OPERATIONS=REPLACE_IDENTICAL,REPLACE_CONFUSABLES |
When a non-off choice is saved, TEXTTRANSFORM is added to FILTERS. Other existing filter identifiers remain in the list.
Simplify complex characters
REPLACE_IDENTICAL removes Unicode format characters, performs Unicode NFKD normalization, and removes simple spaces from the comparison text. This can convert compatibility characters and ligatures into a comparable base representation, for example fi to fi or ‼ to !!.
Decompose complex characters
Activate to decompose complex or special characters in into basic characters. Complex characters are for instance ligatures like 'fi' which will be decomposed into 'fi'. Furthermore special character like long or short hyphens will be normalized to their base character.
Equalize common OCR mistakes
REPLACE_CONFUSABLES also removes Unicode format characters, first normalizes to NFD, and then uses the Unicode TR39 confusables replacement table for visually similar characters. This can equalize typical OCR confusions such as m and rn.
In the strongest sidebar mode, REPLACE_IDENTICAL is applied first and REPLACE_CONFUSABLES second:
Equalize character recognition mistakes
Activate to correct typical text recognition mistakes. An example for a common ambiguousness in text recognition is the character 'm' and the syllable 'rn' which appear very similar depending on print quality and font.
Profile fragment
A possible XML profile fragment for the strongest mode is:
<entry key="FILTERS">TEXTTRANSFORM</entry> <entry key="TRANSFORM_OPERATIONS">REPLACE_IDENTICAL,REPLACE_CONFUSABLES</entry>
Comparison effect
The transformation is applied to extracted words and recursively to structure elements. The normalized representation is used only for comparison and does not replace the text presented to users. The option equalizes different text representations and common OCR artifacts; it is not general content correction.
