OCR using Transkribus
Step plugin for Goobi workflow for generating OCR using Transkribus
Last updated
Step plugin for Goobi workflow for generating OCR using Transkribus
Last updated
Identifier
intranda_step_transkribus
Repository
Licence
Proprietary commercial
Last change
23.12.2024 10:38:52
This documentation describes the installation, configuration and use of the Step plug-in for OCR using Transkribus.
To be able to use the plugin, the following files must be installed:
Once the plugin has been installed, it can be selected within the workflow for the respective work step and thus executed automatically. A workflow could look like the following example:
The work step must be configured so that the plugin is selected and the checkbox for the time delay is set.
The plugin is designed in such a way that it first uploads all images of a process to Transkribus at the start and then leaves the work step in processing. A file processing.json is created in the file system within the ocr subdirectory of the process, which shows the exact status of the respective pages. This looks like this as an example:
Goobi periodically calls all delay plug-ins in the background, including this one. With every automatic call and also every manual call via the administrative interface for the regular tasks, the plugin attempts to download the pages that have not yet been completed.
If all pages could be downloaded, the step is closed and the workflow continues. In the event of an error, however, the step changes its status to an error with corresponding error messages in the journal.
During Transkribus processing, the plugin updates the exact progress within the processing.json file. However, in addition the progress of the processing is also saved as a process property with the name Transkribus Processing. This property can be displayed within the tabular process list by configuring it as an additional column for display.
The display then provides a good overview of the exact status of the processing. If several processes are being processed, not only the number of images still to be processed is displayed, but also the number of Transkribus jobs that still need to be processed before the process can be completed.
If you want to view the exact progress, this can be viewed in the ocr subfolder of the process. This contains a directory with the generated ALTO files and the file processing.json mentioned above.
A simple script that opens the path to the respective directory of the desired process looks like this and could be stored under /usr/local/sbin/ocr-folder.sh:
If a page is not uploaded, processed or downloaded correctly, it is recommended to proceed as follows, depending on the error:
If individual images have failed during processing, they can be completely reprocessed without affecting the other images in the same directory. The best way to do this is as follows:
Open the corresponding processing.json file from the ocr directory of the process
Set the value in status to RETRY for the corresponding image
Set the work step in the workflow to In process again
Manually restart the delay job from the administrative interface for the regular tasks
If there is a problem downloading the results, for example because the internet connection has been lost and the number of download attempts has exceeded the configured maximum, you can have the selected images downloaded again from the Transkribus server:
Open the corresponding processing.json file from the ocr directory of the process
Set the value in status to RUNNING for the corresponding image
Set the work step in the workflow to In process again
Manually restart the delay job from the administrative interface for the regular tasks
If many images are affected by the fact that they are incorrectly in the CANCELED status, this can be easily changed for a process as follows:
The plugin is configured in the file plugin_intranda_step_transkribus.xml as shown here:
The <config> block can occur repeatedly for different projects or work steps in order to be able to perform different actions within different workflows. The other parameters within this configuration file have the following meanings:
project
This parameter defines which project the current block <config> should apply to. The name of the project is used here. This parameter can occur several times per <config> block.
step
This parameter controls which work steps the <config> block should apply to. The name of the work step is used here. This parameter can occur several times per <config> block.
In addition to these general parameters, the following parameters are available for further configuration:
url
URL for the intranda licence server
login
Login for the intranda licence server
password
Password for the intranda licence server
model
Process property to be used, in which the Transkribus model ID is stored. This property is usually called Transkribus Model.
model-default
Transkribus model ID to be used if there is no property with a value in the process. Example: 36202
max-failed
Maximum number of new attempts if an OCR cannot be performed for a page. The workflow step remains in error status if the number of attempts is exceeded.
max-download
Maximum number of attempts to download the OCR result from Transkribus. The workflow step remains in error status if the number of attempts is exceeded.
As the plugin was designed as a delay plugin, it is called periodically by Goobi's scheduler to retrieve new completed pages from Transkribus. The time control of the dailyDelayJob is used for this periodic query. However, this can also be configured differently in the configuration file goobi_config.properties using the following syntax:
