Data import for the Austrian Housing Promotion Fund
Time-controlled plugin for the repeated import of folder structures from an S3 storage for the import of housing subsidy files in Austria.
Last updated
Time-controlled plugin for the repeated import of folder structures from an S3 storage for the import of housing subsidy files in Austria.
Last updated
Name | Wert |
---|---|
Identifier | intranda_quartz_bka_wohnbau |
Repository | |
Licence | GPL 2.0 or newer |
Last change | 24.07.2024 19:50:31 |
This documentation describes the installation, configuration and use of the time-controlled plugin for importing housing subsidy files in Austria into Goobi workflow. The metadata is taken from a provided JSON file and the associated PDF files are extracted. The files are provided via an S3 memory in several deliveries, each of which is taken into account within the METS files.
To be able to use the plugin, the following files must be installed:
After installation, the plugin is available under the menu item Administration
- Periodic tasks
.
This plugin is a so-called Quartz plugin
for repeated automatic execution. With each call, the plugin assumes that configured Collections
within an S3 bucket contain directories. Each directory corresponds to a delivery
for an existing file
. The following example corresponds to the "second delivery" for the file ST-1431
.
Such a delivery contains several data: - a json file
with metadata - one or more PDF files
and full text files
for each document
of a `delivery
When the plugin is executed, all existing 'deliveries' are run through and a check is made to see whether they have already been imported into Goobi. If they have not yet been imported, the file is created as a new process if it does not already exist. The process is created on the basis of the configured 'production template' and within the configured 'project'. All metadata is transferred from the json file to the METS file as defined in the configuration file.
A new structural element is created for the respective delivery within the existing or newly created file, to which the metadata of the delivery is then assigned. Within the 'delivery', a 'document' is then created for each PDF file provided, to which the document metadata is assigned. Each 'document' is converted from the supplied PDF file into image files and the full texts are extracted in ALTO format. The image files imported are given a prefix to indicate the delivery number and a suffix for the respective page number within the PDF file from which they originate.
The image file is saved within the master
directory of the process. The full text files are stored in the alto
directory in the ocr
folder. The json file
is saved within the import
directory.
The configuration covers two areas. On the one hand, the function of the plugin is defined in its configuration file. On the other hand, a central Goobi configuration is used for time control, which defines when this plugin should be started regularly in order to run automatically.
The plugin is configured in the file plugin_intranda_quartz_bka_wohnbau.xml
as shown here:
The plugin can be repeated automatically or executed manually. Manual execution is possible by calling it within the menu item Administration
- Periodic tasks
. Automatic execution, on the other hand, must take place within the configuration file goobi_config.properties
. To do this, the configuration must look like this if the plugin is to be executed once every hour:
As an example, some further configurations for a different execution time are listed here (cron syntax):