June 2021

Documentation home Goobi workflow Handbook Goobi workflow Plugins
Search…
June 2021
Developments and innovations to Goobi workflow in June 2021
Coming soon
Outsourcing of some statistics to plugins with refreshing of the user interface
Implementation of a new search syntax
Revision of the internal folder structure to comply with the Maven convention
Expansion of the plugin documentation
Conversion from Lido to METS/MODS and METS/MODS to Lido now possible
Goobi now supports the conversion of the metadata format LIDO to METS. For example, it is now possible to import a LIDO file and save it internally as a METS file. In the same way, an existing METS file can also be exported again as a LIDO data set.
Conversion from METS to LIDO and back
​https://gitea.intranda.com/intranda/ugh-extension/commit/e9ce2e551e644f9e20ab6caf69c08f60ec37b48d
https://gitea.intranda.com/intranda/ugh-extension/commit/f3b6864a867c28567c8c76af89436daaa6fc65f4​
Additional locations for image referencing in LIDO files.
Previously, images in LIDO were expected as <linkResource> within the <resourceRepresentation> element:
1
<lido:administrativeMetadata>
2
<lido:resourceWrap>
3
<lido:resourceSet>
4
    <lido:resourceRepresentation lido:type="local">
5
    <lido:linkResource>00000001.jpg</lido:linkResource>
6
    </lido:resourceRepresentation>
7
</lido:resourceSet>
8
</lido:resourceWrap>
9
</lido:administrativeMetadata>
Copied!
With this month's developments, it has now become possible for images to be specified as <resourceID> instead. This is particularly helpful for data transfer from the MuseumPlus application:
1
<lido:administrativeMetadata>
2
<lido:resourceWrap>
3
<lido:resourceSet>
4
    <lido:resourceID>00000001.tif</lido:resourceID>
5
</lido:resourceSet>
6
</lido:resourceWrap>
7
</lido:administrativeMetadata>
Copied!
Please note: If an image belongs to a sub-element, the specification must be made within the <lido> element that describes this sub-element.
​https://gitea.intranda.com/intranda/ugh-extension/commit/183ef6f4b93a8b0becd0627df8094babce1a32ef​
Adjustments to the usability of IIIF URLs in script calls
Recently, we had extended the so-called variable-replacer to the effect that it now also allows IIIF URLs for the corresponding images of a process in script calls. We had to make some corrections here so that the encoding of the URLs fits the respective use cases.
​https://github.com/intranda/goobi-workflow/commit/b887afbc47b8871588e0c1f2d7275981e1141eab
 https://github.com/intranda/goobi-workflow/commit/1715a8603f476141b1c13d58385f51bc8641dff8​
Building an Authority Server for Vocabularies
The vocabulary management in Goobi workflow makes it possible to manage controlled vocabularies in the user interface and to use them as choices for metadata fields in the metadata editor. A particularly practical feature is that the description for a particular entity can be changed and, by linking to only one vocabulary entry in the METS file, the entry is also automatically updated in all processes. Because of these advantages, vocabulary management is being used in more and more projects.
Goobi's vocabulary management is finding more and more use
A problem can arise when using the vocabularies if, after exporting the data, Goobi viewer also needs to access the same vocabularies, as the Goobi workflow installation is often inaccessible from outside the facility network because it is located behind a firewall. For this reason, we have developed a new vocabulary server to which Goobi workflow mirrors all changes to the vocabularies configured in this way. In such a case, the URL of the data records in the publicly accessible vocabulary server, which is hosted by intranda and accordingly accessible worldwide, is entered in the metadata of the METS file. This means that the Goobi viewer can then access the vocabularies without any problems and always receive up-to-date data records from the vocabulary.
The code for this server has not yet been published and can be found in the intranda-internal Git repository: https://gitea.intranda.com/intranda/goobi-authority-server. The still internal documentation can also be found there in the repository: https://gitea.intranda.com/intranda/goobi-authority-server/src/branch/master/goobi-authority-server/docs/AuthorityServer.md. We have not yet decided when to publish this project. Before that, we would like to review the source code and also prepare and translate the documentation.
Beautifier for MARC data sets
Since the first days of Goobi, there has been the possibility to create processes based on a catalogue query and thus reuse the data from the catalogue. Unfortunately, however, the response from the respective catalogues is sometimes not such that Goobi can reliably interpret, for example, the publication types from the expected fields. For this reason, it has long been possible to manipulate the catalogue response with the so-called Beautifier before evaluating it. This was already possible in the past with PICA records. What is new is that these possibilities now also exist for MARC records. Such a configuration could look like this, for example:
Manipulations on the leader
In this example, the leader at position 19 is replaced with a space (\u0020) if the value e is in the leader at position 6.
1
<beautify>
2
<setvalue tag="leader19" subtag="" value="\u0020">
3
<condition tag="leader6" subtag="" value="e" />
4
</setvalue>
5
</beautify>
Copied!
Writing Controlfields
This example shows how to create the field <controlfield tag="999"> with the value KXP for a record of an OPAC query.
1
<beautify>
2
<setvalue tag="999" subtag="" value="KXP" />
3
</beautify>
Copied!
Replace field contents:
This example shows how a field content can be changed. Here the field 041$a is replaced with the value ger if it contains the value lat.
1
<beautify>
2
<setvalue tag="041" subtag="a" value="ger">
3
<condition tag="041" subtag="a" value="lat" />
4
</setvalue>
5
</beautify>
Copied!
The following URL documents the configuration for catalogues and also the definition of the Beautifier:
https://docs.goobi.io/goobi-workflow-en/admin/7/7.2
​https://github.com/intranda/goobi-plugin-opac-marc/commit/13de8118b5aff7c3a82127d0d01a31cdf0a5edc1
https://github.com/intranda/goobi-plugin-opac-marc/commit/930386f0838e462ce5d31c1567beb00bc4b637f4
https://github.com/intranda/goobi-plugin-opac-marc/commit/54d86f5246d3cad5b4f437d22a70ef0aee39d67b​
Further accessibility adjustments
And the wild ride continued this month as far as accessibility optimisations are concerned. One gets the impression that this is a life task. But at least it feels like we are on the home stretch. Let's see how it looks next month. :)
Accessibility is more important than many think!
​https://github.com/intranda/goobi-workflow/commit/bcc76fa67841af7a7fe2a50a1fe3398c12554d1c
https://github.com/intranda/goobi-workflow/commit/70f65c14ab97a2394a7d5b0d7baa98c6bc433af2
https://github.com/intranda/goobi-workflow/commit/b6bd4dad318910aa8d0aa98a8e8ea0d54ba469a3
https://github.com/intranda/goobi-workflow/commit/17b88937cc79b71c1914533aa85b3685c5ca01fc
https://github.com/intranda/goobi-workflow/commit/5806a5e8e5e011d2ac5ac6c68da2dfa950ec4a01
https://github.com/intranda/goobi-workflow/commit/106283895a5604de04c809d2e3a7bcfa99d8fc93​
Compilation with Java 11 possible
Compiling Goobi workflow is no longer limited to the already quite old version 8 of Java. Instead, from now on not only operation with Java 11 is possible but also compiling.
Excerpt from the console after compiling Goobi workflow with Java 11
For those who would like to compile Goobi themselves, here is a mini-summary of how to do it yourself:
1
# install needed packages
2
sudo apt install git
3
sudo apt install maven
4
​
5
# checkout Goobi workflow from GitHub
6
git clone https://github.com/intranda/goobi-workflow.git
7
​
8
# go into the Goobi workflow project
9
cd goobi-workflow/Goobi/
10
​
11
# compile Goobi workflow
12
mvn package
Copied!
​https://github.com/intranda/goobi-workflow/commit/2049aa3ff998b4be5d11a8d792d8287cd65bafa7
https://github.com/intranda/goobi-workflow/commit/7cdde2381c9e0ce2d5bd9eb741b2f82c3f4d7fe3
https://github.com/intranda/goobi-workflow/commit/0138cc4c39a6985094e0f61d1c3adce71d1990c6​
Automatic generation of JWT Secrets
JSON Web Tokens (JWTs) are being used more and more in Goobi workflow to allow other programmes or people to perform certain actions for a certain time. Until now, a JWT-Secret had to be pre-configured manually in order to be able to use this functionality. As of this month, Goobi workflow generates the JWT secret itself if it has not yet been pre-configured.
In the standard installation, the JWT secret is still commented out and can be configured individually.
​https://github.com/intranda/goobi-workflow/commit/35b13272d0f514b156eff0627783a87c258f60be​
JWT key Rotation
During the implementation of the automatic generation of the JWT secrets described above, a potential security gap was discovered: The JWT keys were not rotated, which could have allowed an attacker to guess the secret if many messages were intercepted. This has now been changed and the keys are rotated every 24 hours.
​https://github.com/intranda/goobi-workflow/commit/8441eadb77d075d24ba6c448d2ceb27a3c189589​
Adjustments to the standard export for naming changes
It happens very rarely that file names of images of individual processes are renamed bypassing Goobi, so that Goobi was previously unaware of these renaming changes and this could lead to problems when re-exporting the METS files.
Setting the default export for METS files without considering the MIME type
With an adaptation of the standard export, we now circumvent this problem. Because in the case of the project configuration that the METS file should dynamically generate the file names depending on the recognised MIME type (e.g. image/jpeg), an update of the internally stored file names changed in the meantime is now also carried out within the METS file. In this way, such potential errors in the case of renaming carried out independently of Goobi (e.g. adjusted upper and lower case, changed file extension) are intercepted.
Setting the default export for METS files to consider the MIME type
​https://github.com/intranda/goobi-workflow/commit/e58838f80e6bd24fc48e649602eca004ecd1a5ec​
Code optimisations
Less spectacular for non-programmers sounds an elaborate change we are currently making internally for further code optimisations. We are currently using the Lombok program library even more for code generation of boring, bloated but unfortunately necessary source code, which by convention has to be within Java applications.
For those readers who are Java programmers themselves: Lombok is definitely worth a look and takes care of the automatic generation of getters, setters, constructors and a lot more.
Generation of getters and setters by Lombok
​https://github.com/intranda/goobi-workflow/commit/17a0d32698c2c941cc7ec47317b08e0904dac756
https://github.com/intranda/goobi-workflow/commit/ef797453faf8c83a5bb6cd5096d04106cbebae18
https://github.com/intranda/goobi-workflow/commit/a3ee13e8559eceb29f059557fdf80bb093bf73fd
https://github.com/intranda/goobi-workflow/commit/5eb5eecb3dfe7e4bc103cd061b64d2b13dc17384
https://github.com/intranda/goobi-workflow/commit/afd5b82169ed063da7586ed9c329c5465785336f
https://github.com/intranda/goobi-workflow/commit/93b8d5cc45897e949322b79a8e3a7634bc1396c2
https://github.com/intranda/goobi-workflow/commit/16e903327281e247d6e3da84c4bf6224d89175f3
https://github.com/intranda/goobi-workflow/commit/665f4bd3e2d9b94d479f0fbdb89a0bea3f66d264​
Duplicating metadata groups expanded
We have recently massively expanded the metadata groups. For a project in Croatia, we needed some rework here: It was necessary that metadata groups can also be duplicated, so that they now correctly duplicate all associated metadata, persons and subgroups. This saves some unnecessary manual editing.
Metadata groups can now be completely duplicated
​https://github.com/intranda/goobi-workflow/commit/4ae3035f0e6e1582d68235fd71a22fde8079b0b9​