Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
This section contains detailed documentation on how to carry out updates to Goobi workflow. An update should always be carried out in such a way that a backup of the important data is made beforehand. The creation of a backup and other useful preparatory work is documented on this page:
Once the preparatory work has been successfully completed, the actual update can take place. This is described in this documentation so that the Goobi workflow can be updated step by step from the installed version to the latest version.
When you update Goobi workflow, all plug-ins should also be updated. You should also check the page Installed plug-ins in Goobi workflow. If this page does not yet exist, the relevant files must be identified in the file system. It is important not to forget the REST API.
Commands from this manual are best copied by clicking on the corresponding icon. Otherwise there is the danger of copying unwanted whitespaces.
To ensure the machine is in good condition, the following items should be checked before and/or after the update:
It makes sense to issue the Tomcat before the backup. However, it is necessary to check beforehand that no other user is logged into Goobi workflow. In addition to displaying the users who are logged in in the interface, the last entries in the process log or the last logins can be helpful in assessing the situation:
The Tomcat process for Goobi workflow can then be terminated:
Use the following commands to create a backup directory and copy the files to it:
Commit the current state in the Goobi folder:
Save the current status of the archives:
If an update is performed from a version lower than 24.07 to the current version, all EAD files must be exported from BaseX. To do this, log in at https://URL/basex/dba/login. Then select each database under Databases, select the .xml file and download it.
It is recommended to copy the files to /tmp/g3 on the machine to be updated. Move the files to the correct paths:
Especially configuration files of plugins can have a different file name due to a change in the naming scheme.
When the work is complete, the timestamp comparison can be used to determine whether all necessary files have been updated:
Plugins that are built as tar package:
The following things should be checked after an update:
Sometimes errors may occur after the update. The following points help to identify and solve typical problems:
If the Tomcat hangs after starting, the contents of the folder /opt/digiverso/goobi/activemq/ may be to blame. Emptying and restarting helps in this case:
The LayoutWizzard.jar in /opt/digiverso/{goobi,itm}/lib/ must be the same Symbolic links do not work.
After all preliminary work has been carried out, the actual update can take place. This is explained in detail in individual steps here:
The naming scheme has changed with the update. In the config file plugin_intranda_workflow_fileUploadProcessCreation.xml, the / at the beginning and end of the <namingSchema> tag must be removed.
Before:
After:
If grep -e '0.0.0.0' -e ':::61613' -e '0.0.0.0:61613' /opt/digiverso/goobi/config/goobi_activemq.xml && netstat -tulpn | grep -e ':::61613' -e '0.0.0.0:61613' yield no results, no action is required.
In /opt/digiverso/goobi/config/goobi_activemq.xml, the following two lines must be set to localhost:
In some use cases, it may be desirable to have an open activemq.
The Goobi workflow plugin: Barcode Scanner has been renamed to fit our naming scheme. Instead of plugin_intranda_step-barcode_scanner.jar, the file is now called plugin_intranda_step_barcode_scanner.jar.
With the new version, mail sending is configured in a central place. For this purpose, goobi_mail.xml must be stored as a new configuration file and the configuration for sending mails must be transferred to the config. The following plugins send mails: goobi-plugin-step-sendMail and goobi-plugin-step-leiden-file-delivery.
Image comments were previously stored in the images/ folder as a comments_*.json file. Instead, they are now stored as an image comments * process property. They have to be migrated, if they should still be accessible. This can be ignored if the display of image comments has not been explicitly set to true:
Change of parameters in goobi_config.properties Due to the update, the parameters dailyDelayJob, dailyVocabJob and dailyHistoryAnalyser have changed. These were previously specified by seconds past midnight and have now been replaced by a cron syntax. This looks as follows, for example:
An empty entry stands for not activated, which was -1 before. If the following command does not output anything, nothing has to be done:
An update of the plug-in has added a new configuration option for activating and deactivating individual jobs. This option must be added during an update. To do this, enabled="false" or enabled="true" must be added to each <rule> tag in the config file plugin_intranda_administration_data_poller.xml.
In addition, there is now the option to filter metadata with a white or black list. This was previously only possible via <skipField>. Existing configs must be transferred to the new lists. To do this, the <skipField> must be converted to <field> and the <fieldList mode="blacklist"> </fieldList> tag must also be supplemented with the <field> tags.
The following can be used to easily rename the <skipField> tags:
Typically, one would like to extract the CropBox area of a PDF page. Since without further configuration the MediaBox is used, one should set the option <generatorParameter>-cropbox</generatorParameter> in the images block. For example like this:
The vocabulary server must be updated to version 1.1.9 or newer.
The name of the intranda_step_layoutwizzard plugin has changed. It is now written entirely in lowercase. This also applies to the configuration files. To ensure everything works as usual, the process calls need to be adjusted.
This can be done most easily through the database:
Renaming the configuration files:
The plugin goobi-plugin-step-create-full-pdf has a different configuration. This part has changed and must be taken into account when updating:
OLD:
NEW:
Goobi now requires Java 17. Installation in Ubuntu:
To ensure that the Tomcat uses Java 17, it is probably required to add a line to the file /etc/default/tomcat9:
Scripts called from Goobi need to be compatible, and/or the system's default Java needs to be configured.
Access to the Goobi vocabulary server now requires an additional token. Check whether the vocabulary server is in use:
If vocabularies are to be used or were used before an update, adjustments must be made here. Check whether vocabularies are in use:
If this command returns a number >0, it is strongly recommended, to test the vocabulary migration before the Goobi workflow update. If vocabularies are not in use, then there must be no link to a vocabulary in the goobi_metadataDisplayRules.xml. Test: grep vocabulary -i /opt/digiverso/goobi/config/goobi_metadataDisplayRules.xml
Also before the update, check the update status of the following plugins, which where not yet adjusted at the time of release: plugin-step-flex-editor, plugin-workflow-linked-process-creation, plugin-step-metadata-replacement und plugin-workflow-project-export. Prüfen, ob eines dieser Plugins installiert ist:
This step is for Goobi installations, which use the archive management plugin. To check whether it is installed:
The data of the archive management is no longer stored in a BaseX database. It has to be migrated to the Goobi workflow database. This requires the following steps:
The current status of the archives must be saved. Log in at https://URL/basex/dba/login. Then select each database under Databases, select the .xml files and download them.
Stop BaseX:
Adjust configuration: Open the file /opt/digiverso/goobi/config/plugin_intranda_administration_archive_management.xml and remove the following lines:
Add the following blocks instead:
Adjust permissions: After the update, the following permissions must be available for full access:
Open archive plugin and import EAD files.
Optional: Configure export for the viewer: Archives in the viewer can be found at VIEWERURL/archives/. If the EAD files should also be visible in the viewer, configure the <export> area. The files to be exported and the hotfolder must be specified there.
Set up automatic tasks: Execute the following command to configure automatic exports:
Uninstall BaseX: Once the Goobi workflow update has been successfully tested and the archives are working as required, BaseX can be uninstalled:
Additionally remove the following block from the Apache configuration:
The config file plugin_intranda_step_newspaperRecognizer.xml must contain the following block:
The existing configuration must be adopted.
If the following is written in /opt/digiverso/goobi/config/plugin_intranda_administration_archive_management.xml:
this must be replaced by the following:
If useShelfmark is set to false, nothing needs to be adjusted.
With the new version, the plugin for the configuration editor has been integrated into the Goobi workflow core. A few points must be taken into account when updating: The old plugin must be deleted and the configuration file must be renamed.
Correction of the right's name:
Functionality for configuration has been added to the plugin. The following block must also be added to each <config>...</config> so that the plugin behaves in the same way after the update:
This point can be ignored if the plugin is not in use and thus the output of this command is empty:
In the goobi_opac.xml the searched ID used to be appended to the catalog address, now, for a catalog with the opacType="intranda_opac_xml", it is necessary to write {pv.id} for it. Example:
The change allows searching in XML files (address="file:///import/hotfolder/{pv.id}.xml").
The WebAPI is deprecated, instead we recommend the REST-API. The WebAPI is now deactivated by default. It can be reactivated with this option:
By updating the plugin, a type="" must be set for each <rule>. The type of existing rules is always filter. For this, type="filter" must be set in each <rule> tag in the config file plugin_intranda_administration_data_poller.xml.
The Goobi workflow Plugin: Catalogue Poller has been renamed to Data Poller. Instead of plugin_intranda_administration_catalogue-poller.jar the file is now called plugin_intranda_administration_data-poller.jar and is downloadable from GitHub: https://github.com/intranda/goobi-plugin-administration-catalogue-poller
The name of the config file has changed from plugin_intranda_administration_catalogue_poller.xml to plugin_intranda_administration_data_poller.xml.
With the new version, goobi_exportValidation.xml must be stored as a new configuration file.
http_useProxy has been renamed to http_proxyEnabled. There is a new option http_proxyIgnoreHost, with the following default:
Apply this change:
Please adjust the file plugin_intranda_step_createfullpdf.xml accordingly. Old:
New:
In the background something has changed in the escaping. This must be adjusted in the DB.
Check that the tomcat-lib folder is no longer included as PreResources in:
If used, the urn-generator.jar has to be updated.
The options exportFolder and useSubFolderPerProcess are read directly from the project configuration now (Administration -> Projects -> Edit project -> Technical Data). They should be removed from the config file plugin_intranda_export_configurable.xml. The project configuration has to be adjusted: "Export path for XML file" and "Export path for the images" instead of exportFolder, and "Create process directory" instead of useSubFolderPerProcess.
In addition the option includeDerivate was renamed to includeMedia:
Apply this only if the feature is in use. It is unused, if the result of the following SQL-Queryeine is emtpy: mysql goobi -NBse 'select * from journal where type="file" limit 10'
In the configuration file goobi_rest.xml this block must be present to be able to display the images on dockets:
Change the file using xmlstarlet:
Apply this only if the feature is in use. Image comments are only in use, if MetsEditorShowImageComments or ShowImageComments in the goobi_config.properties is active (true). In order to move any existing image comments from the previous location to the new location, the following script can be called (adjust folder names if necessary):
For an ActiveMQ version update, the files from the goobi processing queues should be deleted (if empty, check in Administration -> Processing queues):
In the Apache httpd config there may still be obsolete options that prevent websockets from working. Delete these lines:
The MySQL connector Java library has been updated again:
The MySQL connector Java library has been updated:
The delivery plugin for Leiden University Library has been renamed. Instead of plugin_intranda_step_file-delivery.jar the file is now called plugin_intranda_step_leiden_file_delivery.jar and is downloadable from GitHub: https://github.com/intranda/goobi-plugin-step-leiden-file-delivery
The rulesets must be adapted for the use of image coordinates. The following metadata elements must be created:
This structural element must be created:
And finally, this element must still be allowed within page:
Also the Metadata _dateDigitization might be missing in the rulesets for some document types. To automate all changes:
For the archive management plugin (if installed), an additional xq file must be available in the right place. This can be downloaded like this:
Line breaks in the configuration file goobi_projects.xml can affect the process title generation when creating new processes. The XML file can be reformatted this way (formatting is lost, especially of multiline comments):
The old version:
Needs to be replaced with:
Script for this change:
Goobi workflow now requires Tomcat 9 and MariaDB >= 10.1
If you are missing "Page order" in "Pagination":
The plugin jar file changed from plugin_intranda_export_mycore.jar to plugin_intranda_export_sword.jar.
The plugin config changed accordingly:
The plugin name changed from plugin_intranda_mycore_export to intranda_export_sword:
Goobi workflow has supported WebSockets for some time. To do this, its support in the Apache web server must be included in the configuration:
If the Tomcat is not listening on 8080, please adjust the port accordingly. However, you must always forward to the HTTP connector, not to the AJP connector;
Then activate the required module and restart the service:
The file docket_metadata.xsl must be exchanged:
The individual update steps that must be carried out to update a Goobi workflow installation to the required version are explained below. Please note that before carrying out the work documented here, you must first make a backup and, if necessary, carry out further preparatory work. These are documented here:
In the ImageQA plugin, the configuration of the rotation and deletion commands has changed. These have to be adapted.
This is the old configuration:
The new one looks like this:
Das generische Excel-Import-Plugin wurde von plugin_intranda_import_generic_excel.jarplugin_intranda_import_excel.jar umbenannt. Dies hat ebenso einen Einfluß auf die Konfigurationsdatei. Diese heisst statt plugin_intranda_import_excel_read_headerdata.xmlplugin_intranda_import_excel.xml.
The generic Excel import plugin has been renamed from plugin_intranda_import_generic_excel.jarplugin_intranda_import_excel.jar This also has an effect on the configuration file. Instead of plugin_intranda_import_excel_read_headerdata.xmlplugin_intranda_import_excel.xml.
The default name of the master folder has changed from master_{processtitle}_media to {processtitle}_master Optionally you can now adjust the folder structure in all operations or stay with the previous scheme - a configuration option is added for this:
If other folder names have been used, either adjust the above setting as necessary or perform all operations and steps.
For example, if the orig_{processtitle}_tif and {processtitle}_tif folders were previously used for master and derivatives, i.e. the setting
Then this must now be replaced by the following in /opt/digiverso/goobi/config/goobi_config.properties
If the option MetsEditorDefaultSuffix=jpeg was previously used, this must now be replaced as follows:
With this file this can be automated:
Execute:
The plugin goobi-plugin-step-xmp-header has been renamed from write-xmp to intranda_step_xmp_header. Adjust the database:
The configuration parser was changed so that the commas and backslashes no longer need to be escaped. All occurrences of \, must be replaced by , and \\ by \. Instead | in xml-attributes are now escaped. The following files under /opt/digiverso/goobi/config/ are affected:
Find affected files (backslashes) - please adjust manually:
Find affected files (pipe sign):
Probably only the goobi_projects.xml is affected here, which can be adapted as follows:
For the plugin goobi-plugin-step-xmpheader changes how the parameters for the exiftool are specified:
The folder name has changed:
If the plugin is used for Generic Excel import, the mapping of the metadata must be adjusted in the configuration file. The attribute name was renamed to property:
If the LayoutWizzard plugin is installed, the section from <contentBorder> to </preview> in its configuration file must be replaced by the following section
The style of the dockets was adapted and support for Hebrew characters was added. If the routing slips have not been individually adapted, the files can be replaced 1:1. For this purpose, the generation should be tried out in the user interface before. The following steps are more defensively formulated to prevent accidental automatic overwriting:
The previous standard XSLT files have the following MD5 hash:
After completion of the work, the generation of the dockets should be checked again.
Before removing the LDAP configuration from the Goobi workflow config file, it is important to start with the new goobi.war so that the migration to the database takes place.
The LDAP configuration was migrated from the configuration file to the database. After the update, the corresponding block can be removed from the goobi_config.properties file.
For the ActiveMQ interface the available resources must be specified:
Before these points are processed, Goobi workflow must be started once with the new goobi.war to update the database schema.
Since you can only make other users super-administrators as super-administrators, an account should be set manually in the database when updating. Subsequently, further accounts can be adjusted by means of a checkbox in the user administration.
The following SQL statements will help here:
Goobi workflow now uses log4j2 and comes with its own configuration file. You will therefore need to remove the overwriting entries from /etc/default/tomcat8:
In the config file application.properties, the token security.token=CHANGEME needs to be added. The vocabulary server requires an update:
The vocabulary management of Goobi workflow has been replaced by the vocabulary server. This is an independent application with which Goobi workflow can communicate. Please read the details in the .
Some tag names and types have changed in the config file, e.g. <folder> is now <includeFolders>, and <includeMedia>true</includeMedia> changed to <media enabled="true"/>.
The syntax for the configuration file of the editor for configuration files has changed and should be re-installed on the system and adapted for this purpose. The current version of a sample configuration for the file plugin_intranda_administration_config_file_editor.xml can be found .
The repository name is unchanged:
In this plugin it is now possible to define several changes for each depending on different process properties. Therefore a new XML element was introduced, which has to be added to the config file, see also:
The statistics plugin now exports XLSX instead of XLS, therefore the template file was changed. /opt/digiverso/goobi/plugins/statistics/statistics_template.xls needs to be replaced with https://github.com/intranda/goobi-plugin-statistics-intranda/releases/latest/download/statistics_template.xlsx.
The configuration for configurable delay plugin has been adapted to the same scheme as the other step plugins. For this, the configuration file plugin_intranda_delay_configurable.xml must be adapted as follows as an example:
The configuration for the generateIdentifier plugin needs to have a <type> added to plugin_intranda_step_generateIdentifier.xml because the default has changed from random to uuid:
The compiled file for the plugin plugin_intranda_step_move_folder_to_master has been renamed as the plugin is called. For the installation, this means that the previously installed plug-in file is now replaced by a new one:
For the Catalogue Poller plugin there was a similar change in the configuration file plugin_intranda_administration_catalogue_poller.xml as for the Catalogue Request plugin. Here the field name is now also configurable:
For the Catalogue-Request-Plugin there was a change in the configuration file plugin_intranda_step_catalogue_request.xml so that parameters were merged. Here, of course, attention must be paid to the respective configuration of the installation:
If database tables still use the Row Format Compact, this should be changed to Dynamic. For example, like this (adjust DB name if necessary):
Within the configuration for the xmp-header plugin, the definition of the folders is different. For this, the following change must be made in the configuration file plugin_intranda_step_xmp_header.xml:
Within the configuration file goobi_opac.xml a possible typing error has to be corrected. Instead of MultivolumeWork it should read MultiVolumeWork:
The keystore information is no longer stored from the Goobi workflow user interface but via a configuration within the configuration file goobi_config.properties. The keystore can now be defined there as follows:
There have been changes to SQL queries in Goobi workflow that make the following changes necessary:
Additionally, it is a good idea to delete some stale data. This data is now deleted automatically by Goobi, but there might be old data in the database:
Java 11 is needed from now on to run Goobi workflow. The package openjdk-11-jre-headless is a good option for Ubuntu users.
If there are runnable Jars included in your workflow (e.g. urn-generator.jar), please update these too.
The import plugin plugin_intranda_import_mab_fileplugin_intranda_import_sisis_sunrise_file. Accordingly, the following files must be replaced with new versions and the configuration file renamed accordingly:
In the configuration file plugin_intranda_step_changeWorkflow.xml the name of the property to be checked must be adapted because the variable-replacer is used here. The values used must be checked accordingly and adapted according to this example:
For better performance in database queries (e.g. when calling the process list) execute the following statements:
Creating the indexes and changing the database tables can take several minutes for large datasets, and is therefore performed manually and not automatically during the update.
For the dockets, the expression for displaying the creation date for the process has been changed to homogenise it. Instead of the previous expression goobi:time, it has to be changed to the expression goobi:creationDate within the xsl files. This usually affects the docket files under the following paths:
Here is a detailed example of the change:
The plugin intranda_dashboard_exampleintranda_dashboard_extended. On the system, the previous files must therefore be replaced with the files with the new names during the update:
The configuration file plugin_intranda_dashboard_extended.xml already present on the systems can thus continue to exist unchanged.
For future updates and installations, please note that the Git repository and the Maven artefact from Jenkins have also been adapted to the new name plugin_intranda_dashboard_extended.
The plugin intranda_statistics_sudan_activity_by_userintranda_statistics_sudan_memory_activity_by_user. On the system, the previous files must therefore be replaced with the files with new names during the update: