3.2 Docker

Introduction

The following installation guide for the Goobi viewer refers to Ubuntu Linux 20.04. It is written as a step-by-step guide from top to bottom, meaning that settings and configurations build on each other. If the order is not followed, certain commands may fail.

VIEWER.EXAMPLE.ORG is used as the domain name in this manual, please adapt this to your own DNS name.

For productive use, a virtual machine with at least 4 CPUs and 8GB RAM is recommended.

Preparation

First, log on to the server on which the Goobi viewer is to be installed and obtain root rights:

ssh VIEWER.EXAMPLE.ORG
sudo -i

Then generate a password for the Goobi viewer database and a token and store them as a variable in the session. The DNS name is also stored there:

export VIEWER_HOSTNAME=VIEWER.EXAMPLE.ORG
export PW_SQL_ROOT=SECRETROOTPASSWORD
export PW_SQL_VIEWER=SECRETPASSWORD
export VIEWER_USERNAME=goobi@intranda.com
export VIEWER_USERPASS=SECRETPASSWORD
export VIEWER_IMAGE=intranda/goobi-viewer-theme-reference:develop
export VIEWER_THEME=reference
export TOKEN=$(uuidgen)

Now install the following packages:

apt -y install docker.io docker-compose mariadb-client python3-passlib python3-bcrypt

It is recommended to have a DNS entry for the server set at this time.

Create directory structure

The following commands create the necessary folder structure:

mkdir -p /opt/digiverso/{indexer,logs,viewer/{abbyy,cmdi,db,deleted_mets,dev,hotfolder,media,orig_lido,orig_denkxweb,success,ugc,alto,cms_media,error_mets,indexed_lido,mix,pdf,tei,updated_mets,cache,config/{PDFTitlePage,watermark,docker},fulltext,indexed_mets,oai/token,ptif,themes,wc}}
mkdir -p /opt/digiverso/zookeeper/{data,datalog}/
mkdir -p /opt/digiverso/solr/data/solr
sudo chown -R 8983:8983 /opt/digiverso/solr

Configuration files

Before the containers are started, the directories and configuration files for productive operation must be available in the file system.

config_viewer.xml

Various settings must be stored in the local config_viewer.xmlif necessary, the protocol must be adapted:

sed -e "s|VIEWER.EXAMPLE.ORG|${VIEWER_HOSTNAME}/viewer|g" -e "s|TOKEN|${TOKEN}|g" -e "s|VIEWERTHEME|${VIEWER_THEME}|g" << "EOF" >/opt/digiverso/viewer/config/config_viewer.xml
<?xml version="1.0" encoding="UTF-8" ?>
<config>
        <urls>
                <metadata>
                        <mets>https://VIEWER.EXAMPLE.ORG/oai?verb=GetRecord&amp;metadataPrefix=mets&amp;identifier=</mets>
                        <marc>https://VIEWER.EXAMPLE.ORG/oai?verb=GetRecord&amp;metadataPrefix=marcxml&amp;identifier=</marc>
                        <dc>https://VIEWER.EXAMPLE.ORG/oai?verb=GetRecord&amp;metadataPrefix=oai_dc&amp;identifier=</dc>
                        <ese>https://VIEWER.EXAMPLE.ORG/oai?verb=GetRecord&amp;metadataPrefix=europeana&amp;identifier=</ese>
                </metadata>

                <contentServerWrapper>https://VIEWER.EXAMPLE.ORG/content/</contentServerWrapper>
                <download>https://VIEWER.EXAMPLE.ORG/download/</download>
                <rest>https://VIEWER.EXAMPLE.ORG/api/v1/</rest>
                <connectorVersion>http://viewer:8080/oai/tools?action=getVersion</connectorVersion>
        </urls>

        <viewer>
                <theme mainTheme="VIEWERTHEME" discriminatorField="" />
        </viewer>

        <rss>
                <numberOfItems>50</numberOfItems>
                <title>Goobi viewer RSS Feed</title>
                <description>new items</description>
                <copyright>(c) Goobi viewer using institution </copyright>
        </rss>

        <webapi>        
                <authorization>
                        <token>TOKEN</token>
                </authorization>
        </webapi>
</config>
EOF

config_oai.xml

A local config_oai.xml is also required:

sed -e "s|VIEWER.EXAMPLE.ORG|${VIEWER_HOSTNAME}|g" << "EOF" >/opt/digiverso/viewer/config/config_oai.xml
<?xml version="1.0" encoding="UTF-8"?>
<config>
        <identifyTags>
                <baseURL useInRequestElement="true">https://VIEWER.EXAMPLE.ORG/viewer/oai</baseURL>
                <adminEmail>support@intranda.com</adminEmail>
        </identifyTags>
        <urnResolverUrl>https://VIEWER.EXAMPLE.ORG/viewer/resolver?urn=</urnResolverUrl>
        <piResolverUrl>https://VIEWER.EXAMPLE.ORG/viewer/piresolver?id=</piResolverUrl>
</config>
EOF

OAI xsl files

The OAI interface requires two more files:

wget -O /opt/digiverso/viewer/oai/MARC21slimUtils.xsl.xml https://raw.githubusercontent.com/intranda/goobi-viewer-connector/master/goobi-viewer-connector/src/main/resources/MARC21slimUtils.xsl
wget -O /opt/digiverso/viewer/oai/MODS2MARC21slim.xsl https://raw.githubusercontent.com/intranda/goobi-viewer-connector/master/goobi-viewer-connector/src/main/resources/MODS2MARC21slim.xsl

config_indexer.xml

Execute the following commands to download the current config.xml and adjust it accordingly:

wget -O /opt/digiverso/indexer/config_indexer.xml https://raw.githubusercontent.com/intranda/goobi-viewer-indexer/master/goobi-viewer-indexer/src/main/resources/config_indexer.xml
sed -e 's|<solrUrl>.*</solrUrl>|<solrUrl>http://solr:8983/solr/collection1</solrUrl>|' -e 's|C:/|/|g' -e "s|<viewerUrl>.*</viewerUrl>|<viewerUrl>https://${VIEWER_HOSTNAME}/viewer/</viewerUrl>|" -i /opt/digiverso/indexer/config_indexer.xml

Cronjob

Cronjobs must be set up for regular tasks:

sed -e "s|VIEWER.EXAMPLE.ORG|${VIEWER_HOSTNAME}|g" -e "s|TOKEN|${TOKEN}|g" << "EOF" >/etc/cron.d/intranda-goobiviewer
PATH=/usr/bin:/bin:/usr/sbin/
MAILTO=admin@intranda.com

#
# Regular cron jobs for the Goobi viewer
#

## These two scripts pull the theme git repository regulary. The @daily part is only 
## a reminder for the 1-minute schedule
#*/1 *       * * *   root    cd /opt/digiverso/viewer/themes/goobi-viewer-theme-reference; git pull | grep -v -e "Already up.to.date." -e "Bereits aktuell."
#@daily              root    echo "Please look at the git checkout interval for the Goobi viewer theme" | mail -s "Reference: Theme repository is checked out every minute" admin@intranda.com 

## Optimize the Solr search index once a month
@monthly            root    curl -s 'http://solr/solr/collection1/update?optimize=true&waitFlush=false'
EOF

ATTENTION: If the installation is not available via HTTPS, then the files just created must be opened at this point and the schema must be adjusted manually.

my.cnf

For easier access to the database later, a my.cnf with the corresponding access data is stored for the root user:

test -e /root/.my.cnf && echo 'WARNING, .my.cnf already exists!' ||
echo -e "[client]\nhost=127.0.0.1\npassword=${PW_SQL_ROOT}" >> /root/.my.cnf
chmod 600 /root/.my.cnf

stopwords.txt

For the Goobi viewer to be able to read the Solr stop words the corresponding language file must be stored, here the German one as an example:

cp /opt/digiverso/viewer/dev/goobi-viewer-docker/solr/goobiviewer/conf/lang/stopwords.txt /opt/digiverso/viewer/config/stopwords.txt

Variables and aliases

Add the following aliases to /root/.bash_aliases:

cat << "EOF" >>/root/.bash_aliases
alias cata='docker-compose -f /opt/digiverso/viewer/config/docker/docker-compose.yml logs -f --tail="500" viewer'
alias ind='docker-compose -f /opt/digiverso/viewer/config/docker/docker-compose.yml logs -f --tail="500" indexer'
EOF

Apply the changes in the current session:

. /root/.bashrc

Installation

For the installation, the Goobi viewer Docker git repository must be cloned:

cd /opt/digiverso/viewer/dev/
git clone https://github.com/intranda/goobi-viewer-docker

If you have your own theme, you must ensure that it can be loaded. If, for example, it is stored in its own Nexus server where access to the container is password-protected, then the access data must first be stored locally, for example:

docker login -u INTRANDACUSTOMER nexus.intranda.com:4443

The setup knows two possible environments. The default is the one for testing. For an installation in productive operation, files must be copied and possibly adapted.

First, copy the .env-production to the local Docker configuration directory, rename it and adjust values in it:

cp /opt/digiverso/viewer/dev/goobi-viewer-docker/.env-production /opt/digiverso/viewer/config/docker/.env
sed -i "s|VIEWER_DOMAIN=VIEWER.EXAMPLE.ORG|VIEWER_DOMAIN=${VIEWER_HOSTNAME}|g" /opt/digiverso/viewer/config/docker/.env
sed -i "s|DB_ROOT_PASSWORD=viewer|DB_ROOT_PASSWORD=${PW_SQL_ROOT}|g" /opt/digiverso/viewer/config/docker/.env
sed -i "s|DB_VIEWER_PASSWORD=viewer|DB_VIEWER_PASSWORD=${PW_SQL_VIEWER}|g" /opt/digiverso/viewer/config/docker/.env
sed -i "s|VIEWER_IMAGE=intranda/goobi-viewer-theme-reference:develop|VIEWER_IMAGE=${VIEWER_IMAGE}|g" /opt/digiverso/viewer/config/docker/.env

Then copy the docker-compose.yml and the solr folder and copy and rename the docker-compose.production.yml:

cp /opt/digiverso/viewer/dev/goobi-viewer-docker/docker-compose.yml /opt/digiverso/viewer/config/docker/docker-compose.yml
cp -a /opt/digiverso/viewer/dev/goobi-viewer-docker/solr /opt/digiverso/viewer/config/docker/
cp /opt/digiverso/viewer/dev/goobi-viewer-docker/docker-compose.production.yml /opt/digiverso/viewer/config/docker/docker-compose.override.yml

Now the containers can be started for the first time with the following commands:

cd /opt/digiverso/viewer/config/docker/
docker-compose up -d viewer-db solr
docker-compose exec solr solr zk upconfig -n goobiviewer -d /opt/goobiviewer
docker-compose exec solr solr create -c collection1 -n goobiviewer
docker-compose up -d

Create account

The Goobi viewer has a backend. The following command adds a test account to the database with the user name goobi@intranda.com and the password set at the beginning:

VIEWER_USERPASS_HASH=$(python3 -c "from passlib.hash import bcrypt; print(bcrypt.using(rounds=10, ident='2a').hash('${VIEWER_USERPASS}'))")
mysql -u viewer -p -D viewer -h 127.0.0.1 -e "INSERT INTO users (active,email,password_hash,score,superuser) VALUES (1,'${VIEWER_USERNAME}','${VIEWER_USERPASS_HASH}',0,1);"

Updates

To apply updates, the goobi-viewer-docker repository must first be pulled:

cd /opt/digiverso/viewer/dev/goobi-viewer-docker
git pull

After that, the differences of the following three files have to be compared and taken over or adjusted accordingly:

diff -u ../../config/docker/.env .env-production  | colordiff
diff -u ../../config/docker/docker-compose.override.yml docker-compose.production.yml  | colordiff
diff -u ../../config/docker/docker-compose.yml docker-compose.yml  | colordiff

At the end, it is still necessary to check if there have been any changes in the Solr directory.

These points are in addition to the existing core and theme changelog.

General commands

Here are some commands documented to get into the containers, etc.

# Start all containers in the background.
# remove -d to run them in the foreground
docker-compose up -d

# Show logfiles of a container
# Possible values are solr, zookeeper, viewer, viewer-db, indexer, connector, proxy
docker-compose logs indexer -f

# Go into a docker container
docker-compose exec viewer /bin/bash

# Restart a docker container
docker-compose restart indexer

# Shutdown
docker-compose down

# Update docker container
docker-compose pull

For example, vim can be installed in a container with the following command:

apt update && apt install vim-nox

Previous3.1 Classical Next4. Development environment

Last updated 1 year ago