3. Installation guide

Introduction
The following installation guide for the Goobi viewer refers to Ubuntu Linux 20.04. It is written as a step-by-step guide from top to bottom, meaning that settings and configurations build on each other. If the order is not followed, certain commands may fail.
VIEWER.EXAMPLE.ORG is used as the domain name in this manual, please adapt this to your own DNS name.
For productive use, a virtual machine with at least 4 CPUs and 8GB RAM is recommended.
Preparation
First, log on to the server on which the Goobi viewer is to be installed and obtain root rights:
1
ssh VIEWER.EXAMPLE.ORG
2
sudo -i
Copied!
Then generate a password for the Goobi viewer database and a token and store them as a variable in the session. The DNS name is also stored there:
1
export VIEWER_HOSTNAME=VIEWER.EXAMPLE.ORG
2
export PW_SQL_VIEWER=SECRETPASSWORT
3
export VIEWER_USERNAME=[email protected]
4
export VIEWER_USERPASS=SECRETPASSWORD
5
export TOKEN=$(uuidgen)
6
export install=/tmp/install
Copied!
Now install the following packages:
1
apt -y install git patch openjdk-11-jdk-headless tomcat9 mariadb-server apache2 ttf-mscorefonts-installer unzip zookeeperd python3-passlib python3-bcrypt
Copied!
At the end of the preparation, a temporary directory for the installation must be created and the Goobi viewer Core Config Repository must be cloned. This contains the files necessary for the installation:
1
install=/tmp/install
2
mkdir -p $install
3
cd $install
4
git clone https://github.com/intranda/goobi-viewer-core-config.git
Copied!
It is recommended to have set a DNS entry for the server at this time.
Variables and aliases
Add the following aliases to /root/.bash_aliases:
1
cat << "EOF" >>/root/.bash_aliases
2
alias cata='journalctl -u tomcat9 -n 1000 -f'
3
alias ct='chown tomcat:tomcat *'
4
alias ctr='chown -R tomcat:tomcat *'
5
alias ind='tail -n 1000 -f /opt/digiverso/logs/indexer.log'
6
alias vl='tail -n 1000 -f /opt/digiverso/logs/viewer.log'
7
EOF
Copied!
Adopt the changes in the current session:
1
. /root/.bashrc
Copied!
Install packages
Install the following packages including all dependencies:
1
apt -y install openjdk-11-jdk-headless tomcat9 mariadb-server apache2 ttf-mscorefonts-installer unzip
Copied!
Create directory structure
The following commands create the necessary folder structure:
1
mkdir -p /opt/digiverso/{indexer,logs,viewer/{abbyy,cmdi,deleted_mets,hotfolder,media,orig_lido,orig_denkxweb,success,ugc,alto,cms_media,error_mets,indexed_lido,mix,pdf,tei,updated_mets,cache,config/{PDFTitlePage,watermark},fulltext,indexed_mets,oai/token,ptif,themes,wc}}
2
chown -R tomcat: /opt/digiverso/{indexer,logs,viewer}
Copied!
Configure services
Tomcat
1
SYSTEMD_EDITOR=tee systemctl edit tomcat9 << "EOF"
2
[Service]
3
LogsDirectoryMode=755
4
CacheDirectoryMode=755
5
ProtectSystem=true
6
NoNewPrivileges=true
7
ReadWritePaths=
8
EOF
Copied!
In /etc/default/tomcat9 adjust the memory under -Xmx to the available machine memory, choose reasonable garbage collector options and use urandom for faster Tomcat startup:
1
patch /etc/default/tomcat9 << "EOF"
2
@@ -11,6 +11,15 @@
3
 # To enable remote debugging uncomment the following line.
4
 # You will then be able to use a Java debugger on port 8000.
5
 #JAVA_OPTS="${JAVA_OPTS} -agentlib:jdwp=transport=dt_socket,address=8000,server=y,suspend=n"
6
+JAVA_OPTS="-Djava.awt.headless=true -Xmx4g -Xms4g"
7
+JAVA_OPTS="${JAVA_OPTS} -XX:+UseG1GC"
8
+JAVA_OPTS="${JAVA_OPTS} -XX:+ParallelRefProcEnabled"
9
+JAVA_OPTS="${JAVA_OPTS} -XX:+DisableExplicitGC"
10
+JAVA_OPTS="${JAVA_OPTS} -XX:+CMSClassUnloadingEnabled"
11
+JAVA_OPTS="${JAVA_OPTS} -Djava.security.egd=file:/dev/./urandom"
12
+JAVA_OPTS="${JAVA_OPTS} -Dfile.encoding='utf-8'"
13
+
14
+UMASK=0022
15
 
16
 # Java compiler to use for translating JavaServer Pages (JSPs). You can use all
17
 # compilers that are accepted by Ant's build.compiler property.
18
@@ -20,4 +29,4 @@
19
 #SECURITY_MANAGER=true
20
 
21
 # Whether to compress logfiles older than today's
22
-#LOGFILE_COMPRESS=1
23
+LOGFILE_COMPRESS=1
24
EOF
Copied!
Further configurations are made in /etc/tomcat9/server.xml. This will disable the appContextProtection for the JreMemoryLeakPreventionListener, set up a HTTP connector (with correct proxyName) and an AJP connector on localhost. A Crawler Session Manager Valve is also activated:
1
sed -e "s|VIEWER.EXAMPLE.ORG|${VIEWER_HOSTNAME}|g" << "EOF" | patch /etc/tomcat9/server.xml
2
@@ -66,59 +66,22 @@
3
          APR (HTTP/AJP) Connector: /docs/apr.html
4
          Define a non-SSL/TLS HTTP/1.1 Connector on port 8080
5
     -->
6
-    <Connector port="8080" protocol="HTTP/1.1"
7
+    <Connector address="127.0.0.1" port="8080" protocol="HTTP/1.1"
8
                connectionTimeout="20000"
9
-               redirectPort="8443" />
10
-    <!-- A "Connector" using the shared thread pool-->
11
-    <!--
12
-    <Connector executor="tomcatThreadPool"
13
-               port="8080" protocol="HTTP/1.1"
14
+	       redirectPort="8443"
15
+               maxThreads="400"
16
+               URIEncoding="UTF-8"
17
+               enableLookups="false"
18
+               disableUploadTimeout="true"
19
+               proxyName="VIEWER.EXAMPLE.ORG"
20
+               proxyPort="80" />	    
21
+
22
+    <Connector address="127.0.0.1" port="8009" protocol="AJP/1.3"
23
+               secretRequired="false"
24
                connectionTimeout="20000"
25
-               redirectPort="8443" />
26
-    -->
27
-    <!-- Define an SSL/TLS HTTP/1.1 Connector on port 8443
28
-         This connector uses the NIO implementation. The default
29
-         SSLImplementation will depend on the presence of the APR/native
30
-         library and the useOpenSSL attribute of the
31
-         AprLifecycleListener.
32
-         Either JSSE or OpenSSL style configuration may be used regardless of
33
-         the SSLImplementation selected. JSSE style configuration is used below.
34
-    -->
35
-    <!--
36
-    <Connector port="8443" protocol="org.apache.coyote.http11.Http11NioProtocol"
37
-               maxThreads="150" SSLEnabled="true">
38
-        <SSLHostConfig>
39
-            <Certificate certificateKeystoreFile="conf/localhost-rsa.jks"
40
-                         type="RSA" />
41
-        </SSLHostConfig>
42
-    </Connector>
43
-    -->
44
-    <!-- Define an SSL/TLS HTTP/1.1 Connector on port 8443 with HTTP/2
45
-         This connector uses the APR/native implementation which always uses
46
-         OpenSSL for TLS.
47
-         Either JSSE or OpenSSL style configuration may be used. OpenSSL style
48
-         configuration is used below.
49
-    -->
50
-    <!--
51
-    <Connector port="8443" protocol="org.apache.coyote.http11.Http11AprProtocol"
52
-               maxThreads="150" SSLEnabled="true" >
53
-        <UpgradeProtocol className="org.apache.coyote.http2.Http2Protocol" />
54
-        <SSLHostConfig>
55
-            <Certificate certificateKeyFile="conf/localhost-rsa-key.pem"
56
-                         certificateFile="conf/localhost-rsa-cert.pem"
57
-                         certificateChainFile="conf/localhost-rsa-chain.pem"
58
-                         type="RSA" />
59
-        </SSLHostConfig>
60
-    </Connector>
61
-    -->
62
+               maxThreads="400"
63
+               URIEncoding="UTF-8" />	
64
 
65
-    <!-- Define an AJP 1.3 Connector on port 8009 -->
66
-    <!--
67
-    <Connector protocol="AJP/1.3"
68
-               address="::1"
69
-               port="8009"
70
-               redirectPort="8443" />
71
-    -->
72
 
73
     <!-- An Engine represents the entry point (within Catalina) that processes
74
          every request.  The Engine implementation for Tomcat stand alone
75
@@ -161,9 +124,14 @@
76
         <!-- Access log processes all example.
77
              Documentation at: /docs/config/valve.html
78
              Note: The pattern used is equivalent to using pattern="common" -->
79
+	<!--
80
         <Valve className="org.apache.catalina.valves.AccessLogValve" directory="logs"
81
                prefix="localhost_access_log" suffix=".txt"
82
                pattern="%h %l %u %t &quot;%r&quot; %s %b" />
83
+	-->
84
+       <Valve className="org.apache.catalina.valves.CrawlerSessionManagerValve"
85
+              crawlerUserAgents=".*[bB]ot.*|.*Yahoo! Slurp.*|.*Feedfetcher-Google.*|.*Apache-HttpClient.*|.*[Ss]pider.*|.*[Cc]rawler.*|.*nagios.*|.*Yandex.*"
86
+              sessionInactiveInterval="60"/>
87
 
88
       </Host>
89
     </Engine>
90
EOF
Copied!
Disable session persistence in /etc/tomcat9/context.xml by running the following patch:
1
patch /etc/tomcat9/context.xml << "EOF"
2
@@ -25,7 +25,5 @@
3
     <WatchedResource>${catalina.base}/conf/web.xml</WatchedResource>
4
 
5
     <!-- Uncomment this to disable session persistence across Tomcat restarts -->
6
-    <!--
7
     <Manager pathname="" />
8
-    -->
9
 </Context>
10
EOF
Copied!
Apache
Apache must be set up to make the viewer available externally. Activate the following modules for this:
1
a2enmod proxy_ajp
2
a2enmod proxy_http
3
a2enmod proxy_wstunnel
4
a2enmod rewrite
5
a2enmod expires
6
a2enmod headers
Copied!
Place the following file at /etc/apache2/sites-available/${VIEWER_HOSTNAME}.conf:
1
sed -e "s|VIEWER.EXAMPLE.ORG|${VIEWER_HOSTNAME}|g" << "EOF" >/etc/apache2/sites-available/${VIEWER_HOSTNAME}.conf
2
<VirtualHost *:80>
3
        ServerAdmin [email protected]
4
        ServerName VIEWER.EXAMPLE.ORG
5
        DocumentRoot /var/www
6
 
7
        ## make sure rewrite is enabled
8
        RewriteEngine On 
9
 
10
        ## search engines: do not follow certain urls
11
        RewriteCond %{HTTP_USER_AGENT} ^.*bot.*$ [NC,OR]
12
        RewriteCond %{HTTP_USER_AGENT} ^.*Yandex.*$ [NC,OR]
13
        RewriteCond %{HTTP_USER_AGENT} ^.*spider.*$ [NC,OR]
14
        RewriteCond %{HTTP_USER_AGENT} ^.*rawler.*$ [NC]
15
        ReWriteRule ^(.*);jsessionid=[A-Za-z0-9]+(.*)$ $1$2 [L,R=301]
16
 
17
        RewriteCond %{HTTP_USER_AGENT} ^.*bot.*$ [NC,OR]
18
        RewriteCond %{HTTP_USER_AGENT} ^.*Yandex.*$ [NC,OR]
19
        RewriteCond %{HTTP_USER_AGENT} ^.*spider.*$ [NC,OR]
20
        RewriteCond %{HTTP_USER_AGENT} ^.*rawler.*$ [NC]
21
        ReWriteRule ^(.*)viewer/!(.*)$ $1viewer/$2 [L,R=301]
22
 
23
        RewriteCond %{HTTP_USER_AGENT} ^.*bot.*$ [NC,OR]
24
        RewriteCond %{HTTP_USER_AGENT} ^.*Yandex.*$ [NC,OR]
25
        RewriteCond %{HTTP_USER_AGENT} ^.*spider.*$ [NC,OR]
26
        RewriteCond %{HTTP_USER_AGENT} ^.*rawler.*$ [NC]
27
        ReWriteRule ^(.*)/[Ll][Oo][Gg]_(.*)$ $1/ [L,R=301]
28
 
29
        ## compress output
30
        <IfModule mod_deflate.c>
31
                AddOutputFilterByType DEFLATE text/plain text/html text/xml
32
                AddOutputFilterByType DEFLATE text/css text/javascript
33
                AddOutputFilterByType DEFLATE application/xml application/xhtml+xml
34
                AddOutputFilterByType DEFLATE application/rss+xml
35
                AddOutputFilterByType DEFLATE application/javascript application/x-javascript
36
        </IfModule>
37
 
38
        ## general proxy settings
39
        ProxyPreserveHost On
40
        <Proxy *>
41
                Require local
42
        </Proxy>
43
 
44
        ## CORS for IIIF
45
        Header set Access-Control-Allow-Origin "*"
46
        Header always set Access-Control-Allow-Methods "GET, OPTIONS"
47
        Header always set Access-Control-Max-Age "600"
48
        Header always set Access-Control-Allow-Headers "Authorization, Content-Type"
49
        Header always set Access-Control-Expose-Headers "Content-Security-Policy, Location"
50
 
51
        RewriteCond %{REQUEST_METHOD} OPTIONS
52
        RewriteRule ^(.*)$ $1 [R=200,L]
53
 
54
        # make sure ETag headers are forwarded correctly
55
        # Post Apache 2.4 have a look at
56
        # https://httpd.apache.org/docs/trunk/mod/mod_deflate.html#deflatealteretag
57
        RequestHeader edit "If-None-Match" '(.*)-gzip"#x27; '$1", $1-gzip"'
58
 
59
        ## Enable WebSockets to check concurrent access
60
        RewriteCond %{HTTP:Upgrade} websocket [NC]
61
        RewriteCond %{HTTP:Connection} upgrade [NC]
62
        RewriteRule /?(.*) ws://localhost:8080/$1 [P,L]
63
 
64
        ## Viewer
65
        redirect 301 /index.html http://VIEWER.EXAMPLE.ORG/viewer/
66
        redirect 301 /viewer http://VIEWER.EXAMPLE.ORG/viewer/
67
 
68
        RewriteRule ^/viewer/oai/oai2.xsl$ http://VIEWER.EXAMPLE.ORG/viewer/oai2.xsl
69
        ProxyPassMatch ^/viewer/(oai.*)$ ajp://localhost:8009/M2M/$1 retry=0
70
        <LocationMatch ^/viewer/(oai.*)gt;
71
                Forcetype text/xml
72
                ProxyPassReverse ajp://localhost:8009/M2M/$1
73
        </LocationMatch>
74
 
75
        ProxyPassMatch ^/viewer/(sru.*)$ ajp://localhost:8009/M2M/$1 retry=0
76
        <LocationMatch ^/viewer/(sru.*)gt;
77
                Forcetype text/xml
78
                ProxyPassReverse ajp://localhost:8009/M2M/$1
79
        </LocationMatch>
80
 
81
        ProxyPassMatch ^/viewer/(.*)$ ajp://localhost:8009/viewer/$1 retry=0
82
        <LocationMatch ^/viewer/(.*)gt;
83
                ProxyPassReverse ajp://localhost:8009/viewer/$1
84
 
85
                <IfModule mod_expires.c>
86
                        ExpiresActive on
87
 
88
                        ExpiresByType image/jpg "access plus 1 months"
89
                        ExpiresByType image/gif "access plus 1 months"
90
                        ExpiresByType image/jpeg "access plus 1 months"
91
                        ExpiresByType image/png "access plus 1 months"
92
 
93
                        ExpiresByType font/ttf "access plus 1 year"
94
                        ExpiresByType application/x-font-woff "access plus 1 year"
95
                        ExpiresByType application/vnd.ms-fontobject "access plus 1 year"
96
                </IfModule>
97
 
98
                Require all granted  
99
        </LocationMatch>
100
 
101
 
102
        ## solr
103
        redirect 301 /solr http://VIEWER.EXAMPLE.ORG/solr/
104
        <Location /solr/>
105
                Require local
106
                ProxyPass ajp://localhost:8009/solr/ retry=0
107
                ProxyPassReverse ajp://localhost:8009/solr/
108
        </Location>
109
 
110
 
111
        ## logging
112
        CustomLog /var/log/apache2/VIEWER.EXAMPLE.ORG_access.log combined
113
        ErrorLog /var/log/apache2/VIEWER.EXAMPLE.ORG_error.log
114
 
115
        # Possible values include: debug, info, notice, warn, error, crit, alert, emerg.
116
        LogLevel warn
117
 
118
</VirtualHost>
119
EOF
Copied!
Enable the Goobi viewer vhost, disable the default vhost and restart the Apache web server:
1
a2dissite 000-default
2
a2ensite ${VIEWER_HOSTNAME}.conf
3
systemctl restart apache2.service
Copied!
Finally create a robots.txt:
1
sed -e "s|VIEWER.EXAMPLE.ORG|${VIEWER_HOSTNAME}|g" << "EOF" >>/var/www/robots.txt
2
User-agent: *
3
Disallow: /viewer/content*action=pdf
4
Disallow: /viewer/search/
5
Disallow: /viewer/tags/
6
Disallow: /viewer/term/
7
Disallow: /viewer/oai
8
Disallow: /viewer/nextHit/
9
Disallow: /viewer/prevHit/
10
Disallow: /viewer/login/
11
Disallow: /viewer/crowd/
12
Disallow: /error/
13
​
14
#Sitemap: http://VIEWER.EXAMPLE.ORG/viewer/sitemap_index.xml
15
 
16
Crawl-delay: 10
17
EOF
Copied!
If UFW is used, Apache must be enabled:
1
ufw allow Apache\ Full
Copied!
MySQL / MariaDB
The Goobi viewer requires a database and its own user. This is created with the following command:
1
mysql -e "CREATE DATABASE viewer;
2
CREATE USER 'viewer'@'localhost' IDENTIFIED BY '$PW_SQL_VIEWER';
3
GRANT ALL PRIVILEGES ON viewer.* TO 'viewer'@'localhost' WITH GRANT OPTION;
4
FLUSH PRIVILEGES;"
Copied!
The database schema is created automatically the first time the application is started.
NFS
This point is only relevant if Goobi workflow is/is also installed and this is not done on the same machine.
Then the /opt/digiverso/viewer folder must be exported to the Goobi workflow server. NFS is used for this. The adjustments for this are here:
1
apt install nfs-kernel-server -y
Copied!
Export of the hotfolder:
1
export IP_GOOBI=1.2.3.4  # IP-Adresse des Goobi workflow Servers
2
echo "/opt/digiverso/viewer/hotfolder ${IP_GOOBI}/255.255.255.255(rw,sync,no_subtree_check,all_squash,anonuid=$(id -u tomcat),anongid=$(id -g tomcat))" >> /etc/exports
3
exportfs -av
Copied!
If UFW is used, TCP 2049 must be enabled for NFSv4:
1
ufw allow from $IP_GOOBI proto tcp to any port 2049
Copied!
The adjustments for Goobi workflow can be found in the installation instructions there:
​https://docs.intranda.com/goobi-workflow-en/admin/8#einrichtung-von-nfs​
Installation
The installation of the required components is described below.
Apache Solr
Download Solr 8.5.2 and unzip the installation script:
1
cd $install
2
wget http://archive.apache.org/dist/lucene/solr/8.5.2/solr-8.5.2.tgz
3
tar -xzf solr-8.5.2.tgz solr-8.5.2/bin/install_solr_service.sh --strip-components=2
Copied!
Now create the folder /opt/digiverso/solr and install it there:
1
mkdir -p /opt/digiverso/solr/
2
./install_solr_service.sh solr-8.5.2.tgz -i /opt/digiverso/solr -d /opt/digiverso/solr -u solr -s solr -p 8983 -n
Copied!
Now adjust the limits for the user solr:
1
cat << "EOF" >/etc/security/limits.d/solr.conf
2
solr hard nofile 65535
3
solr soft nofile 65535
4
solr hard nproc 65535
5
solr soft nproc 65535
6
EOF
Copied!
To use streaming expressions Solr is not installed in the standalone version but in the SolrCloud variant. However, we only use one node. For the configuration Zookeeper is used:
Make sure that Zookeeper only listens on localhost:
1
patch /etc/zookeeper/conf/zoo.cfg << "EOF"
2
@@ -15,6 +15,7 @@
3
 
4
 # the port at which the clients will connect
5
 clientPort=2181
6
+clientPortAddress=127.0.0.1
7
 
8
 # specify all zookeeper servers
9
 # The fist port is used by followers to connect to the leader
10
EOF
11
systemctl restart zookeeper
Copied!
Then make the installation known to Solr and adjust the memory, the garbage collector options and the log level. Furthermore, the Solr should only listen on localhost:
1
patch /etc/default/solr.in.sh << "EOF"
2
@@ -30,3 +30,3 @@
3
 # Increase Java Heap as needed to support your indexing / query needs
4
-#SOLR_HEAP="512m"
5
+SOLR_HEAP="2048m"
6
 
7
@@ -48,15 +48,15 @@
8
 # These GC settings have shown to work well for a number of common Solr workloads
9
-#GC_TUNE=" \
10
-#-XX:SurvivorRatio=4 \
11
-#-XX:TargetSurvivorRatio=90 \
12
-#-XX:MaxTenuringThreshold=8 \
13
-#-XX:+UseConcMarkSweepGC \
14
-#-XX:ConcGCThreads=4 -XX:ParallelGCThreads=4 \
15
-#-XX:+CMSScavengeBeforeRemark \
16
-#-XX:PretenureSizeThreshold=64m \
17
-#-XX:+UseCMSInitiatingOccupancyOnly \
18
-#-XX:CMSInitiatingOccupancyFraction=50 \
19
-#-XX:CMSMaxAbortablePrecleanTime=6000 \
20
-#-XX:+CMSParallelRemarkEnabled \
21
-#-XX:+ParallelRefProcEnabled \
22
+GC_TUNE=" \
23
+-XX:SurvivorRatio=4 \
24
+-XX:TargetSurvivorRatio=90 \
25
+-XX:MaxTenuringThreshold=8 \
26
+-XX:+UseConcMarkSweepGC \
27
+-XX:ConcGCThreads=4 -XX:ParallelGCThreads=4 \
28
+-XX:+CMSScavengeBeforeRemark \
29
+-XX:PretenureSizeThreshold=64m \
30
+-XX:+UseCMSInitiatingOccupancyOnly \
31
+-XX:CMSInitiatingOccupancyFraction=50 \
32
+-XX:CMSMaxAbortablePrecleanTime=6000 \
33
+-XX:+CMSParallelRemarkEnabled \
34
+-XX:+ParallelRefProcEnabled"
35
 #-XX:-OmitStackTraceInFastThrow  etc.
36
@@ -66,3 +66,3 @@
37
 # Leave empty if not using SolrCloud
38
-#ZK_HOST=""
39
+ZK_HOST="127.0.0.1:2181"
40
 
41
@@ -73,3 +73,3 @@
42
 # for production SolrCloud environments to control the hostname exposed to cluster state
43
-#SOLR_HOST="192.168.1.1"
44
+SOLR_HOST="127.0.0.1"
45
 
46
@@ -115,3 +115,3 @@
47
 # This is an alternative to changing the rootLogger in log4j2.xml
48
-#SOLR_LOG_LEVEL=INFO
49
+SOLR_LOG_LEVEL=ERROR
50
 
51
@@ -212,3 +212,3 @@
52
 # host checking can be disabled by using the system property "solr.disable.shardsWhitelist"
53
-#SOLR_OPTS="$SOLR_OPTS -Dsolr.shardsWhitelist=http://localhost:8983,http://localhost:8984"
54
+SOLR_OPTS="$SOLR_OPTS -Dsolr.disable.shardsWhitelist"
55
 
56
@@ -234,2 +234,3 @@
57
 SOLR_PORT="8983"
58
+SOLR_OPTS="$SOLR_OPTS -Djetty.host=127.0.0.1"
59
 
60
EOF
Copied!
Now make the Solr script executable:
1
chmod 755 /opt/digiverso/solr/solr/bin/solr
Copied!
Now the configuration is stored in a new configset:
1
cd /opt/digiverso/solr/solr/server/solr/configsets/
2
cp -a _default/ goobiviewer
3
cd goobiviewer/conf/
4
rm managed-schema
5
wget -O schema.xml https://raw.githubusercontent.com/intranda/goobi-viewer-indexer/master/goobi-viewer-indexer/src/main/resources/other/schema.xml
6
cp lang/stopwords_de.txt lang/stopwords.txt
7
wget https://raw.githubusercontent.com/intranda/goobi-viewer-indexer/master/goobi-viewer-indexer/src/main/resources/other/mapping-ISOLatin1Accent.txt
8
patch solrconfig.xml << "EOF"
9
@@ -21,6 +21,8 @@
10
      this file, see http://wiki.apache.org/solr/SolrConfigXml.
11
 -->
12
 <config>
13
+  <schemaFactory class="ClassicIndexSchemaFactory"/>
14
+
15
   <!-- In all configuration below, a prefix of "solr." for class names
16
        is an alias that causes solr to search appropriate packages,
17
        including org.apache.solr.(search|update|request|core|analysis)
18
@@ -57,6 +59,10 @@
19
 
20
               <lib dir="./lib" />
21
     -->
22
+  <lib dir="${solr.install.dir:../../../..}/contrib/analysis-extras/lib" regex=".*\.jar" />
23
+  <lib dir="${solr.install.dir:../../../..}/contrib/analysis-extras/lucene-libs" regex=".*\.jar" />
24
+  <lib dir="${solr.install.dir:../../../..}/dist/" regex="solr-analysis-extras-\d.*\.jar" />
25
+
26
 
27
   <!-- A 'dir' option by itself adds any files found in the directory
28
        to the classpath, this is useful for including all jars in a
29
@@ -773,7 +779,7 @@
30
 
31
   <initParams path="/update/**,/query,/select,/spell">
32
     <lst name="defaults">
33
-      <str name="df">_text_</str>
34
+      <str name="df">DEFAULT</str>
35
     </lst>
36
   </initParams>
37
 
38
@@ -1108,7 +1114,7 @@
39
   </updateProcessor>
40
 
41
   <!-- The update.autoCreateFields property can be turned to false to disable schemaless mode -->
42
-  <updateRequestProcessorChain name="add-unknown-fields-to-the-schema" default="${update.autoCreateFields:true}"
43
+  <updateRequestProcessorChain name="add-unknown-fields-to-the-schema" default="${update.autoCreateFields:false}"
44
            processor="uuid,remove-blank,field-name-mutating,parse-boolean,parse-long,parse-double,parse-date,add-schema-fields">
45
     <processor class="solr.LogUpdateProcessorFactory"/>
46
     <processor class="solr.DistributedUpdateProcessorFactory"/>
47
​
48
EOF
49
chown -R solr. *
Copied!
Upload the new configset to Zookeeper now:
1
cd /opt/digiverso/solr/solr/
2
sudo -u solr bin/solr zk upconfig -n goobiviewer -d server/solr/configsets/goobiviewer/
Copied!
The JTS library is still needed for the search for geocoordinates:
1
wget -O /opt/digiverso/solr/solr/server/solr-webapp/webapp/WEB-INF/lib/jts-core-1.17.0.jar https://github.com/locationtech/jts/releases/download/1.17.0/jts-core-1.17.0.jar
2
chown solr. /opt/digiverso/solr/solr/server/solr-webapp/webapp/WEB-INF/lib/jts-core-1.17.0.jar
Copied!
Finally, start the service and create a new collection for the Goobi viewer with its configuration files:Solr in Tomcat:
1
systemctl start solr
2
sudo -u solr bin/solr create -c collection1 -n goobiviewer
Copied!
Goobi viewer Indexer
First the application is downloaded. Then copy the required files to /opt/digiverso/indexer/ and rename the indexerconfig_solr.xml to solr_indexerconfig.xml, check the Tomcat port and replace C: with /opt if necessary. Finally the systemd service unit is activated:
1
wget -O /opt/digiverso/indexer/solrIndexer.jar https://github.com/intranda/goobi-viewer-indexer/releases/latest/download/solrIndexer.jar
2
wget -O /opt/digiverso/indexer/solr_indexerconfig.xml https://raw.githubusercontent.com/intranda/goobi-viewer-indexer/master/goobi-viewer-indexer/src/main/resources/indexerconfig_solr.xml
3
sed -e 's|<solrUrl>.*</solrUrl>|<solrUrl>http://localhost:8983/solr/collection1</solrUrl>|' -e 's|C:/|/|g' -i /opt/digiverso/indexer/solr_indexerconfig.xml
4
wget -O /etc/systemd/system/solrindexer.service https://raw.githubusercontent.com/intranda/goobi-viewer-indexer/master/goobi-viewer-indexer/src/main/resources/other/solrindexer.service
5
systemctl enable solrindexer.service
Copied!
Goobi viewer
Stop the Tomcat service:
1
systemctl stop tomcat9
Copied!
Then put the following file to /etc/tomcat9/Catalina/localhost/viewer.xml:
1
sed -e "s|PW_SQL_VIEWER|${PW_SQL_VIEWER}|g" << "EOF" >/etc/tomcat9/Catalina/localhost/viewer.xml
2
<?xml version='1.0' encoding='UTF-8'?>
3
<Context>
4
    <Resources>
5
        <PreResources 
6
            className="org.apache.catalina.webresources.DirResourceSet"
7
            base="/opt/digiverso/viewer/themes/goobi-viewer-theme-reference/goobi-viewer-theme-reference/WebContent/resources/themes/"
8
            webAppMount="/resources/themes" />  
9
    </Resources>
10
    <Resource
11
        name="viewer" 
12
        auth="Container" 
13
        factory="org.apache.tomcat.jdbc.pool.DataSourceFactory" 
14
        type="javax.sql.DataSource" 
15
        driverClassName="org.mariadb.jdbc.Driver" 
16
        username="viewer" 
17
        password="PW_SQL_VIEWER" 
18
        maxActive="100" 
19
        maxIdle="30" 
20
        minIdle="4" 
21
        maxWait="10000" 
22
        testOnBorrow="true" 
23
        testWhileIdle="true" 
24
        validationQuery="SELECT SQL_NO_CACHE 1" 
25
        removeAbandoned="true" 
26
        removeAbandonedTimeout="600" 
27
        url="jdbc:mysql://localhost/viewer?characterEncoding=UTF-8&amp;autoReconnect=true&amp;autoReconnectForPools=true" />
28
</Context>
29
EOF
Copied!
Now adjust the file rights so that the file is not deleted with an update:
1
chown -R root.tomcat /etc/tomcat9/Catalina
2
chmod -R g-w /etc/tomcat9/Catalina
Copied!
The theme is integrated as an external theme:
1
mkdir -p /opt/digiverso/viewer/themes/
2
cd /opt/digiverso/viewer/themes/
3
git clone https://github.com/intranda/goobi-viewer-theme-reference.git
Copied!
Now download the Goobi viewer and place it in the expected location in the file system:
1
wget -O /var/lib/tomcat9/webapps/viewer.war https://github.com/intranda/goobi-viewer-theme-reference/releases/latest/download/viewer.war
Copied!
Finally move the last configuration files and restart the Tomcat service:
1
mv $install/goobi-viewer-core-config/goobi-viewer-core-config/src/main/resources/install/* /opt/digiverso/viewer/config/
2
chown -R tomcat: /opt/digiverso/viewer/
3
systemctl start tomcat9
Copied!
Connector - (OAI & SRU)
For the installation of the OAI and SRU interface, the application must first be placed in the expected location in the file system:
1
wget -O /var/lib/tomcat9/webapps/M2M.war https://github.com/intranda/goobi-viewer-connector/releases/latest/download/M2M.war
Copied!
Then download further required files and adjust the rights:
1
wget -O /opt/digiverso/viewer/oai/MARC21slimUtils.xsl https://raw.githubusercontent.com/intranda/goobi-viewer-connector/master/goobi-viewer-connector/src/main/resources/MARC21slimUtils.xsl
2
wget -O /opt/digiverso/viewer/oai/MODS2MARC21slim.xsl https://raw.githubusercontent.com/intranda/goobi-viewer-connector/master/goobi-viewer-connector/src/main/resources/MODS2MARC21slim.xsl
3
chown -R tomcat: /opt/digiverso/viewer/oai/
Copied!
Finally, place a local config_oai.xml in the file system:
1
sed -e "s|VIEWER.EXAMPLE.ORG|${VIEWER_HOSTNAME}|g" << "EOF" >/opt/digiverso/viewer/config/config_oai.xml
2
<?xml version="1.0" encoding="UTF-8"?>
3
<config>
4
        <identifyTags>
5
                <baseURL useInRequestElement="true">http://VIEWER.EXAMPLE.ORG/viewer/oai</baseURL>
6
                <adminEmail>[email protected]</adminEmail>
7
        </identifyTags>
8
        <solr>
9
                <solrUrl>http://localhost:8983/solr/collection1</solrUrl>
10
        </solr>
11
        <urnResolverUrl>http://VIEWER.EXAMPLE.ORG/viewer/resolver?urn=</urnResolverUrl>
12
        <piResolverUrl>http://VIEWER.EXAMPLE.ORG/viewer/piresolver?id=</piResolverUrl>
13
</config>
14
EOF
Copied!
Adjust the settings to the correct URL so that the resolver links work and ensure that the port of the tomcat on which the Solr service is running is 8080:
1
sed -i -e "s|http://localhost:8080/viewer/oai|http://${VIEWER_HOSTNAME}/viewer/oai|g" -e "s|http://localhost:8080/viewer/resolver|http://${VIEWER_HOSTNAME}/viewer/resolver|g" -e "s|http://localhost:8080/viewer/piresolver|http://${VIEWER_HOSTNAME}/viewer/piresolver|g" -e "s|http://localhost:8081/solr|http://localhost:8983/solr|g" /opt/digiverso/viewer/config/config_oai.xml
Copied!
Settings
Cronjob
A cronjob must be set up for regular tasks:
1
sed -e "s|VIEWER.EXAMPLE.ORG|${VIEWER_HOSTNAME}|g" -e "s|TOKEN|${TOKEN}|g" << "EOF" >/etc/cron.d/intranda-goobiviewer
2
PATH=/usr/bin:/bin:/usr/sbin/
3
[email protected]
4
 
5
#
6
# Regular cron jobs for the Goobi viewer
7
#
8
 
9
## This REST call triggers the email notification about new search hits for users, 
10
## that enabled notifications for saved searches
11
42 8,12,17  * * *   root    curl -s -H "Content-Type: application/json" -H "token:TOKEN" -d '{"type":"NOTIFY_SEARCH_UPDATE"}' http://localhost:8080/viewer/api/v1/tasks/ 1>/dev/null
12
 
13
## This REST call creates an XML sitemap for the Goobi viewer instance. Please always 
14
## call it on it's external URL because otherwise the protocol (http/https) might not 
15
## be detected correctly
16
18 1        * * *   root    curl -s -H "Content-Type: application/json" -H "token:TOKEN" -d '{"type":"UPDATE_SITEMAP"}' http://localhost:8080/viewer/api/v1/tasks/ 1>/dev/null
17
 
18
## These two scripts pull the theme git repository regulary. The @daily part is only 
19
## a reminder for the 1-minute schedule
20
#*/1 *       * * *   root    cd /opt/digiverso/viewer/themes/goobi-viewer-theme-reference; git pull | grep -v -e "Already up.to.date." -e "Bereits aktuell."
21
#@daily              root    echo "Please look at the git checkout interval for the Goobi viewer theme" | mail -s "Reference: Theme repository is checked out every minute" [email protected] 
22
 
23
## Optimize the Solr search index once a month
24
@monthly            root    curl -s 'http://localhost:8080/solr/collection1/update?optimize=true&waitFlush=false'
25
EOF
Copied!
config_viewer.xml
In the local config_viewer.xml different settings have to be stored:
1
sed -e "s|VIEWER.EXAMPLE.ORG|${VIEWER_HOSTNAME}/viewer|g" -e "s|TOKEN|${TOKEN}|g" << "EOF" >/opt/digiverso/viewer/config/config_viewer.xml
2
<?xml version="1.0" encoding="UTF-8" ?>
3
<config>
4
        <urls>
5
                <metadata>
6
                        <mets>http://VIEWER.EXAMPLE.ORG/oai?verb=GetRecord&amp;metadataPrefix=mets&amp;identifier=</mets>
7
                        <marc>http://VIEWER.EXAMPLE.ORG/oai?verb=GetRecord&amp;metadataPrefix=marcxml&amp;identifier=</marc>
8
                        <dc>http://VIEWER.EXAMPLE.ORG/oai?verb=GetRecord&amp;metadataPrefix=oai_dc&amp;identifier=</dc>
9
                        <ese>http://VIEWER.EXAMPLE.ORG/oai?verb=GetRecord&amp;metadataPrefix=europeana&amp;identifier=</ese>
10
                </metadata>
11
 
12
                <contentServerWrapper>http://VIEWER.EXAMPLE.ORG/content/</contentServerWrapper>
13
                <download>http://VIEWER.EXAMPLE.ORG/download/</download>
14
                <rest>http://VIEWER.EXAMPLE.ORG/api/v1/</rest>
15
                <solr>http://localhost:8983/solr/collection1</solr>
16
        </urls>
17
 
18
        <viewer>
19
                <theme mainTheme="reference" discriminatorField="">
20
                        <rootPath>/opt/digiverso/goobi-viewer-theme-reference/goobi-viewer-theme-reference/WebContent/resources/themes/</rootPath>
21
                </theme>
22
        </viewer>
23
 
24
        <rss>
25
                <numberOfItems>50</numberOfItems>
26
                <title>Goobi viewer RSS Feed</title>
27
                <description>new items</description>
28
                <copyright>(c) Goobi viewer using institution </copyright>
29
        </rss>
30
 
31
        <webapi>        
32
                <authorization>
33
                        <token>TOKEN</token>
34
                </authorization>
35
        </webapi>
36
</config>
37
EOF
Copied!
Create user account
The Goobi viewer has a backend. The following command inserts a test account with the user name [email protected] and the password specified in the beginning into the database:
1
VIEWER_USERPASS_HASH=$(python3 -c "from passlib.hash import bcrypt; print(bcrypt.using(rounds=10, ident='2a').hash('${VIEWER_USERPASS}'))")
2
mysql viewer -e "INSERT INTO users (active,email,password_hash,score,superuser) VALUES (1,'${VIEWER_USERNAME}','${VIEWER_USERPASS_HASH}',0,1);"
Copied!