Table of Contents
Solr Replication
Solr may be configured to replicate between servers. This can be useful for separating your indexing processes from your running search system, for load balancing, etc. Newer versions of Solr also support a more complex “Solr Cloud” configuration for robust multi-server searching; see VUFIND-834 for some notes.
solrconfig.xml
For basic Solr replication, in which one or more servers clone the index from a single authoritative source of truth, the following configuration can be added to the $VUFIND_HOME/solr/$core/conf/solrconfig.xml file (uncomment the appropriate master or slave sections):
<requestHandler name="/replication" class="solr.ReplicationHandler" > <!-- To enable simple master/slave replication, uncomment one of the sections below, depending on whether this solr instance should be the "master" or a "slave". If this instance is a "slave" you will also need to fill in the masterUrl to point to a real machine. --> <!-- <lst name="master"> <str name="replicateAfter">commit</str> <str name="replicateAfter">startup</str> <str name="replicateAfter">optimize</str> <str name="confFiles">schema.xml,stopwords.txt</str> </lst> --> <!-- <lst name="slave"> <str name="masterUrl">http://master:8983/solr/biblio</str> <str name="pollInterval">00:00:60</str> </lst> --> </requestHandler>
pull-browse.sh (AlphaBrowse Synchronization)
A Bash script to pull browse indexes from another host. The idea is that if you are running a Solr slave to keep the indexing load away from searches, you probably want to do the same for the alphabetical browse indexes. But there's no replication for SQLite! Prereqs are curl and passwordless ssh/scp between the hosts for the administrative user.
#!/bin/bash # Tod Olson <tod@uchicago.edu> # Much is stolen shamelessly from scripts by Keith Waclena # # Copy browse index from a remote host to local host via scp. Sets up # the .db-ready file semaphore and calls browse handler to move the # index into place. # ME=`basename $0 .sh` USAGE="Usage: $ME [-t] [-V remote_home] [user@]host index; -H for help" HELP="OPTIONS: -H: print this help message -B: use scp batch and quiet modes -V: VUFIND_HOME on remote host (if different from localhost) -t: testing only - NYI ARGUMENTS: host: name of the remote host to copy from, can accept SSH-style user@host syntax index: name of the index (less the _browse suffix) ENVIRONMENT The following variables must be set: VUFIND_HOME" TAB="$(printf '\t')" TESTING=false # prerequisites prereqs () { satisfied= missing= for p in $@ do if type "$p" > /dev/null 2>&1 then satisfied="$satisfied| $p${TAB}$(which $p)" else missing="$missing $p" fi done if [ "$missing" != "" ] then ( echo "$ME: missing prerequisites:" echo "$missing" echo -n "$ME: satisfied prerequisites:" echo "$satisfied" | tr \| '\012' ) >&2 exit 69 else : ok fi } noreq () { fatal "DRYROT: missing prerequisite!" } prereqs scp # handy functions usage () { echo "$USAGE" 1>&2 case "$1" in '') ;; 0) exit 0 ;; *) exit "$1" ;; esac } nyi () { echo "$1: not yet implemented" 1>&2 exit 2 } warning () { echo "$*" 1>&2 } fatal () { if expr "$1" : '[0-9][0-9]*$' > /dev/null then X=$1; shift else X=1 fi echo "$*" 1>&2 exit $X } prefix () { if $TESTING then echo "TEST$1" else echo "$1" fi } # # Check environment # if [ -z "$VUFIND_HOME" ] then fatal "VUFIND_HOME not set" fi REMOTE_VUFIND_HOME="$VUFIND_HOME" SCP_BATCH="" SSH_QUIET="" while true do case "$1" in -H) shift; usage; echo "$HELP"; exit 0 ;; -t) shift; TESTING=true ;; -B) shift; SCP_BATCH="-Bq"; SSH_QUIET="-q";; -V) REMOTE_VUFIND_HOME=$2; shift; shift;; --) shift; break ;; -) break ;; -*) usage 1 ;; *) break ;; esac done [ $# -eq 2 ] || usage 1 REMOTE_HOST=$1 INDEX=$2 # # Main # INDEX_BASE=${INDEX}_browse REMOTE_BASE=$REMOTE_VUFIND_HOME/solr/alphabetical_browse/${INDEX_BASE} LOCAL_BASE=$VUFIND_HOME/solr/alphabetical_browse/${INDEX_BASE} # .db-ready is the signal that .db-updated is complete, but # .db-updated could move into place and .db-ready could be deleted at # any time if ssh $SSH_QUIET $REMOTE_HOST test -f ${REMOTE_BASE}.db-ready then scp ${SCP_BATCH} -p ${REMOTE_HOST}:${REMOTE_BASE}.db-updated ${LOCAL_BASE}.db-updated || scp ${SCP_BATCH} -p ${REMOTE_HOST}:${REMOTE_BASE}.db ${LOCAL_BASE}.db-updated else scp ${SCP_BATCH} -p ${REMOTE_HOST}:${REMOTE_BASE}.db ${LOCAL_BASE}.db-updated fi if [ $? -ne 0 ] then fatal "failed to copy $INDEX_BASE from $REMOTE_HOST" fi touch ${LOCAL_BASE}.db-ready && curl "http://localhost:8983/solr/biblio/browse?source=${INDEX}" > /dev/null # Local Variables: # End: