Warning: This page has not been updated in over over a year and may be outdated or deprecated.
administration:performance
Differences
This shows you the differences between two versions of the page.
Both sides previous revisionPrevious revisionNext revision | Previous revision | ||
administration:performance [2017/08/01 18:43] – demiankatz | administration:performance [2022/05/06 12:52] (current) – [Garbage Collection] demiankatz | ||
---|---|---|---|
Line 4: | Line 4: | ||
====== Java Tuning ====== | ====== Java Tuning ====== | ||
- | After the initial setup and importing of records, you may notice a significant degradation of performance. This means its time to do some JVM tuning. However, if you look around the Internet, you'll quickly realize performance tuning for Java applications is a bit of a black art. However, we'll provide some basic settings to help improve your performance, | + | After the initial setup and importing of records, you may notice a significant degradation of performance. This means it' |
As a short side-note, these recommendations are for a " | As a short side-note, these recommendations are for a " | ||
- | ===== JAVA_OPTIONS | + | ===== Setting Options |
- | + | ||
- | :!: This section refers to VuFind 2.x and earlier, which include Solr 4 or earlier. Starting with VuFind 3.0, Solr 5 is bundled with VuFind, and it uses a different startup routine. To adjust the heap memory size in these versions, simply change the SOLR_HEAP environment variable. | + | |
- | + | ||
- | If you are using the included distribution of Jetty, the tuning options are set in the JAVA_OPTIONS environmental variable. You can either set this for the user running the instance of Jetty (recommended) or in / | + | |
- | + | ||
- | It's recommended that you run the most recent [[http:// | + | |
+ | :!: This section refers to VuFind 3.x and later. For instructions on earlier releases, see the change history of the page and read an earlier version. | ||
+ | It's recommended that you run the most recent [[https:// | ||
==== Heap Space ==== | ==== Heap Space ==== | ||
- | The heap space in Java is the amount of memory allocated to Java for objects. One of the more common issues with a full index of records is an java.lang.OutOfMemoryError exception due to the heap space size. By default, | + | The heap space in Java is the amount of memory allocated to Java for objects. One of the more common issues with a full index of records is an java.lang.OutOfMemoryError exception due to the heap space size. By default, |
- | + | ||
- | It's a good idea to set both of these to the same value as follows (assumes 4GB of RAM): | + | |
< | < | ||
- | -Xms3800m -Xmx3800m | + | SOLR_HEAP=3800M $VUFIND_HOME/ |
</ | </ | ||
- | Note: For very large indices (> 4 million documents) you will need more than 4GB of RAM, and a 64 bit OS to utilize it. You must also tell Java to use the 64 bit " | + | Note: For very large indices (> 4 million documents) you will need more than 4GB of RAM, and a 64 bit OS to utilize it. |
- | + | ||
- | < | + | |
- | -d64 -Xms8192m -Xmx8192m | + | |
- | </ | + | |
- | + | ||
- | Failure to use the " | + | |
- | " | + | |
- | The specified size exceeds the maximum representable size. | + | |
- | Could not create the Java virtual machine." | + | |
Please note that simply increasing the heap size to a very large one is not recommended, | Please note that simply increasing the heap size to a very large one is not recommended, | ||
Line 44: | Line 28: | ||
for more information). | for more information). | ||
==== Garbage Collection ==== | ==== Garbage Collection ==== | ||
- | :!: Starting with VuFind 3.0, VuFind includes | + | :!: Since version 5 (included in VuFind 3), Solr has done some garbage collection auto-tuning by default; the suggestions below are probably not necessary |
- | + | ||
- | Garbage collection is how aggressive the JVM is with clearing out unneeded objects. For the purposes of Vufind, parallel garbage collection should work nicely, but read Sun's [[http:// | + | |
- | + | ||
- | < | + | |
- | -XX: | + | |
- | </ | + | |
- | + | ||
- | === Young Generation === | + | |
- | Young generation is a type of garbage collection that has three object spaces, the new object space (Eden) and two survivor spaces. Newer objects are created in Eden, while longer lived objects are moved to the old generation survivor (tenured) spaces. Young generation uses fast copying garbage collection for more frequent clearing of Eden, and more spaced out full garbage collections in the tenured space (which is slower). There are a few switches that will help with this. | + | |
- | + | ||
- | < | + | |
- | -XX: | + | |
- | </ | + | |
- | Note: if you use the -XX: | + | Garbage collection is how aggressive |
=== Collecting Data === | === Collecting Data === | ||
- | If you want to learn more about your garbage collection performance, | + | If you want to learn more about your garbage collection performance, |
< | < | ||
- | -Xloggc: | + | SOLR_ADDITIONAL_JVM_OPTIONS=" |
</ | </ | ||
- | Once you have a log file, you can use the [[https:// | + | Once you have a log file, you can use the [[https:// |
+ | :!: Note that as of VuFind 5.0 (and possibly earlier), the default Solr configuration in VuFind creates a garbage collection log in $VUFIND_HOME/ | ||
- | ==== Setting JAVA_OPTIONS ==== | + | :!: Note that in older versions of Java (8 and earlier), -Xlog:gc was called |
- | For a good base setting for JAVA_OPTIONS, | + | ==== Setting SOLR_ADDITIONAL_JVM_OPTIONS |
- | + | ||
- | < | + | |
- | JAVA_OPTIONS=" | + | |
- | </ | + | |
- | + | ||
- | For more information on Java Tuning, see [[http:// | + | |
- | + | ||
- | NOTE: The -server switch needs to be the first argument in the string in order for the JDK to pick up the setting. | + | |
- | + | ||
- | ==== Reminders ==== | + | |
- | Be sure to check that the settings are getting applied by running | + | |
- | + | ||
- | < | + | |
- | ./vufind.sh check | + | |
- | </ | + | |
- | + | ||
- | If all went well, you should see output along the lines of | + | |
- | + | ||
- | < | + | |
- | Checking arguments to VuFind: | + | |
- | VUFIND_HOME | + | |
- | SOLR_HOME | + | |
- | SOLR_DATA_DIR | + | |
- | JETTY_HOME | + | |
- | JETTY_LOG | + | |
- | JETTY_CONF | + | |
- | JETTY_RUN | + | |
- | JETTY_PID | + | |
- | JETTY_CONSOLE | + | |
- | JETTY_PORT | + | |
- | CONFIGS | + | |
- | JAVA_OPTIONS | + | |
- | JAVA = / | + | |
- | CLASSPATH | + | |
- | RUN_CMD | + | |
- | </ | + | |
- | + | ||
- | ===== User Contributed Settings ===== | + | |
- | William and Mary: Dell Precision 2950, 2x2 dual core Xeon 3.2 GHz processors, 4GB RAM | + | |
- | < | + | |
- | JAVA_OPTIONS=" | + | |
- | </ | + | |
+ | As noted above, if you need to send extra options to your Java startup command line, you can use the SOLR_ADDITIONAL_JVM_OPTIONS environment variable. You can export this to your environment, | ||
+ | Best practices for Java tuning have evolved over time and with new releases of Java; try a search for "java tuning" | ||
===== Resources ===== | ===== Resources ===== | ||
- | Here's a list of useful | + | Here's a list of websites on Java performance tuning; note that some of these are now quite old and may be less relevant to newer JVMs: |
* [[http:// | * [[http:// | ||
Line 130: | Line 62: | ||
====== PHP Tuning ====== | ====== PHP Tuning ====== | ||
- | On PHP 5.4, installing a PHP cache like [[http:// | + | On PHP 5.4, installing a PHP cache like [[http:// |
- | + | ||
- | If you created a custom module, remember to create an autoloader_classmap.php by calling | + | |
- | <code php> | + | |
- | ../ | + | |
- | </ | + | |
- | from module directory. | + | |
===== Asset Pipeline ===== | ===== Asset Pipeline ===== | ||
Line 154: | Line 79: | ||
Note: NEVER EDIT THE COMPILED THEME. Instead, edit the source theme and recompile it when you need to make changes. You must use the --force switch of the compiler if you need to overwrite your compiled theme with a new version. | Note: NEVER EDIT THE COMPILED THEME. Instead, edit the source theme and recompile it when you need to make changes. You must use the --force switch of the compiler if you need to overwrite your compiled theme with a new version. | ||
- | ===== Troubleshooting | + | ===== Session Handling |
- | There have been some reports of VuFind errors when running APC. | + | When using database sessions, deleting expired sessions may be a surprisingly heavy process. On a busy site it may be beneficial to turn off PHP's session garbage collection |
- | // | + | ====== Apache Tuning ====== |
- | Inserting the line in index.php | + | |
- | // | + | |
- | <code php> | + | ===== GZIP Compression ===== |
- | register_shutdown_function(' | + | |
- | </ | + | |
- | // | + | For a productive environment, |
- | immediately before requiring the session handler, seems to fix the problem with APC. Since this is the result of trawling the web and not of any deep understanding on my part, I can't guarantee this won't have sideffects (I'll report back if I find any), or that it will always work. | + | |
- | // | + | |
- | + | ||
- | Note: As of VuFind 1.4, the default VuFind code will register the shutdown function, so this modification should no longer be necessary. | + | |
- | + | ||
- | ====== Apache Tuning ====== | + | |
- | For a productive environment, | + | |
<code bash> | <code bash> | ||
Line 183: | Line 97: | ||
</ | </ | ||
- | One thing you should also do is minifying JS and CSS files. This removes whitespaces and line breaks. It can by done by your IDE. Netbeans for example has a plugin Js CSS Minify Compress. The minified files must be configured in theme.config.php and in some cases in the theme files. Just search for " | + | Note that the " |
+ | |||
+ | ===== Minification ===== | ||
+ | |||
+ | One thing you should also do is minifying JS and CSS files. This removes whitespaces and line breaks. It can by done by your IDE. Netbeans for example has a plugin Js CSS Minify Compress. The minified files must be configured in theme.config.php and in some cases in the theme files. Just search for " | ||
+ | |||
+ | :!: Manual minification should not be necessary if you turn on VuFind' | ||
====== Solr Tuning ====== | ====== Solr Tuning ====== | ||
Line 252: | Line 172: | ||
See the [[configuration: | See the [[configuration: | ||
+ | ===== Limits ===== | ||
+ | |||
+ | Certain operating system limits can impact Solr performance, | ||
===== Further Reading ===== | ===== Further Reading ===== | ||
Line 268: | Line 191: | ||
---- struct data ---- | ---- struct data ---- | ||
+ | properties.Page Owner : | ||
---- | ---- | ||
administration/performance.txt · Last modified: 2022/05/06 12:52 by demiankatz