Taking Solr to Production

We recommend separating your live Solr files, such as logs and index files, from the files included in the Solr distribution bundle, as that makes it easier to upgrade Solr and is considered a good practice to follow as a system administrator.

Running Solr as root is not recommended for security reasons, and the bin/solr start command will refuse to do so. Consequently, you should determine the username of a system user that will own all of the Solr files and the running Solr process. By default, the installation script will create the solr user, but you can override this setting using the -u option. If your organization has specific requirements for creating new user accounts, then you should create the user before running the script. The installation script will make the Solr user the owner of the /opt/solr and /var/solr directories.

You are now ready to run the installation script.

To run the script, you’ll need to download the latest Solr distribution archive and then do the following:

The previous command extracts the install_solr_service.sh script from the archive into the current directory. If installing on Red Hat, please make sure lsof is installed before running the Solr installation script (sudo yum install lsof). The installation script must be run as root:

By default, the script extracts the distribution archive into /opt, configures Solr to write files into /var/solr, and runs Solr as the solr user. Consequently, the following command produces the same result as the previous command:

You can customize the service name, installation directories, port, and owner using options passed to the installation script. To see available options, simply do:

Once the script completes, Solr will be installed as a service and running in the background on your server (on port 8983). To verify, you can do:

If you do not want to start the service immediately, pass the -n option. You can then start the service manually later, e.g., after completing the configuration setup.

We’ll cover some additional configuration settings you can make to fine-tune your Solr setup in a moment. Before moving on, let’s take a closer look at the steps performed by the installation script. This gives you a better overview and will help you understand important details about your Solr installation when reading other pages in this guide; such as when a page refers to Solr home, you’ll know exactly where that is on your system.

In the next section, we cover some additional environment settings to help you fine-tune your production setup. However, before we move on, let’s review what we’ve achieved thus far. Specifically, you should be able to control Solr using /etc/init.d/solr. Please verify the following commands work with your setup:

The status command should give some basic information about the running Solr node that looks similar to:

If the status command is not successful, look for error messages in /var/solr/logs/solr.log.

The Merge Scheduler is configured in solrconfig.xml and defaults to ConcurrentMergeScheduler. This scheduler uses multiple threads to merge Lucene segments in the background.

By default, the ConcurrentMergeScheduler auto-detects defaults for maxThreadCount and maxMergeCount accordingly. maxThreadCount is set to 4 or half the number of processors available to the JVM whichever is greater and maxMergeCount is set to maxThreadCount+5.

If you have a spinning disk, it is best to explicitly set values for maxThreadCount and maxMergeCount in the IndexConfig section of SolrConfig.xml so that values appropriate to your hardware are used.

By default, the bin/solr script sets the maximum Java heap size to 512M (-Xmx512m), which is fine for getting started with Solr. For production, you’ll want to increase the maximum heap size based on the memory requirements of your search application; values between 8 and 16 gigabytes are not uncommon for production servers. When you need to change the memory settings for your Solr server, use the SOLR_HEAP variable in the include file, such as:

Also, the Solr Control Script comes with a set of pre-configured Garbage First Garbage Collection settings that have shown to work well with Solr for a number of different workloads. However, these settings may not work well for your specific use of Solr. Consequently, you may need to change the GC settings, which should also be done with the GC_TUNE variable in the /etc/default/solr.in.sh include file.

For more information about garbage collection settings refer to following articles: . https://cwiki.apache.org/confluence/display/solr/ShawnHeisey . https://www.oracle.com/technetwork/articles/java/g1gc-1984535.html

You can also refer to JVM Settings for tuning your memory and garbage collection settings.

To run Solr in SolrCloud mode, you need to set the ZK_HOST variable in the include file to point to your ZooKeeper ensemble. Running the embedded ZooKeeper is not supported in production environments. For instance, if you have a ZooKeeper ensemble hosted on the following three hosts on the default client port 2181 (zk1, zk2, and zk3), then you would set:

When the ZK_HOST variable is set, Solr will launch in "cloud" mode.

Use the SOLR_HOST variable in the include file to set the hostname of the Solr server.

Setting the hostname of the Solr server is recommended, especially when running in SolrCloud mode, as this determines the address of the node when it registers with ZooKeeper.

To guard against accidentally doing changes to the wrong cluster, you may configure a visual indication in the Admin UI of whether you currently work with a production environment or not. To do this, edit your solr.in.sh or solr.in.cmd file with a -Dsolr.environment=prod setting, or set the cluster property named environment. Allowed values are dev, test, stage or prod. Each of these have default labels and colors predefined. To specify label and/or color, use a comma-delimited format as below. Label has to be defined before color. The + character can be used instead of space to avoid quoting. Colors may be valid CSS colors or numeric, e.g., #ff0000 for bright red. Examples of valid environment configs:

Solr allows configuration properties to be overridden using Java system properties passed at startup using the -Dproperty=value syntax. For instance, in solrconfig.xml, the default auto soft commit settings are set to:

In general, whenever you see a property in a Solr configuration file that uses the ${solr.PROPERTY:DEFAULT_VALUE} syntax, then you know it can be overridden using a Java system property. For instance, to set the maxTime for soft-commits to be 10 seconds, then you can start Solr with -Dsolr.autoSoftCommit.maxTime=10000, such as:

The bin/solr script simply passes options starting with -D on to the JVM during startup. For running in production, we recommend setting these properties in the SOLR_OPTS variable defined in the include file. Keeping with our soft-commit example, in /etc/default/solr.in.sh, you would do:

There are several settings that should be monitored and set as high as possible, "unlimited" by preference. On most *nix operating systems, you can see the current values by typing the following at a command prompt.

These four settings in particular are important to have set very high, unlimited if possible.

We strongly recommend that these settings be permanently raised. The exact process to permanently raise them will vary per operating system. Some systems require editing configuration files and restarting your server. Consult your system administrators for guidance in your particular environment.

When running a Java application like Solr, having the OS swap memory to disk is a very bad situation. We usually prefer a hard crash so other healthy Solr nodes can take over, instead of letting a Solr node swap, causing terrible performance, timeouts and an unstable system. So our recommendation is to disable swap on the host altogether or reduce the "swappiness". These instructions are valid for Linux environments. Also note that when running Solr in a Docker container, these changes must be applied to the host.