System Directories and Logs

There are several important directories in the LucidWorks Search installation. System activities are recorded in several log files. Knowing where files and logs are located will make system configuration and troubleshooting easier.

Locating Files and Directories

The following table shows the default location of some directories that may be needed to effectively work with LucidWorks Search. These paths are all relative to the "LucidWorks Home" directory ($LWE_HOME) which is specified during installation.

Editing Configuration Files on Windows LucidWorks Search holds configuration files open after reading them, which may cause problems on Windows systems that do not allow editing open files. In this case, stop LucidWorks Search before editing files on Windows to be sure the edits are saved properly.

Configuring LucidWorks Search Directories

After you have installed LucidWorks Search, you can configure the location of of the app, conf, data, and logs directories by passing these parameters to the start script (start.sh or start.bat):

• -lwe_app_dir
• -lwe_conf_dir
• -lwe_data_dir
• -lwe_log_dir

For example, to change the location of the data directory, pass the following parameter to your start script:

start.sh -lwe_data_dir /var/data

See the section on Starting and Stopping LucidWorks Search for more information about the start scripts.

Temporary Files

By default, LucidWorks Search uses standard system directories (as detected by the JVM) for creating temporary files. This can be changed by adding a system property to the master.conf for java.io.tmpdir in the section that controls each JVM for the system. For example, to change the location of temporary files for the LucidWorks Core component, you would follow these steps:

1. Shut down LucidWorks using the instructions found in the section on Starting and Stopping LucidWorks Search.
2. Open master.conf with a text editor (found in $LWE_HOME/conf.
3. Find the section for lwecore.jvm.params and add -Djava.io.tmpdir=/tmp/files/.
4. Start LucidWorks.

The directory chosen as the location for temporary files should exist before starting LucidWorks Search, and must be writable by the user running LucidWorks.

Back to Top

System Logs

LucidWorks Search records system activities to rolling log files located in the $LWE_HOME/data/logs directory of the installation by default. The table below describes the main purpose of the various log files.

Log files are available through the Admin UI, by going to the Server Logs page for a collection and clicking the link at the bottom of the page. If for some reason the Admin UI is not available, log files can be downloaded with a curl command to the Core component such as:

curl http://localhost:8888/logs/<log_file_name>

Note, however, if the LucidWorks Search Core component is down, that curl command will not work.

Log Properties

The LucidWorks Search Core log is configured by the $LWE_HOME/conf/log4j-core.xml properties file. The default is to create a distinct log per date (server time).

The LucidWorks Search UI log is configured by the $LWE_HOME/conf/log4j-ui.xml properties file. The default is to create a distinct log per date (server time).

The LucidWorks Search Connector log is configured by the $LWE_HOME/conf/log4j-connectors.xml properties file. The default is to create a distinct log per date (server time).

The LucidWorks Search Connectors log includes information about crawl activities such as attempts to access a file or URL and the results of those attempts. By default, the log does not record the collection or data source associated with crawl activities. However, if you would like to record that information for later review, you can edit the $LWE_HOME/conf/log4j-connectors.xml file. In the file, find the section that begins with a comment to "Use the pattern below to log additional context info...", as below: <!-- Use the pattern below to log additional context info like collection and data source name --> <!-- <param value="%d{ISO8601} %p %c{2} - %X %m%n" name="ConversionPattern"/> --> Uncomment <param value="%d{ISO8601} %p %c{2} - %X %m%n" name="ConversionPattern"/> and save the file. You should restart LucidWorks Search after making this change.

More information on how to modify log4j settings for the Core and UI log files is available at http://logging.apache.org/log4j/1.2/manual.html.

Back to Top

LucidWorksLogs Collection

LucidWorks Search records log files for your Solr indexes in a collection called LucidWorksLogs, which contains a pre-configured data source also called lucidworkslogs. You can view the data for the LucidWorksLogs collection as you would for any other collection. You can also access the log files directly in the $LWE_HOME/data/solr/cores/LucidWorksLogs/ directory.

The LucidWorksLogs collection powers the error log and all statistics about recent query and indexing activity that is shown in the Admin UI.

The log files on a LWE-Core server are accessible via HTTP at the URL "http://server:port/logs". This URL lists all files currently in the logs directory, and provides links for downloading them individually. This can be useful in situations where you do not have direct shell access to the LWE-Core machine, but would like to review the log files for troubleshooting purposes.

If you are using LucidWorks Search in SolrCloud mode or with each component installed on a different server, please see the section Log Indexing with Separated Components for details on how to make sure your logs are fully indexed.

When securing the HTTP Port of LWE-Core installation, consideration should be taken as to whether the "/logs" directory should be secured or not.

Deleting the LucidWorksLogs Collection It is possible to delete the LucidWorksLogs collection if desired; however, this will disable the server log page within other collections, all activity graphing, and all calculations of Most Popular and Most Recent queries. If the collection was deleted in error, or if you'd like to restore it at a later time, go to the Server log page within any collection and click Recreate the log collection. It is also possible to remove the LucidWorksLogs data source from the LucidWorksLogs collection (i.e., retain the collection for possible later use, but remove the mechanism that indexes the logs). However, at the current time it will automatically be re-created and re-scheduled on server restart. If you wish to disable log crawling, you must either remove the entire LucidWorksLogs collection, or modify the LucidWorksLogs data source so that the schedule is not active (you can modify the schedule with the Data Source Schedules API or in the Schedules screen of the Admin UI).

Related Topics

• [doc:Working With LucidWorks Search Components]
• Starting and Stopping LucidWorks Search

Back to Top

Working With LucidWorks Search Components

Understanding LucidWorks Search

Starting and Stopping LucidWorks Search