06 August 2020

How To Troubleshoot Common Apache Errors

Introduction

There are three main commands, and a set of common log locations that you can use to get started troubleshooting Apache errors. Generally when you are troubleshooting Apache, you will use these commands in the order indicated here, and then examine log files for specific diagnostic data.

The commands that you will commonly use to troubleshoot Apache across most Linux distributions are:

  • systemctl - Used to control and interact with Linux services via the systemd service manager.
  • journalctl - Used to query and view the logs that are generated by systemd.
  • apachectl - When troubleshooting, this command is used to check Apache’s configuration.

These commands, how to use them, and Apache’s log locations where you can find additional information about errors are described in further detail in the following sections.

<$>[note] Note: On Debian and Ubuntu systems, the Apache service and process name is apache2, whereas on CentOS, Fedora, and other RedHat-derived systems, Apache’s service and process name is httpd. Apart from the differences between the service and running process names, starting, stopping, and checking Apache’s status, as well as logs with journalctl should work the same on any Linux system that uses systemd to manage the Apache service. Be sure to use the correct name for your Linux distribution. <$>

systemctl Commands for Apache

To troubleshoot common Apache errors using the systemd service manager, the first step is to inspect the state of the Apache processes on your system. The following systemctl commands will query systemd for the state of Apache’s processes.

On Ubuntu and Debian systems run:

sudo systemctl status apache2.service -l --no-pager

The -l flag will ensure that output is not truncated or ellipsized. The --no-pager flag will make sure that output will go directly to your terminal without requiring any interaction on your part to view it. You should receive output like this:

[secondary_label Output]
● apache2.service - The Apache HTTP Server
   Loaded: loaded (/lib/systemd/system/apache2.service; enabled; vendor preset: enabled)
  Drop-In: /lib/systemd/system/apache2.service.d
           └─apache2-systemd.conf
   Active: active (<^>running<^>) since Mon 2020-07-13 14:43:35 UTC; 1 day 4h ago
  Process: 929 ExecStart=/usr/sbin/apachectl start (code=exited, status=0/SUCCESS)
 Main PID: 1346 (apache2)
    Tasks: 55 (limit: 4702)
   CGroup: /system.slice/apache2.service
           ├─1346 /usr/sbin/apache2 -k start
. . .

To inspect the Apache process on CentOS and Fedora systems run:

sudo systemctl status httpd.service -l --no-pager

You should receive output like this:

[secondary_label Output]
● httpd.service - The Apache HTTP Server
   Loaded: loaded (/usr/lib/systemd/system/httpd.service; enabled; vendor preset: disabled)
   Active: <^>active (running)<^> since Tue 2020-07-14 19:46:52 UTC; 3s ago
     Docs: man:httpd.service(8)
 Main PID: 21217 (httpd)
   Status: "Started, listening on: port 80"
    Tasks: 213 (limit: 2881)
   Memory: 16.6M
   CGroup: /system.slice/httpd.service
           ├─21217 /usr/sbin/httpd -DFOREGROUND
. . .
Jul 14 19:46:52 localhost.localdomain httpd[21217]: Server configured, listening on: port 80

In either case, make a note of the Active line in the output. If your Apache server does not show active (running) as highlighted in the previous examples but you expect it should, there may be an error. Typically if there is a problem, you will have a line like the following in your output (note the highlighted failed portion):

[secondary_label Example Error Output]
Active: <^>failed<^> (Result: exit-code) since Tue 2020-07-14 20:01:29 UTC; 1s ago

If there is a problem with your Apache process or configuration you can troubleshoot it further using the journalctl command.

journalctl Commands for Apache

To inspect the systemd logs for Apache, you can use the journalctl command. The systemd logs for Apache will usually indicate whether there is a problem with starting or managing the Apache process.

These logs are separate from Apache’s request and error logs. journalctl displays logs from systemd that describe the Apache service itself, from startup to shutdown, along with any process errors that may be encountered along the way.

On Ubuntu and Debian systems use the following command to examine the logs:

sudo journalctl -u apache2.service --since today --no-pager

The --since today flag will limit the output of the command to log entries beginning at 00:00:00 of the current day only. Using this option will help restrict the volume of log entries that you need to examine when checking for errors. You should receive output like the following:

[secondary_label Output]
Jul 14 20:12:14 ubuntu2004 systemd[1]: Starting The Apache HTTP Server...
Jul 14 20:12:14 ubuntu2004 systemd[1]: Started The Apache HTTP Server.

If you are using a CentOS or Fedora based system, use this version of the command:

sudo journalctl -u httpd.service --since today --no-pager

You should receive output like the following:

[secondary_label Output]
Jul 14 20:13:09 centos8 systemd[1]: Starting The Apache HTTP Server...
. . .
Jul 14 20:13:10 centos8 httpd[21591]: Server configured, listening on: port 80

If there is an error, you will have a line in the output that is similar to the following, with the main difference between Linux distributions being the highlighted yourhostname portion:

[secondary_label Example Error Output]
Jul 14 20:13:37 <^>yourhostname<^> systemd[1]: Failed to start The Apache HTTP Server.

If your Apache server has errors in the journalctl logs like the previous example, then the next step to troubleshoot possible issues is investigating Apache’s configuration using the apachectl command line tool.

Troubleshooting with apachectl

Most Linux distributions include the apachectl utility with Apache. apachectl is an invaluable tool to help detect and diagnose Apache configuration problems.

To troubleshoot issues using apachectl, test your Apache configuration using the apachectl configtest command. The tool will parse your Apache files and detect any errors or missing settings before attempting to start the server.

Run the command like this on Ubuntu, Debian, CentOS, and Fedora based distributions:

sudo apachectl configtest

A working Apache configuration will result in output like the following:

[secondary_label Output]
Syntax OK

Depending on your Linux distribution, there may be other lines mixed in with the output, but the important line is the one that says Syntax OK.

If there is an error in your Apache configuration, like a directive that references a module that is not enabled or even a single typo, apachectl will detect it and attempt to notify you about the problem.

For example, attempting to use directives for an Apache module that is not enabled will result in apachectl configtest messages like the following:

[secondary_label Example Error Output]
AH00526: Syntax error on line 232 of /etc/apache2/apache2.conf:
Invalid command 'SSLEngine', perhaps misspelled or defined by a module not included in the server configuration
Action 'configtest' failed.
The Apache error log may have more information.

In this example the ssl module is not enabled, so the SSLEngine directive generates an error when the configuration is tested. The last line also indicates that The Apache error log may have more information, which is the next place to look for more detailed debugging information.

Apache Log Files

Apache log files are a very helpful resource for troubleshooting. Generally, any error that you receive in a browser or other HTTP client will have a corresponding entry in Apache’s logs. Sometimes Apache will also output errors related to configuration, built-in modules, and other debugging information to its log files.

To examine log files for errors while troubleshooting Apache on a Fedora, CentOS, or RedHat server, examine the /var/log/httpd/error_log file.

If you are troubleshooting a Debian or Ubuntu derived system, examine /var/log/apache2/error.log for errors using a tool like tail or less. For example, to view the last two lines of the error log using tail, run the following command:

sudo tail -n 2 /var/log/apache2/error.log

Substitute the number of lines that you would like to examine in place of the number 2 in the command. On a CentOS or Fedora system, the log file to examine is /var/log/httpd/error_log.

An example error will resemble something like the following lines, regardless of which Linux distribution you are using to run your Apache server:

[secondary_label Error Log Examples]
[Wed Jul 15 01:34:12.093005 2020] [proxy:error] [pid 13949:tid 140150453516032] (13)Permission denied: AH00957: HTTP: attempt to connect to 127.0.0.1:9090 (127.0.0.1) failed
[Wed Jul 15 01:34:12.093078 2020] [proxy_http:error] [pid 13949:tid 140150453516032] [client 127.0.0.1:42480] AH01114: HTTP: failed to make connection to backend: 127.0.0.1

The two lines in this output are distinct error messages. They both reference the module causing the error (proxy in the first line, proxy_http in the second) and include an error code that is specific to the module. The first one, AH00957, indicates that the Apache server attempted to connect to a backend server (127.0.0.1 on port 9090 in this case) using the proxy module but failed to do so.

The second error is derived from the first: AH01114 is a proxy_http module error that also indicates that Apache was unable to connect to the configured backend server to make an HTTP request.

These example lines are just for illustration purposes. If you are diagnosing errors with your Apache server, chances are the error lines in your logs will have different contents than these. Regardless of your Linux distribution, the format of any error lines in your logs will include the relevant Apache module and error code, as well as a text description of the error.

Once you have an idea of what might be causing problems with your Apache server you can continue researching and troubleshooting the issue. The error code and text description are especially useful, since they give you explicit and specific terms that you can use to narrow down the range of possible causes of a problem.

Conclusion

Troubleshooting Apache errors can range from diagnosing errors with the service itself, to locating misconfigured options for modules, or to examining customized access control rules in detail. This introduction to diagnosing issues with Apache explained how to use a number of utilities to help narrow down the possible causes of errors. Usually, you will use these utilities in the same order, although you can always skip some, or start directly with examining logs if you have a general idea of what the problem might be.

However, as a general sequence for troubleshooting, it helps to be methodical and use these tools in the order described. Start troubleshooting with systemctl to examine the state of the Apache server. If you need more information, examine the systemd logs for Apache using the journalctl command. If the issue is still not apparent after checking journalctl, testing Apache’s configuration using apachectl configtest is the next step. Finally, for in-depth troubleshooting, examining Apache’s log files will usually indicate a specific error, with helpful diagnostic messages and error codes.

The rest of the tutorials in this series will examine some common errors that you may encounter when using Apache in more detail.