Logging Considerations

Overview: 

For many years, logs have been an essential part of troubleshooting application and infrastructure performance. They help provide visibility into how our applications are running on each of the various infrastructure components.

Log data contains information such as exceptions or hard disk errors. This is very helpful information that will help us identify the “why” behind a problem either that a user has brought to our attention or that we have uncovered.

From a security point of view, the purpose of a log is to act as a red flag when something bad is happening. Reviewing logs regularly could help identify malicious attacks on your system.

Logging in Spring Boot:

Spring Boot uses Commons Logging for all internal logging but leaves the underlying log implementation open. By default, if you use the “Starters”, Logback is used for logging. Appropriate Logback routing is also included to ensure that dependent libraries that use Java Util Logging, Commons Logging, Log4J, or SLF4J all work correctly.

Log levels and Configurations:

Let us take an example of log4j logging utility. A log request of level p in a logger with level q is enabled if p >= q. This rule is at the heart of log4j. It assumes that levels are ordered. For the standard levels, we have ALL < DEBUG < INFO < WARN < ERROR < FATAL < OFF. 

Logging utilities provide configurations to set logger levels. 

All the supported logging systems can have the logger levels set in the Spring Environment (for example, in application.properties) by using logging.level.<logger-name>=<level> where level is one of TRACE, DEBUG, INFO, WARN, ERROR, FATAL, or OFF. The root logger can be configured by using logging.level.root.

The following example shows potential logging settings in application.properties:

logging.level.root=WARN

logging.level.org.springframework.web=DEBUG

logging.level.org.hibernate=ERROR

Recommendations:

1. As a rule of thumb, let INFO be the global default log level.

2. For a development environment, it is a good idea to have TRACE level logs.

3. For QA environments, it is recommended to have DEBUG level logs.

4. For production environments, it is recommended to stick to WARN and ERROR level logs for saving storage capacity and performance.

Best Practices to be followed: 

1. Use a standard and easily configurable logging framework - Logging frameworks like log4j, log4net etc, allow faster config changes than hardcoded or proprietary frameworks.

2. Making logging descriptive and contextual - Logging should be descriptive and must provide the context which would help with troubleshooting.

3. Making log data structured - A common best practice for logging is to ensure that the log data is structured. Unstructured logs make it tougher to store, index and search for log events when you’re troubleshooting. Having the data in a structured format requires specifying not just that something bad has happened within your application but to which application, uniquely identified by its application number, it happened to. This allows us to pinpoint which applications were impacted by the issue.

4. Avoiding extreme logging - The more relevant data you log, the better off you’ll be. Extreme logging can make it difficult to search and find relevant data when there’s an issue you need to debug and solve.

5. Don’t log sensitive information - Make sure you don’t log sensitive information such as: