Guide
When the author cooperates with other students in his daily life, he often finds unreasonable log configuration and various logging methods, and the subsequent author intends to share Java logs within the team, and this article is the first in a series of articles.
sequence
The original intention of writing this article was to share the Java log within the team, because when I work with other students, I often find unreasonable log configurations and various logging methods. However, in the process of preparing to share and add details, I further found that the current blog-related articles only focus on a certain aspect, or talk about history and principles, or solve package conflicts, but do not connect the entire Java log knowledge. In the end, this article surpasses the previous positioning, and the more I write, the richer it becomes, and in order to make everyone not tired of reading it, my article will be presented in the form of a series.
I. Preface
Today, logs have been abstracted into three layers: the interface layer, the implementation layer, and the adaptation layer.
- Interface layer: or log façade, that is, interface, only define the interface, waiting for others to implement.
- Implementation layer: A tool that actually does the work and can record the content of the logs. Note, however, that it is not an implementation of the upper interface, as it is not aware of or directly implements the interface, but is only a stand-alone implementation.
- 适配层：一般称为Adapter，它才是上边接口的implements。 因为接口层和适配层并非都出自一家之手，它们之间无法直接匹配。 而鲁迅曾经说过：「计算机科学领域的任何问题都可以通过增加一个中间层来解决」（All problems in computer science can be solved by another level of indirection. -- David Wheeler[1]），所以就有了适配层。
The adaptation layer can be divided into two capabilities: binding and bridging:
- Binding: Bind an interface layer to an implementation layer (implement an interface layer and call the methods of the implementation layer)
- Bridging: Bridging an interface layer to another interface layer (implementing an interface layer and calling the interfaces of another interface layer) is mainly used to facilitate low-cost migration between interface layers and adaptation layers
If you think the above description is more abstract and blunt, you can skip it first, and you will understand it naturally after reading this article.
Next, we will review the history of Java logging in chronological order, which will help guide our subsequent practice and truly understand why it is true.
2. Historical evolution
2.1 Standard output (<1999)
Java didn't have a dedicated logging tool in the beginning, everyone used System.out and System.err to output logs. However, they are simply information outputs, unable to distinguish error levels, unable to control output granularity, and have little management or filtering capabilities. As Java becomes more engineering, their capabilities become somewhat stretched.
While System.out and System.err output to the console by default, they have the ability to save output to a file:
System.setOut(new PrintStream(new FileOutputStream("log.txt", true)));
System.out.println("这句将输出到 log.txt 文件中");
System.setErr(new PrintStream(new FileOutputStream("error.txt", true)));
System.err.println("这句将输出到 error.txt 文件中"); 2.2 Log4j (1999)
IN 1996, A EUROPEAN COMPANY CALLED SEMPER DECIDED TO DEVELOP A TOOL FOR LOGGING. After many iterations, it eventually developed into Log4j. The main author of this tool is a Russian programmer named Ceki Gülcü[2], remember his name: Ceki, and he will be mentioned several times later.
By 1999, Log4j had been widely used, and as the number of users grew, so did their demands. So Ceki chose to open source Log4j in 2001, hoping to grow Log4j with the help of the community. Soon after, the Apache Foundation extended an olive branch to Log4j, and Ceki joined Apache to continue the development of Log4j, and since then Log4j has been renamed Apache Log4j[3] and has entered the fast lane of development.
Log4j offers more powerful capabilities than System.out, and many of the ideas are still widely accepted today, such as:
- Logs can be output to consoles, files, databases, and even remote servers and e-mails (known as Appenders);
- The log output format (called Layout) allows for customization, such as using different presentation forms for error logs and normal logs;
- Logs are divided into 5 levels (called Levels), from lowest to highest, debug, info, warn, error, fatal, and the configured allowed level will be verified before output, and logs below this level will be ignored. In addition, there are two special levels, all, off, which indicate that the log output is completely released and completely closed;
- Different loggers (called loggers) can be specified at any time in the project, and independent recording locations and log levels can be configured for them.
- It can be configured through properties or XML files;
With the success of Log4j, Apache has incubated Log4Net[4], Log4cxx[5], and Log4php[6] products, and the open source community has also imitated and launched many projects such as Log4c[7], Log4cpp[8], Log4perl[9], and many others. It also confirms the influence of Log4j in the field of log processing.
However, Log4j had obvious performance shortcomings, and gradually declined after the introduction of Logback and Log4j 2, and finally Apache announced in 2015 that it would end the development of Log4j and fully migrate to Log4j 2 [10] (see [2.7 Log4j 2 (2012)]).
2.3 JUL (2002.2)
As the Java project evolved, Sun also realized the importance of logging and believed that this capability should be natively supported by the JRE. So in 1999 Sun submitted a proposal for JSR 047 [11] called "Logging API Specification". However, it wasn't until two years later, in 2002, that Java's official logging system was released with Java 1.4. This system is called Java Logging API, and the package path is java.util.logging, or JUL for short.
According to some retrospective articles, "Apache wanted to add Log4j to the JRE as the default logging implementation, but the arrogant Sun didn't say yes and soon introduced its own logging system." I have not found a source for this statement, and I cannot confirm its veracity.
However, judging from the actual products launched, JUL, which came out later, lagged behind Log4j in terms of functionality and performance, and it has the flavor of being hastily released because of high expectations, maybe that gossip is not groundless, haha. Although JUL has made great progress in Java 5.0 (1.5) [12], which was introduced in 2004, it still has not many bright spots in front of Log4j, and the majority of developers have no incentive to migrate, resulting in JUL never becoming a climate.
We don't recommend JUL's plans later in the article, so I won't go into much detail here (mainly I won't either).
2.4 JCL (2002.8)
In addition to Log4j and JUL, there were logging tools such as Apache Avalon [13] (a server-side development framework) and Lumberjack [14] (an open-source logging tool running on JDK 1.2/1.3).
For stand-alone and lightweight projects, developers can use a certain logging solution according to their preference. But more often than not, a business system relies on a large number of third-party tools, and many third-party tools will use different log implementations, and when they are integrated, it will inevitably lead to logging confusion.
To this end, Apache introduced an interface in 2002, Jakarta Commons Logging[15], or JCL for short, and its main author is still Ceki. This interface actively supports many logging tools such as Log4j, JUL, Apache Avalon, and Lumberjack. If developers want to print logs, they only need to call the interface of JCL, and the final log implementation is determined by the top-level business system. As we can see, this is a typical interface and implementation separation design.
However, because it is an interface (JCL) that comes first (Log4j, JUL), JCL provides an adaptation layer for interfaces and implementations (the latest version of it is not used, and the reasons will be mentioned in [1.2.7 Log4j2 (2012)]):
A brief introduction to the adaptation/implementation layers that JCL comes with:
- AvalonLogger/LogKitLogger: The adaptation layer used to bind Apache Avalon
- Jdk13LumberjackLogger:用于绑定Lumberjack的适配层
- Jdk14Logger: The adaptation layer used to bind JUL (since JUL has been provided since JDK 1.4).
- Log4JLogger：用于绑定Log4j的适配层
- NoOpLog: JCL's own logging implementation, but it's an empty implementation that doesn't do anything
- SimpleLog: JCL's built-in log implementation allows users to print logs even if they don't rely on other tools, but the function is very simple
At the time, the project was prefixed with the name Jakarta because it was part of the Jakarta Project [16] (email [17]) launched by Apache and Sun. At present, JCL is a sub-project of Apache Commons [18], called Apache Commons Logging, which is a brother to Commons Lang [19] and Commons Collections [20], which we commonly use. However, the abbreviated name of JCL was retained and not changed to ACL.
2.5 Slf4j (2005)
Ceki, the author of Log4j, saw a lot of shortcomings of Log4j and JCL, but was unable to promote rapid iteration of the project, and was dissatisfied with the management of Apache, believing that he had lost control of the Log4j project (blog [21], email [22]), so he chose to set up his own portal in 2005 and soon launched a new work, Simple Logging Facade for Java [23], or Slf4j for short.
Slf4j也是一个接口层，接口设计与JCL非常接近（毕竟有师承关系）。 相比JCL有一个重要的区别是日志实现层的绑定方式：JCL是动态绑定，即在运行时执行日志记录时判定合适的日志实现；而Slf4j选择的是静态绑定，应用编译时已经确定日志实现，性能自然更好。 这就是常被提到的classloader问题，更详细地讨论可以参考What is the issue with the runtime discovery algorithm of Apache Commons Logging[24]以及Ceki自己写的文章Taxonomy of class loader problems encountered when using Jakarta Commons Logging[25]。
At the time of the launch of Slf4j, there was already another set of interface layers on the market, JCL, and in order to give the choice to the user (and I guess to dig into the corner of JCL), Slf4j introduced two bridge layers:
- jcl-over-slf4j: The purpose is to make it easy for users who are already using JCL to migrate to Slf4j, you think you are tuning to the JCL interface, but you are moving to the Slf4j interface behind the scenes. I don't think it's too much to say that's digging into the corner of JCL, right?
- slf4j-jcl: Let users who are using Slf4j easily migrate to JCL, and dig their own corners, the main focus is a fair, just and open.
SLF4J basically meets all the scenarios of users by launching various adaptation layers, let's take a look at its whole family bucket:
Articles about Slf4j on the Internet often cite two images from its official website:
Interested students can also refer to it.
Here is the name slf4j-log4j12, which indicates the adaptation layer of Slf4j + Log4j 1.2 (the last version of Log4j). Similarly, slf4j-jdk14 represents the adaptation layer for Slf4j + JDK 1.4 (JUL).
2.6 Logback (2006)
However, Ceki's goal is not limited to Slf4j, and in the face of Log4j, which he created, as the original author, he naturally knows what problems it has. So in 2006, Ceki introduced a logging implementation: Logback [26]. Logback is better than Log4j in terms of ease of use, functionality, and performance, and naturally supports Slf4j without the need for an additional adaptation layer. Logback has become the most widely accepted logging implementation layer in the Java community (Logback itself has a market share of 48% in 2021 [27]).
Compared to Log4j, Logback provides a number of new features that we now take for granted:
- Log file cutting and scrolling records and asynchronous writing are supported
- For historical logs, it supports automatic cleaning by time or disk usage, and supports automatic compression to save disk space
- Support branch syntax, , <if><then>, You <else>can configure different log output logic according to conditions, such as determining that more detailed log information is only output in the development environment
- A large number of log filters can even identify each user through the logged-in user session and output independent log files
- The exception stack supports printing jar package information, so that we can not only know which file and which line the call came from, but also know which jar package the file came from
Logback is mainly composed of three parts (various articles on the Internet describe classic and access in vague terms, I had to go directly to the official website documentation to find a clearer explanation):
- logback-core：记录/输出日志的核心实现
- logback-classic：适配层，完整实现了Slf4j接口
- logback-access[28]: Used to integrate Logback into servlet containers (Tomcat, Jetty), so that HTTP access logs from these containers can also be output via powerful Logback
2.7 Log4j 2 (2012)
Watching the rise of Slf4j + Logback, Apache naturally did not sit idly by, and finally held back a big move in 2012: Apache Log4j 2 [29], which naturally has a lot of highlights:
- The plug-in structure [30] allows users to develop their own plug-ins and implement Appender, Logger, and Filter to complete the extension
- Based on the asynchronous output of LMAX Disruptor [31], the performance is about 10 times higher than that of Logback in multi-threaded scenarios, and Apache officials also promote this part as the main selling point, see Log4j 2 Performance for details[32].
Log4j 2 consists of two main parts:
- log4j-core：核心实现，功能类似于logback-core
- log4j-api：接口层，功能类似于Slf4j，里面只包含Log4j 2的接口定义
You will find that Log4j 2 is designed to provide a third interface layer in addition to JCL and Slf4j (log4j-api, albeit only its own interface), which is explained in the API Separation[33] section of the official website, which allows users to use different interface layers and implementation layers in a single project.
However, at present, Log4j 2 is generally regarded as the implementation layer, and JCL or Slf4j is introduced as the interface layer. In particular, JCL released version 1.3.0 [34] at the end of 2023 after nearly a decade, adding adaptation for Log4j 2. Remember when we didn't use the latest version of JCL in [1.2.4 JCL (2002.8)], because this version 10 years later @Deprecated out of the log adaptation layer that has been "obsolete".
In fact, Logback and Slf4j are like log4j-core and log4j-api, and if you want to use Logback, you can only use Slf4j. But who makes them happen at the right time, everyone will discuss them separately and think that they are two products.
Although it has been ten years since the release of Log4j 2 (this article was written in 2024), it still can't shake Logback's status, and I personally summarize two main points:
- Although Log4j 2 bears the name of Log4j, it is a completely rewritten logging system, and the upgrade cannot be completed only by changing the Log4j version number, and historical users are not willing to upgrade
- Log4j 2 came out 6 years later than Logback, but it did not provide enough bright and differentiated capabilities (the two highlights introduced above are not attractive enough for ordinary users), while the combination of Slf4j + Logback is already very good, and the first-mover advantage is obvious
For example, it was suggested that Spring Boot should switch the logging system from Logback to Log4j2 [35], but this was rejected by Phil Webb [36] (a core contributor to Spring Boot). The reasons he gave in his reply included the fact that Spring Boot needed to be backwards compatible to facilitate user upgrades, and that switching Log4j 2 was disruptive; At present, the vast majority of users do not face log performance problems, and the performance advantages advocated by Log4j 2 are not the core concerns of the framework and users. It is also very convenient if you want to switch to Log4j 2 in Spring Boot (see the official documentation [37]).
2.8 spring-jcl (2017)
Since most of the applications are built on Spring/Spring Boot, I will introduce the spring-jcl [38] package, which currently uses the spring-jcl + Logback solution.
Spring曾在它的官方Blog《Logging Dependencies in Spring》[39]中提到，如果可以重来，Spring会选择李白Slf4j而不是JCL作为默认日志接口。
Now Spring wants to support Slf4j and ensure forward compatibility to support JCL, so the spring-jcl package has been provided since 5.0 (Spring Boot 2.0). It bears the name of Spring, but the package name in the code is the same as JCL (org.apache.commons.logging), and the role is naturally the same as that of JCL, but it additionally adapts to Slf4j and puts Slf4j in the first priority of the lookup, so as to achieve "both and want" (you can go back to the section [1.2.4 JCL (2002.8)] for comparison).
If you're creating a new application based on Spring Initialize [40], you can leave this package alone, it's already working behind the scenes; If you encounter package conflicts during project development, or need to choose your own logging interface and implementation, you can treat spring-jcl as a JCL and boldly exclude it.
2.9 Miscellaneous
In addition to the logging solutions we mentioned above, there are some less common ones, such as:
- Flogger[41]: A logging interface layer introduced by Google in 2018. The initial F stands for Fluent, and that's exactly what it has to offer: chained calls (or streaming APIs, Slf4j 2.0 also supports Fluent APIs, which we'll cover in a later series of articles)
- JBoss Logging[42]: Launched by RedHat around 2010, it includes a complete interface layer, implementation layer, and adaptation layer
- slf4j-reload4j[43]: Ceki is based on the version forked from Log4j 1.2.7 to solve the security problems of Log4j. (However, not all security issues can be solved, please refer to the link above for details)
Because we rarely use these logging frameworks in actual development, I won't repeat them in this article (mainly I won't).
3. Summary
The history is over, but the story is not over. Two interfaces (JCL, Slf4j) and four implementations (Log4j, JUL, Logback, Log4j2), plus countless adaptation layers, they are connected into a network, I drew a diagram:
Explain/add to this diagram:
- Modules of the same color have the same groupId, which can be referred to in the legend.
- JCL's adaptation layer is provided directly in its own package, which we have already introduced earlier, and can be found in [1.2.4 JCL (2002.8)].
- If you want to use Logback, you must not be able to bypass Slf4j (referencing its adaptation layer is also counted); Similarly, if you want to use Log4j 2, you can't get around the log4j-api.
If you felt that it was too abstract when you read "1.1 Introduction" before, then I suggest you look back at this time, I believe you will have more experience.
From this history, I also found several interesting details:
- For a long time before and after the release of Log4j 2, Slf4j and Logback were slow to update because they had no competitors. Heroes have no opponents and can only slowly dwilight, and only when they meet opponents can they laugh proudly.
- The kindness and stubbornness of technical people: the products that came out late are all supported by the predecessors; The products that came out early did not care about its "juniors".
- Any problem in computer science can be solved by adding an intermediate layer, or two if not (the bridging layer does the job).
- Ceki has been carrying half of the Java log for 25 years (and still growing), and the true god is also. (Of course, there are many such gods in the coding world, such as Linus Torvalds[44], who has been maintaining Linux for 33 years, although he has been mainly involved as a product manager in the later years, and the late Bram Moolenaar[45] who has been maintaining Vim for 32 years.)
Reference Links:
[1]https://codedocs.org/what-is/david-wheeler-computer-scientist
[2]https://github.com/ceki
[3]https://logging.apache.org/log4j/1.2/
[4]https://logging.apache.org/log4net/
[5]https://logging.apache.org/log4cxx/
[6]https://logging.apache.org/log4php/
[7]https://log4c.sourceforge.net/
[8]https://log4cpp.sourceforge.net/
[9]https://mschilli.github.io/log4perl/
[10]https://news.apache.org/foundation/entry/apache_logging_services_project_announces
[11]https://jcp.org/en/jsr/detail
[12]https://www.java.com/releases/
[13]https://avalon.apache.org/
[14]https://javalogging.sourceforge.net/
[15]https://commons.apache.org/proper/commons-logging/
[16]https://jakarta.apache.org/
[17]https://lists.apache.org/thread/53otcqljjfnvjs3hv8m4ldzlgz59yk6k
[18]https://commons.apache.org/
[19]https://commons.apache.org/proper/commons-lang/
[20]https://commons.apache.org/proper/commons-collections/
[21]http://ceki.blogspot.com/2010/05/forces-and-vulnerabilites-of-apache.html
[22]https://lists.apache.org/thread/dyzmtholjdlf3h32vvl85so8sbj3v0qz
[23]https://www.slf4j.org/
[24]https://stackoverflow.com/questions/3222895/what-is-the-issue-with-the-runtime-discovery-algorithm-of-apache-commons-logging
[25]https://articles.qos.ch/classloader.html
[26]https://logback.qos.ch/
[27]https://qos.ch/
[28]https://logback.qos.ch/access.html
[29]https://logging.apache.org/log4j/2.x/
[30]https://logging.apache.org/log4j/2.x/manual/extending.html
[31]https://logging.apache.org/log4j/2.x/manual/async.html
[32]https://logging.apache.org/log4j/2.x/performance.html
[33]https://logging.apache.org/log4j/2.x/manual/api-separation.html
[34]https://commons.apache.org/proper/commons-logging/changes-report.html
[35]https://github.com/spring-projects/spring-boot/issues/16864
[36]https://spring.io/team/philwebb
[37]https://docs.spring.io/spring-boot/docs/3.2.x/reference/html/howto.html
[38]https://docs.spring.io/spring-framework/reference/core/spring-jcl.html
[39]https://spring.io/blog/2009/12/04/logging-dependencies-in-spring
[40]https://start.spring.io/
[41]https://google.github.io/flogger/
[42]https://github.com/jboss-logging
[43]https://reload4j.qos.ch/
[44]https://github.com/torvalds
[45]https://moolenaar.net/ Author: Shang Zuo
Source-WeChat public account: Alibaba Cloud Developer
Source: https://mp.weixin.qq.com/s/eIiu08fVk194E0BgGL5gow