Chen Jiaxin, School of Law, Ocean University of China
First of all, I am very grateful to Mr. Huang Wenxu for his recommendation and the internship opportunity given by Mr. Guo, the magic weapon of Peking University, as a researcher who still has a lot of room for improvement in all aspects, I am honored to be able to participate in the preliminary research work of the "Digital Intelligence Sirui" large model application research results exhibition jointly organized by Beijing No. 3 Intermediate People's Court and Peking University Law School, and have the opportunity to observe and learn offline. The content of the work and observation is as shown by the purpose of this exhibition - "to jointly study and evaluate the capabilities and prospects of artificial intelligence in assisting judicial trials, promote the in-depth application of artificial intelligence technology in the judicial field, comprehensively promote the digitization and intelligence of court work, and serve the modernization of the court trial system and trial capacity" - extremely open innovation and futurism. So even though the process was only a few weeks, I still learned a lot.
The exhibition of the research results of the application of the large model of "Digital Intelligence Sirui" was held in the lecture hall of Peking University Law School
1. Preliminary research: interaction with big data and AI
The exhibition of research results on the application of the "Digital Sirui" large model is divided into similar case retrieval, automatic document generation, generative artificial intelligence judicial question and answer, digital policing, In the preliminary research work, I am mainly responsible for some specific tasks on the business side, mainly three: first, to formulate the objective element evaluation standards for the enforcement ruling documents generated in the automatic generation of documents; second, to determine the scope of the case data set of the similar case retrieval track; and third, to find and sort out the corresponding similar cases in the determined scope of the case data set. In the process of completing the above tasks, I gradually felt the special features of the magic weapon work of Peking University and the work of the project:
The first task is to formulate evaluation standards for automatically generating enforcement rulings: according to the document format standards for enforcement cases, combined with test examples, sort out and clarify the objective elements that are used as the basis for evaluation in enforcement rulings, and form scoring content and evaluation standards. By searching the litigation document style column on the official website of the Supreme People's Court, I found that the document style on the enforcement of rulings covers all kinds of different situations of enforcement, so there are a large number of them. Therefore, in addition to extracting the objective elements of the basic information, the intersection elements and common factor elements of different styles of enforcement rulings are extracted, and the scoring content of the objective elements is finally determined, that is, the verification set of objective elements for the machine. At the same time, because the objective element evaluation is based on the machine's comparison of the hit rate and relevance of the elements, this determines that this task is different from other general standard-setting tasks: it is necessary to integrate the perspective of the machine to a certain extent, to refine and present the elements, to design the scoring rules, to deal with the parts that are closely intertwined with the subjective element evaluation, and to better transform the legal text into a machine-readable and processable model.
The second task is to determine the case data set to be retrieved for similar cases: according to the determined causes of action for civil and criminal acts, the results are retrieved within the data range of the relevant courts, and the corresponding number of reference cases in the four categories (Guiding Cases, Gazette Cases, Typical Cases, and Reference Cases) is counted, as well as the search status of each cause of action or charge in the people's court's case database. Based on the above results, the total amount is further considered, and the judgment of whether the dataset is suitable or not suitable as a similar case dataset for machine learning verification is given, and a quantitative optimization direction and clear suggestions are given for the unsuitable dataset. There is a process of constantly adjusting search queries, analysis, and selection, and consideration is also needed to consider how to design the table to present the results and relevant dimensions in a clear and visual package. (Note: Data security and information security are ensured at each track.) The content and process of the competition are completely isolated from the intranet system and do not directly intervene in the intranet system, and the competition uses public data or declassified information, which meets the confidentiality requirements. ）
The third task is to find and sort out similar cases: according to the data range of several causes of action or crimes identified, control the test cases, and manually sort out similar cases for machine learning through keyword search, including substantive and formal similar cases. Before starting this task, I first reviewed the course notes on similar case retrieval and the content of the legal retrieval course that graduate students are studying at the 2023 "Big Data and Legal Retrieval" Hunan Graduate Summer School. Since several test cases mainly present the basic facts, the matching keywords are extracted based on the basic facts to screen similar cases, and the possible focus of the dispute and the application of the law involved are roughly summarized to expand the keywords searched. Then, on the basis of satisfying structural similarities, a further distinction is made between substantive and formal cases. Finally, this task should also take into account that the object-oriented results are artificial intelligence, and the search track for similar cases aims to test the ability of artificial intelligence products to search and match similar cases, and test its level of similarity screening and judgment for multiple legal texts.
In general, in terms of the experience of participating in this project, compared with the previous internship work in law firms and courts, the work content and characteristics of Peking University Fabao are very different:
- The work is more comprehensive and diverse, and requires the initiation and application of a variety of different thinking and abilities. In addition to the legal knowledge base, search ability, case analysis ability, the most important thing is the ability to collect, analyze, process and present information. There is no dirty work that is too repetitive, and there is no fixed or ready-made reference template, so you need to keep your mind active and flexible. And in the process of completion, there will be many points worthy of further extension and excavation (for example, the reference cases in the case database of the people's courts are cases that have been reviewed and entered into the database after being solicited by the Supreme People's Court from December 2023, so they are different from the reference cases originally determined by the court itself, and the similarities and differences between the two can be further analyzed). Therefore, it is also a work with research and creative elements, which can discover new problems and gain new experiences.
- The requirements for a work style are rigorous and thorough. This rigor is not manifested in the screening and proofreading of forms and formats, but in the need to take into account many aspects, different possibilities, and details that can easily cause deviations in the results, and the need to be open and a little imaginative. You may encounter some specific small problems that are not knowledgeable and practical (such as the number of similar cases is too small or even almost no similar cases), so you should maintain a sense of analysis, think about how to resolve them, and give a certain optimization plan.
Third, the object of the work is special. This is a job that needs to interact with big data and AI, the basic resource of the work is big data, and the final object of the work is not judges or lawyers, but AI models. Therefore, it is necessary to switch and integrate the machine's perspective, not only to be familiar with the database, but also to consider the better readability, processability, and applicability of the machine when composing, organizing, and presenting the final file of the task.
I also give special thanks to Mr. Guo: every time Mr. Guo assigns work, he will explain the background and goals of the work very clearly, and then inform me of the final results of the work. This makes me not feel that this is a particularly trivial work, and I understand what steps and positions the specific work I have to undertake in the overall work, to what extent and to what extent. The whole process also made me feel more about what effective communication is.
In addition to the above-mentioned direct experience of the work itself, I have also gained a lot of other things:
Directly feel the subjectivity and control that humans have in the face of machines. The degree of intelligence is directly proportional to the degree of human input. The design, training, R&D, and application stages of legal large models require human participation, especially in the feeding and training of basic data and corpus for machines, which are inseparable from manual annotation and human thinking, and this process can give full play to human subjectivity and realize the "value alignment" of AI to human beings.
I also have a better understanding of what is "good at fake things" - this is a profound test and exercise of my search ability: analyze the problem, split the understanding of the case into a single search condition, use factual information to supplement the problem and keywords, carry out a combination search, and adjust the search formula, search method or re-conversion of the search task according to the in-depth adjustment of the degree of understanding, and go through the cycle of multiple searches from small to large, from large to small...... Retrieval is a skill that is not the best, only better, and there is always room for improvement in accuracy and recall. The training involved in identifying, retrieval, obtaining, managing, applying information, and optimizing my own information and knowledge system involved in the whole problem-solving process not only made me further familiar with the perspective and space of Peking University's magic weapon and resource library that can be used to solve problems, but also improved my information literacy.
Finally, the work of the legal knowledge project of Peking University Fabao made me feel very clearly that retrieval and annotation itself is part of the research. A large number of real data and cases have a very direct impact, and the persuasiveness of data and cases is far greater than words. Although I only read about 300 cases with different causes of action when I completed the third task, I found that there are still many points worthy of summary and analysis. A large amount of data is still "sleeping", it deserves and needs to be "awakened", and its potential value needs to be further explored and transformed into real productivity.
2. Offline observation: infinite expectations for legal AI
On the afternoon of Saturday, March 23, I went to the lecture hall on the first floor of the Kaiyuan Building of Peking University Law School to observe the exhibition of the research results of the application of the "Digital Intelligence Sirui" large model.
The legal models from major AI leading enterprises, information/technology service companies, and university research institutes displayed and competed in five tracks: similar case retrieval, automatic document generation, generative artificial intelligence judicial Q&A, digital policing, and big data supervision and management, which can be said to be a "grand gathering of China's legal artificial intelligence". I benefited a lot.
The first track - similar case retrieval
In the first similar case retrieval track, the operation process of each team's similar case retrieval intelligent large model is basically as follows: by filling in the case description, basic facts and keywords, etc., after comparing each key element, similar cases are analyzed and detected in the database. On the basis of the above, the more eye-catching model design is: first, it can be adapted differently according to the actual use of the scene; second, it can calculate and comprehensively calculate the similarity of all aspects separately and comprehensively, and present it with specific values; third, it can detect similar cases without extracting keywords from the human brain, and there is a large model in the track that can directly enter the entire fact or the entire case, and the AI can directly conduct overall reasoning on the massive case library based on the entered whole piece of information, so as to obtain similar cases. This function has moved me very much by selecting keywords to search and filter similar cases through various thoughts.
As for how the large model of case retrieval can be optimized, two aspects were pointed out in the comments of the judges: first, a typical feature of the generative large model is the interactive question and answer style, so the large model of similar case retrieval can be considered to use this feature to design an interactive question and answer style, and finally approach the similar case you want through continuous questioning of the results of the similar cases detected; Different controversies, etc., are the basic and traditional structural elements of similar cases, which can be designed to identify more features. The more features there are, the more precise the search will be, but this aspect needs to integrate more judicial-level features.
The second track - automatic generation of documents
In the automatic document generation track, the large model automatically generates relevant templated documents through document input or dialogue mode according to the given materials and case information, which tests the ability of artificial intelligence products to identify and extract information and apply document templates. Each type of document has a corresponding template embedded in the system in advance, and the required document can be generated by performing corresponding operations according to the needs. In addition to the automatic generation of documents, individual large models also have the function of aggregating documents, which can be organized into dossiers on the basis of the given materials. (P.S. The scoring of the objective elements of the track was done by the judges of the three machines. ）
My feeling is that in this track, it is very intuitive which large model is better: first, it can steal more laziness for you, and even help you think about and expand the laziness you can steal in advance, if there is a large model that can realize the conversion of various formats, and some large models can carry out personalized services;
The third track is generative AI judicial Q&A
The generative AI judicial Q&A track aims to test the natural language recognition ability and basic legal knowledge learning ability of AI products, using short-answer questions adapted from real cases, including civil, criminal, and administrative common basic cases, covering the prediction of criminal sentences, the determination of civil tort liability, the judgment of the legality of administrative acts, etc., please answer on the spot by artificial intelligence. This kind of large model is more complex than the design of the large model of similar case retrieval and the large model of automatic document generation, involves more technologies, and has higher learning and training costs.
As a viewer, my experience is that: first, for generative AI judicial Q&A products, the design of its training model can be regarded as the most important question, which directly affects the AI's understanding of the question and the pattern of the answers generated. Secondly, different question and answer models have their own advantages and characteristics. From the user's point of view, if it can provide an integrated, multi-model integration platform, it is undoubtedly the most efficient and fast. The intelligent question and answer system in the new AI application of Peking University Magic Weapon has been connected to all mainstream large models in China, including the original large model and the large model that integrates magic weapon data, and users can freely combine and select multiple models, and at the same time get answers from different models and select models according to their needs.
For the relevant legal basis provided by the AI application when outputting the answer, it can also be connected to the magic weapon database to directly "trace" and "enhance" the answer. In the future, it is also very possible to have an integrated platform for artificial intelligence products such as judicial questions and answers. In addition, no matter how smart AI and large models are or how unfortunate or unsatisfactory they are, asking questions is a very critical ability for people. How to assign tasks to AI, express to AI, communicate with AI, and promote mutual understanding with AI? So that it can efficiently generate satisfactory answers, and improve the value of AI as a tool for oneself? This is worth thinking about and training continuously.
The fourth track is digital policing
The digital policing track tests the ability of artificial intelligence in the people's courts to improve the work efficiency of the judicial police team, focusing on solving the outstanding problems in judicial policing work, such as high pressure on criminal case protection, prominent contradictions in litigation-related petitions, and low level of intelligent risk early warning. The most typical large-scale model application of this track, such as the bailiff integrated business management system, has functions such as intelligent patrol prevention and control, artificial intelligence-assisted face information recognition and early warning, etc., which can delineate different shapes of prevention and control areas, mark key prevention and control parts, and realize early warning of abnormal behaviors of control personnel through face recognition technology.
There is also a very interesting large police model application on this track - the bailiff physical and mental health detection platform, which can detect the fatigue, emotional status and energy value of bailiffs through wearable devices. I think that if the original intention of the design is to evaluate the physical and mental health of the bailiff, so as to understand the relevant situation to prevent and relieve the realization of humanistic care, then in fact, outside the track, the object of this application design can not only be limited to the bailiff, but also all kinds of groups in society, whether it is managers or workers, or even contemporary college students, in fact, they can have a special physical and mental health detection system. In other words, it does not reflect the particularity of this platform for bailiffs. If we continue to discuss the application of such platforms in the context of public security, procuratorate, and law, can we consider the detection of the emotional state of detainees?
The fifth track - big data supervision and management
The big data supervision and management model covers a number of work scenarios, mainly aiming at problems such as inconsistent application of laws, flaws in adjudication work, non-standard enforcement work, non-standard documents, non-coordination of information, and idling of procedures in case handling. Taking the large model of quality supervision of judgment documents as an example, embedding relevant rules, standards, norms, etc. into the system can supervise the input of judgment documents, such as the accuracy of citation of laws and regulations, the standardization of the structure of the style, and the consistency of the context logic of the judgment documents, which can undoubtedly achieve efficient error correction and quality inspection of judgment documents. There is also a large model of big data trial supervision and management, such as the large model of data supervision of high-risk cases, which can comprehensively calculate high-risk cases through the detection, collision and integration of specific element data, carry out "automatic error prevention" and "full early warning", and realize the transformation from "post-event supervision" to "ex-ante supervision" and "ex-post supervision".
There are also teams that start with the analysis of specific types of cases and design a large model with stronger professionalism and dispute resolution. In addition to showing the basic information such as the overall number and geographical distribution of this type of case, this large model can also analyze the reasons for the formation of the case and the key provisions involved in the case, so as to summarize the common focus of disputes, and refine the general adjudication rules on the basis of big data, and give countermeasures and suggestions. Although some of the conclusions are relatively large, we can still see the extraordinary inductive and summary analysis ability and potential of large models. This kind of large model can meet the needs of court scene construction and has very strong functional scalability: on the basis of big data analysis, it can reasonably screen and determine whether the behavior involved in the case to be tested belongs to a certain type of behavior; at the same time, it can also help judges quickly determine the focus of the dispute and directly carry out substantive review of the disputed facts; it can also carry out "data portraits" of the defendant; further, it can be based on the weak links of social governance, establish relevant data models, form special analysis reports, and provide reference for decision-making.
I deeply feel that the application of the court's digital model is directly and closely related to the construction of the digital court, and the application of the legal model makes the case handling no longer simply the wisdom and thinking of individual judges, but the support of a complete set of big data intelligent analysis system. Moreover, when this kind of large model is applied, it is also a process of absorbing, learning, transforming new data, further expanding and building a data warehouse, realizing the benign interaction between data fully empowering the court business and the court business continuously feeding back data. At the same time, I also realized the key role of human work in this link - the establishment of this kind of large model is inseparable from the learning of a large number of similar cases, and the combing of similar cases cannot be separated from manual work, and the high quality and efficiency of the final generated content also requires high-quality data annotation to be realized.
——————
The display of the five tracks was not so much a competition as an exchange of views and a collision of ideas, and in the end, all the participating teams won different types of awards, such as the Best Design Award, the Best Creativity Award, and so on. Perhaps this is why the final name is a "exhibition" of research results using large models, rather than a "competition", and it is more about the process of sharing experience and wisdom than competition.
From the perspective of the five tracks, as summarized by Sun Lingling, vice president of the Beijing Higher People's Court, the court's demand for legal artificial intelligence is nothing more than three points: first, to reduce the workload of judges' non-value judgments and reduce repetitive labor; second, through massive data, to achieve the whole chain and all-round management and supervision of the court; third, the court has a large number of judgment documents, and through legal artificial intelligence, the use and use of this resource advantage, and then make suggestions and references for social governance.
After this observation, I also have an intuitive experience of the construction of digital courts: the construction of digital courts is different from the construction of court informatization and smart courts, and it has achieved fundamental changes that are different from the traditions. The all-round change of the system is also a change in concept and thinking - "digital governance" can become a way and means of governance.
At the same time, the five tracks selected by the exhibition vividly demonstrated the five very important aspects of "transforming data into productivity and helping to modernize trial work", the integration of industry, academia, research and practice, and the combination of knowledge and application, which also made people have stronger expectations for the potential of AI: just like the most impressive declaration of the undergraduate team of Beijing Institute of Technology - "Our large model only needs a computer of 8,000 yuan" The training cost of artificial intelligence is very high, and it takes time to enter the stage of self-learning and updating, so how to achieve lightweight and reduce costs is of great significance for the promotion and application of large models. In addition, AI cannot solve the problem of value judgment at present, but can it be designed so that AI can pay attention to more effects, such as social effects, public opinion influence, etc., and the role that practice-oriented can play is worth further considering.
In addition to the above-mentioned observation feelings, the most vivid emotions are indescribable excitement. It is not only because I feel wonderful to see the work I have participated in and the results of my work become part of the foundation of the application and display of large models, not only because of the "eight immortals crossing the sea, each showing their magical powers" of each track, each team and each model, but also because of the future full of infinite possibilities. In the speech of Professor Shi Jianzhong, Vice President of China University of Political Science and Law, "Infinite expectations, unlimited expectations, for AI, we have opened the perspective of coping and the way to deal with opportunities...... Faster than imagined, we have entered the era of strong artificial intelligence, facing unprecedented opportunities and challenges, and the social relations adjusted by law are undergoing digital changes, which have given rise to new legal issues. If there is no supply of legislation, the challenges and burdens of justice and law enforcement will be heavier, so the results of digitalization should be used to actively respond to problems in judicial practice.
Peking University's magic weapon won the "Special Contribution Award" at the "Digital Intelligence Sirui" Large Model Application Research Results Exhibition
In the evening, on the subway to Beijing South Railway Station, combined with the harvest of the daytime exhibition, I discussed the whole way with Sister Rui Xue, the product manager of the magic weapon, about the functional design of the magic weapon GPT "moot court" - it can rely on the magic weapon database to provide the ...... for lawyers, judges, and law students through the "moot court" Forecasting, preparation, simulation..... There are also AI lawyers, simulated arbitration, simulated mediation...... AI should be used to augment the capabilities of legal professionals, not to replace legal judgments...... It's exciting to think about a future full of creative possibilities.
3. Reflections on Non-Finality: AI, Law and the Future
Intelligence is the inevitable trend of future technology, not the development and iteration of technology, but also the choice of human beings. Human beings have chosen to develop technology, and they have also chosen to apply the developed technology and deal with the risks it brings. As Mr. Gong Xiangrui and Premier Li Keqiang wrote at the end of the article "Computerization of Legal Work" in 1983, "the computerization of legal work is the inevitable product of this new era", in the Internet era, the digital age, and the era of artificial intelligence, from the digitization of legal information, the digitization of legal work to the intelligence of legal operation, legal intelligence is also an inevitable product of this era.
By participating in this work and observing the exhibition of research results on the application of large models, I have a better understanding of the connotation and significance of the six words "make the law smarter". From the bright, unclear, and "what a high-level expression!" when I used the magic weapon of Peking University for the first time, to the performance, problems, methods and paths of "legal intelligence" that I recognized and learned by participating in the "Summer School of Big Data and Legal Retrieval" jointly sponsored by Hunan Normal University and Peking University in 2023, until now, I realized that these six words are in no way intended to highlight the functions and advantages of the magic weapon of Peking University, let alone just a slogan. The goal and vision are a heavy practice that is really being implemented and developed with heart, which has been ongoing, updated and promoted - "making the law smarter".
In the era of artificial intelligence, how to use AI technology to make the law more intelligent, Zhao Xiaohai, founder and general manager of Peking University Fabao, said the most important point: "Legal professionals should actively accept and use AI technology with passion and curiosity." Maintain curiosity and desire to explore the latest developments in science and technology, actively engage with new information, and then think about how to apply it to the legal industry", the future era is the same for future technology, "making the law smarter" is not completed, only in progress......
With great opportunities come profound challenges. How to deal with the security risks of open source large models? How to choose cases to be fed to machine learning training to ensure fairness? How to control the degree of "assistance" in AI-assisted trials? Will AI reverse the judicial structure due to machine probability correlation ≠ legal causality? How to develop AI value judgments and define the scope of AI intervention? Will human selection and use of different legal models cause an inequality?...... Not only limited to the development and application of large legal models, the speed of AI iteration is beyond imagination, ChatGPT4.0 has been proud, Sora has made a stunning debut, and ChatGPT5.0 is ready to go...... AI will only develop faster, and the speed of each qualitative change far exceeds that of every wave of quantitative change that humans have to deal with, and some risks are becoming clearer and clearer, but sometimes, before we have even traced the specific face of risk, we are already in the whirlpool of risk......
How will humanity move towards the era of strong artificial intelligence, is it really ready...... I'm sometimes optimistic and passionate, looking forward to a future full of possibilities and opportunities, and sometimes I'm torn between AI deliverers and adventists with concerns about the risks that are spreading everywhere....... However, as a post-00s generation who is still young and vigorous in 2024, I thought about it, after all, young people still maintain an overall positive outlook and a controllable view of risks for the future.
Overall, it was a hearty learning experience, and I am grateful for the opportunity to have such an experience.
Facing AI and the future, "courage, passion, and desire to explore are indispensable". Using AI to empower legal professionals, as legal professionals empower themselves through AI, let legal people shape AI to a greater extent than they are shaped by AI - I think that they should have the mentality and attitude of the "weak", and continue to optimize their own knowledge, thinking and skills, just like the continuous feeding, training and upgrading iteration of large models, people should never stop learning and optimizing themselves. In the era of strong AI (and beyond, the era of super AI), every industry, every platform, and every person can and need to "keep thinking about how to make the future more exciting".