Spelled out! Tencent AI simulates and splices Sanxingdui cultural relics for the first time

The protagonist of today's story is a bronze statue.

It is it - the national treasure-level cultural relic from Sanxingdui "Bronze Beast Kneeling Bronze Statue" (or "Beast Topping Kneeling Bronze Statue").

For the first time in China and even the world, such unique shapes and intricate decorations of cultural relics have been discovered.

This is what it looks like when it was just unearthed.

This is what it looks like after it has been cleaned up.

It can be seen that it is damaged, cracked, deformed, and corroded seriously, and many fragments are missing, making it extremely difficult to repair.

And this is not a complete cultural relic, this top kneeling bronze statue unearthed in pit 3, and there is a base, which is a large standing bronze divine beast extracted from pit 8, put together, is a complete bronze statue of the bronze beast kneeling seat.

Theoretically, they could fit together, but archaeologists don't recommend that. Because it has undergone more than 3,000 years of grinding, the body is very fragile, the height is more than 2 meters, if you do the right experiment, it may cause irreversible damage to cultural relics.

This is the challenge for most excavated artifacts: fragile and mutilated, difficult to repair, and irreversible damage if not paid attention to. As of November 2022, more than 17,000 cultural relics have been unearthed in the six sacrificial pits of Sanxingdui newly discovered by the Sichuan Provincial Institute of Cultural Relics and Archaeology, and there are less than 200 front-line personnel engaged in the protection and restoration of cultural relics in the entire Sichuan Province, which is also a long-term problem faced by our national archaeological community.

Moreover, for archaeologists, it is necessary to restore the original appearance of cultural relics through rigorous and reasonable deduction and empirical judgment in the case of incompleteness, which is time-consuming, heavy-duty, and very challenging.

Can you still have a chance to see the "real face" of these national treasure-level cultural relics?

Our technical team said, give it a try!

In March this year, Tencent SSV Digital Culture Lab, Tencent PCG ARC Lab, and Sichuan Institute of Cultural Relics and Archaeology cooperated to set up a joint team to use Tencent's AI intelligent computing analysis, computer vision and other technologies to start the first pilot cultural relics simulation splicing work based on professional literature.

Today, under the guidance of cultural relics and archaeology experts from the Sichuan Provincial Institute of Archaeology, we have achieved phased results, and this "bronze statue of a copper beast kneeling and sitting on top of a human being" has successfully completed the simulation splicing.

This is what it looks like in the digital world.

We can look at its unique line and exquisite decoration in a 360° flip, and feel its splendor when it was cast thousands of years ago through techniques such as color reproduction.

How does AI do it?

First of all, "brothers recognize each other". To see if the artifacts from these two different pits can be put together.

In the past, it may not be so rigorous to rely on naked eye observation and experience judgment, but now we analyze the three-dimensional models of two cultural relics through AI intelligent calculation, extract the geometric feature information of the contact surface of the mythical beast and the portrait, and calculate the similarity of the features of the two to obtain the matching degree of the splicing.

(The matching degree is as high as 75%, and the "interface" has been confirmed, and it is a brother who is right)

Next, "orthopedic surgery" is done. According to geometric analysis and deformation, crack detection and orthopedic algorithms can correct the deformed parts of sacred beasts and portraits.

After more "clinical trials", we hope that AI will provide cultural relics archaeologists with explainable and interactive references for crack orthopedics and overall posture orthopedics in the future, and have a more accurate restoration basis.

Then, "fill in the gaps." According to the symmetry completion algorithm of shape analysis, it can provide experts with a variety of conjecture references for the original appearance of cultural relics.

For example, the divine beast usually has symmetrical two ears, so it also completes the other ear for this large standing bronze divine beast.

Finally, "fit"! Here is the geometric splicing knowledge of classic graphics learned in the textbook.

Through quantitative data and geometric splicing algorithms, multiple fragments are quickly spliced and compared to give a reasonable splicing method. Then analyze the overall force and judge how the two "tall people" can stand firmly together and it is not easy to "fall flat".

(There is an illusion that AI is building Lego)

In this way, in the small digital world, AI can easily provide feasible solutions for cultural relics restoration. At present, these algorithm models have been validated on simulation datasets and reached the industry-leading level.

In the future, it will further learn more advanced skills - how to perform stitching calculations when there are missing fragments.

We also use AI algorithms to assist in the analysis of cultural relics diseases.

According to the 3D model, the crack length, cross-sectional data, overall attitude and force of cultural relics are measured and calculated, so that experts can have more quantitative supporting data when analyzing cultural relics deterioration diseases.

In the next step, our joint project team with the Sichuan Archaeological Institute will also develop AI-assisted archaeological line drawing tools.

Because the current archaeological mapping is highly dependent on labor, such as Sanxingdui has tens of thousands of cultural relics waiting for draftsmen to draw, and there are few draftsmen in the industry, which will seriously affect the release progress of archaeological research reports.

(Draftsman: Can't finish drawing, can't finish drawing at all)

Through deep learning, AI can help quickly generate archaeological line drawings in various dimensions. It is planned to be applied to the excavated cultural relics in Sanxingdui on a pilot basis, and then gradually open to the whole industry in the future to help improve the efficiency of archaeological research on cultural relics in the mainland.

Tang Fei, President of the Sichuan Institute of Cultural Relics and Archaeology, said:

We hope that our AI will continue to work hard to learn and help archaeologists carry out archaeological and cultural research work better and more efficiently.

Cultural relics are by no means static antiques, they are living history. Hopefully, the artifacts will "restore" them to their original appearance, taking us to peek into the stories of the past.