Reporting by XinZhiyuan
EDIT: Good sleepy LRS
【New Zhiyuan Introduction】"Beauty Meta Universe" is coming? South Korea's latest AI will be beautiful women video seconds into webcomics, in addition to assisting creators, may also be the technical reserve to achieve virtual avatars.
Recently, Naver webtoon(s), South Korea's largest manga platform website, launched a new tool, WebtoonMe, that can convert selfies or videos into the style of Korean-style webcomics.
It is better to "eat" with the sound
Here's the first wave of effects:
As soon as the project was launched, it sparked a buzz on Reddit, garnering more than 1,700 support in less than a day.
On the one hand, Korean beautiful girls bring their own traffic, on the other hand, after converting photos into comics, there is still a lot of room for improvement in the picture.
First of all, the comic character's eyeballs do not move, nor do they blink, squinting and other actions, which do not seem particularly "real".
Some netizens pointed out that the hypothetical scene of the "uncanny valley theory" is particularly obvious here.
Uncanny Valley theory (Uncanny Valley) was proposed by Japanese roboticist Masahiro Mori in 1970, which is simply that because robots and humans are very similar in appearance and action, humans naturally have positive emotions for robots; but when robots are real to a certain extent, even if there is a little difference from humans, they will appear very dazzling, making robots appear stiff and terrifying, making people feel like facing zombies.
The resulting webcomic characters have dull eyes, and the "window of the mind" is closed, which naturally looks very strange.
Cartoonists who don't develop AI are not good alchemists
In 2018, 36-year-old popular webcomic writer Kawa Ichi-kuen used AI technology to publish a new concept comic "Met" on Naver Webtoon, and the cumulative number of views on the day after its release exceeded 40 million.
By incorporating the reader's image into the webcomic, the reader can make eye contact, talk directly, and develop emotional stories with the characters in the comic in the school environment, because the reader advances the plot by interacting with the work, so this type of comic is also called interactive webtoon.
Met is also the industry's first interactive webcomic.
This type of comic is particularly popular, and many people really like this immersive sense of storytelling rather than simply reading the story given by the author.
Inspired by Met, the WebtoonMe project will take the transformation technology a step further, generating webcomic-style pictures while retaining the characteristics of people photos.
What's most curious when you think of "techniques for turning photos into comics"? How does a three-dimensional face become a two-dimensional face?
Driven by curiosity, the team has finally begun to have a brow after countless experiments on the face conversion technology based on deep learning.
However, when trying to change the background to the "two-dimensional" style, it encountered an unexpected problem, and the direct use of the previous technology would lead to a very unnatural result.
In analyzing the data, the team came to the conclusion that the objects in the comic dataset have unique "stylistic" properties, which is different from the natural photo datasets typically used for deep learning research.
In addition, unlike face swaps, the various objects in the photo (such as faces, body parts, backgrounds, etc.) also need different "styles", which the team did not realize at first.
So the team decided to dig deeper into the webcomic creation process to see how different works stylized different objects.
After interviewing several creators, the team learned that "foreground (including faces)" and "background" were created with different tools.
First, a webcomic dataset of various "parts" such as faces, bodies, objects, and backgrounds needs to be built, and then various deep learning models need to be created to reduce stylistic differences between foreground and background. Finally, several post-processing algorithms that mimic the retouching techniques of different creators are applied, thereby improving the quality of the results more naturally.
From AI coloring to AI face swapping
As a leader in the webcomic industry, Naver Webtoon, the largest manga platform in South Korea, has been working on developing new content IP, such as the fire idol drama "My ID is Gangnam Beauty" from this platform.
As AI technology becomes more and more practical, the Webtoon AI team has developed several projects using deep learning technology to not only improve the reading experience of webcomic readers, but also to help creators create.
On October 27, 2021, Naver Webtoon announced the launch of Webtoon AI Painter's testing service, which helps creators color sketches.
The AI algorithm first divides the area of the picture, and when the creator selects a color and touches the corresponding area, the area can be automatically colored. Different from the existing manual coloring, AI shading can complete the full image coloring with only a few touches, which can greatly improve the coloring time of webcomic creators and thus improve creative efficiency.
Webtoon AI Painter took 3 years to develop, and the main technology is a deep learning model, using about 300,000 data to learn the characteristics of each area of the image and various coloring styles, such as the human face, body and background. Model-related work on this coloring technique was also published in WACV 2022.
In addition, Webtoon AI Painter also provides better technical services for creators' post-editing work by applying PSD (Photoshop file format) storage functions for layer separation and original resolution maintenance technology.
Layout beauty meta universe
In a completely virtual metaverse world, users need a virtual doppelganger, and a virtual doppelganger generated based on a selfie can better retain both the sense of reality and the sense of virtuality.
In August 2018, Naver Z launched the meta-universe virtual fashion social platform Zepeto, where users can freely create personal digital images in the software, focusing on a "fashion"!
Users need to "pinch their faces" on the platform first, pinch out the face shape and facial features according to their own preferences; then use virtual currency on the platform to buy fashion clothing and accessories, etc., dress up their avatars; they can also buy and unlock different poses, actions, choose different photo backgrounds to take photos with friends; at the same time, users can also produce content on the Zepeto platform and earn revenue.
Users can also upload selfies within Zepeto and then convert them into 3D avatars within social platforms.
In addition to virtual characters, Zepeto has created a virtual game world in which users can manipulate their own virtual characters to move around the world. The social scenes set up by the platform are rich and diverse, including haunted houses, classrooms, schools, famous attractions, etc., and users can attend parties, camping or even virtual weddings of friends.
Zepeto works closely with the fashion entertainment sector, often inviting famous fashion brands to release virtual clothing decorations on the platform, and also cooperating with k-pop entertainment companies to launch events.
In February last year, Gucci unveiled a virtual "Gucci Villa" on Zepeto that mimicked its Italian headquarters and unveiled an NFT fashion item. Users can use avatars to try on or purchase virtual costumes released by Gucci, and they can also punch in to roam the "Gucci Villa".
In addition, famous brands such as D&G, Dior, and Nike have also entered the platform. The globally popular k-pop groups Blackpink, ITZY and SF9 have also held virtual events on Zepeto to publish k-pop-related content.
The presence of the brand has indeed brought considerable revenue to Zepeto. At present, hundreds of companies and 1.5 million studio creators have promoted their virtual products on the Zepeto platform, and 50 million studio virtual designs are being sold.
In 2021, the company's monthly revenue hovered between $2.117 million and $2.794 million, an increase of about 70% over the same period in 2020, most of which came from advertising cooperation with well-known brands such as Gucci, Dior, Samsung, Hyundai and so on.
Ruby Lee, Naver Z's chief strategy officer, revealed that Zepeto's monthly active users grew from 10 million in May last year to 20 million in January this year. Zepeto has 290 million registered users, almost doubling in the last 18 months. And in its virtual world, the average user session time can reach about 30 minutes.
Although compared with the game head metaverse platform Roblox daily active users have more than 47 million data, Zepeto in the entire metaverse field is far from the volume of the first echelon.
However, most of the current meta-universe platforms are still dominated by adult male users, as the world's largest virtual fashion platform, Zepeto has attracted a large number of young female users, and has played a full differentiation effect in the meta-universe field.
Zepeto has more than 240 million users, 90% of whom are from outside Of Korea, 80% are teenagers and 70% are women.
At the end of 2021, Zepeto, a subsidiary of Naver Z, announced a Series B round of financing of 223.5 billion won (about $189 million), with Japan's SoftBank Group Vision Fund 2 leading an investment of up to $150 million.
SoftBank's position is very supportive of the development of the metacosmity, so obtaining softbank investment can explain the value of Zepeto to a certain extent, Androg Moon, the managing partner of SoftBank Investment, has said that he believes that the metacosm may become the next evolution of the Internet.
Other investors include South Korean financial firm Mirae Asset, webcomic publisher Naver Webtoon, and K-pop agencies JYP, YG Entertainment and HYBE funding.
In the same month, Naver Z also received a 5 billion won ($4.2 million) investment from South Korean game company Krafton, the developer of PUBG.
In today's metaverse concept fire, there are still many technical problems that have not been completely solved, AI technology solutions can accelerate the speed of metaverse development, and in turn, the investment obtained by the metacosm will also accelerate the development of AI technology.
In the torrent of science and technology, look forward to a better real world and virtual world.
Resources:
https://webtoon.github.io/WebtoonMe/en
https://www.reddit.com/r/MachineLearning/comments/sfbtds/p_webtoonme_project_selfie_to_webtoon_style/
https://m.jiemian.com/article/6893200.html