还记得网友们用PS的Generative Fill花式整活的盛况吗？
Next, it may be the turn of PR to be played bad by netizens.
Whether you're a professional video editor or an amateur, you're no stranger to Premiere Pro (PR).
A few days ago, Adobe officially released a video that demonstrated the three AI functions that will be launched in PR quickly, accurately, and ruthlessly - object addition, object deletion, and generation extensions.
Users can directly call third-party AI video models such as Sora, Pika, and Runway in the PR to generate video clips.
At the same time, Adobe's self-developed Firefly series models will also be integrated into PR.
Compared with the minor repairs and minor repairs in recent years, this update can be said to be a major refactoring.
Ashley Still, senior vice president of creative products at Adobe, said Adobe is reimagining every step of the video creation process to provide creators with efficient, convenient and easy-to-use features.
PR+AI, P video like PS
Adobe's generative AI model, Firefly, is now widely used in Photoshop, illustrator, and Adobe Express.
In response to generative AI video models such as OpenAI's Sora and Google's Imagen 2, Adobe said it is training a new model capable of generating and editing video on top of Firefly, but has not yet revealed when the model will be launched.
For now, let's take a look at a couple of AI features that PR will be updating later this year.
Video element replacement
If you want to modify a specific object or area of the video instead of replacing the entire video footage, a common practice is to use a mask/layer to overwrite the area you want to modify, or use the Clone Repair tool to select an area similar to the area you want to modify, copy and overlay it onto the target area.
The process usually involves more delicate video editing work, such as masking, effect application, or local pixel adjustments, which requires a certain amount of video editing skill and patience to ensure that local changes are made as consistent and natural as possible with the rest of the video.
In Adobe's demo video, if you want to change the density of the diamond in the suitcase, you only need to circle the area you want to modify with the pen tool, and then enter the prompt text "A lot of gems", and Firefly will generate 3 different video clips, select one to complete the replacement.
It's okay to add a tie to a man in a suit and a watch that looks a little more expensive:
Judging by the demo video, the effect of the replacement is very natural, and there are hardly any flaws.
Video element removal
For the object you want to remove—it could be a boom mic, a sign, clutter on the floor—simply select it, right-click on "Object Removal", and it's easy to remove it from the video, although the result may not be perfect.
Video generation extensions
As for the generation extension, all you need to do is open the Adobe Firefly video generation plugin, drag the stop frame of the video, and Firefly will naturally extend the video for a few seconds.
This is known as video stop-frame expansion, and the traditional practice is tedious: determine which stop-motion frame to expand, create a new video sequence or timeline, and adjust the duration and speed of the extended frame according to the overall rhythm of the video and the desired effect. You may also want to generate a series of intermediate frames before and after the selected frame to smooth the transition to a dynamic effect.
If you're looking for a more detailed effect, post-processing, including color correction, special effects addition, and more, is followed by dynamic processing to ensure that the expanded frame is visually in harmony with the rest of the original video.
Now, with the help of graphical video technology, Sora is even able to simulate the motion of the real physical world, which is very useful when doing stop-motion extensions.
Adobe has integrated the image generation video capability into PR, so that users do not need to go to video editing software and AI video generation tools.
However, judging from the video examples, the extended video is very short, basically within 3 seconds, and some netizens think that "at most 72 frames, there is not much demonstration value".
In terms of generating video materials, users can directly call third-party models such as Sora, Runway, and Pika in the PR, and generate the required video materials by entering text prompts, such as "cityscape in the night rain", which can generate 3 videos at a time.
To prevent AI-generated videos from being used in deepfakes or spreading false information, Adobe will introduce Content Credentials, which indicate what content is AI-generated and which AI model was used to generate it.
But in reality, what Adobe demonstrates in the video can be achieved by other tools, Adobe just wins in integration, integration, and optimization of the interactive experience.
Soon after the release of the PR AI demo video, someone used Stable Diffusion's ComfyUI to realize the operation of changing diamonds.
In addition, Adobe is paying photographers and artists on its Adobe Stock platform up to $120 to submit short video clips that will be used to train Adobe's video generation model, according to people familiar with the matter. Depending on the quality of the submission, the remuneration paid ranges from approximately $2.62 to $7.25 per minute, with high-quality video clips being paid more.
Non-AI-native incumbents are "preemptive"?
Some people say that the biggest beneficiaries of this wave of AI technology are actually non-AI-native veteran players such as Microsoft and Adobe.
There's a common belief in Silicon Valley venture capital circles that "incumbents, rather than startups, are better equipped to leverage AI." AI is a sustaining technology that strengthens existing players rather than giving new entrants disruptive power. ”
Judging by the speed at which companies like Microsoft and Adobe are integrating AI capabilities, there is some truth to this view.
Veteran players have a strong brand influence and a stable customer base, and it seems that they only need to call an OpenAI API to achieve "late strike".
However, from the perspective of "model as application", as AI models are increasingly integrated into applications, the improvement of application product capabilities and user experience is increasingly constrained by the improvement of basic model capabilities. Developers can customize and optimize the model according to the needs of a specific application to make it more suitable for the application scenario, and the model can also learn from the interaction with users to continuously optimize its own performance.
This means that the so-called moat of existing products is not unbreakable, and it is easy for basic model companies to catch up, especially in the particularly competitive AI video track.
Although compared with Midjourney, OpenAI's DALL-E 3, etc., Adobe's Firefly image generation model is not highly rated in the industry, and is even considered to be mediocre.
But in this race for generative AI, failure to act means losing users and missing out on important potential revenue streams.
Not to mention, in addition to star players such as OpenAI's Sora and Stable Diffusion's SVD 3D, there are also plenty of generative AI video startups on the rise.
In early April, AI video startup Higgsfield announced that it had raised $8 million in seed funding. Higgsfield plans to use the seed money to improve the company's video editing tools, with the goal of allowing users to modify the people and objects in their videos. In addition, Higgsfield will train a more powerful video generation model specifically for social media.
Higgsfield's differentiated competitive strategy is "mobile-first, social-first," prioritizing iOS and Android apps over desktop workflows, prioritizing ease of use and consumer-friendly features that enable creators to create compelling social media content on the go.
At the China AIGC Industry Summit held on April 17, Wu Xinhong, founder of Meitu, said that there are three key points in the competition of domestic video models.
Creativity beyond reality: Large video models need to generate images beyond reality, which complement the actual shooting and become a new way to realize special effects.
Workflow integration: Video generation has limited application scenarios and needs to be combined with existing AI capabilities and traditional video technologies to form workflows.
Vertical scenarios: The "usability" of video in vertical scenarios such as e-commerce, advertising, games, animation, and film and television is extremely critical.
On the surface, foreign companies such as Microsoft and Adobe, and domestic companies such as Alibaba, Byte, and Meitu have only updated their own products with the help of AI technology, but the most important thing is that they have mastered the most critical link from product landing to market demand matching: scenarios.
Therefore, in the short term, generative AI video startups do not compete in the same dimension as creative software leaders such as Adobe. By introducing generative AI technology and providing a one-stop solution, it will not only greatly improve the efficiency of creative workflows, but also make it easy for non-professional video editors to get started with PR, further expanding Adobe's potential market.