快科技日(5月14日)消息,Today,腾讯旗下混元文生图大模型(混元DiT,Diffusion Models with Transformers)宣布全面开源。
At present, it has been released on Hugging Face and Github, including complete models such as model weights, inference code, and model algorithms, for free commercial use by enterprise and individual developers.
According to media reports, Lu Qinglin, head of Tencent's mixed yuan Wensheng map, said that the value of mixed yuan DiT open source has two aspects.
On the one hand, this is the industry's first Chinese native DiT architecture, which makes up for the gap in the open source community, and on the other hand, the hybrid DiT is fully open and completely consistent with the existing network version.
According to reports, the open source hybrid DiT adopts the same key technology behind Sora - DiT architecture, and also supports 256-word Chinese understanding, not only supporting Wensheng diagrams, but also as the basis for multimodal visual generation such as videos.
Tencent said that in order to build a hybrid DiT, it designed a Transformer structure, a text encoder and position encoding, and built a complete data pipeline for updating and evaluating data to help model optimization and iteration.
In order to achieve fine-grained text understanding, a multimodal large language model is also trained to optimize the text description of images.
As a result, DiT is able to engage in multiple rounds of dialogue with the user, generating and refining images based on context.