Huatai Securities: Looking at the financial GPT opportunities from BloombergGPT

On March 30, 2023, financial information provider Bloomberg released the Bloomberg GPT, a large language model (LLM) for the financial sector. Relying on a large number of financial data sources from Bloomberg, the model builds a dataset of 363 billion labels, supports various tasks in the financial industry, far exceeds the existing models in performing financial tasks, and can compete with existing models in general scenarios.

Xie Chunsheng (practice: S0570519080006), an analyst at Huatai Securities, believes that domestic manufacturers that master financial data are also expected to copy the path of Bloomberg GPT and realize the effective empowerment of big language models in financial scenarios.

The core breakthrough lies in the financial corpus

Although the model parameters of Bloomberg GPT are between GPT-2 and GPT-3. But BloombergGPT's financial vertical capabilities far exceed those of the GPT series.

The analyst noted:

According to the model introduction of the paper "Bloomberg GPT: A Large Language Model for Finance", Bloomberg GPT is also based on a typical Transformer architecture, Bloomberg GPT model parameters are between GPT-2 and GPT-3, GPT-2 model parameters are 150 million, GPT-3 model parameters are 175 billion, Bloomberg GPT has a model parameter of 50 billion.

The test results in the official paper "Bloomberg GPT: A Large Language Model for Finance" show that Bloomberg GPT's performance in performing financial tasks exceeds the existing general LLM model, and the performance in the general scenario is basically the same as the existing general LLM model capability.

Although the model parameters of Bloomberg GPT are smaller than GPT-3, analysts said that relying on a large number of financial data sources from Bloomberg, Bloomberg GPT obtained a large number of high-quality financial data in pre-training, and carried out a series of cleaning and labeling of pre-training data, and Bloomberg GPT achieved a significant enhancement of financial vertical capabilities under the condition that the general ability was basically the same as GPT-3.

Develop new ideas of LLM of open source model + vertical data

What makes Bloomberg GPT unique in its development approach? Analysts believe that in the construction of the model, Bloomberg GPT shows excellent innovation, providing a meaningful path reference for domestic financial data companies to develop large models.

Specifically, it is mainly reflected in five aspects:

1) Vertical domain language model: In the past, the large language model was mostly a general model based on general text training, and the vertical domain model was mostly based on vertical domain data to train vertical models, Bloomberg GPT created a general + vertical hybrid training method, so that the model is both versatile and professional;

2) Training data: In the past, the pre-training data of large language models relied heavily on web scraping data, such as C4, ThePile, Wikipedia, etc., and Bloomberg built its own high-quality large-scale financial dataset;

3) Model evaluation: In addition to the public and financial NLP benchmarking of the model, Bloomberg also conducted a series of performance tests on the model based on Bloomberg's internal tasks;

4) Tokenizer: Tokenizer is a key step in model training, and Bloomberg uses Unigram models to replace greedymerge-based sub-word models to achieve more intelligent tokenization transformation;

5) Model construction method: The large language models represented by GPT-3 and GPT-4 are developed by large-scale professional artificial intelligence teams, and model training requires a lot of computing power; Benefiting from the open source model BLOOM's project practice and Bloomberg's deep accumulation of high-quality data in vertical fields, Bloomberg GPT has successfully demonstrated that a medium-sized team can produce equally competitive large-language models on specific data in vertical fields.

The future of financial GPT is promising

Analysts believe that Bloomberg GPT is expected to be applied to the following three scenarios in the future:

1) Bloomberg query language generation: Bloomberg GPT can convert user natural language queries into effective Bloomberg query language, making interaction with financial data more natural;

2) Suggestions for news headlines: Bloomberg GPT can provide support for Bloomberg's news application to assist journalists in writing short news headlines;

3) Financial Q&A: Thanks to the input of knowledge in financial verticals, Bloomberg GPT can answer finance-related questions more accurately, for example, Bloomberg GPT's answers are more accurate than the general model in the Q&A to identify the company's CEO.

Analysts pointed out that as a manufacturer that does not focus on artificial intelligence financial vertical fields, Bloomberg provides a useful demonstration with reference value for the development of financial GPT.

Master rich financial vertical knowledge and existing AI product layout, based on high-quality financial data and open source big language models, there is also an opportunity to create a large language model for exclusive financial scenarios, realize the effective landing of large language models in financial scenarios, and make big language models become the underlying AI operating system.

The main point of view of this article is from the report "Financial GPT Opportunities from BloombergGPT" written by Huatai Securities analyst Xie Chunsheng (practice: S0570519080006), which is abridged