Extracting valuable insights from unstructured text is a key application in the financial industry. However, this task often goes beyond simple data extraction and requires advanced reasoning skills.
A typical example is determining the maturity date in a credit agreement, which usually involves deciphering a complex instruction such as "the maturity date should be the last business day before the third anniversary of the effective date". This level of complex inference presents a challenge for large language models (LLMs). It entails incorporating external knowledge, such as a holiday calendar, to accurately interpret and use the given instructions. An integrated knowledge graph is a promising solution with several key advantages.
The advent of Transformer has revolutionized text vectorization, achieving unprecedented precision. These embeddings encapsulate deep semantic meanings beyond previous approaches, which is why large language models (LLMs) are so great at generating text.
LLMs further demonstrate reasoning ability, albeit with limitations: the depth of their reasoning tends to decline rapidly. However, combining knowledge graphs with these vector embeddings can significantly improve reasoning skills. This synergy leverages the inherent semantic richness of embedding to elevate inference to unparalleled heights, marking a major advance in AI.
In the financial sector, LLMs are primarily used through Retrieval Enhanced Generation (RAG), a method that injects new, trained knowledge into LLMs. This process involves encoding text data, indexing for efficient retrieval, encoding queries, and using similar algorithms to obtain relevant paragraphs. These retrieved paragraphs are then used with the query as the basis for the LLM to generate a response.
This approach greatly expands the LLM's knowledge base, making it invaluable for financial analysis and decision-making. While retrieval enhancement generation marks a significant step forward, it also has limitations.
A key drawback is that channel vectors may not fully grasp the semantic intent of the query, resulting in important contexts being overlooked. This oversight occurs because embedding may not capture some of the inferred connections necessary to understand the full scope of the query.
In addition, condensing complex paragraphs into individual vectors can lead to a loss of nuance, obscuring key details distributed throughout the sentence.
There is also the fact that the matching process deals with each paragraph separately and lacks a joint analysis mechanism that can connect different facts. This absence hinders the model's ability to aggregate information from multiple sources, which is often necessary to generate comprehensive and accurate responses to information from different contexts.
There are many efforts to improve the retrieval enhancement generation framework, from optimizing block sizes to using a parent block retriever, hypothetical question embedding, and query rewriting. While these strategies provide improvements, they do not lead to revolutionary outcome changes. Another way is to bypass the retrieval enhancement generation by expanding the context window, as Google Gemini jumped to 1 million token capacity. However, this brings new challenges, including inconsistent focus and large amounts of information in the context of expansion, often with thousands of times the cost increase.
Combining knowledge graphs with dense vectors is the most promising solution. While embeddings effectively compress text of different lengths into fixed-dimensional vectors, enabling the identification of semantically similar phrases, they sometimes fail to distinguish between key nuances. For example, "cash from banks and maturities" and "cash and cash equivalents" produce almost identical vectors, suggesting that similarities ignore substantial differences. The latter includes interest-bearing entities such as "asset-backed securities" or "money market funds," while "bank maturity" refers to interest-free deposits.
A knowledge graph captures the complex interrelationships between concepts. This fosters a deeper level of contextual insight, emphasizing additional unique characteristics through connections between concepts. For example, the U.S. GAAP Knowledge Graph clearly defines the sum of "cash and cash equivalents", "bank interest-bearing deposits", and "bank maturities" as "cash and cash equivalents."
By integrating these detailed contextual clues and relationships, the knowledge graph significantly improves the reasoning capabilities of LLMs. They enable more precise multi-level inference in a single graph and facilitate federated inference across multiple graphs.