LLMs excel in natural language processing tasks thanks to their wealth of knowledge. Some current studies aim to improve their performance within the existing knowledge range, however they still have limitations in terms of information capacity. Therefore, understanding one's own limitations on the unknown is self-knowledge.
The following paper explores and analyzes the self-awareness of LLM.
- The authors developed a dataset called SelfAware, which contains unanswerable questions and their answerable counterparts.
- The authors introduce an automated approach to detect uncertainty in the model's response, providing a novel measure of its self-knowledge.
- In the end, they conducted an extensive analysis involving 20 LLM models, including GPT-3, InstructGPT, and LLaMA, and found that these models have inherent self-knowledge.
- In addition, the authors demonstrated that contextual learning and instruction tuning can further enhance this self-knowledge.
Original paper: https://arxiv.org/pdf/2305.18153.pdf Github: https://github.com/yinzhangyue/SelfAware
Know-Unknow quadrant. The horizontal axis represents the model's ability to remember knowledge, and the vertical axis represents the model's ability to understand and utilize knowledge.
1. SelfAware Dataset
To provide a more comprehensive assessment of the model's self-knowledge, the authors constructed a dataset containing a large number and more types of unanswerable questions, called SelfAware. The dataset includes:
- 1,032 unanswerable questions (from Quora, HowStuffWorks)
- 2,337 answerable questions (from SQuAD, HotpotQA, TriviaQA)
Add image annotations, no more than 140 words (optional)
2. Evaluation methods and experiments
- Use SimCSE to quantify the similarity between the target sentence and the reference sentence
- Use a sliding window of length 5 to parse the target sentence into semantic blocks to counteract potential errors in similarity calculations
- Set the temperature to 0.7 during the build process
- For GPT-4, 100 instances were randomly selected for analysis, while other models were reviewed using the full SelfAware dataset.
As for model comparison, the authors conducted a series of experiments to assess the degree of self-awareness exhibited by various LLMs (language models), including the GPT-3 and InstructGPT series, as well as more recently LLaMA and its derivative models, namely Alpaca and Vicuna. Their survey method used three different forms of input: direct input, instruction input, and contextual learning (ICL).
The F1 score is used as a measure of LLM's self-knowledge. Questions that cannot be answered are classified as positive examples, and questions that can be answered are classified as negative examples.
3. Conclusion
- Model size: Larger model sizes are associated with F1 scores and LLM self-perception, as shown in Figure 2.
- Instruction Tuning: The InstructGPT model demonstrated better self-awareness than the GPT-3 model, and instruction adjustments particularly enhanced self-awareness in the Alpaca and Vicuna models.
- Input form: The incorporation of instructions and examples enhances the self-awareness of the GPT-3 and InstructGPT series. Specifically, the ICL input form that provides richer contextual information significantly enhances the self-awareness ability of the model.
- Compared to humans: GPT-4 performs well but still does not reach human levels, highlighting the room for improvement in language models in terms of self-perception.