Metadata
- Author: Lilian Weng
- Full Title:: Prompt Engineering
- Category:: 🗞️Articles
- URL:: https://lilianweng.github.io/posts/2023-03-15-prompt-engineering/
- Finished date:: 2023-03-20
Highlights
easy-to-use and shared benchmark infrastructure should be more beneficial to the community. Iterative prompting or external tool use would not be trivial to set up. Also non-trivial to align the whole research community to adopt it. (View Highlight)
presents a set of high-quality demonstrations, each consisting of both input and desired output, on the target task. As the model first sees good examples, it can better understand human intention and criteria for what kinds of answers are wanted. Therefore, few-shot learning often leads to better performance than zero-shot (View Highlight)
Many studies looked into how to construct in-context examples to maximize the performance and observed that choice of prompt format, training examples, and the order of the examples can lead to dramatically different performance, from near random guess to near SoTA (View Highlight)
A general suggestion is to keep the selection of examples diverse, relevant to the test sample and in random order to avoid majority label bias and recency bias (View Highlight)