mirror of https://github.com/hwchase17/langchain
You cannot select more than 25 topics
Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
399 lines
14 KiB
Plaintext
399 lines
14 KiB
Plaintext
{
|
|
"cells": [
|
|
{
|
|
"cell_type": "raw",
|
|
"id": "94c3ad61",
|
|
"metadata": {},
|
|
"source": [
|
|
"---\n",
|
|
"sidebar_position: 3\n",
|
|
"---"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "b91e03f1",
|
|
"metadata": {},
|
|
"source": [
|
|
"# How to use few shot examples\n",
|
|
"\n",
|
|
"In this guide, we'll learn how to create a simple prompt template that provides the model with example inputs and outputs when generating. Providing the LLM with a few such examples is called few-shotting, and is a simple yet powerful way to guide generation and in some cases drastically improve model performance.\n",
|
|
"\n",
|
|
"A few-shot prompt template can be constructed from either a set of examples, or from an [Example Selector](https://api.python.langchain.com/en/latest/example_selectors/langchain_core.example_selectors.base.BaseExampleSelector.html) class responsible for choosing a subset of examples from the defined set.\n",
|
|
"\n",
|
|
"This guide will cover few-shotting with string prompt templates. For a guide on few-shotting with chat messages for chat models, see [here](/docs/how_to/few_shot_examples_chat/).\n",
|
|
"\n",
|
|
"```{=mdx}\n",
|
|
"import PrerequisiteLinks from \"@theme/PrerequisiteLinks\";\n",
|
|
"\n",
|
|
"<PrerequisiteLinks content={`\n",
|
|
"- [Prompt templates](/docs/concepts/#prompt-templates)\n",
|
|
"- [Example selectors](/docs/concepts/#example-selectors)\n",
|
|
"- [LLMs](/docs/concepts/#llms)\n",
|
|
"- [Vectorstores](/docs/concepts/#vectorstores)\n",
|
|
"`} />\n",
|
|
"```\n",
|
|
"\n",
|
|
"## Create a formatter for the few-shot examples\n",
|
|
"\n",
|
|
"Configure a formatter that will format the few-shot examples into a string. This formatter should be a `PromptTemplate` object."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 1,
|
|
"id": "4e70bce2",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"from langchain.prompts.prompt import PromptTemplate\n",
|
|
"\n",
|
|
"example_prompt = PromptTemplate.from_template(\"Question: {question}\\n{answer}\")"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "50846ad4",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Creating the example set\n",
|
|
"\n",
|
|
"Next, we'll create a list of few-shot examples. Each example should be a dictionary representing an example input to the formatter prompt we defined above."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 2,
|
|
"id": "a44be840",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"examples = [\n",
|
|
" {\n",
|
|
" \"question\": \"Who lived longer, Muhammad Ali or Alan Turing?\",\n",
|
|
" \"answer\": \"\"\"\n",
|
|
"Are follow up questions needed here: Yes.\n",
|
|
"Follow up: How old was Muhammad Ali when he died?\n",
|
|
"Intermediate answer: Muhammad Ali was 74 years old when he died.\n",
|
|
"Follow up: How old was Alan Turing when he died?\n",
|
|
"Intermediate answer: Alan Turing was 41 years old when he died.\n",
|
|
"So the final answer is: Muhammad Ali\n",
|
|
"\"\"\",\n",
|
|
" },\n",
|
|
" {\n",
|
|
" \"question\": \"When was the founder of craigslist born?\",\n",
|
|
" \"answer\": \"\"\"\n",
|
|
"Are follow up questions needed here: Yes.\n",
|
|
"Follow up: Who was the founder of craigslist?\n",
|
|
"Intermediate answer: Craigslist was founded by Craig Newmark.\n",
|
|
"Follow up: When was Craig Newmark born?\n",
|
|
"Intermediate answer: Craig Newmark was born on December 6, 1952.\n",
|
|
"So the final answer is: December 6, 1952\n",
|
|
"\"\"\",\n",
|
|
" },\n",
|
|
" {\n",
|
|
" \"question\": \"Who was the maternal grandfather of George Washington?\",\n",
|
|
" \"answer\": \"\"\"\n",
|
|
"Are follow up questions needed here: Yes.\n",
|
|
"Follow up: Who was the mother of George Washington?\n",
|
|
"Intermediate answer: The mother of George Washington was Mary Ball Washington.\n",
|
|
"Follow up: Who was the father of Mary Ball Washington?\n",
|
|
"Intermediate answer: The father of Mary Ball Washington was Joseph Ball.\n",
|
|
"So the final answer is: Joseph Ball\n",
|
|
"\"\"\",\n",
|
|
" },\n",
|
|
" {\n",
|
|
" \"question\": \"Are both the directors of Jaws and Casino Royale from the same country?\",\n",
|
|
" \"answer\": \"\"\"\n",
|
|
"Are follow up questions needed here: Yes.\n",
|
|
"Follow up: Who is the director of Jaws?\n",
|
|
"Intermediate Answer: The director of Jaws is Steven Spielberg.\n",
|
|
"Follow up: Where is Steven Spielberg from?\n",
|
|
"Intermediate Answer: The United States.\n",
|
|
"Follow up: Who is the director of Casino Royale?\n",
|
|
"Intermediate Answer: The director of Casino Royale is Martin Campbell.\n",
|
|
"Follow up: Where is Martin Campbell from?\n",
|
|
"Intermediate Answer: New Zealand.\n",
|
|
"So the final answer is: No\n",
|
|
"\"\"\",\n",
|
|
" },\n",
|
|
"]"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "3d1ec9d5",
|
|
"metadata": {},
|
|
"source": [
|
|
"Let's test the formatting prompt with one of our examples:"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 13,
|
|
"id": "8c6e48ad",
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"Question: Who lived longer, Muhammad Ali or Alan Turing?\n",
|
|
"\n",
|
|
"Are follow up questions needed here: Yes.\n",
|
|
"Follow up: How old was Muhammad Ali when he died?\n",
|
|
"Intermediate answer: Muhammad Ali was 74 years old when he died.\n",
|
|
"Follow up: How old was Alan Turing when he died?\n",
|
|
"Intermediate answer: Alan Turing was 41 years old when he died.\n",
|
|
"So the final answer is: Muhammad Ali\n",
|
|
"\n"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"print(example_prompt.invoke(examples[0]).to_string())"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "dad66af1",
|
|
"metadata": {},
|
|
"source": [
|
|
"### Pass the examples and formatter to `FewShotPromptTemplate`\n",
|
|
"\n",
|
|
"Finally, create a [`FewShotPromptTemplate`](https://api.python.langchain.com/en/latest/prompts/langchain_core.prompts.few_shot.FewShotPromptTemplate.html) object. This object takes in the few-shot examples and the formatter for the few-shot examples. When this `FewShotPromptTemplate` is formatted, it formats the passed examples using the `example_prompt`, then and adds them to the final prompt before `suffix`:"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 14,
|
|
"id": "e76fa1ba",
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"Question: Who lived longer, Muhammad Ali or Alan Turing?\n",
|
|
"\n",
|
|
"Are follow up questions needed here: Yes.\n",
|
|
"Follow up: How old was Muhammad Ali when he died?\n",
|
|
"Intermediate answer: Muhammad Ali was 74 years old when he died.\n",
|
|
"Follow up: How old was Alan Turing when he died?\n",
|
|
"Intermediate answer: Alan Turing was 41 years old when he died.\n",
|
|
"So the final answer is: Muhammad Ali\n",
|
|
"\n",
|
|
"\n",
|
|
"Question: When was the founder of craigslist born?\n",
|
|
"\n",
|
|
"Are follow up questions needed here: Yes.\n",
|
|
"Follow up: Who was the founder of craigslist?\n",
|
|
"Intermediate answer: Craigslist was founded by Craig Newmark.\n",
|
|
"Follow up: When was Craig Newmark born?\n",
|
|
"Intermediate answer: Craig Newmark was born on December 6, 1952.\n",
|
|
"So the final answer is: December 6, 1952\n",
|
|
"\n",
|
|
"\n",
|
|
"Question: Who was the maternal grandfather of George Washington?\n",
|
|
"\n",
|
|
"Are follow up questions needed here: Yes.\n",
|
|
"Follow up: Who was the mother of George Washington?\n",
|
|
"Intermediate answer: The mother of George Washington was Mary Ball Washington.\n",
|
|
"Follow up: Who was the father of Mary Ball Washington?\n",
|
|
"Intermediate answer: The father of Mary Ball Washington was Joseph Ball.\n",
|
|
"So the final answer is: Joseph Ball\n",
|
|
"\n",
|
|
"\n",
|
|
"Question: Are both the directors of Jaws and Casino Royale from the same country?\n",
|
|
"\n",
|
|
"Are follow up questions needed here: Yes.\n",
|
|
"Follow up: Who is the director of Jaws?\n",
|
|
"Intermediate Answer: The director of Jaws is Steven Spielberg.\n",
|
|
"Follow up: Where is Steven Spielberg from?\n",
|
|
"Intermediate Answer: The United States.\n",
|
|
"Follow up: Who is the director of Casino Royale?\n",
|
|
"Intermediate Answer: The director of Casino Royale is Martin Campbell.\n",
|
|
"Follow up: Where is Martin Campbell from?\n",
|
|
"Intermediate Answer: New Zealand.\n",
|
|
"So the final answer is: No\n",
|
|
"\n",
|
|
"\n",
|
|
"Question: Who was the father of Mary Ball Washington?\n"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"from langchain.prompts.few_shot import FewShotPromptTemplate\n",
|
|
"\n",
|
|
"prompt = FewShotPromptTemplate(\n",
|
|
" examples=examples,\n",
|
|
" example_prompt=example_prompt,\n",
|
|
" suffix=\"Question: {input}\",\n",
|
|
" input_variables=[\"input\"],\n",
|
|
")\n",
|
|
"\n",
|
|
"print(\n",
|
|
" prompt.invoke({\"input\": \"Who was the father of Mary Ball Washington?\"}).to_string()\n",
|
|
")"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "59c6f332",
|
|
"metadata": {},
|
|
"source": [
|
|
"By providing the model with examples like this, we can guide the model to a better response."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "bbe1f843",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Using an example selector\n",
|
|
"\n",
|
|
"We will reuse the example set and the formatter from the previous section. However, instead of feeding the examples directly into the `FewShotPromptTemplate` object, we will feed them into an implementation of `ExampleSelector` called [`SemanticSimilarityExampleSelector`](https://api.python.langchain.com/en/latest/example_selectors/langchain_core.example_selectors.semantic_similarity.SemanticSimilarityExampleSelector.html) instance. This class selects few-shot examples from the initial set based on their similarity to the input. It uses an embedding model to compute the similarity between the input and the few-shot examples, as well as a vector store to perform the nearest neighbor search.\n",
|
|
"\n",
|
|
"To show what it looks like, let's initialize an instance and call it in isolation:"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 4,
|
|
"id": "80c5ac5c",
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"Examples most similar to the input: Who was the father of Mary Ball Washington?\n",
|
|
"\n",
|
|
"\n",
|
|
"answer: \n",
|
|
"Are follow up questions needed here: Yes.\n",
|
|
"Follow up: Who was the mother of George Washington?\n",
|
|
"Intermediate answer: The mother of George Washington was Mary Ball Washington.\n",
|
|
"Follow up: Who was the father of Mary Ball Washington?\n",
|
|
"Intermediate answer: The father of Mary Ball Washington was Joseph Ball.\n",
|
|
"So the final answer is: Joseph Ball\n",
|
|
"\n",
|
|
"question: Who was the maternal grandfather of George Washington?\n"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"from langchain.prompts.example_selector import SemanticSimilarityExampleSelector\n",
|
|
"from langchain_chroma import Chroma\n",
|
|
"from langchain_openai import OpenAIEmbeddings\n",
|
|
"\n",
|
|
"example_selector = SemanticSimilarityExampleSelector.from_examples(\n",
|
|
" # This is the list of examples available to select from.\n",
|
|
" examples,\n",
|
|
" # This is the embedding class used to produce embeddings which are used to measure semantic similarity.\n",
|
|
" OpenAIEmbeddings(),\n",
|
|
" # This is the VectorStore class that is used to store the embeddings and do a similarity search over.\n",
|
|
" Chroma,\n",
|
|
" # This is the number of examples to produce.\n",
|
|
" k=1,\n",
|
|
")\n",
|
|
"\n",
|
|
"# Select the most similar example to the input.\n",
|
|
"question = \"Who was the father of Mary Ball Washington?\"\n",
|
|
"selected_examples = example_selector.select_examples({\"question\": question})\n",
|
|
"print(f\"Examples most similar to the input: {question}\")\n",
|
|
"for example in selected_examples:\n",
|
|
" print(\"\\n\")\n",
|
|
" for k, v in example.items():\n",
|
|
" print(f\"{k}: {v}\")"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "89ac47fe",
|
|
"metadata": {},
|
|
"source": [
|
|
"Now, let's create a `FewShotPromptTemplate` object. This object takes in the example selector and the formatter prompt for the few-shot examples."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 5,
|
|
"id": "de69a214",
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"Question: Who was the maternal grandfather of George Washington?\n",
|
|
"\n",
|
|
"Are follow up questions needed here: Yes.\n",
|
|
"Follow up: Who was the mother of George Washington?\n",
|
|
"Intermediate answer: The mother of George Washington was Mary Ball Washington.\n",
|
|
"Follow up: Who was the father of Mary Ball Washington?\n",
|
|
"Intermediate answer: The father of Mary Ball Washington was Joseph Ball.\n",
|
|
"So the final answer is: Joseph Ball\n",
|
|
"\n",
|
|
"\n",
|
|
"Question: Who was the father of Mary Ball Washington?\n"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"prompt = FewShotPromptTemplate(\n",
|
|
" example_selector=example_selector,\n",
|
|
" example_prompt=example_prompt,\n",
|
|
" suffix=\"Question: {input}\",\n",
|
|
" input_variables=[\"input\"],\n",
|
|
")\n",
|
|
"\n",
|
|
"print(\n",
|
|
" prompt.invoke({\"input\": \"Who was the father of Mary Ball Washington?\"}).to_string()\n",
|
|
")"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "1b460794",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Next steps\n",
|
|
"\n",
|
|
"You've now learned how to add few-shot examples to your prompts.\n",
|
|
"\n",
|
|
"Next, check out the other how-to guides on prompt templates in this section, the related how-to guide on [few shotting with chat models](/docs/how_to/few_shot_examples_chat), or the other [example selector how-to guides](/docs/how_to/example_selectors/)."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "bf06d2a6",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": []
|
|
}
|
|
],
|
|
"metadata": {
|
|
"kernelspec": {
|
|
"display_name": "Python 3 (ipykernel)",
|
|
"language": "python",
|
|
"name": "python3"
|
|
},
|
|
"language_info": {
|
|
"codemirror_mode": {
|
|
"name": "ipython",
|
|
"version": 3
|
|
},
|
|
"file_extension": ".py",
|
|
"mimetype": "text/x-python",
|
|
"name": "python",
|
|
"nbconvert_exporter": "python",
|
|
"pygments_lexer": "ipython3",
|
|
"version": "3.10.1"
|
|
}
|
|
},
|
|
"nbformat": 4,
|
|
"nbformat_minor": 5
|
|
}
|