"\u001b[33mWARNING: You are using pip version 22.0.4; however, version 24.0 is available.\n",
"You should consider upgrading via the '/Users/jacoblee/.pyenv/versions/3.10.5/bin/python -m pip install --upgrade pip' command.\u001b[0m\u001b[33m\n",
"\u001b[0mNote: you may need to restart the kernel to use updated packages.\n"
"\n",
"\u001b[1m[\u001b[0m\u001b[34;49mnotice\u001b[0m\u001b[1;39;49m]\u001b[0m\u001b[39;49m A new release of pip is available: \u001b[0m\u001b[31;49m23.2.1\u001b[0m\u001b[39;49m -> \u001b[0m\u001b[32;49m24.0\u001b[0m\n",
"\u001b[1m[\u001b[0m\u001b[34;49mnotice\u001b[0m\u001b[1;39;49m]\u001b[0m\u001b[39;49m To update, run: \u001b[0m\u001b[32;49mpython -m pip install --upgrade pip\u001b[0m\n",
"Note: you may need to restart the kernel to use updated packages.\n"
"You might notice above that `parser` actually doesn't block the streaming output from the model, and instead processes each chunk individually. Many of the [LCEL primitives](/docs/how_to#langchain-expression-language-lcel) also support this kind of transform-style passthrough streaming, which can be very convenient when constructing apps.\n",
"Note that we're getting streaming output even though we're using `parser` at the end of the chain above. The `parser` operates on each streaming chunk individidually. Many of the [LCEL primitives](/docs/how_to#langchain-expression-language-lcel) also support this kind of transform-style passthrough streaming, which can be very convenient when constructing apps. \n",
"\n",
"Custom functions can be [designed to return generators](/docs/how_to/functions#streaming), which are able to operate on streams.\n",
"\n",
"Certain runnables, like [prompt templates](/docs/how_to#prompt-templates) and [chat models](/docs/how_to#chat-models), cannot process individual chunks and instead aggregate all previous steps. This will interrupt the streaming process. Custom functions can be [designed to return generators](/docs/how_to/functions#streaming), which"
"Certain runnables, like [prompt templates](/docs/how_to#prompt-templates) and [chat models](/docs/how_to#chat-models), cannot process individual chunks and instead aggregate all previous steps. Such runnables can interrupt the streaming process."
]
},
{
@ -256,10 +299,9 @@
"metadata": {},
"source": [
":::{.callout-note}\n",
"If the above functionality is not relevant to what you're building, you do not have to use the `LangChain Expression Language` to use LangChain and can instead rely on a standard **imperative** programming approach by\n",
"The LangChain Expression language allows you to separate the construction of a chain from the mode in which it is used (e.g., sync/async, batch/streaming etc.). If this is not relevant to what you're building, you can also rely on a standard **imperative** programming approach by\n",
"caling `invoke`, `batch` or `stream` on each component individually, assigning the results to variables and then using them downstream as you see fit.\n",
"\n",
"If that works for your needs, then that's fine by us 👌!\n",
":::"
]
},
@ -283,7 +325,7 @@
},
{
"cell_type": "code",
"execution_count": 6,
"execution_count": 7,
"id": "5ff63cce-715a-4561-951f-9321c82e8d81",
"metadata": {},
"outputs": [
@ -321,7 +363,9 @@
" model | JsonOutputParser()\n",
") # Due to a bug in older versions of Langchain, JsonOutputParser did not stream results from some models\n",
"async for text in chain.astream(\n",
" 'output a list of the countries france, spain and japan and their populations in JSON format. Use a dict with an outer key of \"countries\" which contains a list of countries. Each country should have the key `name` and `population`'\n",
" \"output a list of the countries france, spain and japan and their populations in JSON format. \"\n",
" 'Use a dict with an outer key of \"countries\" which contains a list of countries. '\n",
" \"Each country should have the key `name` and `population`\"\n",
"chain = model | JsonOutputParser() | _extract_country_names\n",
"\n",
"async for text in chain.astream(\n",
" 'output a list of the countries france, spain and japan and their populations in JSON format. Use a dict with an outer key of \"countries\" which contains a list of countries. Each country should have the key `name` and `population`'\n",
" \"output a list of the countries france, spain and japan and their populations in JSON format. \"\n",
" 'Use a dict with an outer key of \"countries\" which contains a list of countries. '\n",
" \"Each country should have the key `name` and `population`\"\n",
"):\n",
" print(text, end=\"|\", flush=True)"
]
@ -407,7 +453,7 @@
},
{
"cell_type": "code",
"execution_count": 8,
"execution_count": 9,
"id": "15984b2b-315a-4119-945b-2a3dabea3082",
"metadata": {},
"outputs": [
@ -451,7 +497,9 @@
"chain = model | JsonOutputParser() | _extract_country_names_streaming\n",
"\n",
"async for text in chain.astream(\n",
" 'output a list of the countries france, spain and japan and their populations in JSON format. Use a dict with an outer key of \"countries\" which contains a list of countries. Each country should have the key `name` and `population`'\n",
" \"output a list of the countries france, spain and japan and their populations in JSON format. \"\n",
" 'Use a dict with an outer key of \"countries\" which contains a list of countries. '\n",
" \"Each country should have the key `name` and `population`\",\n",
"Event Streaming is a **beta** API. This API may change a bit based on feedback.\n",
"\n",
":::{.callout-note}\n",
"Introduced in langchain-core **0.1.14**.\n",
"\n",
"This guide demonstrates the `V2` API and requires langchain-core >= 0.2. For the `V1` API compatible with older versions of LangChain, see [here](https://python.langchain.com/v0.1/docs/expression_language/streaming/#using-stream-events).\n",
":::"
]
},
{
"cell_type": "code",
"execution_count": 12,
"execution_count": null,
"id": "61348df9-ec58-401e-be89-68a70042f88e",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"'0.1.45'"
]
},
"execution_count": 12,
"metadata": {},
"output_type": "execute_result"
}
],
"outputs": [],
"source": [
"import langchain_core\n",
"\n",
@ -647,24 +685,21 @@
"When streaming is implemented properly, the inputs to a runnable will not be known until after the input stream has been entirely consumed. This means that `inputs` will often be included only for `end` events and rather than for `start` events.\n",
"/home/eugene/src/langchain/libs/core/langchain_core/_api/beta_decorator.py:87: LangChainBetaWarning: This API is in beta and may change in the future.\n",
" warn_beta(\n"
]
}
],
"source": [
"events = []\n",
"async for event in model.astream_events(\"hello\", version=\"v1\"):\n",
"async for event in model.astream_events(\"hello\", version=\"v2\"):\n",
" events.append(event)"
]
},
@ -698,13 +742,16 @@
"source": [
":::{.callout-note}\n",
"\n",
"Hey what's that funny version=\"v1\" parameter in the API?! 😾\n",
"Hey what's that funny version=\"v2\" parameter in the API?! 😾\n",
"\n",
"This is a **beta API**, and we're almost certainly going to make some changes to it.\n",
"This is a **beta API**, and we're almost certainly going to make some changes to it (in fact, we already have!)\n",
"\n",
"This version parameter will allow us to minimize such breaking changes to your code. \n",
"\n",
"In short, we are annoying you now, so we don't have to annoy you later.\n",
"\n",
"`v2` is only available for langchain-core>=0.2.0.\n",
" 'data': {'output': AIMessageChunk(content='Hello! How can I assist you today?', id='run-26134ba4-e486-4552-94d9-a31a2dfe7f4a')}}]"
" 'metadata': {}}]"
]
},
"execution_count": 15,
"execution_count": 16,
"metadata": {},
"output_type": "execute_result"
}
@ -798,7 +845,7 @@
},
{
"cell_type": "code",
"execution_count": 16,
"execution_count": 17,
"id": "4328c56c-a303-427b-b1f2-f354e9af555c",
"metadata": {},
"outputs": [],
@ -810,8 +857,10 @@
"events = [\n",
" event\n",
" async for event in chain.astream_events(\n",
" 'output a list of the countries france, spain and japan and their populations in JSON format. Use a dict with an outer key of \"countries\" which contains a list of countries. Each country should have the key `name` and `population`',\n",
" version=\"v1\",\n",
" \"output a list of the countries france, spain and japan and their populations in JSON format. \"\n",
" 'Use a dict with an outer key of \"countries\" which contains a list of countries. '\n",
" \"Each country should have the key `name` and `population`\",\n",
" 'data': {'input': 'output a list of the countries france, spain and japan and their populations in JSON format. Use a dict with an outer key of \"countries\" which contains a list of countries. Each country should have the key `name` and `population`'},\n",
" 'name': 'RunnableSequence',\n",
" 'tags': [],\n",
" 'metadata': {},\n",
" 'data': {'input': 'output a list of the countries france, spain and japan and their populations in JSON format. Use a dict with an outer key of \"countries\" which contains a list of countries. Each country should have the key `name` and `population`'}},\n",
" 'data': {'input': {'messages': [[HumanMessage(content='output a list of the countries france, spain and japan and their populations in JSON format. Use a dict with an outer key of \"countries\" which contains a list of countries. Each country should have the key `name` and `population`')]]}},\n",
" 'data': {'input': {'messages': [[HumanMessage(content='output a list of the countries france, spain and japan and their populations in JSON format. Use a dict with an outer key of \"countries\" which contains a list of countries. Each country should have the key `name` and `population`')]]}}},\n",
" 'output a list of the countries france, spain and japan and their populations in JSON format. Use a dict with an outer key of \"countries\" which contains a list of countries. Each country should have the key `name` and `population`',\n",
" version=\"v1\",\n",
" \"output a list of the countries france, spain and japan and their populations in JSON format. \"\n",
" 'Use a dict with an outer key of \"countries\" which contains a list of countries. '\n",
" \"Each country should have the key `name` and `population`\",\n",
"{'event': 'on_parser_start', 'data': {'input': 'output a list of the countries france, spain and japan and their populations in JSON format. Use a dict with an outer key of \"countries\" which contains a list of countries. Each country should have the key `name` and `population`'}, 'name': 'my_parser', 'tags': ['seq:step:2'], 'run_id': 'e058d750-f2c2-40f6-aa61-10f84cd671a9', 'metadata': {}}\n",
" 'output a list of the countries france, spain and japan and their populations in JSON format. Use a dict with an outer key of \"countries\" which contains a list of countries. Each country should have the key `name` and `population`',\n",
" version=\"v1\",\n",
" \"output a list of the countries france, spain and japan and their populations in JSON format. \"\n",
" 'Use a dict with an outer key of \"countries\" which contains a list of countries. '\n",
" \"Each country should have the key `name` and `population`\",\n",
" version=\"v2\",\n",
" include_names=[\"my_parser\"],\n",
"):\n",
" print(event)\n",
@ -1019,7 +1072,7 @@
},
{
"cell_type": "code",
"execution_count": 20,
"execution_count": 21,
"id": "096cd904-72f0-4ebe-a8b7-d0e730faea7f",
"metadata": {},
"outputs": [
@ -1027,17 +1080,17 @@
"name": "stdout",
"output_type": "stream",
"text": [
"{'event': 'on_chat_model_start', 'name': 'model', 'run_id': '02b68bbd-e99b-4a66-bf5f-6e238bfd0182', 'tags': ['seq:step:1'], 'metadata': {}, 'data': {'input': {'messages': [[HumanMessage(content='output a list of the countries france, spain and japan and their populations in JSON format. Use a dict with an outer key of \"countries\" which contains a list of countries. Each country should have the key `name` and `population`')]]}}}\n",
"{'event': 'on_chat_model_start', 'data': {'input': 'output a list of the countries france, spain and japan and their populations in JSON format. Use a dict with an outer key of \"countries\" which contains a list of countries. Each country should have the key `name` and `population`'}, 'name': 'model', 'tags': ['seq:step:1'], 'run_id': 'db246792-2a91-4eb3-a14b-29658947065d', 'metadata': {}}\n",
" 'output a list of the countries france, spain and japan and their populations in JSON format. Use a dict with an outer key of \"countries\" which contains a list of countries. Each country should have the key `name` and `population`',\n",
" version=\"v1\",\n",
" version=\"v2\",\n",
" include_types=[\"chat_model\"],\n",
"):\n",
" print(event)\n",
@ -1078,7 +1131,7 @@
},
{
"cell_type": "code",
"execution_count": 21,
"execution_count": 22,
"id": "26bac0d2-76d9-446e-b346-82790236b88d",
"metadata": {},
"outputs": [
@ -1086,17 +1139,17 @@
"name": "stdout",
"output_type": "stream",
"text": [
"{'event': 'on_chain_start', 'run_id': '55ab7082-7200-4545-8f45-bb0997b0bce8', 'name': 'RunnableSequence', 'tags': ['my_chain'], 'metadata': {}, 'data': {'input': 'output a list of the countries france, spain and japan and their populations in JSON format. Use a dict with an outer key of \"countries\" which contains a list of countries. Each country should have the key `name` and `population`'}}\n",
"{'event': 'on_chat_model_start', 'name': 'ChatAnthropic', 'run_id': 'd2efdbe8-77e4-4b29-ae68-be163239385e', 'tags': ['seq:step:1', 'my_chain'], 'metadata': {}, 'data': {'input': {'messages': [[HumanMessage(content='output a list of the countries france, spain and japan and their populations in JSON format. Use a dict with an outer key of \"countries\" which contains a list of countries. Each country should have the key `name` and `population`')]]}}}\n",
"{'event': 'on_chain_start', 'data': {'input': 'output a list of the countries france, spain and japan and their populations in JSON format. Use a dict with an outer key of \"countries\" which contains a list of countries. Each country should have the key `name` and `population`'}, 'name': 'RunnableSequence', 'tags': ['my_chain'], 'run_id': 'fd68dd64-7a4d-4bdb-a0c2-ee592db0d024', 'metadata': {}}\n",
"{'event': 'on_chat_model_start', 'data': {'input': {'messages': [[HumanMessage(content='output a list of the countries france, spain and japan and their populations in JSON format. Use a dict with an outer key of \"countries\" which contains a list of countries. Each country should have the key `name` and `population`')]]}}, 'name': 'ChatAnthropic', 'tags': ['seq:step:1', 'my_chain'], 'run_id': 'efd3c8af-4be5-4f6c-9327-e3f9865dd1cd', 'metadata': {}}\n",
" 'output a list of the countries france, spain and japan and their populations in JSON format. Use a dict with an outer key of \"countries\" which contains a list of countries. Each country should have the key `name` and `population`',\n",
" 'output a list of the countries france, spain and japan and their populations in JSON format. Use a dict with an outer key of \"countries\" which contains a list of countries. Each country should have the key `name` and `population`',\n",
" \"output a list of the countries france, spain and japan and their populations in JSON format. \"\n",
" 'Use a dict with an outer key of \"countries\" which contains a list of countries. '\n",
" \"Each country should have the key `name` and `population`\",\n",
"):\n",
" print(chunk, flush=True)"
]
@ -1203,7 +1258,7 @@
},
{
"cell_type": "code",
"execution_count": 24,
"execution_count": 25,
"id": "b08215cd-bffa-4e76-aaf3-c52ee34f152c",
"metadata": {},
"outputs": [
@ -1246,8 +1301,10 @@
"num_events = 0\n",
"\n",
"async for event in chain.astream_events(\n",
" 'output a list of the countries france, spain and japan and their populations in JSON format. Use a dict with an outer key of \"countries\" which contains a list of countries. Each country should have the key `name` and `population`',\n",
" version=\"v1\",\n",
" \"output a list of the countries france, spain and japan and their populations in JSON format. \"\n",
" 'Use a dict with an outer key of \"countries\" which contains a list of countries. '\n",
" \"Each country should have the key `name` and `population`\",\n",
Eugene Yurtsev
2 weeks ago
committed by
GitHub