docs: Added documentation on Anthropic models on vertex (#21070)

Description:Added documentation on Anthropic models on Vertex @lkuligin for review --------- Co-authored-by: adityarane@google.com <adityarane@google.com>
1 month ago · ee2c55ca09
parent 7d451d0041
commit ee2c55ca09
1 changed files with 168 additions and 0 deletions
--- a/docs/docs/integrations/llms/google_vertex_ai_palm.ipynb
+++ b/docs/docs/integrations/llms/google_vertex_ai_palm.ipynb
@ -648,6 +648,174 @@
    "chain = prompt | llm\n",
    "print(chain.invoke({\"thing\": \"life\"}))"
   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Anthropic on Vertex AI"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "> [Anthropic Claude 3](https://cloud.google.com/vertex-ai/generative-ai/docs/partner-models/use-claude) models on Vertex AI offer fully managed and serverless models as APIs. To use a Claude model on Vertex AI, send a request directly to the Vertex AI API endpoint. Because Anthropic Claude 3 models use a managed API, there's no need to provision or manage infrastructure."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "NOTE : Anthropic Models on Vertex are implemented as Chat Model through class `ChatAnthropicVertex`"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "!pip install -U langchain-google-vertexai anthropic[vertex]"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 3,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from langchain_core.messages import (\n",
+    "    AIMessage,\n",
+    "    AIMessageChunk,\n",
+    "    HumanMessage,\n",
+    "    SystemMessage,\n",
+    ")\n",
+    "from langchain_core.outputs import LLMResult\n",
+    "from langchain_google_vertexai.model_garden import ChatAnthropicVertex"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "NOTE : Specify the correct [Claude 3 Model Versions](https://cloud.google.com/vertex-ai/generative-ai/docs/partner-models/use-claude#claude-opus)\n",
+    "- For Claude 3 Opus (Preview), use `claude-3-opus@20240229`.\n",
+    "- For Claude 3 Sonnet, use `claude-3-sonnet@20240229`.\n",
+    "- For Claude 3 Haiku, use `claude-3-haiku@20240307`.\n",
+    "\n",
+    "We don't recommend using the Anthropic Claude 3 model versions that don't include a suffix that starts with an @ symbol (claude-3-opus, claude-3-sonnet, or claude-3-haiku)."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 5,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# TODO : Replace below with your project id and region\n",
+    "project = \"<project_id>\"\n",
+    "location = \"<region>\"\n",
+    "\n",
+    "# Initialise the Model\n",
+    "model = ChatAnthropicVertex(\n",
+    "    model_name=\"claude-3-haiku@20240307\",\n",
+    "    project=project,\n",
+    "    location=location,\n",
+    ")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 6,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# prepare input data for the model\n",
+    "raw_context = (\n",
+    "    \"My name is Peter. You are my personal assistant. My favorite movies \"\n",
+    "    \"are Lord of the Rings and Hobbit.\"\n",
+    ")\n",
+    "question = (\n",
+    "    \"Hello, could you recommend a good movie for me to watch this evening, please?\"\n",
+    ")\n",
+    "context = SystemMessage(content=raw_context)\n",
+    "message = HumanMessage(content=question)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 7,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Since your favorite movies are the Lord of the Rings and Hobbit trilogies, I would recommend checking out some other epic fantasy films that have a similar feel:\n",
+      "\n",
+      "1. The Chronicles of Narnia series - These films are based on the beloved fantasy novels by C.S. Lewis and have a great blend of adventure, magic, and memorable characters.\n",
+      "\n",
+      "2. Stardust - This 2007 fantasy film, based on the Neil Gaiman novel, has an excellent cast and a charming, whimsical tone.\n",
+      "\n",
+      "3. The Golden Compass - The first film adaptation of Philip Pullman's His Dark Materials series, with stunning visuals and a compelling story.\n",
+      "\n",
+      "4. Pan's Labyrinth - Guillermo del Toro's dark, fairy tale-inspired masterpiece set against the backdrop of the Spanish Civil War.\n",
+      "\n",
+      "5. The Princess Bride - A classic fantasy adventure film with humor, romance, and unforgettable characters.\n",
+      "\n",
+      "Let me know if any of those appeal to you or if you'd like me to suggest something else! I'm happy to provide more personalized recommendations.\n"
+     ]
+    }
+   ],
+   "source": [
+    "# Invoke the model\n",
+    "response = model.invoke([context, message])\n",
+    "print(response.content)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 8,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Sure, I'd be happy to recommend a movie for you! Since you mentioned that The Lord of the Rings and The Hobbit are among your favorite movies, I'll suggest some other epic fantasy/adventure films you might enjoy:\n",
+      "\n",
+      "1. The Princess Bride (1987) - A classic fairy tale with adventure, romance, and a lot of wit and humor. It has an all-star cast and very quotable lines.\n",
+      "\n",
+      "2. Willow (1988) - A fun fantasy film produced by George Lucas with fairies, dwarves, and brownies going on an epic quest. Has a similar tone to the Lord of the Rings movies.\n",
+      "\n",
+      "3. Stardust (2007) - An underrated fantasy adventure based on the Neil Gaiman novel about a young man entering a magical kingdom to retrieve a fallen star. Great cast and visuals.\n",
+      "\n",
+      "4. The Chronicles of Narnia series - The Lion, The Witch and The Wardrobe is the best known, but the other Narnia films are also very well done fantasy epics.\n",
+      "\n",
+      "5. The Golden Compass (2007) - First installment of the His Dark Materials trilogy, set in a parallel universe with armored polar bears and truth-seeking devices.\n",
+      "\n",
+      "Let me know if you'd like any other suggestions or have a particular style of movie in mind! I aimed for entertaining fantasy/adventure flicks similar to Lord of the Rings.\n"
+     ]
+    }
+   ],
+   "source": [
+    "# You can choose to initialize/ override the model name on Invoke method as well\n",
+    "response = model.invoke([context, message], model_name=\"claude-3-sonnet@20240229\")\n",
+    "print(response.content)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# Use streaming responses\n",
+    "sync_response = model.stream([context, message], model_name=\"claude-3-haiku@20240307\")\n",
+    "for chunk in sync_response:\n",
+    "    print(chunk.content)"
+   ]
  }
 ],
 "metadata": {