import os
from dotenv import load_dotenv, find_dotenv
_ = load_dotenv(find_dotenv()) # read local .env file
import warnings
warnings.filterwarnings("ignore")1 Introduction
Large language models (LLMs) are emerging as a transformative technology, enabling developers to build applications that they previously could not. But using LLMs in isolation is often not enough in practice to create a truly powerful or useful business application - the real power comes when you are able to combine them with other sources of computation, services or knowledge. LangChain is an intuitive open-source python framework created to simplify the development of useful applications using large language models (LLMs), such as OpenAI or Hugging Face.
In earlier articles we introduced the LangChain library and key components.
In this article, we will use LangChain to create LLM based agents. People often see LLM’s as a knowledge store, but you could also see them as a reasoning engine, where you give it various sources of new information to help answer questions, reason through content or even to decide what to do next. This is what LangChains agent framework helps you to do, using different tools such as DuckDuckGo search and Wikipedia and more.
2 Setup
We will use OpenAI’s ChatGPT LLM for our examples, so lets load in the required libraries.
3 Built-in LangChain tools
So we are going to import some modules, and create a language model with a temperature of 0. This is important because we are going to be using this model as a reasoning engine of an agent where its connecting to other sources of data or computation. So we want this reasoning engine to be as good and precise as possible - with less randomness !
from langchain.agents.agent_toolkits import create_python_agent
from langchain.agents import load_tools, initialize_agent
from langchain.agents import AgentType
from langchain.tools.python.tool import PythonREPLTool
from langchain.python import PythonREPL
from langchain.chat_models import ChatOpenAIllm = ChatOpenAI(temperature=0)Next we are going to load some tools the math and wikipedia tool. The llm-math tool is actually a chain itself, which uses a language model in conjunction with a calculator to do maths problems. The wikipedia tool is an API that connects to wikipedia that allows you to run search queries and get back results.
Then we are going to initialise an agent with the tools, agent and agent type which will be ‘CHAT_ZERO_SHOT_REACT_DESCRIPTION’. The key parts to note in this agent type are:
- CHAT: This is an agent optimised to work with chat models
- REACT: This is a prompting technique designed to get the best reasoning behaviour from a LLM
We will also set handle_parsing_errors=True which is useful when the LLM outputs something that is’nt possible to be parsed into an action output. When this happens we will actually pass the mistaken output back to the language model to allow it to correct itself. We will also set verbose = True so that the agent prints out all the steps its taking to make it really clear what its doing.
So we will then ask it a maths question to start and see what it outputs.
tools = load_tools(["llm-math","wikipedia"], llm=llm)agent= initialize_agent(
tools,
llm,
agent=AgentType.CHAT_ZERO_SHOT_REACT_DESCRIPTION,
handle_parsing_errors=True,
verbose = True)agent("What is the 25% of 300?")
> Entering new AgentExecutor chain...
Thought: We need to calculate 25% of 300, which involves multiplication and division.
Action:
```
{
"action": "Calculator",
"action_input": "300*0.25"
}
```
Observation: Answer: 75.0
Thought:We have the answer to the question.
Final Answer: 75.0
> Finished chain.Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.<locals>._completion_with_retry in 1.0 seconds as it raised RateLimitError: That model is currently overloaded with other requests. You can retry your request, or contact us through our help center at help.openai.com if the error persists. (Please include the request ID 64e4e1ddda90b93a26f42cecc4bd87ad in your message.).{'input': 'What is the 25% of 300?', 'output': '75.0'}So we can see here it first thinks about what it needs to do - it has a thought. It then has an action and the action is a JSON blob corresponding to 2 things: an action and an action input. The action corresponds to the tool to use i.e. Calculator, and action input is the input to that tool so here its the calculation needed to get the answer.
Next we see the observation in a separate colour, this is actually coming from the calculator tool itself.
Then we go back to the language model in green with our final answer.
For our next example we will come up with a question that could use wikipedia, and lets see what it does.
question = "Tom M. Mitchell is an American computer scientist \
and the Founders University Professor at Carnegie Mellon University (CMU)\
what book did he write?"
result = agent(question)
> Entering new AgentExecutor chain...
Thought: I should use Wikipedia to find the answer to this question.
Action:
```
{
"action": "Wikipedia",
"action_input": "Tom M. Mitchell"
}
```
Observation: Page: Tom M. Mitchell
Summary: Tom Michael Mitchell (born August 9, 1951) is an American computer scientist and the Founders University Professor at Carnegie Mellon University (CMU). He is a founder and former Chair of the Machine Learning Department at CMU. Mitchell is known for his contributions to the advancement of machine learning, artificial intelligence, and cognitive neuroscience and is the author of the textbook Machine Learning. He is a member of the United States National Academy of Engineering since 2010. He is also a Fellow of the American Academy of Arts and Sciences, the American Association for the Advancement of Science and a Fellow and past President of the Association for the Advancement of Artificial Intelligence. In October 2018, Mitchell was appointed as the Interim Dean of the School of Computer Science at Carnegie Mellon.
Page: Tom Mitchell (Australian footballer)
Summary: Thomas Mitchell (born 31 May 1993) is a professional Australian rules footballer playing for the Collingwood Football Club in the Australian Football League (AFL). He previously played for the Sydney Swans from 2012 to 2016, and the Hawthorn Football Club between 2017 and 2022. Mitchell won the Brownlow Medal as the league's best and fairest player in 2018 and set the record for the most disposals in a VFL/AFL match, accruing 54 in a game against Collingwood during that season.
Thought:The book that Tom M. Mitchell wrote is "Machine Learning".
Final Answer: Machine Learning.
> Finished chain.We can see it realises it should use wikipedia from the action JSON. The observation comes back in yellow, because different tools have outputs in different colours and its returned a summary of this person from the page. We actually get two page results for this name, for two different people. We can see that the information needed to answer the question, the book he wrote, is in the summary of the first page.
Next it has another thought to look up the book that he wrote, it isnt required and is an indication that agents are not perfectly reliable in only performing the steps needed as yet. Nevertheless it does return the correct answer.
4 Python Agent
If you have seen coding tools such as GitHub Co-Pilot and wondered how they work, or ChatGPT with the code interpreter plugin enabled, one of the things these are doing is getting the language model to write the code and then execute that code. We can do the same thing here by creating a Python Agent.
To do this, we use the same LLM as before as well as a PythonREPL tool which is a way to execute python code, a bit like a Jupyter Notebook - so the agent can execute that code using this and get back some results, and these results will be passed back into the agent so it can decide what to do next.
The problem we will give this agent to solve is to give it a list of names and ask it to sort them and print the output. These outputs are fed back into the model later on, so the model can use these to reason about the output of the code.
agent = create_python_agent(
llm,
tool=PythonREPLTool(),
verbose=True
)customer_list = [["Harrison", "Chase"],
["Lang", "Chain"],
["Dolly", "Too"],
["Elle", "Elem"],
["Geoff","Fusion"],
["Trance","Former"],
["Jen","Ayai"]
]agent.run(f"""Sort these customers by \
last name and then first name \
and print the output: {customer_list}""")
> Entering new AgentExecutor chain...
I can use the sorted() function to sort the list of customers by last name and then first name. I will need to provide a key function to sorted() that returns a tuple of the last name and first name in that order.
Action: Python REPL
Action Input:
```
customers = [['Harrison', 'Chase'], ['Lang', 'Chain'], ['Dolly', 'Too'], ['Elle', 'Elem'], ['Geoff', 'Fusion'], ['Trance', 'Former'], ['Jen', 'Ayai']]
sorted_customers = sorted(customers, key=lambda x: (x[1], x[0]))
for customer in sorted_customers:
print(customer)
```
Observation: ['Jen', 'Ayai']
['Lang', 'Chain']
['Harrison', 'Chase']
['Elle', 'Elem']
['Trance', 'Former']
['Geoff', 'Fusion']
['Dolly', 'Too']
Thought:The customers have been sorted by last name and then first name.
Final Answer: [['Jen', 'Ayai'], ['Lang', 'Chain'], ['Harrison', 'Chase'], ['Elle', 'Elem'], ['Trance', 'Former'], ['Geoff', 'Fusion'], ['Dolly', 'Too']]
> Finished chain."[['Jen', 'Ayai'], ['Lang', 'Chain'], ['Harrison', 'Chase'], ['Elle', 'Elem'], ['Trance', 'Former'], ['Geoff', 'Fusion'], ['Dolly', 'Too']]"We can see in the AgentExecutor chain it realises it can use the sorted() function to sort the list of customers.
As its a different agent type, we can see that the action and action input are formatted differently.
The action it takes uses the python REPL and then the python code is the action input to sort the list and print out the results. Then the agent realises the task is done and returns the names.
4.1 View detailed outputs of the chains
Let’s look a bit deeper at what is going on here, setting langchain.debug=True and then repeating the query.
import langchain
langchain.debug=True
agent.run(f"""Sort these customers by \
last name and then first name \
and print the output: {customer_list}""")
langchain.debug=False[chain/start] [1:chain:AgentExecutor] Entering Chain run with input:
{
"input": "Sort these customers by last name and then first name and print the output: [['Harrison', 'Chase'], ['Lang', 'Chain'], ['Dolly', 'Too'], ['Elle', 'Elem'], ['Geoff', 'Fusion'], ['Trance', 'Former'], ['Jen', 'Ayai']]"
}
[chain/start] [1:chain:AgentExecutor > 2:chain:LLMChain] Entering Chain run with input:
{
"input": "Sort these customers by last name and then first name and print the output: [['Harrison', 'Chase'], ['Lang', 'Chain'], ['Dolly', 'Too'], ['Elle', 'Elem'], ['Geoff', 'Fusion'], ['Trance', 'Former'], ['Jen', 'Ayai']]",
"agent_scratchpad": "",
"stop": [
"\nObservation:",
"\n\tObservation:"
]
}
[llm/start] [1:chain:AgentExecutor > 2:chain:LLMChain > 3:llm:ChatOpenAI] Entering LLM run with input:
{
"prompts": [
"Human: You are an agent designed to write and execute python code to answer questions.\nYou have access to a python REPL, which you can use to execute python code.\nIf you get an error, debug your code and try again.\nOnly use the output of your code to answer the question. \nYou might know the answer without running any code, but you should still run the code to get the answer.\nIf it does not seem like you can write code to answer the question, just return \"I don't know\" as the answer.\n\n\nPython REPL: A Python shell. Use this to execute python commands. Input should be a valid python command. If you want to see the output of a value, you should print it out with `print(...)`.\n\nUse the following format:\n\nQuestion: the input question you must answer\nThought: you should always think about what to do\nAction: the action to take, should be one of [Python REPL]\nAction Input: the input to the action\nObservation: the result of the action\n... (this Thought/Action/Action Input/Observation can repeat N times)\nThought: I now know the final answer\nFinal Answer: the final answer to the original input question\n\nBegin!\n\nQuestion: Sort these customers by last name and then first name and print the output: [['Harrison', 'Chase'], ['Lang', 'Chain'], ['Dolly', 'Too'], ['Elle', 'Elem'], ['Geoff', 'Fusion'], ['Trance', 'Former'], ['Jen', 'Ayai']]\nThought:"
]
}
[llm/end] [1:chain:AgentExecutor > 2:chain:LLMChain > 3:llm:ChatOpenAI] [12.68s] Exiting LLM run with output:
{
"generations": [
[
{
"text": "I can use the sorted() function to sort the list of customers by last name and then first name. I will need to provide a key function to sorted() that returns a tuple of the last name and first name in that order.\nAction: Python REPL\nAction Input:\n```\ncustomers = [['Harrison', 'Chase'], ['Lang', 'Chain'], ['Dolly', 'Too'], ['Elle', 'Elem'], ['Geoff', 'Fusion'], ['Trance', 'Former'], ['Jen', 'Ayai']]\nsorted_customers = sorted(customers, key=lambda x: (x[1], x[0]))\nfor customer in sorted_customers:\n print(customer)\n```",
"generation_info": null,
"message": {
"content": "I can use the sorted() function to sort the list of customers by last name and then first name. I will need to provide a key function to sorted() that returns a tuple of the last name and first name in that order.\nAction: Python REPL\nAction Input:\n```\ncustomers = [['Harrison', 'Chase'], ['Lang', 'Chain'], ['Dolly', 'Too'], ['Elle', 'Elem'], ['Geoff', 'Fusion'], ['Trance', 'Former'], ['Jen', 'Ayai']]\nsorted_customers = sorted(customers, key=lambda x: (x[1], x[0]))\nfor customer in sorted_customers:\n print(customer)\n```",
"additional_kwargs": {},
"example": false
}
}
]
],
"llm_output": {
"token_usage": {
"prompt_tokens": 327,
"completion_tokens": 144,
"total_tokens": 471
},
"model_name": "gpt-3.5-turbo"
}
}
[chain/end] [1:chain:AgentExecutor > 2:chain:LLMChain] [12.68s] Exiting Chain run with output:
{
"text": "I can use the sorted() function to sort the list of customers by last name and then first name. I will need to provide a key function to sorted() that returns a tuple of the last name and first name in that order.\nAction: Python REPL\nAction Input:\n```\ncustomers = [['Harrison', 'Chase'], ['Lang', 'Chain'], ['Dolly', 'Too'], ['Elle', 'Elem'], ['Geoff', 'Fusion'], ['Trance', 'Former'], ['Jen', 'Ayai']]\nsorted_customers = sorted(customers, key=lambda x: (x[1], x[0]))\nfor customer in sorted_customers:\n print(customer)\n```"
}
[tool/start] [1:chain:AgentExecutor > 4:tool:Python REPL] Entering Tool run with input:
"```
customers = [['Harrison', 'Chase'], ['Lang', 'Chain'], ['Dolly', 'Too'], ['Elle', 'Elem'], ['Geoff', 'Fusion'], ['Trance', 'Former'], ['Jen', 'Ayai']]
sorted_customers = sorted(customers, key=lambda x: (x[1], x[0]))
for customer in sorted_customers:
print(customer)
```"
[tool/end] [1:chain:AgentExecutor > 4:tool:Python REPL] [0.69ms] Exiting Tool run with output:
"['Jen', 'Ayai']
['Lang', 'Chain']
['Harrison', 'Chase']
['Elle', 'Elem']
['Trance', 'Former']
['Geoff', 'Fusion']
['Dolly', 'Too']"
[chain/start] [1:chain:AgentExecutor > 5:chain:LLMChain] Entering Chain run with input:
{
"input": "Sort these customers by last name and then first name and print the output: [['Harrison', 'Chase'], ['Lang', 'Chain'], ['Dolly', 'Too'], ['Elle', 'Elem'], ['Geoff', 'Fusion'], ['Trance', 'Former'], ['Jen', 'Ayai']]",
"agent_scratchpad": "I can use the sorted() function to sort the list of customers by last name and then first name. I will need to provide a key function to sorted() that returns a tuple of the last name and first name in that order.\nAction: Python REPL\nAction Input:\n```\ncustomers = [['Harrison', 'Chase'], ['Lang', 'Chain'], ['Dolly', 'Too'], ['Elle', 'Elem'], ['Geoff', 'Fusion'], ['Trance', 'Former'], ['Jen', 'Ayai']]\nsorted_customers = sorted(customers, key=lambda x: (x[1], x[0]))\nfor customer in sorted_customers:\n print(customer)\n```\nObservation: ['Jen', 'Ayai']\n['Lang', 'Chain']\n['Harrison', 'Chase']\n['Elle', 'Elem']\n['Trance', 'Former']\n['Geoff', 'Fusion']\n['Dolly', 'Too']\n\nThought:",
"stop": [
"\nObservation:",
"\n\tObservation:"
]
}
[llm/start] [1:chain:AgentExecutor > 5:chain:LLMChain > 6:llm:ChatOpenAI] Entering LLM run with input:
{
"prompts": [
"Human: You are an agent designed to write and execute python code to answer questions.\nYou have access to a python REPL, which you can use to execute python code.\nIf you get an error, debug your code and try again.\nOnly use the output of your code to answer the question. \nYou might know the answer without running any code, but you should still run the code to get the answer.\nIf it does not seem like you can write code to answer the question, just return \"I don't know\" as the answer.\n\n\nPython REPL: A Python shell. Use this to execute python commands. Input should be a valid python command. If you want to see the output of a value, you should print it out with `print(...)`.\n\nUse the following format:\n\nQuestion: the input question you must answer\nThought: you should always think about what to do\nAction: the action to take, should be one of [Python REPL]\nAction Input: the input to the action\nObservation: the result of the action\n... (this Thought/Action/Action Input/Observation can repeat N times)\nThought: I now know the final answer\nFinal Answer: the final answer to the original input question\n\nBegin!\n\nQuestion: Sort these customers by last name and then first name and print the output: [['Harrison', 'Chase'], ['Lang', 'Chain'], ['Dolly', 'Too'], ['Elle', 'Elem'], ['Geoff', 'Fusion'], ['Trance', 'Former'], ['Jen', 'Ayai']]\nThought:I can use the sorted() function to sort the list of customers by last name and then first name. I will need to provide a key function to sorted() that returns a tuple of the last name and first name in that order.\nAction: Python REPL\nAction Input:\n```\ncustomers = [['Harrison', 'Chase'], ['Lang', 'Chain'], ['Dolly', 'Too'], ['Elle', 'Elem'], ['Geoff', 'Fusion'], ['Trance', 'Former'], ['Jen', 'Ayai']]\nsorted_customers = sorted(customers, key=lambda x: (x[1], x[0]))\nfor customer in sorted_customers:\n print(customer)\n```\nObservation: ['Jen', 'Ayai']\n['Lang', 'Chain']\n['Harrison', 'Chase']\n['Elle', 'Elem']\n['Trance', 'Former']\n['Geoff', 'Fusion']\n['Dolly', 'Too']\n\nThought:"
]
}
[llm/end] [1:chain:AgentExecutor > 5:chain:LLMChain > 6:llm:ChatOpenAI] [8.09s] Exiting LLM run with output:
{
"generations": [
[
{
"text": "The customers have been sorted by last name and then first name, and the output has been printed. \nFinal Answer: [['Jen', 'Ayai'], ['Lang', 'Chain'], ['Harrison', 'Chase'], ['Elle', 'Elem'], ['Trance', 'Former'], ['Geoff', 'Fusion'], ['Dolly', 'Too']]",
"generation_info": null,
"message": {
"content": "The customers have been sorted by last name and then first name, and the output has been printed. \nFinal Answer: [['Jen', 'Ayai'], ['Lang', 'Chain'], ['Harrison', 'Chase'], ['Elle', 'Elem'], ['Trance', 'Former'], ['Geoff', 'Fusion'], ['Dolly', 'Too']]",
"additional_kwargs": {},
"example": false
}
}
]
],
"llm_output": {
"token_usage": {
"prompt_tokens": 526,
"completion_tokens": 75,
"total_tokens": 601
},
"model_name": "gpt-3.5-turbo"
}
}
[chain/end] [1:chain:AgentExecutor > 5:chain:LLMChain] [8.09s] Exiting Chain run with output:
{
"text": "The customers have been sorted by last name and then first name, and the output has been printed. \nFinal Answer: [['Jen', 'Ayai'], ['Lang', 'Chain'], ['Harrison', 'Chase'], ['Elle', 'Elem'], ['Trance', 'Former'], ['Geoff', 'Fusion'], ['Dolly', 'Too']]"
}
[chain/end] [1:chain:AgentExecutor] [20.77s] Exiting Chain run with output:
{
"output": "[['Jen', 'Ayai'], ['Lang', 'Chain'], ['Harrison', 'Chase'], ['Elle', 'Elem'], ['Trance', 'Former'], ['Geoff', 'Fusion'], ['Dolly', 'Too']]"
}First we see the AgentExecutor. Then we start the LLM chain, this is the LLM chain that the agent is using a combination of a prompt and LLM.
At the next level we see the exact call to the language model ‘3:llm:ChatOpenAI] Entering LLM run with input’ so we can see the fully formatted prompt, and the exact output of the model.
It then wraps up the LLM chain, and from there next it calls the REPL tool and the exact input to that tool, and then the output.
We can then see the next input ‘5:chain:LLMChain] Entering Chain run with input’ so the language model can look at the output of the python tool and reasom about what to do next.
The next steps are where the agent realises it has its answer and has finished its job.
This detailed view should give you a better idea of what is going on under the hood of this agent. Sometimes agents do strange things, so having all this information can be very useful in those cases.
5 Define your own tool
So far we have used tools already defined in LangChain, but a big power of LangChain is you can connect it to your own sources of information, your own API’s or sources of computation.
Here will will cover an example of how you can create your own agent tool. We are going to make a tool that tells us what the current date is.
First we import the tool decorator ‘angchain.agents’ and it turns any python function into a tool we can use. Next we return a function called time, to return what todays date is.
It’s also important to write a detailed function docstring for time() as this will actually be used by the agent to know what it can use this tool for, and how it should call this tool.
from langchain.agents import tool
from datetime import date@tool
def time(text: str) -> str:
"""Returns todays date, use this for any \
questions related to knowing todays date. \
The input should always be an empty string, \
and this function will always return todays \
date - any date mathmatics should occur \
outside this function."""
return str(date.today())We will now initilise an agent, and add our time tool to our existing tools.
Finally lets ask the agent what the date is and see what it does.
agent= initialize_agent(
tools + [time],
llm,
agent=AgentType.CHAT_ZERO_SHOT_REACT_DESCRIPTION,
handle_parsing_errors=True,
verbose = True)try:
result = agent("whats the date today?")
except:
print("exception on external access")
> Entering new AgentExecutor chain...
Thought: I need to use the `time` tool to get today's date.
Action:
```
{
"action": "time",
"action_input": ""
}
```
Observation: 2023-06-01
Thought:I have successfully retrieved today's date using the `time` tool.
Final Answer: Today's date is 2023-06-01.
> Finished chain.6 Acknowledgements
I’d like to express my thanks to the wonderful LangChain for LLM Application Development Course by DeepLearning.ai - which i completed, and acknowledge the use of some images and other materials from the course in this article.