import os
import openai
from dotenv import load_dotenv, find_dotenv
_ = load_dotenv(find_dotenv()) # read local .env file
openai.api_key = os.environ['OPENAI_API_KEY']1 Introduction
OpenAI’s API comes with a range of powerful features, one of which is the ability to call functions. This capability, introduced a few months back, opens up a realm of possibilities, making interactions with the language model more dynamic and informative - allowing the model to access external knowledge and functionality.
Open AI has fine tuned some of its most recent models to set additional parameters for function calls. These models are fine tuned to determine if it’s relevant to call one of these functions and if so, what the parameters to the functions should be.
This article aims to explain how to harness the function calling feature and shares some useful tips for building more capable llm-based applications.
2 Setting the Stage
Before diving into the nitty-gritty, we need to set up our environment. This begins with loading our OpenAI API key which acts as the passport to leveraging the OpenAI services.
3 Guess Current Weather Function
Let’s take a leaf out of OpenAI’s book with an example they provided at the launch of this feature - a function to fetch the current weather. Although this example returns hard-coded values, in a real-world scenario, it could be linked to a weather API to provide real-time data. This showcases how language models can be augmented with live data to respond to queries they can’t answer solely based on their training.
import json
# Example dummy function hard coded to return the same weather
# In production, this could be your backend API or an external API
def get_current_weather(location, unit="fahrenheit"):
"""Get the current weather in a given location"""
weather_info = {
"location": location,
"temperature": "72",
"unit": unit,
"forecast": ["sunny", "windy"],
}
return json.dumps(weather_info)4 Defining Functions
So OpenAI has introduced a new parameter called ‘Functions’ which allows us to pass a list of function definitions to the language model. Each function definition is a JSON object comprising various parameters like ‘name’, ‘description’, and ‘parameters’ with properties like ‘location’ and ‘unit’. These parameters play a crucial role as they are passed directly to the language model, aiding it in deciding whether or not to call a function, and how to call it if needed.
Here’s an example below:
# define a function
functions = [
{
"name": "get_current_weather",
"description": "Get the current weather in a given location",
"parameters": {
"type": "object",
"properties": {
"location": {
"type": "string",
"description": "The city and state, e.g. San Francisco, CA",
},
"unit": {"type": "string", "enum": ["celsius", "fahrenheit"]},
},
"required": ["location"],
},
}
]messages = [
{
"role": "user",
"content": "What's the weather like in Boston?"
}
]5 Engaging with the Language Model
Having defined our functions, it’s time to interact with the language model. We’ll create a list of messages, and pose a question related to the weather in Boston. We’ll specify the model, pass in our messages and the functions using the ‘functions’ parameter, and then we trigger the function call and receive a response from the language model.
import openairesponse = openai.ChatCompletion.create(
model="gpt-3.5-turbo-0613",
messages=messages,
functions=functions
)print(response){
"id": "chatcmpl-8GrEksl0bN3TSvQlnQotGNu1uf6ZN",
"object": "chat.completion",
"created": 1699028582,
"model": "gpt-3.5-turbo-0613",
"choices": [
{
"index": 0,
"message": {
"role": "assistant",
"content": null,
"function_call": {
"name": "get_current_weather",
"arguments": "{\n \"location\": \"Boston, MA\"\n}"
}
},
"finish_reason": "function_call"
}
],
"usage": {
"prompt_tokens": 82,
"completion_tokens": 18,
"total_tokens": 100
}
}6 Peeking Under the Hood
The response from the language model can be a bit cryptic at first glance. It contains a ‘function call’ parameter with objects ‘name’ and ‘argument’, which correspond to the function name and arguments to be passed to it. Although the function call isn’t executed directly, it gives us a clear indication of which function to call and the arguments to pass.
response_message = response["choices"][0]["message"]response_message<OpenAIObject at 0x7fab5e67a590> JSON: {
"role": "assistant",
"content": null,
"function_call": {
"name": "get_current_weather",
"arguments": "{\n \"location\": \"Boston, MA\"\n}"
}
}response_message["content"]response_message["function_call"]<OpenAIObject at 0x7fab5e68b720> JSON: {
"name": "get_current_weather",
"arguments": "{\n \"location\": \"Boston, MA\"\n}"
}json.loads(response_message["function_call"]["arguments"]){'location': 'Boston, MA'}args = json.loads(response_message["function_call"]["arguments"])We can now just call the function manually with the parameters suggested by the model.
get_current_weather(args)'{"location": {"location": "Boston, MA"}, "temperature": "72", "unit": "fahrenheit", "forecast": ["sunny", "windy"]}'Let’s try another example for a query that doesn’t require the weather service function call and see what that returns.
messages = [
{
"role": "user",
"content": "hi!",
}
]response = openai.ChatCompletion.create(
model="gpt-3.5-turbo-0613",
messages=messages,
functions=functions,
)print(response){
"id": "chatcmpl-8GrEwA62gbCbaPr43AcMYwdnDJIFQ",
"object": "chat.completion",
"created": 1699028594,
"model": "gpt-3.5-turbo-0613",
"choices": [
{
"index": 0,
"message": {
"role": "assistant",
"content": "Hello! How can I assist you today?"
},
"finish_reason": "stop"
}
],
"usage": {
"prompt_tokens": 76,
"completion_tokens": 10,
"total_tokens": 86
}
}We can see under the ‘choices’ section that it did’nt suggest the weather function to use - as we would hope!
7 Function Call Modes
OpenAI provides flexibility with three modes for function calling - ‘auto’, ‘none’, and ‘force’. While ‘auto’ leaves the decision to the language model, ‘none’ restricts any function calls, and ‘force’ mandates a function call. This flexibility can be handy based on the specifics of your interaction with the language model.
messages = [
{
"role": "user",
"content": "hi!",
}
]
response = openai.ChatCompletion.create(
model="gpt-3.5-turbo-0613",
messages=messages,
functions=functions,
function_call="auto",
)
print(response){
"id": "chatcmpl-8GrEzSzOIoxvVonXkpD5fcfPwWQIF",
"object": "chat.completion",
"created": 1699028597,
"model": "gpt-3.5-turbo-0613",
"choices": [
{
"index": 0,
"message": {
"role": "assistant",
"content": "Hello! How can I assist you today?"
},
"finish_reason": "stop"
}
],
"usage": {
"prompt_tokens": 76,
"completion_tokens": 10,
"total_tokens": 86
}
}messages = [
{
"role": "user",
"content": "hi!",
}
]
response = openai.ChatCompletion.create(
model="gpt-3.5-turbo-0613",
messages=messages,
functions=functions,
function_call="none",
)
print(response){
"id": "chatcmpl-8GrF1yZjffksrWabpGPpwByermf86",
"object": "chat.completion",
"created": 1699028599,
"model": "gpt-3.5-turbo-0613",
"choices": [
{
"index": 0,
"message": {
"role": "assistant",
"content": "Hello! How can I assist you today?"
},
"finish_reason": "stop"
}
],
"usage": {
"prompt_tokens": 77,
"completion_tokens": 9,
"total_tokens": 86
}
}8 Token Usage and Function Calls
It’s important to note espeically when thinking about production implications that the function definitions and descriptions count towards the token usage limit of OpenAI models. Being mindful of the length of function definitions alongside your messages is key to staying within the token limits.
Lets first use the example that will use function calls.
messages = [
{
"role": "user",
"content": "What's the weather in Boston?",
}
]
response = openai.ChatCompletion.create(
model="gpt-3.5-turbo-0613",
messages=messages,
functions=functions,
function_call="none",
)
print(response){
"id": "chatcmpl-8GrF4xcEvyCjD0uc8RhMHWMTs4nGK",
"object": "chat.completion",
"created": 1699028602,
"model": "gpt-3.5-turbo-0613",
"choices": [
{
"index": 0,
"message": {
"role": "assistant",
"content": "I am an AI language model and don't have real-time data. To get the current weather in Boston, you can use an online weather service or ask a voice assistant with access to weather data."
},
"finish_reason": "stop"
}
],
"usage": {
"prompt_tokens": 82,
"completion_tokens": 40,
"total_tokens": 122
}
}So we can note this used a total of 122 tokens. Let’s now try the example that does’nt need function calls.
messages = [
{
"role": "user",
"content": "hi!",
}
]
response = openai.ChatCompletion.create(
model="gpt-3.5-turbo-0613",
messages=messages,
functions=functions,
function_call={"name": "get_current_weather"},
)
print(response){
"id": "chatcmpl-8GrF9SZIBQYuYf0GJDtfzUvNmMJMI",
"object": "chat.completion",
"created": 1699028607,
"model": "gpt-3.5-turbo-0613",
"choices": [
{
"index": 0,
"message": {
"role": "assistant",
"content": null,
"function_call": {
"name": "get_current_weather",
"arguments": "{\n \"location\": \"San Francisco, CA\",\n \"unit\": \"celsius\"\n}"
}
},
"finish_reason": "stop"
}
],
"usage": {
"prompt_tokens": 83,
"completion_tokens": 20,
"total_tokens": 103
}
}We can see this used 103 tokens so fewer than the example that used function calls.
messages = [
{
"role": "user",
"content": "What's the weather like in Boston!",
}
]
response = openai.ChatCompletion.create(
model="gpt-3.5-turbo-0613",
messages=messages,
functions=functions,
function_call={"name": "get_current_weather"},
)
print(response){
"id": "chatcmpl-8GrFCL19xZtycOsRgAwCHbcbG12hX",
"object": "chat.completion",
"created": 1699028610,
"model": "gpt-3.5-turbo-0613",
"choices": [
{
"index": 0,
"message": {
"role": "assistant",
"content": null,
"function_call": {
"name": "get_current_weather",
"arguments": "{\n \"location\": \"Boston, MA\"\n}"
}
},
"finish_reason": "stop"
}
],
"usage": {
"prompt_tokens": 89,
"completion_tokens": 11,
"total_tokens": 100
}
}9 Round-Trip Complete Function Calls
An interesting feature is the ability to pass the results of function calls back into the language model for further processing. This round-trip can be instrumental in scenarios where a function call’s outcome needs to be fed back into the language model to generate a comprehensive response.
Let’s use an example to illustrate this - we will take the function suggested from the previous example, run the function to get the weather response, pass that response back to the model which should give a nicer plain english answer to the origiinal question.
messages.append(response["choices"][0]["message"])args = json.loads(response["choices"][0]["message"]['function_call']['arguments'])
observation = get_current_weather(args)messages.append(
{
"role": "function",
"name": "get_current_weather",
"content": observation,
}
)response = openai.ChatCompletion.create(
model="gpt-3.5-turbo-0613",
messages=messages,
)
print(response){
"id": "chatcmpl-8GrFgVaDSQvldr977wf9rzdzr86O8",
"object": "chat.completion",
"created": 1699028640,
"model": "gpt-3.5-turbo-0613",
"choices": [
{
"index": 0,
"message": {
"role": "assistant",
"content": "The weather in Boston is currently sunny and windy with a temperature of 72\u00b0F."
},
"finish_reason": "stop"
}
],
"usage": {
"prompt_tokens": 77,
"completion_tokens": 17,
"total_tokens": 94
}
}10 Conclusion
Function calling with OpenAI is a powerul feature that when used correctly, and can significantly enhance the capabilities of large language models. By defining functions, engaging with the language model, understanding the response, and managing token usage, we can create a dynamic interaction that can be fine-tuned to our requirements and generate better responses to llm-based applications.
11 Acknowledgements
I’d like to express my thanks to the wonderful Functions, Tools and Agents with LangChain by DeepLearning.ai - which i completed, and acknowledge the use of some images and other materials from the course in this article.