Langchain and OpenAI Functions - Tools and Routing

This article will delve into the mechanics of OpenAI’s functions, how to use it to select the appropriate tools for an LLM, and how to execute them efficiently. This can give you the knowledge to create your own tools tailored to specific tasks and to utilize them effectively within the OpenAI framework and Langchain to create powerful LLM based applications.
natural-language-processing
langchain
openai
Author

Pranath Fernando

Published

November 7, 2023

1 Introduction

This article will delve into the mechanics of OpenAI’s functions, how to use it to select the appropriate tools for an LLM, and how to execute them efficiently. This can give you the knowledge to create your own tools tailored to specific tasks and to utilize them effectively within the OpenAI framework and Langchain to create powerful LLM based applications.

2 Understanding Langchain & OpenAI Tools & Functions

When we consider having an OpenAI function, there are two parts to it. The first is to let a language model pick which function to employ and what its input should be. The next step is to use those inputs to call that function. Langchain combines those two ideas into a tool, which is really a schema definition for the function that we can then convert into the OpenAI functions type, and a callable, which allows us to actually call that model.

Many of these tools are incorporated into the package search tools, mask tools, and SQL tools in Langchain, but we’ll focus on constructing our own tools in this piece. One thing that has been discovered is that when constructing your own chain and agents, a lot of it is dependent on really creating your own tools because what you’re attempting to do is probably pretty specific to your own work. So we’ll go over how to simply design our own tools, and then we’ll go over how to use a layer form to select which tools to use, and then call those tools.

We’ll begin with the standard setup, followed by the import of the tool decorator in Langchain. So we can use this decorator on top of the function, and then when we define the function, we’ll go over how to use the code. So we may use this tool decorator on top of the function that we define. This automatically transforms the function into a Langchain tool that we can use and analyse. So this search engine now has a name, a description, and arguments.

As a result, all of these will be used to create the open definition action. We may further improve on this by creating a more explicit input schema structure. This is particularly significant since, once again, the language model uses the description of the input to determine what the input should be. As a result, having a very explicit specification for the input becomes critical.

import os
import openai
import warnings
warnings.filterwarnings('ignore')

from dotenv import load_dotenv, find_dotenv
_ = load_dotenv(find_dotenv()) # read local .env file
openai.api_key = os.environ['OPENAI_API_KEY']
#!pip install langchain==0.0.319 openapi-pydantic==0.3.2
from langchain.agents import tool
@tool
def search(query: str) -> str:
    """Search for weather online"""
    return "42f"
search.name
'search'
search.description
'search(query: str) -> str - Search for weather online'
search.args
{'query': {'title': 'Query', 'type': 'string'}}

We can accomplish this by first creating an Pydantic model, and then when we define the function, we can include our schema, which will search for input. So the search input class is the one that is generated here. It has the same structure as the function, therefore it has a query parameter that corresponds to the query parameter there.

from pydantic import BaseModel, Field
class SearchInput(BaseModel):
    query: str = Field(description="Thing to search for")
@tool(args_schema=SearchInput)
def search(query: str) -> str:
    """Search for the weather online."""
    return "42f"
search.args
{'query': {'title': 'Query',
  'description': 'Thing to search for',
  'type': 'string'}}
search.run("sf")
'42f'

The primary distinction here is that we’ve added a description for that query option. This is a method of providing description and other information, therefore we want to put it in the field, to the tool’s args_schema. So, if we run this, and then do a search, we can see that we get the description of what we entered. So, if we do this and pass in a string, it is evaluated. And this isn’t actually doing anything beneath the hood, but we’ll see how you can use it later. The first practical tool we’ll build is one that returns the current temperature given a latitude and longitude.

3 Defining Input Schemas

So, first and foremost, we’ll create the input schema. So, latitude and longitude, and then the description. This function will then be defined. We’ll use this tool decorator. We’ll pass in args_schema equals open meteo input, and open meteo is the API we’ll be using behind the scenes. We’re dialling cdi.metio.com. We’re referring to the forecast endpoint. So now we have a forecast. And we’re receiving one forecast day ahead of schedule. We’re passing in the longitude and latitude and either passing them into the function or saying that we want the temperature back. We then make this call with the request library, parsing out the response as we proceed.

import requests
from pydantic import BaseModel, Field
import datetime

# Define the input schema
class OpenMeteoInput(BaseModel):
    latitude: float = Field(..., description="Latitude of the location to fetch weather data for")
    longitude: float = Field(..., description="Longitude of the location to fetch weather data for")

@tool(args_schema=OpenMeteoInput)
def get_current_temperature(latitude: float, longitude: float) -> dict:
    """Fetch current temperature for given coordinates."""
    
    BASE_URL = "https://api.open-meteo.com/v1/forecast"
    
    # Parameters for the request
    params = {
        'latitude': latitude,
        'longitude': longitude,
        'hourly': 'temperature_2m',
        'forecast_days': 1,
    }

    # Make the request
    response = requests.get(BASE_URL, params=params)
    
    if response.status_code == 200:
        results = response.json()
    else:
        raise Exception(f"API Request failed with status code: {response.status_code}")

    current_utc_time = datetime.datetime.utcnow()
    time_list = [datetime.datetime.fromisoformat(time_str.replace('Z', '+00:00')) for time_str in results['hourly']['time']]
    temperature_list = results['hourly']['temperature_2m']
    
    closest_time_index = min(range(len(time_list)), key=lambda i: abs(time_list[i] - current_utc_time))
    current_temperature = temperature_list[closest_time_index]
    
    return f'The current temperature is {current_temperature}°C'

3.1 Building Functional Tools

Then, the next day, we obtain the forecast and locate the spot in the forecast that is closest to the current time. Once we’ve done that and got that temperature, we’ll respond with the current temperature in degrees Celsius. We call it current temperature if we look at the name of the description for this. So, given the description, we can see that it contains both the signature and the doc string. Looking at the args, we can see that it has the longitude and latitude. This tool can also be converted into the exact opening error function definition.

get_current_temperature.name
'get_current_temperature'
get_current_temperature.description
'get_current_temperature(latitude: float, longitude: float) -> dict - Fetch current temperature for given coordinates.'
get_current_temperature.args
{'latitude': {'title': 'Latitude',
  'description': 'Latitude of the location to fetch weather data for',
  'type': 'number'},
 'longitude': {'title': 'Longitude',
  'description': 'Longitude of the location to fetch weather data for',
  'type': 'number'}}
from langchain.tools.render import format_tool_to_openai_function
format_tool_to_openai_function(get_current_temperature)
{'name': 'get_current_temperature',
 'description': 'get_current_temperature(latitude: float, longitude: float) -> dict - Fetch current temperature for given coordinates.',
 'parameters': {'title': 'OpenMeteoInput',
  'type': 'object',
  'properties': {'latitude': {'title': 'Latitude',
    'description': 'Latitude of the location to fetch weather data for',
    'type': 'number'},
   'longitude': {'title': 'Longitude',
    'description': 'Longitude of the location to fetch weather data for',
    'type': 'number'}},
  'required': ['latitude', 'longitude']}}
get_current_temperature({"latitude": 13, "longitude": 14})
'The current temperature is 35.8°C'

So, when we do this on a particular tool, we get a JSON blob that combines all of these parts and now has the name, description, and parameter that includes the longitude and latitude values. This is also the format that the opening error function wants. And, once again, we have the latitude and longitude from this tool. This is now a genuine request to open media, with an immediate answer.

4 Advanced Tool Usage: The Wikipedia Example

The second tool we’ll use is the Wikipedia tool. So it will use doc search to access the Wikipedia library and return a list of pages. It will iterate over the first three items on that list. And it will obtain further information about that page. So it’ll call Wikipedia.page with the page title we got back, and it’ll get this Wikipedia element. What we are not going to do is create a list of summaries. So we’ll put page and page title, then summary, then finally summary the page.

import wikipedia
@tool
def search_wikipedia(query: str) -> str:
    """Run Wikipedia search and get page summaries."""
    page_titles = wikipedia.search(query)
    summaries = []
    for page_title in page_titles[: 3]:
        try:
            wiki_page =  wikipedia.page(title=page_title, auto_suggest=False)
            summaries.append(f"Page: {page_title}\nSummary: {wiki_page.summary}")
        except (
            self.wiki_client.exceptions.PageError,
            self.wiki_client.exceptions.DisambiguationError,
        ):
            pass
    if not summaries:
        return "No good Wikipedia Search Result was found"
    return "\n\n".join(summaries)
search_wikipedia.name
'search_wikipedia'
search_wikipedia.description
'search_wikipedia(query: str) -> str - Run Wikipedia search and get page summaries.'
format_tool_to_openai_function(search_wikipedia)
{'name': 'search_wikipedia',
 'description': 'search_wikipedia(query: str) -> str - Run Wikipedia search and get page summaries.',
 'parameters': {'title': 'search_wikipediaSchemaSchema',
  'type': 'object',
  'properties': {'query': {'title': 'Query', 'type': 'string'}},
  'required': ['query']}}

And we’re going to make that list, and then we’ll answer with just a list of those summaries. So, if we look at this tool, we can look at the name and the description. This is transformed to the function definition, which we can then use to provide a query. So let’s look up a langchain and see what Wikipedia knows about it; it returns a slew of them. The langchain page is the first thing that returns. So it has the page as well as the summary.

search_wikipedia({"query": "langchain"})
'Page: LangChain\nSummary: LangChain is a framework designed to simplify the creation of applications using large language models (LLMs). As a language model integration framework, LangChain\'s use-cases largely overlap with those of language models in general, including document analysis and summarization, chatbots, and code analysis.\n\nPage: Prompt engineering\nSummary: Prompt engineering is the process of structuring text that can be interpreted and understood by a generative AI model. A prompt is natural language text describing the task that an AI should perform.A prompt for a text-to-text model can be a query such as "what is Fermat\'s little theorem?", a command such as "write a poem about leaves falling", a short statement of feedback (for example, "too verbose", "too formal", "rephrase again", "omit this word") or a longer statement including context, instructions, and input data. Prompt engineering may involve phrasing a query, specifying a style, providing relevant context or assigning a role to the AI such as "Act as a native French speaker". A prompt may include a few examples for a model to learn from, such as "maison -> house, chat -> cat, chien -> dog", an approach called few-shot learning.When communicating with a text-to-image or a text-to-audio model, a typical prompt is a description of a desired output such as "a high-quality photo of an astronaut riding a horse" or "Lo-fi slow BPM electro chill with organic samples". Prompting a text-to-image model may involve adding, removing, emphasizing and re-ordering words to achieve a desired subject, style, layout, lighting, and aesthetic.\n\n\n\nPage: Sentence embedding\nSummary: In natural language processing, a sentence embedding refers to a numeric representation of a sentence in the form of a vector of real numbers which encodes meaningful semantic information.State of the art embeddings are based on the learned hidden layer representation of dedicated sentence transformer models. BERT pioneered an approach involving the use of a dedicated [CLS] token prepended to the beginning of each sentence inputted into the model; the final hidden state vector of this token encodes information about the sentence and can be fine-tuned for use in sentence classification tasks. In practice however, BERT\'s sentence embedding with the [CLS] token achieves poor performance, often worse than simply averaging non-contextual word embeddings. SBERT later achieved superior sentence embedding performance by fine tuning BERT\'s [CLS] token embeddings through the usage of a siamese neural network architecture on the SNLI dataset. \nOther approaches are loosely based on the idea of distributional semantics applied to sentences. Skip-Thought trains an encoder-decoder structure for the task of neighboring sentences predictions. Though this has been shown to achieve worse performance than approaches such as InferSent or SBERT. \nAn alternative direction is to aggregate word embeddings, such as those returned by Word2vec, into sentence embeddings. The most straightforward approach is to simply compute the average of word vectors, known as continuous bag-of-words (CBOW). However, more elaborate solutions based on word vector quantization have also been proposed. One such approach is the vector of locally aggregated word embeddings (VLAWE), which demonstrated performance improvements in downstream text classification tasks.\n\n'

Looking down, we can notice that the following page of those results is about quick engineering in general. The third page that appears is regarding patent embedding. As a result, it provides three distinct results to this enormous list text block.

5 Integrating OpenAI Functions with APIs

So far, we’ve generated functions in our article and then open AI function definitions for those functions. Functions with which we want to engage are frequently exposed behind the API. In addition, APIs frequently have a defined specification for its inputs and outputs known as an open API specification. So what we’re going to demonstrate now is how to take one of these open API steps and convert it into a list of open AI function calls. Again, this is quite handy because much functionality is hidden behind API.

As a result, having a general straightforward way to communicate with those APIs has proven to be really beneficial. So we’ll bring in two items. To begin, we will import a function named open API spec and open AI function.

from langchain.chains.openai_functions.openapi import openapi_spec_to_openai_fn
from langchain.utilities.openapi import OpenAPISpec

And then there’s this open API spec class. Then, in the first place, we’ll load the open API spec. Take a look at this sample open API specification. The pet specification comes first, followed by a get endpoint. We can then see that there is a post endpoint with the same path, which results in the creation of a pet. Finally, we can see that there is another endpoint that accepts a pet ID and then returns information on that individual pet.

text = """
{
  "openapi": "3.0.0",
  "info": {
    "version": "1.0.0",
    "title": "Swagger Petstore",
    "license": {
      "name": "MIT"
    }
  },
  "servers": [
    {
      "url": "http://petstore.swagger.io/v1"
    }
  ],
  "paths": {
    "/pets": {
      "get": {
        "summary": "List all pets",
        "operationId": "listPets",
        "tags": [
          "pets"
        ],
        "parameters": [
          {
            "name": "limit",
            "in": "query",
            "description": "How many items to return at one time (max 100)",
            "required": false,
            "schema": {
              "type": "integer",
              "maximum": 100,
              "format": "int32"
            }
          }
        ],
        "responses": {
          "200": {
            "description": "A paged array of pets",
            "headers": {
              "x-next": {
                "description": "A link to the next page of responses",
                "schema": {
                  "type": "string"
                }
              }
            },
            "content": {
              "application/json": {
                "schema": {
                  "$ref": "#/components/schemas/Pets"
                }
              }
            }
          },
          "default": {
            "description": "unexpected error",
            "content": {
              "application/json": {
                "schema": {
                  "$ref": "#/components/schemas/Error"
                }
              }
            }
          }
        }
      },
      "post": {
        "summary": "Create a pet",
        "operationId": "createPets",
        "tags": [
          "pets"
        ],
        "responses": {
          "201": {
            "description": "Null response"
          },
          "default": {
            "description": "unexpected error",
            "content": {
              "application/json": {
                "schema": {
                  "$ref": "#/components/schemas/Error"
                }
              }
            }
          }
        }
      }
    },
    "/pets/{petId}": {
      "get": {
        "summary": "Info for a specific pet",
        "operationId": "showPetById",
        "tags": [
          "pets"
        ],
        "parameters": [
          {
            "name": "petId",
            "in": "path",
            "required": true,
            "description": "The id of the pet to retrieve",
            "schema": {
              "type": "string"
            }
          }
        ],
        "responses": {
          "200": {
            "description": "Expected response to a valid request",
            "content": {
              "application/json": {
                "schema": {
                  "$ref": "#/components/schemas/Pet"
                }
              }
            }
          },
          "default": {
            "description": "unexpected error",
            "content": {
              "application/json": {
                "schema": {
                  "$ref": "#/components/schemas/Error"
                }
              }
            }
          }
        }
      }
    }
  },
  "components": {
    "schemas": {
      "Pet": {
        "type": "object",
        "required": [
          "id",
          "name"
        ],
        "properties": {
          "id": {
            "type": "integer",
            "format": "int64"
          },
          "name": {
            "type": "string"
          },
          "tag": {
            "type": "string"
          }
        }
      },
      "Pets": {
        "type": "array",
        "maxItems": 100,
        "items": {
          "$ref": "#/components/schemas/Pet"
        }
      },
      "Error": {
        "type": "object",
        "required": [
          "code",
          "message"
        ],
        "properties": {
          "code": {
            "type": "integer",
            "format": "int32"
          },
          "message": {
            "type": "string"
          }
        }
      }
    }
  }
}
"""

We’ll load the open API spec onto this pet and then incorporate that step into the open API stage. First, we’ll get the function definition back, the open AI function definition that we may utilise. Second, we’ll get a collection of callables that we may use to invoke those functions. Because it’s a made-up step, the callables won’t be genuine, but if this was a true function in the spec, they’d work.

As can be seen, we have three functions. We have the list pets function first, then the create pets method, and finally the show pet by ID function.

spec = OpenAPISpec.from_text(text)
Attempting to load an OpenAPI 3.0.0 spec.  This may result in degraded performance. Convert your OpenAPI spec to 3.1.* spec for better support.
pet_openai_functions, pet_callables = openapi_spec_to_openai_fn(spec)
pet_openai_functions
[{'name': 'listPets',
  'description': 'List all pets',
  'parameters': {'type': 'object',
   'properties': {'params': {'type': 'object',
     'properties': {'limit': {'type': 'integer',
       'maximum': 100.0,
       'schema_format': 'int32',
       'description': 'How many items to return at one time (max 100)'}},
     'required': []}}}},
 {'name': 'createPets',
  'description': 'Create a pet',
  'parameters': {'type': 'object', 'properties': {}}},
 {'name': 'showPetById',
  'description': 'Info for a specific pet',
  'parameters': {'type': 'object',
   'properties': {'path_params': {'type': 'object',
     'properties': {'petId': {'type': 'string',
       'description': 'The id of the pet to retrieve'}},
     'required': ['petId']}}}}]

6 Intelligent Function Selection and Execution

We’ll now demonstrate how to use a language model to decide which of these functions to invoke. So we’re going to import an OpenAI model and then generate a simplified version of it. And we’ll use temperature equals zero once more because we want to choose amongst functions in a predictable manner. Then we’ll look for it in the function arguments.

We’ll use the open AI model to determine which function to execute, and then we’ll do the invocation. This results in something called routing, in which we use the language model to identify which path to take as well as the inputs to that path. First, we’ll compile a list of open AI functions that we’ll employ. So we’ll call format tools and open AI functions on our two tools, starting with wikipedia and getting the current temperature. Then we’ll build a model, set the temperature to zero, and bind to those functions.

from langchain.chat_models import ChatOpenAI
model = ChatOpenAI(temperature=0).bind(functions=pet_openai_functions)

Let’s give that model a few sentences. So, let’s base it on what’s in our step right now. It appears to use the get current temperature tool with parameters as well as the current latitude and longitude. If we look at what is langchain, we can see that it uses the wikipedia search tool with the query langchain. Before proceeding to the model call, we’ll take the next step and add a prompt.

model.invoke("what are three pets names")
AIMessage(content='', additional_kwargs={'function_call': {'name': 'listPets', 'arguments': '{\n  "params": {\n    "limit": 3\n  }\n}'}})
model.invoke("tell me about pet with id 42")
AIMessage(content='', additional_kwargs={'function_call': {'name': 'showPetById', 'arguments': '{\n  "path_params": {\n    "petId": "42"\n  }\n}'}})
functions = [
    format_tool_to_openai_function(f) for f in [
        search_wikipedia, get_current_temperature
    ]
]
model = ChatOpenAI(temperature=0).bind(functions=functions)
model.invoke("what is the weather in sf right now")
AIMessage(content='', additional_kwargs={'function_call': {'name': 'get_current_temperature', 'arguments': '{\n  "latitude": 37.7749,\n  "longitude": -122.4194\n}'}})
model.invoke("what is langchain")
AIMessage(content='', additional_kwargs={'function_call': {'name': 'search_wikipedia', 'arguments': '{\n  "query": "langchain"\n}'}})

So we’re going to make a very simple adjustment in which this prompt and its model are very simple, just a system message that says you’re a helpful assistant.

But we’re showing this because if you need to customise the prompt to be more relevant to the type of path you’re attempting to solve, this will show you how to do so. And now, when we apply the adjustment to the same inputs, we observe that we receive the same result. This is good and beneficial, but there is still a minor issue because we have this AI message with null content, an additional inquiry with self-dictionary content, function calls with self-dictionary content, and so on.

from langchain.prompts import ChatPromptTemplate
prompt = ChatPromptTemplate.from_messages([
    ("system", "You are helpful but sassy assistant"),
    ("user", "{input}"),
])
chain = prompt | model
chain.invoke({"input": "what is the weather in sf right now"})
AIMessage(content='', additional_kwargs={'function_call': {'name': 'get_current_temperature', 'arguments': '{\n  "latitude": 37.7749,\n  "longitude": -122.4194\n}'}})

So what we’re going to do is transform this into a bit more readable output. Consider the various outcomes of this answer as well. The two key ones are when it decides to call a tool and when it decides not to call a tool. When it doesn’t decide to call a tool, the major thing we’re looking at is the worth of content. When it does decide to call a tool, the important item we’re interested in is both the tool that it calls and the input to that tool.

It would be ideal if the input to that tool was not just a string of this JSON block, but rather parsed out into a dictionary, like we saw in the tagging and extractions article. This is possible with a new output parser. This will take the output and parse it into a format containing that information. First, determine whether it is simply a function call or a response. Second, if it is a function call, what function should be called, and what is the input? We’ll make a chain by combining the above prompt with the model and then exit the parser.

from langchain.agents.output_parsers import OpenAIFunctionsAgentOutputParser
chain = prompt | model | OpenAIFunctionsAgentOutputParser()

The chain will then be invoked on the same input as previously. Let’s see what the end result looks like.

So, based on the type of effect, it is an agent activity.

And this is due to the fact that it will be calling one of these tools.

result = chain.invoke({"input": "what is the weather in sf right now"})
type(result)
langchain.schema.agent.AgentActionMessageLog
result.tool
'get_current_temperature'

So, if we call result.tool, we can now see if we succeeded in determining what that tool is. And we have this chunk of inputs that we want to send to the tool if we use result.tool.input.

We can then do things like send the tool input into the function itself. So we can feed this into the system and obtain the reaction to the current temperature in the cell.

What happens when the tool is called? What if they’re the same kind?

result.tool_input
{'latitude': 37.7749, 'longitude': -122.4194}
get_current_temperature(result.tool_input)
'The current temperature is 8.3°C'
result = chain.invoke({"input": "hi!"})
type(result)
langchain.schema.agent.AgentFinish

Looking at this and the type of result, we can see that we receive it in agent finish.

So we’ll have a look at result.returnValue, which is available on all agent finish types. So far, we’ve seen how, depending on the input, it might be an agent action or an agent finish.

What exactly is happening inside? What’s going on is fairly straightforward. If a function is called, we consider it an agent activity. If no function is called and the response is merely a standard response, we describe it as an agent finish.

result.return_values
{'output': 'Hello! How can I assist you today?'}
from langchain.schema.agent import AgentFinish
def route(result):
    if isinstance(result, AgentFinish):
        return result.return_values['output']
    else:
        tools = {
            "search_wikipedia": search_wikipedia, 
            "get_current_temperature": get_current_temperature,
        }
        return tools[result.tool].run(result.tool_input)

We’ve seen how we can use a language to determine what actions to do or whether to take any actions at all, and how this can be represented as an agent finish or an agent action.

The final step will be to carry out the right action. To accomplish this, we will define a route function. This route function will act on the language model’s output and do the necessary steps.

So we’ll check to see whether it’s an agent finish, and if it is, we’ll simply return the values and output. If it’s not an agent finish, therefore if it’s an agent action, we’ll look up the appropriate tool for you and then run that tool with the tool input as stated.

Let’s make a new chain that is identical to the previous one, except that we are now including the final step of this route function.

chain = prompt | model | OpenAIFunctionsAgentOutputParser() | route

We’ll put together the prompt. This will be used as a model. We’ll translate that output into an agent finish or an agent action and then pass it through this route method.

Let’s use the weather as an example of what we’re asking. So, how is the weather and temperature today?

result = chain.invoke({"input": "What is the weather in san francisco right now?"})
result
'The current temperature is 8.3°C'

Let’s try it on an input where we want it to call Wikipedia, so we’ll execute it with langchain and inspect the result before passing it.

We get this enormous block back, which is what we got when we first looked up what langchain is on Wikipedia.

result = chain.invoke({"input": "What is langchain?"})
result
'Page: LangChain\nSummary: LangChain is a framework designed to simplify the creation of applications using large language models (LLMs). As a language model integration framework, LangChain\'s use-cases largely overlap with those of language models in general, including document analysis and summarization, chatbots, and code analysis.\n\nPage: Prompt engineering\nSummary: Prompt engineering is the process of structuring text that can be interpreted and understood by a generative AI model. A prompt is natural language text describing the task that an AI should perform.A prompt for a text-to-text model can be a query such as "what is Fermat\'s little theorem?", a command such as "write a poem about leaves falling", a short statement of feedback (for example, "too verbose", "too formal", "rephrase again", "omit this word") or a longer statement including context, instructions, and input data. Prompt engineering may involve phrasing a query, specifying a style, providing relevant context or assigning a role to the AI such as "Act as a native French speaker". A prompt may include a few examples for a model to learn from, such as "maison -> house, chat -> cat, chien -> dog", an approach called few-shot learning.When communicating with a text-to-image or a text-to-audio model, a typical prompt is a description of a desired output such as "a high-quality photo of an astronaut riding a horse" or "Lo-fi slow BPM electro chill with organic samples". Prompting a text-to-image model may involve adding, removing, emphasizing and re-ordering words to achieve a desired subject, style, layout, lighting, and aesthetic.\n\n\n\nPage: Sentence embedding\nSummary: In natural language processing, a sentence embedding refers to a numeric representation of a sentence in the form of a vector of real numbers which encodes meaningful semantic information.State of the art embeddings are based on the learned hidden layer representation of dedicated sentence transformer models. BERT pioneered an approach involving the use of a dedicated [CLS] token prepended to the beginning of each sentence inputted into the model; the final hidden state vector of this token encodes information about the sentence and can be fine-tuned for use in sentence classification tasks. In practice however, BERT\'s sentence embedding with the [CLS] token achieves poor performance, often worse than simply averaging non-contextual word embeddings. SBERT later achieved superior sentence embedding performance by fine tuning BERT\'s [CLS] token embeddings through the usage of a siamese neural network architecture on the SNLI dataset. \nOther approaches are loosely based on the idea of distributional semantics applied to sentences. Skip-Thought trains an encoder-decoder structure for the task of neighboring sentences predictions. Though this has been shown to achieve worse performance than approaches such as InferSent or SBERT. \nAn alternative direction is to aggregate word embeddings, such as those returned by Word2vec, into sentence embeddings. The most straightforward approach is to simply compute the average of word vectors, known as continuous bag-of-words (CBOW). However, more elaborate solutions based on word vector quantization have also been proposed. One such approach is the vector of locally aggregated word embeddings (VLAWE), which demonstrated performance improvements in downstream text classification tasks.\n\n'

And then finally, let’s try it out on something really simple.

chain.invoke({"input": "hi!"})
'Hello! How can I assist you today?'

7 Conclusion

In this post, we’ve outlined the intricacies of OpenAI’s advanced tool usage, from the langchain concept to the practical creation of tools and their integration with APIs. We’ve also shown how the OpenAI model can act as a smart selector and executor of functions, culminating in a comprehensive system capable of routing and performing actions.

8 Acknowledgements

I’d like to express my thanks to the wonderful Functions, Tools and Agents with LangChain by DeepLearning.ai - which i completed, and acknowledge the use of some images and other materials from the course in this article.

Subscribe