Creating Better Chatbots using Chained Prompts and Quality Checks

Here, we will put together chained prompts, moderation and other quality checks to create a better customer services chatbot using ChatGPT

Pranath Fernando


June 24, 2023

1 Introduction

Large language models such as ChatGPT can generate text responses based on a given prompt or input. Writing prompts allow users to guide the language model’s output by providing a specific context or topic for the response. This feature has many practical applications, such as generating creative writing prompts, assisting in content creation, and even aiding in customer service chatbots.

In earlier articles i’ve looked at how you can use ChatGPT to solve some of these tasks with simple prompts. But in many use cases, what is required is not just one prompt but a sequence of prompts where we need to also consider the outputs at each stage, before providing a final output - for example with a customer service chatbot.

In this article, we will put together chained prompts, moderation and other quality checks to create a better customer services chatbot using ChatGPT.

2 Setup

2.1 Load the API key and relevant Python libaries.

First we need to load certain python libs and connect the OpenAi api.

The OpenAi api library needs to be configured with an account’s secret key, which is available on the website.

You can either set it as the OPENAI_API_KEY environment variable before using the library: !export OPENAI_API_KEY='sk-...'

Or, set openai.api_key to its value:

import openai
openai.api_key = "sk-..."
import os
import openai
import sys
import utils

import panel as pn  # GUI

from dotenv import load_dotenv, find_dotenv
_ = load_dotenv(find_dotenv()) # read local .env file

openai.api_key  = os.environ['OPENAI_API_KEY']

The utils python module and json used here can be found in this github location.

# Define helper function
def get_completion_from_messages(messages, model="gpt-3.5-turbo", temperature=0, max_tokens=500):
    response = openai.ChatCompletion.create(
    return response.choices[0].message["content"]

3 System of chained prompts for processing the user query

So, we will construct a complete example of a customer support assistant. We’ll perform these subsequent actions.

  • First, we’ll see if the input raises any moderation API flags.
  • Second, we’ll extract the list of products if it doesn’t.
  • Third, we’ll attempt to search them up if the products are discovered.
  • Four, we’ll use the model to respond to the user’s query.

We’ll then run the response through the moderation API. Additionally, we’ll give it back to the user if it is not flagged.

So we have our user input that we’ve been using.

Tell me about the smartx pro phone and the camera.


tell me about TVs.

So let’s run this.

def process_user_message(user_input, all_messages, debug=True):
    delimiter = "```"
    # Step 1: Check input to see if it flags the Moderation API or is a prompt injection
    response = openai.Moderation.create(input=user_input)
    moderation_output = response["results"][0]

    if moderation_output["flagged"]:
        print("Step 1: Input flagged by Moderation API.")
        return "Sorry, we cannot process this request."

    if debug: print("Step 1: Input passed moderation check.")
    category_and_product_response = utils.find_category_and_product_only(user_input, utils.get_products_and_category())
    # Step 2: Extract the list of products
    category_and_product_list = utils.read_string_to_list(category_and_product_response)

    if debug: print("Step 2: Extracted list of products.")

    # Step 3: If products are found, look them up
    product_information = utils.generate_output_string(category_and_product_list)
    if debug: print("Step 3: Looked up product information.")

    # Step 4: Answer the user question
    system_message = f"""
    You are a customer service assistant for a large electronic store. \
    Respond in a friendly and helpful tone, with concise answers. \
    Make sure to ask the user relevant follow-up questions.
    messages = [
        {'role': 'system', 'content': system_message},
        {'role': 'user', 'content': f"{delimiter}{user_input}{delimiter}"},
        {'role': 'assistant', 'content': f"Relevant product information:\n{product_information}"}

    final_response = get_completion_from_messages(all_messages + messages)
    if debug:print("Step 4: Generated response to user question.")
    all_messages = all_messages + messages[1:]

    # Step 5: Put the answer through the Moderation API
    response = openai.Moderation.create(input=final_response)
    moderation_output = response["results"][0]

    if moderation_output["flagged"]:
        if debug: print("Step 5: Response flagged by Moderation API.")
        return "Sorry, we cannot provide this information."

    if debug: print("Step 5: Response passed moderation check.")

    # Step 6: Ask the model if the response answers the initial user query well
    user_message = f"""
    Customer message: {delimiter}{user_input}{delimiter}
    Agent response: {delimiter}{final_response}{delimiter}

    Does the response sufficiently answer the question?
    messages = [
        {'role': 'system', 'content': system_message},
        {'role': 'user', 'content': user_message}
    evaluation_response = get_completion_from_messages(messages)
    if debug: print("Step 6: Model evaluated the response.")

    # Step 7: If yes, use this answer; if not, say that you will connect the user to a human
    if "Y" in evaluation_response:  # Using "in" instead of "==" to be safer for model output variation (e.g., "Y." or "Yes")
        if debug: print("Step 7: Model approved the response.")
        return final_response, all_messages
        if debug: print("Step 7: Model disapproved the response.")
        neg_str = "I'm unable to provide the information you're looking for. I'll connect you with a human representative for further assistance."
        return neg_str, all_messages

user_input = "tell me about the smartx pro phone and the fotosnap camera, the dslr one. Also what tell me about your tvs"
response,_ = process_user_message(user_input,[])
Step 1: Input passed moderation check.
Step 2: Extracted list of products.
Step 3: Looked up product information.
Step 4: Generated response to user question.
Step 5: Response passed moderation check.
Step 6: Model evaluated the response.
Step 7: Model approved the response.
The SmartX ProPhone is a powerful smartphone with a 6.1-inch display, 128GB storage, 12MP dual camera, and 5G capabilities. The FotoSnap DSLR Camera is a versatile camera with a 24.2MP sensor, 1080p video, 3-inch LCD, and interchangeable lenses. As for our TVs, we have a range of options including the CineView 4K TV with a 55-inch display, 4K resolution, HDR, and smart TV capabilities, the CineView 8K TV with a 65-inch display, 8K resolution, HDR, and smart TV capabilities, and the CineView OLED TV with a 55-inch display, 4K resolution, HDR, and smart TV capabilities. Do you have any specific questions about these products or would you like me to recommend a product based on your needs?

You can see that we are following the processes to respond to the user’s question. Extraction of the product list is the first phase, followed by moderation, and then second step. The third stage is to research the product details.

The model is attempting to respond to the query as we’ve seen in the earlier articles now that we have this product information. Finally, it runs the response once again through the moderation API to make sure the user can see it without risk. Thus, this is the response that we are now accustomed to. So let’s discuss what is going a little bit.

So we have our helper function “process_user_message”. It takes in the user input, which is the current message, and an array of all of the messages so far and this is when we build the chatbot UI.

So the first step is to see if the input raises a flag for the moderation API. This was discussed in earlier articles. We inform the user that the request cannot be processed if the input is flagged.

We attempt to extract the list of items as we did in the last video if it is not highlighted. Next, we attempt to research the products. Additionally, in this instance, this will merely be an empty string if no products are identified. After that, we respond to the user’s query by providing the conversation history and any new messages that contain relevant product information. So after receiving the response, we put it through the moderation API.

If it’s flagged, we tell the user that we can’t provide this information. Maybe we’ll say something like, let me connect you, and you could take some subsequent step

4 Function that collects user and assistant messages over time

And so, let’s put this all together with a nice UI, and try to have a conversation. So here we have a function that will just accumulate the messages as we interact with the assistant.

def collect_messages(debug=False):
    user_input = inp.value_input
    if debug: print(f"User Input = {user_input}")
    if user_input == "":
    inp.value = ''
    global context
    #response, context = process_user_message(user_input, context, utils.get_products_and_category(),debug=True)
    response, context = process_user_message(user_input, context, debug=False)
    context.append({'role':'assistant', 'content':f"{response}"})
        pn.Row('User:', pn.pane.Markdown(user_input, width=600)))
        pn.Row('Assistant:', pn.pane.Markdown(response, width=600, style={'background-color': '#F6F6F6'})))
    return pn.Column(*panels)

So if we run this, now let’s try and have a conversation with the customer service assistant.

5 Chat with the chatbot

So this example, we’ve combined the methods we learnt in earlier articles to produce a comprehensive system with a series of phases that assesses user inputs, processes them, and finally checks the output.

You can change the procedures and improve the system’s overall performance by keeping track of the system’s performance over a wider range of inputs. Perhaps you will discover that some of the stages don’t even need to be taken, or that our instructions could be better for those steps. A better retrieval technique might be discovered, etc.

panels = [] # collect display 

context = [ {'role':'system', 'content':"You are Service Assistant"} ]  

inp = pn.widgets.TextInput( placeholder='Enter text here…')
button_conversation = pn.widgets.Button(name="Service Assistant")

interactive_conversation = pn.bind(collect_messages, button_conversation)

dashboard = pn.Column(
    pn.panel(interactive_conversation, loading_indicator=True, height=300),


6 Acknowledgements

I’d like to express my thanks to the wonderful Building Systems with the ChatGPT API Course by and OpenAI - which i completed, and acknowledge the use of some images and other materials from the course in this article.