Iterative Prompt Development for Large Language Models

Here we look at how to develop prompts for large language models iteratively
natural-language-processing
deep-learning
openai
prompt-engineering
Author

Pranath Fernando

Published

May 2, 2023

1 Introduction

Large language models such as ChatGPT can generate text responses based on a given prompt or input. Writing prompts allow users to guide the language model’s output by providing a specific context or topic for the response. This feature has many practical applications, such as generating creative writing prompts, assisting in content creation, and even aiding in customer service chatbots.

For example, a writing prompt such as “Write a short story about a time traveler who goes back to the medieval period” could lead the language model to generate a variety of unique and creative responses. Additionally, prompts can be used to generate more specific and relevant responses for tasks such as language translation or summarization. In these cases, the prompt would provide information about the desired output, such as the language to be translated or the key points to be included in the summary. Overall, prompts provide a way to harness the power of large language models for a wide range of practical applications.

However, creating effective prompts for large language models remains a significant challenge, as even prompts that seem similar can produce vastly different outputs.

In my previous article, we looked at two prompting principles and their related tactics in order to write effective prompts for large language models to get better results.

In this article, we will iteratively analyze and refine prompts to generate marketing copy from a product fact sheet.

2 Prompt Development

The process for writing prompts can be similar in that you start with an idea for what you want to accomplish, make a first attempt at writing a prompt that is hopefully clear and specific, and perhaps, if appropriate, gives the system time to think, before running it and observing the outcome.

And if it doesn’t work well enough the first time, you can always go back and make adjustments to the idea and prompt until you find one that works for your application by iteratively determining why the instructions, for instance, weren’t clear enough or the algorithm wasn’t given enough time to think.

Because there probably isn’t a perfect prompt for every situation, it may not be worth paying as much attention to internet publications that list ‘great prompts’. It’s more crucial that you have a method for creating a strong prompt for your particular application.

You will be able to develop a prompt that is effective for the activity you want to accomplish as long as you have a good procedure for iteratively improving your prompt.

3 Setup

3.1 Load the API key and relevant Python libaries.

First we need to load certain python libs and connect the OpenAi api.

The OpenAi api library needs to be configured with an account’s secret key, which is available on the website.

You can either set it as the OPENAI_API_KEY environment variable before using the library: !export OPENAI_API_KEY='sk-...'

Or, set openai.api_key to its value:

import openai
openai.api_key = "sk-..."
import openai
import os

from dotenv import load_dotenv, find_dotenv
_ = load_dotenv(find_dotenv()) # read local .env file

openai.api_key  = os.getenv('OPENAI_API_KEY')

3.2 Helper function

We will use OpenAI’s gpt-3.5-turbo model and the chat completions endpoint.

This helper function will make it easier to use prompts and look at the generated outputs:

We’ll simply define this helper function to make it easier to use prompts and examine outputs that are generated. GetCompletion is a function that just accepts a prompt and returns the completion for that prompt.

def get_completion(prompt, model="gpt-3.5-turbo"):
    messages = [{"role": "user", "content": prompt}]
    response = openai.ChatCompletion.create(
        model=model,
        messages=messages,
        temperature=0, # this is the degree of randomness of the model's output
    )
    return response.choices[0].message["content"]

4 Generate a marketing product description from a product fact sheet

Here is a fact sheet for a chair that describes it as being a member of a lovely family that is mid-century influenced, among other things. discusses the design, includes the measurements, offers choices for the chair, lists the materials, and so forth. originates in Italy.

Let’s imagine that you wish to use this fact sheet to assist a marketing team in creating a website description for an online retailer.

My prompt here says your objective is to assist a marketing team in developing the description for a retail website or product based on a techie fact sheet, write a product description, and so on.

fact_sheet_chair = """
OVERVIEW
- Part of a beautiful family of mid-century inspired office furniture, 
including filing cabinets, desks, bookcases, meeting tables, and more.
- Several options of shell color and base finishes.
- Available with plastic back and front upholstery (SWC-100) 
or full upholstery (SWC-110) in 10 fabric and 6 leather options.
- Base finish options are: stainless steel, matte black, 
gloss white, or chrome.
- Chair is available with or without armrests.
- Suitable for home or business settings.
- Qualified for contract use.

CONSTRUCTION
- 5-wheel plastic coated aluminum base.
- Pneumatic chair adjust for easy raise/lower action.

DIMENSIONS
- WIDTH 53 CM | 20.87”
- DEPTH 51 CM | 20.08”
- HEIGHT 80 CM | 31.50”
- SEAT HEIGHT 44 CM | 17.32”
- SEAT DEPTH 41 CM | 16.14”

OPTIONS
- Soft or hard-floor caster options.
- Two choices of seat foam densities: 
 medium (1.8 lb/ft3) or high (2.8 lb/ft3)
- Armless or 8 position PU armrests 

MATERIALS
SHELL BASE GLIDER
- Cast Aluminum with modified nylon PA6/PA66 coating.
- Shell thickness: 10 mm.
SEAT
- HD36 foam

COUNTRY OF ORIGIN
- Italy
"""
prompt = f"""
Your task is to help a marketing team create a 
description for a retail website of a product based 
on a technical fact sheet.

Write a product description based on the information 
provided in the technical specifications delimited by 
triple backticks.

Technical specifications: ```{fact_sheet_chair}```
"""
response = get_completion(prompt)
print(response)
Output

Introducing our stunning mid-century inspired office chair, the perfect addition to any home or business setting. Part of a beautiful family of office furniture, including filing cabinets, desks, bookcases, meeting tables, and more, this chair is available in several options of shell color and base finishes to suit your style. Choose from plastic back and front upholstery (SWC-100) or full upholstery (SWC-110) in 10 fabric and 6 leather options.

The chair is constructed with a 5-wheel plastic coated aluminum base and features a pneumatic chair adjust for easy raise/lower action. It is available with or without armrests and is qualified for contract use. The base finish options are stainless steel, matte black, gloss white, or chrome.

Measuring at a width of 53 cm, depth of 51 cm, and height of 80 cm, with a seat height of 44 cm and seat depth of 41 cm, this chair is designed for ultimate comfort. You can also choose between soft or hard-floor caster options and two choices of seat foam densities: medium (1.8 lb/ft3) or high (2.8 lb/ft3). The armrests are available in either an armless or 8 position PU option.

The materials used in the construction of this chair are of the highest quality. The shell base glider is made of cast aluminum with modified nylon PA6/PA66 coating and has a shell thickness of 10 mm. The seat is made of HD36 foam, ensuring maximum comfort and durability.

This chair is made in Italy and is the perfect combination of style and functionality. Upgrade your workspace with our mid-century inspired office chair today!

5 Issue 1: The text is too long

  • Limit the number of words/sentences/characters.

So this is my first try. However, when I look at this, I think, wow, this is really long. It’s done a wonderful job writing a description, office chair, ideal edition, and so on. It did a good job of carrying out my request, which was to produce a product description starting from the technical information sheet.

But i’d like it to be a little bit shorter. In order to provide greater direction on the intended length of this, I will then clarify my prompt and suggest use no more than 50 words. Let’s run it again.

prompt = f"""
Your task is to help a marketing team create a 
description for a retail website of a product based 
on a technical fact sheet.

Write a product description based on the information 
provided in the technical specifications delimited by 
triple backticks.

Use at most 50 words.

Technical specifications: ```{fact_sheet_chair}```
"""
response = get_completion(prompt)
print(response)
Output

Introducing our mid-century inspired office chair, part of a beautiful furniture family. Available in various shell colors and base finishes, with plastic or full upholstery options in fabric or leather. Suitable for home or business use, with a 5-wheel base and pneumatic chair adjust. Made in Italy.

This actually seems like a much neater, succinct description of the product, introducing an office chair with mid-century design inspiration, and so forth. But its actually slightly longer than 50 words if you count.

Large language models can follow instructions regarding a very precise word count, but they’re not very good at it. Nevertheless, this is not a negative thing.

6 Issue 2. Text focuses on the wrong details

  • Ask it to focus on the aspects that are relevant to the intended audience.

As we continue to hone the language for our website, we might realise that it’s not intended to sell directly to consumers but rather to furniture dealers who would be more interested in the chair’s technical specifications and construction materials. In that scenario, you can change this question by stating that you want it to be more specific regarding the technical information.

So lets change the prompt again. I’ll add that since furniture dealers are the target audience for this description, it should be technical and concentrate on the items, materials, and construction methods used.

So maybe I can make this prompt even better. And I can add this instruction at the conclusion of the description, “Include every 7 character product ID in the technical specification,” to have it give me the product IDs. Let’s try running it now to see what happens. And thus it reads, “Let me introduce you to our mid-century inspired office chair,” discussing the two product IDs, the shell colours, and the plastic covering and aluminium base.

prompt = f"""
Your task is to help a marketing team create a 
description for a retail website of a product based 
on a technical fact sheet.

Write a product description based on the information 
provided in the technical specifications delimited by 
triple backticks.

The description is intended for furniture retailers, 
so should be technical in nature and focus on the 
materials the product is constructed from.

Use at most 50 words.

Technical specifications: ```{fact_sheet_chair}```
"""
response = get_completion(prompt)
print(response)
prompt = f"""
Your task is to help a marketing team create a 
description for a retail website of a product based 
on a technical fact sheet.

Write a product description based on the information 
provided in the technical specifications delimited by 
triple backticks.

The description is intended for furniture retailers, 
so should be technical in nature and focus on the 
materials the product is constructed from.

At the end of the description, include every 7-character 
Product ID in the technical specification.

Use at most 50 words.

Technical specifications: ```{fact_sheet_chair}```
"""
response = get_completion(prompt)
print(response)
Output

Introducing our mid-century inspired office chair, perfect for both home and business settings. With a range of shell colors and base finishes, including stainless steel and matte black, this chair is available with or without armrests. The 5-wheel plastic coated aluminum base and pneumatic chair adjust make it easy to raise and lower. Made in Italy with a cast aluminum shell and HD36 foam seat.

Quite good. Pneumatic chair and a coated aluminium base are mentioned. superior components. Therefore, by altering the prompt, you may have it concentrate more on certain characters or particular traits you want it to. Additionally, after considering this, I might decide that I also wanted to include the product ID to the description at the conclusion. SWC 110 and SOC 100 are the two options this chair provides. Therefore, perhaps I can make this prompt even better.

7 Issue 3. Description needs a table of dimensions

  • Ask it to extract information and organize it in a table.

Lets look at an example of an even more complex prompt that might give you a sense of what ChatGPT can do, which is I’ve just added a few extra instructions here. After description, include a table that gives the product dimensions, and then you’ll format everything as HTML.

prompt = f"""
Your task is to help a marketing team create a 
description for a retail website of a product based 
on a technical fact sheet.

Write a product description based on the information 
provided in the technical specifications delimited by 
triple backticks.

The description is intended for furniture retailers, 
so should be technical in nature and focus on the 
materials the product is constructed from.

At the end of the description, include every 7-character 
Product ID in the technical specification.

After the description, include a table that gives the 
product's dimensions. The table should have two columns.
In the first column include the name of the dimension. 
In the second column include the measurements in inches only.

Give the table the title 'Product Dimensions'.

Format everything as HTML that can be used in a website. 
Place the description in a <div> element.

Technical specifications: ```{fact_sheet_chair}```
"""

response = get_completion(prompt)
print(response)
Output

Mid-Century Inspired Office Chair

Introducing our mid-century inspired office chair, part of a beautiful family of office furniture that includes filing cabinets, desks, bookcases, meeting tables, and more. This chair is available in several options of shell color and base finishes, allowing you to customize it to your liking. You can choose between plastic back and front upholstery or full upholstery in 10 fabric and 6 leather options. The base finish options are stainless steel, matte black, gloss white, or chrome. The chair is also available with or without armrests, making it suitable for both home and business settings. Plus, it’s qualified for contract use, ensuring its durability and longevity.

The chair’s construction features a 5-wheel plastic coated aluminum base and a pneumatic chair adjust for easy raise/lower action. You can also choose between soft or hard-floor caster options and two choices of seat foam densities: medium (1.8 lb/ft3) or high (2.8 lb/ft3). The armrests are also customizable, with the option of armless or 8 position PU armrests.

The materials used in the chair’s construction are of the highest quality. The shell base glider is made of cast aluminum with modified nylon PA6/PA66 coating, with a shell thickness of 10 mm. The seat is made of HD36 foam, ensuring maximum comfort and support.

Made in Italy, this mid-century inspired office chair is the perfect addition to any office space. Order yours today!

Product IDs:

  • SWC-100
  • SWC-110
Product Dimensions
Dimension Measurement (inches)
Width 20.87”
Depth 20.08”
Height 31.50”
Seat Height 17.32”
Seat Depth 16.14”

In reality, it would take several repetitions before you arrived at a prompt like this. The first time someone tried to get the system to process a fact sheet, I don’t think I know anyone who would use this identical prompt.

The process of prompt development is iterative. To get closer to getting the outcomes you want, try something, evaluate how it falls short of exactly what you want, and then consider how to make your instructions clearer or, in some circumstances, consider how to give it more time to deliberate. And I believe that having a robust methodology in place to create prompts that work well for your application rather than knowing the ideal prompt is the key to being an excellent prompt engineer.

You might occasionally have a large sample size for more complicated applications, such a list of 10, 50, or 100 fact sheets. The prompt can then be developed iteratively and evaluated against a variety of scenarios. However, many people are constructing it somewhat similarly to this with just one example in the initial stages of the majority of apps. To compare prompts to a larger collection of samples, however, may occasionally be helpful for more sophisticated applications. Consider testing various prompts on a large number of fact sheets to determine their average or worst-case performance.

8 Acknowledgements

I’d like to express my thanks to the wonderful ChatGPT Prompt Engineering for Developers Course by DeepLearning.ai and OpenAI - which i completed, and acknowledge the use of some images and other materials from the course in this article.

Subscribe