What it Takes to be Effective with Gen AI – Adopting Roles and Providing Structure to Increase Accuracy

Part 2 of a 3-part series on AI in CCM

In the first installment of my series, “What it Takes to Be Effective with Gen AI – Tips for Great Prompt Engineering,” we delved into the foundational aspects of prompt engineering, exploring principles like clarity, context and user-centric design. In this second article, we will explore the innovative techniques and methodologies in prompt engineering to shed light on the technical progress that is shaping the future of our interactions with generative AI.

First, it is important to understand that generative AI systems like ChatGPT leverage Large Language Models (LLMs) to produce human-like text based on vast amounts of training data. LLMs like ChatGPT are trained on massive datasets containing diverse language patterns, enabling them to understand context, predict subsequent words, and generate coherent and contextually appropriate responses to prompts. Organizations may choose to create a specific version of an LLMs by taking a model like ChatGPT and training it with their own proprietary data. This involves feeding the LLM their unique datasets and industry-specific language, jargon, and context. This customization can certainly enhance an LLMs and accuracy for the organization’s particular needs, enabling it to generate responses and insights that are more aligned with the company’s specific industry, policies, products, and brand. While this is an effective approach, it is very time-consuming, costly, can pose security risks, and requires a team of experts to properly train, test and maintain the LLM. For many organizations and applications of AI, this approach is out of reach.

One of the primary advantages of prompt-based techniques in working with LLMs is their flexibility and efficiency. Unlike methods that require retraining or fine-tuning the base model like creating a corporate model mentioned above, prompt engineering relies on crafting semantically rich prompts to guide the LLM. This approach effectively aligns the model to perform specialized tasks without altering its underlying structure. This flexibility is a significant win as it allows the same technique to be applied across various LLMs, regardless of their individual architectures or training datasets. It democratizes the use of advanced AI by making it more accessible and adaptable to a wide range of applications and users.

However, this approach is not without its challenges. Using complex prompts to guide AI can be tricky. If prompts are too detailed, they might become too complicated and take up too much space. This is a problem because the AI can only take in so much information at once. In practice, this means there’s a cap on how much information can be included in a single prompt or interaction. Furthermore, in longer conversations or interactions, repeating the prompt to maintain context can be problematic since LLMs have a finite input size. This limitation can pose challenges in maintaining the continuity and depth of interactions with the AI tool, particularly in more complex or extended dialogues.

Let’s begin by exploring different strategies for interacting with LLMs to achieve desired outcomes. Asking the model to assume a particular role or perspective can be particularly effective in shaping a response. Approaches might include:

Expert role adoption:

In this technique, the LLM is prompted to assume the role of an expert in a specific field to guide and provide specialized responses. For instance, in the insurance sector, you might prompt the LLM with, “As an insurance underwriter, rewrite this policy in plain language without losing meaning.” Here, the LLM takes on the role of an insurance professional and customizes its response to provide detailed information on life insurance options.

Audience role adoption

You may wish to prompt the LLM to tailor its responses to a specific audience considering their knowledge level, interests, and needs. For instance, in the finance/banking domain, if the target audience is first-time investors under 30 years old with small portfolios, the prompt might be, “Rewrite this newsletter on the latest market trends for investors between the ages of 18 and 30 years old with a portfolio of less than $100,000 in a manner that would resonate with a first-time investor.” The LLM then adjusts its response to be informative, beginner-friendly and engaging, tailored to the needs of someone new to financial investment decisions.

Meta-prompting

This involves providing guidelines, context, and structure to guide the LLM to steer the responses. For example, a user might input, “Describe the process of applying for a personal loan.” A meta-prompt approach might involve the LLM internally rephrasing or expanding the prompt to something like, “As a financial advisor, explain the step-by-step process of applying for a personal loan, including necessary documentation, credit requirements and typical approval timelines.” This not only guides the LLM to provide a more comprehensive answer but also aligns the response more closely with what might be the underlying intent of the user’s query.

To further enhance the effectiveness of any of the above approaches in prompting, additional techniques can be applied to guide an LLM’s responses using examples or context provided within the prompts. One such technique is called “in-context learning” and involves including instructions, background information, or examples in the prompt to generate a more precise response. This might be done in a single prompt, or through a series of prompts to refine output. For example, if you ask an LLM to shorten a letter so that it can be more effectively communicated via email, you might give it an example of similar email you wish it to mimic.

Another technique that can be employed to increase accuracy is called “few-shot learning”. Few-shot learning involves giving the LLM a small number of examples to help it learn how to perform a specific task. This is particularly effective in situations where large-scale training isn’t feasible. For instance, you might provide an LLM with a few examples of well-written customer service emails with the sentiment you are looking for. The prompt would then instruct the LLM to craft responses in a similar sentiment and tone. By analyzing these few examples, the LLM learns the desired format and content for the task at hand and applies this understanding to generate similar responses.

When combined with the basic techniques of persona adoption and meta-prompting, these techniques can dramatically improve the efficiency and effectiveness of interactions with LLMs. In-context learning helps the model understand and utilize additional information provided in the prompts, while few-shot learning helps it quickly adapt to new tasks or styles with minimal examples. Together, they allow for more dynamic, accurate and contextually relevant responses from the LLM, enhancing the overall user experience.

Despite these improvements, challenges such as knowledge freshness (the ability of LLMs to provide up-to-date information) and hallucinations (the generation of incorrect or irrelevant responses) remain. Knowledge freshness is critical when using LLMs for new or recent information and, while better prompts can mitigate hallucinations, they do not entirely eliminate them. In the next and final article of this 3-part series, we will explore more sophisticated prompt engineering techniques designed to address these limitations. These advanced strategies aim to further refine user control over LLM output, pushing the boundaries of what we can achieve with this cutting-edge technology.

Click here to see part 1 of this 3-part series on AI in CCM.

Originally published in Document Media