Introducing LLMs with
Template Augmented Generation (TAG)

Pavel Simakov, 2/13/2024

In this article we introduce Template Augmented Generation (or TAG). A new simple technique to inject custom domain knowledge and data into LLM prompts.

Conventional Prompt Engineering

The Prompt-Engineering journey towards truly personalized LLMs entails injecting custom domain knowledge and data into the prompt to enhance its grounding. This involves incorporating the most pertinent facts as contextual information into the prompt, which is then passed to the LLM. The LLM analyzes the user query and contextual data together, generating a response tailored to the user's needs. Two specific techniques are popular today.

Few-Shot Prompting involves providing a small number of labeled examples in the prompt to enhance grounding. While effective for a limited number of examples, the custom domain knowledge dataset may outgrow the capacity of the prompt.

Retrieval Augmented Generation (RAG) emerges as a solution for larger datasets. This technique employs a vector database to retrieve relevant custom domain knowledge facts that match the user's query. RAG effectively handles datasets of any size, providing a scalable approach for leveraging custom domain knowledge.

Introducing Template Augmented Generation (TAG)

Template Augmented Generation (TAG) is a technique for dynamic customization of prompts based upon the domain knowledge and data represented in a structured document. The prompts become templates that are evaluated in the context of this structured document at runtime, resulting in fully materialized prompts, which are then passed to the underlying LLMs

Diagram: A general data flow in Template Augmented Generation (TAG).

Few notes on the diagram:

  • All domain knowledge and data are represented in a structured document – in this case, a JSON document.
  • Prompts are templatized to refer to values in this structured document; any templating system of your choice will work well.
  • Each individual prompt template is bound to a specific subset of data in the structured document based on the prompt's purpose.
  • Multiple prompts are defined, and because they are all bound to the same underlying document instance, their outputs are similarly grounded by the same underlying document.
  • At runtime, when requested by the user, prompt templates are evaluated into materialized templates based on the current content of the structured document and are passed to the underlying LLM.
  • The LLM returns the outputs and passes them to the user.

Failures of Conventional Prompt Engineering

Conventional prompt engineering excels at solving a narrow set of use cases modeled around a large number of users asking an LLM a variety of questions and expecting quality answers. This is exciting, but not the only way to use LLMs.

We believe there are much larger classes of software applications that can bring users unmatched personalized GenAI experiences using LLMs, for which conventional prompt engineering does not work. Let us introduce an example of such software application and illustrate alternative approaches to LLM grounding.

A Case Study

Envision developing an online web service to let users create all the necessary content to host their own customized instance of a Murder Mystery Game (link). The application will run in a web browser and will be backed by a set of cloud services, including the capabilities of Generative AI Large Language Models (LLMs).

A user would enter the tagline and location of the party, names of the invited guests and provide few facts about them. Then GenAI LLMs would help write all the content, including character guides for each guest, along with motives, evidence, clues and yes – the details of the actual murder.

We believe for this type of application conventional prompt engineering techniques do not work well. So we propose a different approach -- Template Augmented Generation (TAG), which excells in this type of applications.

Source Code & Results

The complete source code is available in the companion Google Colab Notebook (link). Here are the higlights:

  • we define the structured document that captured the definition of the party itself, four invited guests, and facts about them


     'party': {

         'tagline': 'Oak Stump Murder',

         'when_and_where': 'somewhere in rural Indiana'


     'guests': [


           'name': 'Alexander',

           'pronouns': 'he, him',

           'personality': 'driver',

           'sports_team': 'San Jose Sharks',

           'places': 'Japan, Greece',

           'education': 'business major',


             'Alexander and victim went to the same high '

             'school together. Alexander was bullied by the victim.'



           'name': 'Anna',

           'pronouns': 'she, her',

           'personality': 'amiable',

           'sports_team': 'does not like sports',

           'places': 'Chipotle, Starbucks',

           'education': 'dance major',


             'Anna was victim\'s fiance, but ran away from '

             'the ceremony on the wedding day.'



           'name': 'Sandra',

           'pronouns': 'she, her',

           'personality': 'analytical',

           'sports_team': 'chess',

           'places': 'library',

           'education': 'computer science',


             'Sandra borrowed a lot of money from the victim. '

             'She did not do well on the stock market...'



           'name': 'Seth',

           'pronouns': 'he, him',

           'personality': 'expressive',

           'sports_team': 'anything in boxing',

           'places': 'Metropolitan Opera',

           'education': 'litigation lawyer',


             'Seth was representing victim in court and lost '

             'the case. Victim threatened to sue and destroy Seth\'s firm.'




  • to help author the murder mystery game with LLMs, we develop three template augmented generators:
  • victim name generator template
  • guest introduction generator template
  • credible motive generator template

  • victim name generator
  • template

   We are holding a party.

   Party name is "{{ }}".

   It will be held {{ }}.

   Suggest 10 unique names for a fictional party guest.

   Each name must derrived from the words used in

   the description of the party and its location.

   The name must be short and funny.

   Only return the names.

   Use "*" in front of each name.

  • output

* Stumpy

* Tree Hugger

* Branch

* Acorn

* Hoosier

* Corn Fed

* Corn Husker

* Farmer John

* Granny Smith

* Sheriff

  • guest introduction generator
  • template

   Improve the text below.

   Make it funnier and use hyperbolas.

   Format it as an introduction of a guest at the party.

   Return improved text only.

   Text size should be around 300 words, 3 paragraphs.


   Party name is "{{ }}".

   Guest name is: {{ }}.

   Their pronouns are: {{ model.guest.pronouns }}.

   Their personality is: {{ model.guest.personality }}.

   Their sports team is: {{ model.guest.sports_team }}.

   Places they like to visit or hangout are: {{ model.guest.places }}.

   Their education is: {{ }}.

  • output

Introduction for Alexander

Oak Stump Murder is about to get a whole lot more interesting with the arrival of Alexander! This driver is a die-hard San Jose Sharks fan and has been known to travel to Japan and Greece just to catch a game. Alexander is a business major, so he's always looking for a good deal. But don't let his mild-mannered exterior fool you, he's a force to be reckoned with on the dance floor. So if you're looking for a good time, be sure to say hello to Alexander at Oak Stump Murder!

Alexander is a man of many talents. He's a talented driver, a die-hard sports fan, and a savvy business major. But what he's really known for is his killer dance moves. Alexander can dance circles around anyone on the dance floor, and he's always up for a good time. So if you're looking for someone to make your night at Oak Stump Murder unforgettable, be sure to look for Alexander!

Introduction for Anna

Oak Stump Murder is a party like no other. It's a night of debauchery and mayhem, and the only rule is that there are no rules. And there's no one who embodies the spirit of Oak Stump Murder more than Anna.

Anna is a force of nature. She's always up for anything, and she never fails to liven up the party. She's also incredibly amiable, and she's always quick with a smile. If you're looking for someone to have a good time with, Anna is the perfect person to go to.

Anna doesn't care about sports, so you won't have to worry about her talking your ear off about the latest game. Instead, she'll be more interested in talking about her favorite dance moves or her latest trip to Chipotle. She's a great listener, and she's always up for a good conversation.

If you're looking for someone to make your night at Oak Stump Murder unforgettable, Anna is the perfect person to invite. She's the life of the party, and she's sure to make you laugh until your sides hurt. So what are you waiting for? Invite Anna to Oak Stump Murder today!

Introduction for Sandra

Oak Stump Murder is a party like no other. There will be blood, gore, and a whole lot of chess. And if you're lucky, you might even get to meet Sandra. Sandra is a brilliant analytical mind who is always up for a good game of chess. She's also a bit of a loner, and she prefers to spend her time in the library or at home with her computer. But don't let her quiet demeanor fool you, Sandra is a force to be reckoned with. She's a skilled chess player and she's not afraid to get her hands dirty. So if you're looking for a challenge, Sandra is the perfect opponent. Just be prepared to lose.

Introduction for Seth

Oak Stump Murder is going to be lit AF this year. Everyone who's anyone will be there. But the most anticipated guest of all will be Seth. Seth is the most expressive person you'll ever meet. He's always got something to say and he's never afraid to show his emotions. He's also a huge boxing fan and loves to go to the Metropolitan Opera. But what Seth is really passionate about is litigation law. He's a litigation lawyer and he's damn good at it. So if you're looking for someone to talk to about boxing, the opera, or litigation law, Seth is your guy. But be prepared for an animated conversation. Seth is not one to hold back.

  • credible motive generator
  • template

   The guest at the party "{{ }}" was found murdered.

   {{ }} is a possible suspect.


   This is what we know about the relationships of

   {{ }} and the victim:

   {{ model.guest.relation_to_victim }}


   What could be a possible motive for {{ }}

   to commit the murder?

  • output

Credible Motive for Alexander

Alexander could have murdered the victim because he was bullied by the victim in high school. This could have led to Alexander feeling angry and resentful towards the victim, and he may have decided to take revenge by killing the victim.

Credible Motive for Anna

1. Anna and the victim were engaged to be married, but Anna ran away from the ceremony on the wedding day. This could be a possible motive for Anna to commit the murder. If Anna was still in love with the victim, she may have killed him in a fit of rage or jealousy. Alternatively, Anna may have killed the victim in order to get his money or property.

Credible Motive for Sandra

Sandra may have committed the murder because she was in debt to the victim. She may have felt that she had no other way to pay off her debt. Sandra may have also been angry with the victim for not giving her more time to pay back the money.

Credible Motive for Seth

Seth could have been motivated to commit the murder by the victim's threats to sue and destroy his firm. If the victim had been successful in suing Seth's firm, it could have caused financial ruin for Seth and his colleagues. The murder could have been an attempt to prevent the victim from following through on his threats.


We found Template Augmented Generation (TAG) to be very effective in the above use cases and types of software applications. We wanted to share our findings with a broad community to promote adoption of LLMs across knowledge domains.

About the Author

Dr. Pavel Simakov is an uber Tech Lead at Google, currently working on Google Cloud Platform (profile). Previously he contributed to the technical architecture of Bridgewater Associates's investment research, Google's Area 120 Startup Incubator, Google's Payments Platform, and PayPal Merchant Services. Above all he loves building intelligent software systems.

This article reflects the personal views of the author. It does not reflect the official position of Google LLC.