Section 3: Techniques to improve gen AI model output

3.1 Proactively overcome foundation model limitations

Exam Guide §3.1 Common limitations (data dependency, knowledge cutoff, bias, fairness, hallucinations, edge cases) · Google-recommended practices (grounding, RAG, prompt engineering, fine-tuning, HITL) · continuous monitoring and evaluation (automatic model upgrades, KPIs, security patches and updates, versioning, performance tracking, drift monitoring, Agent Platform Feature Store).

Limitations of foundation models

• Six limitations according to Exam Guide/Study Guide: data dependency, knowledge cutoff, bias, fairness, hallucinations, edge cases.
• Data dependency: A foundation model's performance depends heavily on its training data. Biased or incomplete data shows up in the outputs.
• Knowledge cutoff: Models are trained up to a specific cutoff date. They know nothing about events after that date; questions about them are answered incompletely, with outdated information, or incorrectly.

Google-recommended practices against limitations

• Prompt engineering: write precise prompts to steer the model toward the desired output. → 3.2
• Grounding: connect the model's output to verifiable sources of information, e.g. company documents or the web. Result: more accurate, more relevant, company-specific and verifiable answers. → 3.3
• RAG is one grounding method: relevant information is retrieved from a knowledge base and passed to the LLM as context, in the steps retrieval → augmentation → generation (optionally iteration). Retrieval is typically done via semantic search, i.e. by meaning rather than by keywords alone. → 3.3

Continuous monitoring and evaluation

• Google Cloud offers tools for the entire lifecycle of ML models. Monitoring and evaluation are ongoing, not one-off: data, user behavior and the models themselves change.
• Automatic model upgrades: Anyone using managed Gemini models continuously benefits from the provider's model improvements and new features. The flip side: model versions have a release date and a retirement date; when switching, Google recommends testing all business-critical functions before rollout.
• Security patches and updates: keep AI systems, their dependencies and the infrastructure secure and up to date. With managed services, the provider takes care of part of this; the company's own applications, libraries and access rights remain the company's responsibility.

3.2 Prompt engineering techniques and how they drive better results

Exam Guide §3.2 Defining prompt engineering and its significance for LLMs · prompting techniques and use cases (zero-shot, one-shot, few-shot, role prompting, prompt chaining) · advanced techniques and when to use them (chain-of-thought, ReAct).

Definition and significance

• Prompt engineering = "the art and science of creating effective inputs, known as prompts, for generative AI models to maximize their value and tailor responses to specific needs". Basic definition: → 1.1.
• Significance for LLMs:
• Building blocks of good prompts according to Google documentation: task (mandatory), system instructions, few-shot examples, contextual information. In addition: role/persona, target audience, format/length requirements, constraints ("answer only on the basis of the attached documents").

Basic techniques: zero-, one-, few-shot, role, chaining

• Zero-shot prompting: pose the task without examples. The model relies solely on existing knowledge and the instruction. Example: "Summarize the following claim report in two sentences: …"
• One-shot prompting: provide exactly one example for the model to follow. Example: a publisher shows one fully formatted blurb and asks for a second one in the same style.
• Few-shot prompting: provide several examples. The model picks up pattern, format, tone and classification logic; according to Google, examples are especially effective for setting the style and tone of the response. More examples give more consistency but cost more tokens/context. Example: a logistics provider supplies five customer emails, each with the correct category ("delivery problem", "change of address", "complaint"), and then has new emails classified.

Advanced: chain-of-thought, ReAct, metaprompting

• Chain-of-thought (CoT) prompting: guide the LLM through a problem-solving process by providing examples with intermediate reasoning steps. Instead of just "question → answer", the examples show the line of reasoning; the model then breaks the problem down into intermediate steps itself. This works through prompting alone, without changing the model weights.
• ReAct (Reason + Act): enables the LLM to reason about a user request and take actions. Thinking and acting alternate: the model reflects, calls a tool (e.g. search, API, database), evaluates the result and reflects again.
• CoT vs. ReAct: CoT relies on internal reasoning, ReAct enables interaction with the outside world. The two can be combined (a ReAct agent often reasons CoT-style in its "thought" steps).

Reasoning loop (prompt engineering in agents)

• Reasoning loop: an iterative process in which the agent observes, interprets, reasons and acts, often with the help of prompt engineering. According to Study Guide p. 8, ReAct, CoT and metaprompting are among the prompt engineering techniques of the reasoning loop.

3.3 Grounding techniques and their use cases

Exam Guide §3.3 Concept of grounding; grounding with first-party enterprise data, third-party data, world data · how RAG affects generated output · Google Cloud grounding offerings: (a) pre-built RAG with Agent Search, (b) RAG APIs, (c) Grounding with Google Search · sampling parameters and settings (token count, temperature, top-p, safety settings, output length).

Grounding: concept and types of data

• Grounding = connecting the model's output to verifiable sources of information.
• Effect according to Google documentation: fewer hallucinations, answers anchored to your own data sources, verifiability via source links (grounding support/citations). In addition: more current answers beyond the knowledge cutoff, and inclusion of knowledge the model has never seen (internal company information).
• Grounding is the umbrella term, RAG is one method of it.

Retrieval-augmented generation (RAG)

• RAG in steps:
  • Retrieval: The LLM retrieves relevant information from external sources with the help of tools. Typical tools/sources: data stores, vector databases, search engines, knowledge graphs.
  • Augmentation: The retrieved information is built into the prompt to the LLM. The augmented prompt contains the original user question + relevant context.
  • Generation: The LLM processes the augmented prompt and produces the answer (ideally with citations).
  • Iteration (optional): The LLM can repeat the retrieval step if needed (different search terms, different sources) to increase quality and relevance.
• RAG combines classic information retrieval (databases, search) with generative LLMs to make answers more accurate, more current and more relevant.
• Semantic search finds content by meaning rather than by keywords alone. To do this, texts are stored as embeddings, i.e. numerical vectors that represent meaning, and searched for similarity in a vector database (or a vector index such as Vector Search). Google explicitly names the Embeddings API and Vector Search as building blocks for semantic search and RAG.

Google Cloud grounding offerings

• Agent Search (on Gemini Enterprise Agent Platform; formerly: Vertex AI Search): Google's search and recommendations solution for enterprises and at the same time the ready-made, managed RAG route. The documentation describes it as a product for building secure, enterprise-ready gen AI applications for search and recommendations with Google-quality results.
• RAG APIs = building blocks for building your own, individually controlled RAG pipeline ("build-your-own RAG") when the pre-built Agent Search does not offer enough control. According to Google documentation (some names still in Vertex AI branding):
  • Document AI Layout Parser (capture documents in a structured way and split them into context-aware chunks)
  • Embeddings API (generate vectors for text or multimodal content)
  • Vector Search (find semantically similar entries at large scale)
  • Ranking API (re-rank hits by relevance to the query)
  • Grounded generation API (generate answers based on Agent Search data sources, your own data or Google Search)
• What matters most for the exam is the distinction: pre-built (Agent Search) = fast, managed, little effort vs. RAG APIs = flexible, customizable, more development effort.

Sampling parameters and settings

• Sampling parameters = settings that influence the model's behavior and enable customized results. They are set per request via the API or in Google AI Studio / Agent Studio.
• Token count: tokens are meaningful pieces of text (words, parts of words, punctuation). Rule of thumb for Gemini: 1 token ≈ 4 characters, 100 tokens ≈ 60–80 English words. Models have a limit on how many tokens they can process at once. Tokens are also the billing unit.
• Temperature controls the "creativity" or randomness of word choice. Low = focused, more predictable (facts, extraction, code). High = more varied, more surprising (brainstorming, creative writing). At 0, the most probable token is practically always chosen, although small variations are still possible.