Prompt Engineering Guide: Chain of Thought, Few-Shot and RAG

Overview

This tutorial covers three foundational prompt engineering techniques used with Anthropic's Claude API: Chain of Thought (CoT) prompting, Few-Shot learning, and Retrieval-Augmented Generation (RAG). By the end, you will be able to implement each technique in Python, understand when to use each approach, and confidently answer related questions on the Claude AI Developer Certification exam.

Prerequisites

• Python 3.10+ installed locally
• An Anthropic API key (sign up at console.anthropic.com)
• Basic familiarity with REST APIs and Python
• pip and virtualenv knowledge

These techniques are heavily tested on the Claude AI Developer Certification because they represent the difference between a naive prompt and a production-grade AI integration. Mastering them will improve model output quality, reduce hallucinations, and lower token costs.

Getting Started

Create a virtual environment and install the required packages:

# Create and activate a virtual environment
python3 -m venv claude-env
source claude-env/bin/activate  # Windows: claude-env\Scripts\activate

# Install dependencies
pip install anthropic==0.25.0 chromadb==0.4.24 sentence-transformers==2.7.0

Set your API key as an environment variable — never hardcode secrets in source files:

export ANTHROPIC_API_KEY="sk-ant-your-key-here"

Verify your setup with a minimal ping to the Claude API:

import anthropic
import os

client = anthropic.Anthropic(api_key=os.environ["ANTHROPIC_API_KEY"])

message = client.messages.create(
    model="claude-3-5-sonnet-20241022",
    max_tokens=64,
    messages=[{"role": "user", "content": "Say hello in one sentence."}]
)

print(message.content[0].text)

Chain of Thought Prompting

Chain of Thought (CoT) prompting instructs the model to reason step-by-step before producing a final answer. This dramatically improves accuracy on multi-step reasoning tasks such as math problems, code debugging, and logical inference.

There are two variants: Zero-Shot CoT (append "Let's think step by step") and Explicit CoT (embed reasoning steps directly in the prompt). For Claude, explicit CoT inside the system block is the production-preferred pattern.

import anthropic
import os

client = anthropic.Anthropic(api_key=os.environ["ANTHROPIC_API_KEY"])

def chain_of_thought_query(user_problem: str) -> str:
    """
    Uses explicit Chain of Thought to solve a multi-step problem.
    """
    response = client.messages.create(
        model="claude-3-5-sonnet-20241022",
        max_tokens=512,
        system=(
            "You are a precise analytical assistant. "
            "For every problem, follow this exact structure:\n"
            "1. Restate the problem in your own words.\n"
            "2. Identify all known variables and constraints.\n"
            "3. Work through each reasoning step explicitly.\n"
            "4. State your final answer on a line beginning with 'ANSWER:'"
        ),
        messages=[{"role": "user", "content": user_problem}]
    )
    return response.content[0].text

problem = (
    "A cloud team deploys 3 microservices. Each service needs 2 vCPUs and 4 GB RAM. "
    "The EC2 instance type m5.xlarge provides 4 vCPUs and 16 GB RAM. "
    "How many m5.xlarge instances are needed to run all services with no resource sharing?"
)

print(chain_of_thought_query(problem))

Common pitfall: Asking Claude to think step-by-step without giving it a structured output format often results in verbose reasoning with an ambiguous final answer. Always define where the answer should appear (e.g., a line prefixed with ANSWER:) so downstream parsing is reliable.

Few-Shot Learning

Few-Shot prompting provides the model with labelled input-output examples before the actual task. This aligns the model's output format, tone, and logic to your domain without fine-tuning — making it extremely cost-effective for production systems.

The key is example quality over quantity. Two well-crafted examples consistently outperform six mediocre ones. Each example must be representative of the real input distribution.

import anthropic
import os

client = anthropic.Anthropic(api_key=os.environ["ANTHROPIC_API_KEY"])

# Build few-shot examples as a conversation history
FEW_SHOT_EXAMPLES = [
    {
        "role": "user",
        "content": "Classify this cloud alert: 'CPU utilization on prod-web-01 exceeded 95% for 10 minutes.'"
    },
    {
        "role": "assistant",
        "content": "SEVERITY: HIGH | CATEGORY: Compute | ACTION: Scale out ASG or investigate runaway process."
    },
    {
        "role": "user",
        "content": "Classify this cloud alert: 'S3 bucket policy updated by IAM user devops-temp.'"
    },
    {
        "role": "assistant",
        "content": "SEVERITY: MEDIUM | CATEGORY: Security | ACTION: Review policy change in CloudTrail and verify intent."
    }
]

def classify_alert(alert_text: str) -> str:
    """
    Classifies a cloud alert using few-shot examples.
    """
    messages = FEW_SHOT_EXAMPLES + [
        {"role": "user", "content": f"Classify this cloud alert: '{alert_text}'"}
    ]

    response = client.messages.create(
        model="claude-3-5-sonnet-20241022",
        max_tokens=128,
        system="You are a cloud operations AI. Classify alerts using the exact format shown.",
        messages=messages
    )
    return response.content[0].text

# Test with a new alert
new_alert = "RDS instance db-prod-mysql storage is 92% full."
print(classify_alert(new_alert))

Best practices for Few-Shot:

• Keep examples in chronological conversation format using the messages array — not concatenated strings
• Cover edge cases in your examples, not just the happy path
• Maintain consistent formatting across all examples so Claude can infer the output schema
• Limit to 3-6 examples; beyond that, consider fine-tuning or RAG instead

Retrieval-Augmented Generation (RAG)

RAG grounds Claude's responses in real, up-to-date documents by retrieving relevant context at query time and injecting it into the prompt. This eliminates hallucinations on domain-specific knowledge and removes the need to bake facts into the system prompt — which wastes tokens.

The pipeline has three steps: index your documents into a vector store, retrieve top-K relevant chunks at query time, then augment the Claude prompt with those chunks.

import anthropic
import os
import chromadb
from sentence_transformers import SentenceTransformer

client = anthropic.Anthropic(api_key=os.environ["ANTHROPIC_API_KEY"])
embedder = SentenceTransformer("all-MiniLM-L6-v2")

# --- Step 1: Index documents into ChromaDB ---
chroma_client = chromadb.Client()
collection = chroma_client.create_collection("cloud_docs")

documents = [
    "CloudFront distributions cache content at edge locations to reduce latency.",
    "AWS Shield Standard protects all AWS customers from common DDoS attacks at no extra cost.",
    "S3 Intelligent-Tiering automatically moves objects between access tiers to optimize costs.",
    "Route 53 health checks can trigger DNS failover when an endpoint becomes unhealthy."
]

doc_embeddings = embedder.encode(documents).tolist()
collection.add(
    documents=documents,
    embeddings=doc_embeddings,
    ids=[f"doc_{i}" for i in range(len(documents))]
)

# --- Step 2: Retrieve top-K relevant chunks ---
def retrieve_context(query: str, top_k: int = 2) -> list[str]:
    query_embedding = embedder.encode([query]).tolist()
    results = collection.query(query_embeddings=query_embedding, n_results=top_k)
    return results["documents"][0]

# --- Step 3: Augment Claude prompt with retrieved context ---
def rag_query(user_question: str) -> str:
    context_chunks = retrieve_context(user_question)
    context_block = "\n".join(f"- {chunk}" for chunk in context_chunks)

    augmented_prompt = (
        f"Use ONLY the following context to answer the question. "
        f"If the answer is not in the context, say 'I don't have that information.'\n\n"
        f"CONTEXT:\n{context_block}\n\n"
        f"QUESTION: {user_question}"
    )

    response = client.messages.create(
        model="claude-3-5-sonnet-20241022",
        max_tokens=256,
        messages=[{"role": "user", "content": augmented_prompt}]
    )
    return response.content[0].text

print(rag_query("How does Route 53 handle endpoint failures?"))

Common pitfall: Injecting too many chunks floods the context window and introduces noise. Always cap retrieval at 3-5 chunks and instruct Claude explicitly to use only the provided context. This also prevents the model from mixing retrieved facts with its training data.

Exam Tips for Claude AI Developer Certification

The following concepts appear consistently across practice exams and the official assessment:

• CoT vs Few-Shot selection: Use CoT for multi-step reasoning tasks; use Few-Shot for format/output consistency tasks. Expect scenario questions asking you to pick the right technique.
• RAG vs Fine-tuning: RAG is preferred when knowledge changes frequently or must be auditable. Fine-tuning suits stable, high-volume domain adaptation. Know the cost/latency tradeoffs.
• Token budgeting: The exam tests awareness that longer prompts cost more. CoT increases input tokens; RAG trades retrieval latency for token efficiency versus embedding everything in system prompts.
• Claude message structure: Know that system sets persistent instructions, user and assistant alternate in the messages array, and you cannot place two consecutive messages with the same role.
• Model selection: claude-3-5-sonnet-20241022 is the current high-performance default; claude-3-haiku is the latency-optimized choice for high-throughput RAG pipelines.

Quick reference pattern:

1. Complex logic problem → Chain of Thought with structured output format
2. Custom output schema needed → Few-Shot with conversation-history examples
3. Domain knowledge grounding → RAG with top-K vector retrieval
4. Both format and grounding needed → Combine Few-Shot examples with RAG context block

Summary

You have implemented all three core prompt engineering techniques against the Claude API:

• Chain of Thought — forces step-by-step reasoning via structured system instructions, improving accuracy on complex tasks
• Few-Shot Learning — aligns model output format and logic using labelled examples in the messages array, no fine-tuning required
• RAG — grounds responses in real documents using vector similarity search, eliminating hallucinations and keeping prompts lean

The natural next step is combining all three: use Few-Shot examples to define output format, inject RAG context for factual grounding, and wrap it in a CoT system prompt for reasoning-heavy queries. This hybrid pattern is the gold standard for enterprise Claude integrations.

Continue your certification preparation with the related guide: Claude API Tools and Function Calling — CloudNinjas.ca.