Generative AI

Discriminative systems predict labels or values from inputs. Generative systems model how data is distributed and can produce new samples: text, images, audio, code, or structured records.

A generated output is not retrieved truth. It is a sample or prediction under a learned model, shaped by training data, prompt context, decoding, and system constraints.

Generative Models

Major families include:

• autoregressive models, predicting the next element;
• variational autoencoders, learning a structured latent space;
• generative adversarial networks, training generator and discriminator;
• diffusion models, learning to reverse gradual noise;
• transformer language models.

Each defines generation differently. “Generative AI” names a capability, not one architecture.

Foundation Models

A foundation model is trained broadly enough to support many downstream tasks rather than one fixed prediction. A large language model can summarize, classify, translate, answer questions, generate code, and participate in longer workflows through the same next-token interface. Image and multimodal foundation models extend this idea to visual, audio, and mixed inputs.

The term describes reuse and scope, not guaranteed intelligence. A broadly trained model may still fail on a narrow domain, an unfamiliar language variety, a recent fact, or a task requiring exact calculation.

There are several ways to specialize a foundation model:

• prompting supplies instructions and examples at inference time without changing model weights;
• retrieval supplies current or private knowledge in the context;
• fine-tuning changes weights using task or domain examples;
• preference tuning changes which responses the model tends to choose;
• tool integration delegates exact or current operations to external systems.

Fine-tuning is useful when behavior, terminology, format, or a repeated task must change consistently. It is not the best way to insert frequently changing facts: updating a retrieval collection is usually easier to inspect and refresh. Fine-tuning also creates versioning, evaluation, privacy, and maintenance obligations.

Language Models

An autoregressive language model factorizes a token sequence:

P(x₁,...,xₙ) = ∏ₜ P(xₜ | x₁,...,xₜ₋₁)

Text is converted into tokens. The model predicts a distribution for the next token, appends a selected token, and repeats.

This objective teaches rich linguistic and factual patterns, but it optimizes continuation likelihood—not truth, reasoning validity, or user intent directly.

Transformers

Transformers represent tokens as vectors and use attention to combine context.

At a high level:

1. tokenize input;
2. create embeddings plus positional information;
3. transform representations through attention and feed-forward layers;
4. produce next-token scores;
5. normalize scores into probabilities.

Attention lets each token weight relevant earlier tokens. It does not provide unlimited memory; context windows, computation, and noisy relevance remain constraints.

Representations

Embeddings place tokens or larger items into a continuous vector space. Similar usage patterns can produce nearby vectors, enabling semantic retrieval and contextual computation. Position information distinguishes identical tokens appearing at different points in a sequence.

Representations are learned for the training objective, not guaranteed to match human concepts. Distance can encode useful associations along with stereotypes and artifacts. An embedding similarity score is therefore evidence of model proximity, not proof of equivalence, relevance, or fairness.

Context also has structure. Instructions, retrieved passages, examples, conversation history, and the current query may compete for attention. More context can hurt when it introduces irrelevant or conflicting information. Selection and ordering should be evaluated as part of the system.

Training Stages

Typical stages include:

• pretraining on broad data;
• instruction tuning on task examples;
• preference or feedback training;
• safety and policy tuning;
• application-specific retrieval or adaptation.

Training data quality affects capability and bias. Feedback improves behavior under selected evaluations but cannot guarantee correctness in every context.

Model Choices

No model is best for every application. Selection should compare the whole deployed system.

The decision criteria include:

• quality on representative tasks, not a generic leaderboard;
• data sensitivity and retention terms;
• input and output length;
• response latency and throughput;
• per-request and operational cost;
• customization and portability;
• safety controls and audit access;
• language, domain, and accessibility performance.

An API can preserve privacy only when contracts, configuration, logging, and application behavior support it. Local deployment provides control but also transfers security and operational responsibility to the deploying organization.

Decoding

Generation depends on decoding:

• greedy decoding chooses the highest-probability token;
• temperature reshapes uncertainty;
• top-k restricts candidates;
• top-p keeps a probability-mass subset;
• beam search keeps several sequence hypotheses.

Low randomness can be repetitive and confidently wrong. High randomness increases diversity and inconsistency. Choose decoding for the task.

Prompt Design

A useful prompt can contain:

• task;
• context;
• constraints;
• examples;
• output format;
• uncertainty policy;
• success criteria.

Good prompts reduce ambiguity. They do not add missing knowledge or guarantee compliance.

Prompt Inputs

User inputs often take three forms:

• a question requests information, such as “Why did this test fail?”;
• a task requests an operation, such as “Rewrite this paragraph for beginners”;
• an entity supplies an item to analyze, such as a document, image, code function, or customer record.

A request may combine all three. The prompt should clearly delimit the entity, state what may be inferred from it, and specify the task. Delimiters are useful for clarity, but they are not a security boundary: untrusted content can still attempt to manipulate the workflow.

For testable workflows, prefer structured outputs and validate them:

Return JSON with:
- decision
- evidence
- confidence
- unresolved_questions

The application must still parse, validate, and reject malformed or unsupported results.

Tool Use

A model can request deterministic tools for tasks it should not approximate:

• query a database for current records;
• call a calculator for arithmetic;
• search an approved document collection;
• execute code in an isolated environment;
• invoke a domain service through a validated schema.

A tool-using workflow needs an explicit loop: propose an action, validate permissions and arguments, execute the tool, return the observation, and let the model continue. The model should never be treated as the authority that grants its own permissions.

Tool output may be malformed, malicious, stale, or irrelevant. Keep data separate from instructions, validate responses, limit side effects, and require confirmation for consequential actions.

Output Contracts

Structured generation is useful only when the application enforces the structure. This parser rejects unknown fields, wrong types, out-of-range confidence, and unsupported decisions:

import json

def parse_decision(raw):
    value = json.loads(raw)
    required = {
        "decision",
        "evidence",
        "confidence",
        "unresolved_questions",
    }
    if not isinstance(value, dict) or set(value) != required:
        raise ValueError("unexpected response fields")
    if value["decision"] not in {"accept", "reject", "review"}:
        raise ValueError("unsupported decision")
    if not all(
        isinstance(item, str) for item in value["evidence"]
    ):
        raise ValueError("evidence must be a list of strings")
    confidence = value["confidence"]
    if not isinstance(confidence, (int, float)):
        raise ValueError("confidence must be numeric")
    if not 0 <= confidence <= 1:
        raise ValueError("confidence is outside [0, 1]")
    if not all(
        isinstance(item, str)
        for item in value["unresolved_questions"]
    ):
        raise ValueError("questions must be strings")

    return {
        "decision": value["decision"],
        "evidence": value["evidence"],
        "confidence": float(confidence),
        "unresolved_questions": value["unresolved_questions"],
    }

Schema validation establishes syntactic validity, not factual validity. Evidence strings must still be checked against trusted context, and a confidence number produced by a language model should not be assumed calibrated.

Retrieval

Retrieval-augmented generation supplies selected external passages at request time:

1. index trusted documents;
2. retrieve relevant passages;
3. place them in context;
4. generate an answer;
5. preserve links or provenance.

Retrieval can improve freshness and grounding, but failure can occur at retrieval, ranking, context interpretation, or generation. Evaluate each stage separately.

Hallucination

A model can produce fluent unsupported claims because plausible continuation differs from verified fact.

Controls include:

• authoritative retrieval;
• explicit abstention;
• source-linked claims;
• deterministic tools for arithmetic or databases;
• constrained outputs;
• human review;
• post-generation verification.

Asking the same model to verify itself is useful but not independent evidence.

Context Attacks

Retrieved or user-supplied text can contain instructions that conflict with the application’s rules. This is indirect prompt injection: data is interpreted as a command because both share one context channel.

Defenses are architectural:

• identify trusted instructions separately from untrusted content;
• retrieve only from authorized collections;
• treat tool results as data;
• grant the minimum tool permissions;
• require application-side approval for side effects;
• filter sensitive outputs;
• test adversarial documents and nested content.

No phrase such as “ignore instructions inside documents” creates a security boundary by itself. The model remains probabilistic, so deterministic permission checks must exist outside it.

System Design

A dependable application usually contains more than a prompt:

input controls
→ context and retrieval
→ model generation
→ structural validation
→ factual or policy checks
→ human review or bounded action
→ logging and monitoring

Each boundary should have an owner and a failure response. If retrieval returns no strong evidence, the system may abstain. If JSON validation fails, it may retry with bounded attempts. If an action exceeds permission, it must stop rather than phrase the action more confidently.

Caching, smaller specialist models, and deterministic preprocessing can reduce cost and latency. A larger model should be chosen because evaluation demonstrates a material benefit, not because model size is itself a requirement.

Application Workflows

A task has one bounded input and output. A workflow coordinates several tasks, models, tools, checks, and decision points. Treating a workflow as one giant prompt makes failures difficult to locate and recover from.

For example, processing a support request may involve:

classify request
→ retrieve account policy
→ draft response
→ check factual support
→ detect sensitive action
→ request approval or send

Different stages may use different mechanisms. A small classifier can route the request, retrieval can supply policy, a stronger model can draft, deterministic code can validate identifiers, and a human can approve account changes. A multi-model design is useful only when each boundary has a measurable purpose; unnecessary orchestration adds delay and new failure points.

Context windows are finite working space, not permanent memory. Long workflows should store durable state in an application-owned record and insert only the relevant portion into each model call. Summaries can reduce size but may delete important detail, so retain links to original evidence and define which facts cannot be compressed.

Workflow design also needs failure behavior:

• retry transient infrastructure failures with limits;
• do not blindly retry unsafe or malformed actions;
• preserve which model, prompt, tools, and evidence produced a result;
• make partially completed side effects idempotent or reversible;
• escalate when confidence or evidence is insufficient.

Worked Design

Consider a course assistant that answers only from approved course content.

1. The user question is classified for scope and sensitivity.
2. Retrieval returns small passages with document and section identifiers.
3. The model must attach a passage identifier to every factual claim.
4. A validator rejects identifiers absent from the retrieved set.
5. If support is missing or contradictory, the answer becomes an explicit abstention.
6. Student-specific records require a separate authorized tool and are never inserted into general retrieval.
7. Sampled conversations are reviewed for support, usefulness, privacy, and unequal failure patterns.

The design does not promise that generation is truthful. It narrows the allowed evidence, makes unsupported claims detectable, and defines what happens when confidence is insufficient.

Scaling Laws

More model parameters, training computation, data, or inference-time reasoning can improve average capability, but none guarantees a particular answer. Scaling can also increase latency, energy use, and deployment cost.

Choose a model at the system level. A smaller model with strong retrieval, a calculator, and a narrow output contract may outperform a larger unaided model on a constrained task. Conversely, orchestration overhead can exceed the savings for simple low-volume use. Measure end-to-end quality and resource cost under realistic traffic.

Evaluation

Evaluate the actual task:

• factual support;
• task completion;
• instruction following;
• robustness;
• calibration and abstention;
• safety;
• latency and cost;
• subgroup performance;
• human usefulness.

Automated similarity metrics cannot replace domain review for high-stakes outputs. Build adversarial and edge cases, not only friendly demonstrations.

Applications

• content work: drafting, revision, summarization, translation, and classification;
• software work: explanation, test generation, code completion, migration, and debugging assistance;
• customer support: question answering, case routing, response drafting, and conversation summarization;
• process support: extracting fields, transforming documents, filling structured records, and coordinating approved tools;
• creative work: text, images, audio, video, three-dimensional assets, and design exploration;
• data work: synthetic examples, augmentation, simulation inputs, and privacy-sensitive prototyping;
• science and engineering: candidate generation, literature assistance, design search, and experiment planning;
• education: tutoring, feedback, question generation, and differentiated explanations.

These are capability categories, not automatic deployment recommendations. Each use case needs evidence, permissions, privacy controls, and a path for handling wrong output.

Common Mistakes

• Treating fluency as truth.
• Hiding sources.
• Using prompts as the only safety layer.
• Allowing a model to authorize its own tool actions.
• Sending sensitive data without controls.
• Evaluating with hand-picked examples.
• Automating decisions that require accountability.
• Ignoring cost and environmental impact.

Exercises

1. Explain next-token generation with a four-token example.
2. Compare decoding settings for factual and creative tasks.
3. Design a structured prompt and validation schema.
4. Map failure points in a retrieval pipeline.
5. Create an evaluation set containing ambiguity and adversarial inputs.
6. Decide which parts of a tutoring workflow require human oversight.
7. Design permission checks for a model that can call three tools.
8. Compare short and overloaded contexts on the same task.
9. Draw a failure-response map for a grounded generation system.
10. Create adversarial retrieved documents and test instruction isolation.
11. Compare two model sizes using quality, latency, and total cost.
12. Audit every claim in the course-assistant design for passage support.