LLM Agents Academy

Overview

In the previous lessons, you learned what agents are, how LLMs power them, and the observe-think-act loop at their core. But building a real agent means choosing from a rich and fast-moving ecosystem of providers, frameworks, protocols, and infrastructure. This lesson gives you the map.

Think of it as a field guide: rather than mastering each technology now, you will understand what exists, how the pieces fit together, and where in this academy each one gets covered in depth. By the end, you will know enough to navigate conversations about agent tooling and make informed choices about your own stack.

The Ecosystem at a Glance

The agent ecosystem organizes into layers. At the bottom sit the foundation models that provide intelligence. Above them, frameworks and SDKs handle the engineering of agent loops, tool calls, and memory. Cutting across those layers are protocols and standards that define how agents connect to tools and to each other. Supporting everything is infrastructure for data retrieval, observability, and deployment. At the top, evaluation benchmarks tell us whether our agents actually work.

graph TD
  subgraph Eval["Evaluation & Benchmarks"]
      SWE["SWE-Bench"]
      GAIA["GAIA"]
      WEB["WebArena"]
  end

  subgraph Apps["Agent Applications"]
      CODE["Coding Agents"]
      RES["Research Agents"]
      CHAT["Assistants & Copilots"]
  end

  subgraph Frameworks["Frameworks & SDKs"]
      LG["LangChain / LangGraph"]
      CREW["CrewAI"]
      AUTO["AutoGen"]
      OAISDK["OpenAI Agents SDK"]
      CSDK["Claude Agent SDK"]
  end

  subgraph Protocols["Protocols & Standards"]
      MCP["MCP — Model Context Protocol"]
      A2A["A2A — Agent-to-Agent"]
  end

  subgraph Infra["Infrastructure"]
      VEC["Vector Databases"]
      OBS["Observability & Tracing"]
      DEPLOY["Deployment Platforms"]
  end

  subgraph Providers["LLM Providers & Foundation Models"]
      OAI["OpenAI — GPT-4o, o3"]
      ANT["Anthropic — Claude 4, Sonnet"]
      GOOG["Google — Gemini 2.5"]
      OSS["Open Source — Llama, Mistral, Qwen"]
  end

  Providers --> Frameworks
  Providers --> Protocols
  Frameworks --> Apps
  Protocols --> Frameworks
  Infra --> Frameworks
  Apps --> Eval

  style Providers fill:#ebf8ff,stroke:#3182ce
  style Frameworks fill:#f0fff4,stroke:#38a169
  style Protocols fill:#fefcbf,stroke:#d69e2e
  style Infra fill:#fff5f5,stroke:#e53e3e
  style Eval fill:#faf5ff,stroke:#805ad5
  style Apps fill:#e2e8f0,stroke:#4a5568

Let's walk through each layer from the bottom up.

LLM Providers and Foundation Models

Every agent starts with a foundation model -- the LLM that provides reasoning, language understanding, and generation. The choice of model shapes what your agent can do, how fast it responds, and what it costs.

Major providers include:

OpenAI -- GPT-4o for general-purpose tasks, o3 and o4-mini for complex reasoning chains. Pioneer of the function-calling API pattern that most agent frameworks build on.
Anthropic -- Claude Opus and Sonnet models, known for long context windows (up to 200K tokens), strong instruction following, and extended thinking capabilities. Originators of the Model Context Protocol.
Google -- Gemini 2.5 Pro and Flash models with million-token context windows and native multimodal support (text, images, audio, video).
Open-source models -- Llama (Meta), Mistral, and Qwen (Alibaba) can be self-hosted for full control over data privacy and cost. They are closing the gap with proprietary models, especially for focused agent tasks.

The key insight is that agents are model-agnostic in principle but model-sensitive in practice. The same agent architecture can work across providers, but the quality of tool calling, instruction following, and reasoning varies significantly. We explored these capabilities in Lesson 3.

Tip: You do not need to pick one provider forever. Good agent architectures abstract the model layer so you can swap providers as the landscape evolves.

Agent Frameworks and SDKs

Building an agent from raw API calls is educational (you will do exactly that in the next lesson), but production agents benefit from agent frameworks that handle the repetitive engineering: managing the agent loop, orchestrating tool calls, persisting memory, and structuring multi-step reasoning.

Here is a snapshot of the major frameworks:

| Framework | Key Strength | Best For | |-----------|-------------|----------| | LangChain / LangGraph | Rich ecosystem, graph-based workflows | Complex multi-step agents with branching logic | | CrewAI | Role-based multi-agent collaboration | Teams of specialized agents working together | | AutoGen (Microsoft) | Conversational multi-agent patterns | Agents that debate, review, and iterate | | OpenAI Agents SDK | Tight GPT integration, built-in guardrails | OpenAI-native agent applications | | Claude Agent SDK | Native MCP support, extended thinking | Anthropic-native agents with tool integration |

These frameworks differ in philosophy. LangGraph models agent behavior as a state graph with explicit transitions. CrewAI thinks in terms of crews, roles, and tasks. AutoGen structures agents as participants in a conversation. Each abstraction fits different problem shapes.

We will build with these frameworks hands-on in Module 8: Agent Frameworks & SDKs, comparing their strengths on the same problem.

Protocols and Standards

As agents proliferate, two challenges emerge: how does an agent connect to the world's tools, and how do agents talk to each other? Two protocols are defining the answers.

Model Context Protocol (MCP)

The Model Context Protocol is an open standard created by Anthropic that provides a universal interface between LLMs and external data sources and tools. Think of it as USB-C for AI -- one standard connector instead of custom integrations for every tool.

Before MCP, connecting an agent to a database, an API, or a file system meant writing custom glue code for each integration. MCP defines a client-server architecture where:

MCP servers expose tools, resources, and prompts through a standard interface
MCP clients (built into frameworks and applications) discover and invoke those tools
The protocol handles capability negotiation, schema description, and result formatting

This means a tool built as an MCP server works with any MCP-compatible agent -- regardless of which framework or LLM powers it.

We will dive deep into MCP in Module 3: Tool Use & Function Calling, where you will build your own MCP server and connect it to an agent.

Agent-to-Agent Protocol (A2A)

While MCP connects agents to tools, the Agent-to-Agent (A2A) protocol (introduced by Google) addresses a different problem: how do independently built agents communicate and collaborate?

A2A defines a standard for agents to:

Discover each other's capabilities through Agent Cards
Exchange structured tasks and messages
Stream progress updates and partial results
Handle long-running operations with status tracking

This matters as organizations deploy multiple specialized agents that need to coordinate -- a research agent handing off findings to a writing agent, or a coding agent requesting a review from a testing agent.

We will explore A2A in depth in Module 10: Multi-Agent Systems, where you will build agents that discover and collaborate with each other.

Infrastructure

Agents need more than models and frameworks. A production agent stack includes several infrastructure components.

Vector Databases

Vector databases store and search over embeddings -- numerical representations of text, images, and other data. They power Retrieval-Augmented Generation (RAG), where an agent retrieves relevant context from a knowledge base before generating a response. Popular options include Pinecone, Weaviate, Chroma, and pgvector.

Covered in Module 7: Memory & Knowledge, where you will build RAG pipelines and compare retrieval strategies.

Observability and Tracing

When an agent takes ten steps to answer a question, you need to see each step. Observability tools like LangSmith, Arize Phoenix, and Braintrust provide tracing, logging, and debugging for agent runs. They show you the full chain of thought, tool calls, and model responses so you can diagnose failures and optimize performance.

Covered in Module 11: Evaluation & Testing and Module 12: Production, Deployment & Safety.

Deployment Platforms

Getting agents from notebook to production requires containerization, scaling, and monitoring. Platforms like modal, AWS Bedrock Agents, and Google Vertex AI Agent Builder provide managed infrastructure for running agents at scale. Even a simple Docker container with a well-structured agent can go a long way.

Covered in Module 12: Production, Deployment & Safety.

Evaluation and Benchmarks

How do you know if your agent actually works? The field has developed several benchmarks that test agents on realistic tasks:

SWE-Bench -- gives agents real GitHub issues from open-source projects and measures whether they can produce correct code patches. The gold standard for coding agents.
GAIA -- tests general AI assistants on questions that require multi-step reasoning, tool use, and web browsing. Designed so a human can solve each task but a bare LLM cannot.
WebArena -- evaluates agents navigating real websites to complete tasks like shopping, managing content, or filling out forms.

These benchmarks matter because traditional LLM evaluations (multiple choice, summarization quality) do not capture what makes an agent effective. An agent might use a brilliant model but fail because it calls the wrong tool, gets stuck in a loop, or misinterprets an observation.

We will cover evaluation strategies, including building your own task-specific benchmarks, in Module 11: Evaluation & Testing.

How the Pieces Fit Together

To build an agent, you typically assemble components from multiple layers:

Choose a foundation model based on your task requirements (reasoning depth, context length, cost, latency)
Pick a framework that matches your architecture pattern (single agent, multi-agent, graph-based workflows)
Connect tools via MCP or native function calling for the capabilities your agent needs
Add memory and retrieval with vector databases for knowledge-intensive tasks
Instrument with observability so you can debug and improve agent behavior
Evaluate with benchmarks that match your real-world use case

No single vendor owns the whole stack. The ecosystem is deliberately modular -- protocols like MCP ensure that your tool integrations survive even if you swap models or frameworks.

Your Roadmap Through This Academy

This lesson's ecosystem map is also your roadmap for the remaining modules:

| Ecosystem Layer | Academy Coverage | |----------------|-----------------| | Foundation models & capabilities | Module 1 (done) and Module 2 (prompting) | | Tool use & MCP | Module 3 | | Agent architectures | Module 4 | | Design patterns | Module 6 | | Memory, RAG, vector stores | Module 7 | | Frameworks & SDKs | Module 8 | | Multimodal agents | Module 9 | | Multi-agent systems & A2A | Module 10 | | Evaluation & benchmarks | Module 11 | | Production & safety | Module 12 | | Advanced patterns | Module 13 |

Each module builds on the foundations you have established here. By the end, you will have worked with every layer of this ecosystem.

Summary

The agent ecosystem is a layered stack: foundation models provide intelligence, frameworks handle engineering complexity, protocols like MCP and A2A enable interoperability, infrastructure supports retrieval and observability, and benchmarks measure real-world effectiveness. No single tool does everything -- the art of building agents is assembling the right combination for your problem.

In the next lesson, you will move from maps to code. You will build your first agent from raw API calls, experiencing firsthand the agent loop you studied in Lesson 4 -- and you will appreciate why the frameworks and tools from this lesson exist.

The Agent Ecosystem