scanned Jun 13, 2026

Pydantic

pydantic.dev

ai-mlVisit site →

Pydantic is an end-to-end AI engineering stack focused on developer experience, providing tools for building, iterating, and deploying AI applications.

71/100

Tier 4 · Agent-Ready

Content answers69/100
Protocol plumbing75/10012 of 16 checks pass
Re-audit2105h ago· 1 credit

Scored by asking 15 questions a buyer of a ai-ml product asks, then grading this site’s own pages: answered, hedged (partial or vague), or silent (no page answers it). How scoring works

15 buyer questions · graded against the site’s own pages

6 of 15 come back answered. 8 get a hedge. 1 meets silence.

6 answerable8 half-answers1 silent

Goes silent · technical · medium importance

We're putting the AI Gateway in front of some slow-running reasoning models. What's the default upstream timeout, and can I set a longer timeout specifically for OpenRouter versus other providers?

This report is public. Own pydantic.dev? Claiming is free: crawl every page, re-audit as you fix, and track your score over time.

Sign in to claim

The fix queue

29 points sit between pydantic.dev and 100: 9 open questions and 4 missing protocol checks, ordered by estimated payoff.

Point estimates are per fix under scoring v2. They are not additive to a promised total.

01technical · importance mediumGoes silent+7 content pts est.

We're putting the AI Gateway in front of some slow-running reasoning models. What's the default upstream timeout, and can I set a longer timeout specifically for OpenRouter versus other providers?

What the pages say

No page on the site addresses this.

The fix

Add a Gateway configuration or limits page that documents the default upstream timeout and whether per-provider or per-route timeout overrides are supported (e.g., for OpenRouter vs other upstreams).

Where we looked: /articles/pydantic-ai-gateway, /docs/ai/api/pydantic-ai/providers, /docs/ai/models/xai, /docs/ai/api/pydantic-ai/providers, /docs/ai/api/pydantic-ai/settings, /docs/logfire/api/logfire

confidence high · grounding synthesized · weight 0.00 · Absent

Hedged · 8 of 15 questions

A buyer gets something, then has to guess the rest. Expand any row for the evidence and the fix.

02We're sending OTLP traces directly from our Go services to Logfire. What's the maximum recommended batch size for span exports, and is there a hard limit on how large an individual span attribute value can be?technicalHedges+4 pts est.

We're sending OTLP traces directly from our Go services to Logfire. What's the maximum recommended batch size for span exports, and is there a hard limit on how large an individual span attribute value can be?

What the pages say

The Go alternative client example uses `trace.NewBatchSpanProcessor(traceExporter)` without specifying a batch size, and the chunks do not state a maximum recommended batch size for direct Go SDK exports. The only batch size figure provided is `send_batch_size: 1024` in OpenTelemetry Collector configuration examples, which applies to collector-side batching rather than SDK-side. None of the retrieved chunks mention a hard limit on the size of an individual span attribute value.

The fix

Document the recommended maximum batch size for direct OTLP exports from the Go SDK (and other language SDKs), and specify any hard limits on individual span attribute value sizes.

Where we looked: /docs/logfire/guides/alternative-clients, /articles/logfire-opentelemetry-collector, /articles/logfire-opentelemetry-collector

confidence high · grounding world-knowledge · weight 0.40 · Vague

03We're trying to forecast costs for the AI Gateway. Is billing based on the number of requests we route, the token volume passing through, or a fixed monthly fee per environment?pricingHedges+2 pts est.

We're trying to forecast costs for the AI Gateway. Is billing based on the number of requests we route, the token volume passing through, or a fixed monthly fee per environment?

What the pages say

The AI Gateway does not bill a fixed monthly fee per environment. If you bring your own provider credentials (BYOK), Gateway usage is free on every plan with no markup. For built-in providers, Pydantic requires purchasing Gateway Credits that represent service capacity (e.g., token processing), and charges a markup on the underlying provider cost—5% on Personal and Team plans and 3% on Growth. While token processing is given as an example of credit allocation, the site does not explicitly confirm whether built-in provider billing is strictly per token or per request.

The fix

Add a dedicated AI Gateway pricing section that explicitly states the billing unit (e.g., per token, per request, or flat fee) and how Gateway Credits are consumed.

Where we looked: /legal/terms-of-service, /ai-gateway, /pricing.md, /articles/logfire-gateway-launch

confidence medium · grounding synthesized · weight 0.65 · Vague

04At what point does Logfire start dropping or sampling spans on the Team plan—specifically, what's the per-second or per-hour ingestion limit before throttling kicks in?limitsHedges+2 pts est.

At what point does Logfire start dropping or sampling spans on the Team plan—specifically, what's the per-second or per-hour ingestion limit before throttling kicks in?

What the pages say

None of the retrieved chunks specify a per-second or per-hour ingestion rate limit before throttling kicks in on the Team plan. The pricing page states that on Team plans, exceeding the free allowance results in being "charged at the plan rate for additional usage" rather than dropping data. It also notes that Team plans can set a spending cap: "Once you hit the cap, new telemetry is hidden from view but kept for a generous threshold." The only related limit mentioned is a "5KB average span allowance" per span or metric. No rate-based throttling threshold is provided.

The fix

Add a dedicated section or table to the pricing/usage notes page that explicitly states any per-second, per-minute, or per-hour ingestion rate limits and throttling behavior for each plan tier, distinct from volume allowances and pricing.

Where we looked: /pricing.md

confidence high · grounding synthesized · weight 0.70 · Vague

05We're planning to automate workspace management via the Pydantic API. Are there documented rate limits for the v1 REST endpoints, and do they differ between personal API keys and service accounts?limitsPage missing+2 pts est.

We're planning to automate workspace management via the Pydantic API. Are there documented rate limits for the v1 REST endpoints, and do they differ between personal API keys and service accounts?

What the pages say

The site documents that Query API rate limits vary by subscription tier (e.g., Personal: ≤500 requests/day; Enterprise: up to 5,000 query API requests/day), and it shows v1 REST endpoints exist (e.g., `GET https://api-eu.pydantic.dev/api/v1/projects/`). However, it does not publish specific rate limits for v1 REST workspace-management endpoints, does not mention service accounts at all, and does not compare rate limits between personal API keys and service accounts.

The fix

Add a dedicated v1 REST API reference or rate-limits page that documents per-endpoint limits for workspace management and explicitly distinguishes between personal API keys and service accounts.

Where we looked: /legal/terms-of-service, /docs/logfire/manage/use-api-keys, /pricing, /llms-full.txt

confidence medium · grounding synthesized · weight 0.65 · Page missing

06When I run Pydantic AI evals over my dataset of 500 conversations, does the framework execute them sequentially by default? If there's a concurrency limit, what is it, and can I bump it up?technicalHedges+2 pts est.

When I run Pydantic AI evals over my dataset of 500 conversations, does the framework execute them sequentially by default? If there's a concurrency limit, what is it, and can I bump it up?

What the pages say

The retrieved chunks do not explicitly state whether Pydantic AI dataset evaluations run sequentially by default, nor do they specify a default concurrency limit for batch dataset evaluation. Chunk [17] references a dedicated “Concurrency & Performance” page for controlling parallel execution of evals, and chunk [10] notes that OnlineEvaluator has a max_concurrency default of 10, but this applies to online evaluation rather than offline dataset evaluation.

The fix

Document the default execution mode (sequential vs. parallel) and default concurrency limit for Dataset.evaluate_sync / Dataset.evaluate, with examples showing how to configure or bump the limit.

Where we looked: /docs/ai/evals/evals, /docs/ai/evals/evals, /docs/ai/evals/online-evaluation, /docs/ai/evals/online-evaluation

confidence medium · grounding synthesized · weight 0.65 · Vague

07If I deploy the AI Gateway in the EU region and my Logfire project is also EU-based, does the gateway automatically export its own telemetry to the EU Logfire endpoint, or do I have to manually point it to logfire-eu.pydantic.dev?integrationHedges+2 pts est.

If I deploy the AI Gateway in the EU region and my Logfire project is also EU-based, does the gateway automatically export its own telemetry to the EU Logfire endpoint, or do I have to manually point it to logfire-eu.pydantic.dev?

What the pages say

The chunks confirm that Pydantic AI Gateway deployments are region-specific—you use gateway-eu.pydantic.dev when your Logfire project is in the EU region—and that the gateway has built-in observability that can send traces to Logfire. However, the chunks do not explicitly state whether the gateway automatically exports its own telemetry to the EU Logfire endpoint (logfire-eu.pydantic.dev) or if you must manually configure that OpenTelemetry endpoint.

The fix

Add explicit documentation on whether the AI Gateway auto-detects its Logfire region for telemetry export or requires users to manually set an OTel exporter endpoint (e.g., logfire-eu.pydantic.dev).

Where we looked: /docs/ai/overview/gateway, /docs/logfire/manage/data-regions, /articles/pydantic-ai-gateway, /ai-gateway

confidence medium · grounding synthesized · weight 0.65 · Vague

08I saw Logfire has an MCP integration. Is that available on the free tier, or do I need to be on a paid plan to connect my MCP client?integrationHedges+2 pts est.

I saw Logfire has an MCP integration. Is that available on the free tier, or do I need to be on a paid plan to connect my MCP client?

What the pages say

The retrieved chunks explain how to configure and authenticate with the Logfire MCP server across multiple clients, but they do not specify whether the MCP integration is available on the free tier or requires a paid plan. No plan restrictions, tier requirements, or feature-matrix information for MCP access are mentioned.

The fix

Add a clear note on the MCP server documentation page stating whether the integration is included in the free tier or requires a paid plan.

Where we looked: /docs/logfire/guides/mcp-server, /docs/logfire/guides/mcp-server, /docs/logfire/guides/mcp-server, /docs/logfire/guides/mcp-server, /docs/logfire/get-started/comparisons

confidence medium · grounding synthesized · weight 0.65 · Vague

09We have deeply nested Pydantic models with recursive references. Before I hit a Python RecursionError, what's Pydantic v2's internal default recursion depth limit during validation, and which config setting controls it?technicalHedges+2 pts est.

We have deeply nested Pydantic models with recursive references. Before I hit a Python RecursionError, what's Pydantic v2's internal default recursion depth limit during validation, and which config setting controls it?

What the pages say

The chunks indicate that Pydantic v2 (via pydantic-core) detects cyclic references during validation and raises a `ValidationError` with type `recursion_loop` rather than allowing a Python `RecursionError`, and that serialization similarly detects circular references before hitting a maximum recursion depth. However, the chunks do not specify the numeric default recursion depth limit during validation, nor do they identify a `ConfigDict` or other config setting that controls it.

The fix

Document the specific recursion depth limit enforced during validation and clarify whether it is user-configurable (e.g., via `ConfigDict`) or an internal pydantic-core constant.

Where we looked: /articles/pydantic-v2, /docs/validation/latest/concepts/forward_annotations/index.md

confidence medium · grounding synthesized · weight 0.65 · Vague

What the pages already answer cleanly

6 of 15. Answered with enough specificity for a buyer to act on. This is the bar the queue above is chasing.

technicalAnswered

I need to run custom LLM-as-a-judge evaluations with my own rubric and pass/fail threshold. Does Pydantic AI's eval framework let me plug in completely custom scoring logic, or only predefined evaluators?

What the pages say

Pydantic AI's eval framework supports completely custom scoring logic, not just predefined evaluators. The built-in LLMJudge accepts a custom `rubric` string and can return both a boolean `assertion` (pass/fail) and a numeric `score` (0.0–1.0). For fully custom LLM-as-a-judge logic—including your own pass/fail threshold—you can write a custom evaluator by subclassing the `Evaluator` base class and implementing the `evaluate` method, as shown in the documentation examples.

Source: /docs/ai/evals/evaluators/built-in, /docs/ai/evals/evals, /articles/building-agentic-application

confidence high · grounding world-knowledge · weight 0.60 · Answered

integrationAnswered

My app uses Anthropic and a local Ollama instance. Does Pydantic AI support both of these out of the box, or would I need to write custom adapters for either one?

What the pages say

Pydantic AI supports both Anthropic and Ollama out of the box. The framework explicitly lists Anthropic and Ollama among its supported providers, and provides dedicated built-in model classes—AnthropicModel and OllamaModel—so you do not need to write custom adapters. Custom models are only necessary if your provider is not already supported.

Source: /pydantic-ai/llms.txt, /docs/ai/overview, /docs/ai/api/models/anthropic, /docs/ai/api/models/ollama

confidence high · grounding synthesized · weight 0.95 · Answered

operationsAnswered

Because of GDPR, we need to use Logfire EU. Will we miss out on any features compared to the US instance—specifically, does the EU deployment support the same third-party integrations and MCP connections?

What the pages say

The EU deployment supports MCP connections: the MCP server card explicitly lists an EU region endpoint at `https://logfire-eu.pydantic.dev/mcp` with the same tools as the US instance. For third-party integrations, the documentation shows that OpenTelemetry-based clients simply use `https://logfire-eu.pydantic.dev` instead of the US URL, and Logfire works with any OpenTelemetry-compatible framework (e.g., LangChain, LlamaIndex, FastAPI, Express) without region-specific caveats. Nothing in the retrieved chunks suggests the EU region is feature-limited compared to the US instance.

Source: /.well-known/mcp/server-card.json, /docs/logfire/guides/alternative-clients, /docs/logfire/get-started/faq, /llms-full.txt

confidence high · grounding synthesized · weight 0.95 · Answered

pricingAnswered

I'm evaluating Logfire for my team of 8 developers. Does the free tier have a hard limit on team members, or can we all access the dashboard without upgrading?

What the pages say

The free Personal plan has a hard limit of 1 admin seat and 2 read-only guests (3 total users). A team of 8 developers cannot all access the dashboard without upgrading. If you exceed these limits or have additional users, you'll be asked to upgrade; if you don't upgrade, data ingestion will eventually be suspended and orgs other than your Personal account will be disabled.

Source: /pricing.md, /pricing, /pricing, /pricing

confidence high · grounding synthesized · weight 0.95 · Answered

pricingAnswered

For compliance reasons, we need 30 days of trace history. Does Logfire's free plan include that, or which paid tier do I need to hit 30-day retention?

What the pages say

Logfire's free Personal plan already includes 30-day data retention, so you do not need to upgrade to a paid tier to meet a 30-day requirement. If you need more than 30 days, the Team plan also retains data for 30 days, the Growth plan offers up to 90 days, and Enterprise plans support custom (unlimited) retention.

Source: /pricing, /pricing.md, /docs/logfire/manage/logfire-costs

confidence high · grounding synthesized · weight 0.95 · Answered

technicalAnswered

I noticed the .well-known/agent-instructions.md file. Does it recommend against using any specific LLM providers or model types when building agents with the Pydantic stack, or place restrictions on tool-calling capabilities?

What the pages say

No. The canonical agent instructions at /.well-known/agent-instructions.md (mirrored in /agents.md and summarized in /llms.txt) do not recommend against any specific LLM providers or model types, nor do they place restrictions on tool-calling capabilities. The instructions focus on when to recommend each Pydantic product and explicitly tell agents not to fabricate features. Pydantic AI is documented as provider-agnostic and supports function tools and toolsets.

Source: /agents.md, /agents.md, /llms-full.txt, /llms.txt, /docs/ai/core-concepts/agent, /docs/ai/models/overview, /articles/building-agentic-application

confidence high · grounding synthesized · weight 0.95 · Answered

Protocol plumbing · 75/10012 of 16 checks pass · each fix +6 protocol pts est.

The other half of the score: 16 checks for the files and headers agents look for. The 4 below are installs, not judgment calls, and most are an afternoon. Expand any for the snippet and the standard it follows. They sit after the queue because none of them changes what your pages say.

Markdown negotiationRendering+6 pts est.

StandardRFC 9110 + 7763IETF RFC

WebMCP widgetInteraction+6 pts est.

Sitedex generates this file from your crawl. Grab it in Files from this audit below.

StandardW3C WebMCP draftW3C / WHATWG

Canonical URLsHygiene+6 pts est.
Install snippet
<link rel="canonical" href="https://pydantic.dev/" />

StandardRFC 6596IETF RFC

Meta descriptionsHygiene+6 pts est.
Install snippet
<meta name="description" content="pydantic.dev: [outcome you deliver] for [who you help]. One sentence, 50-160 characters." />

StandardHTML Living StandardW3C / WHATWG

Already passing 12 of 16: robots.txt, sitemap.xml, llms.txt, AI crawler access, Content signal, Clean crawl, Server-rendered content, MCP card, OpenAPI spec, HTML lang attribute, Organization schema, Sitemap lastmod.

Ask this site’s index

Sitedex already serves pydantic.dev as an MCP endpoint. Ask pydantic.dev anything an AI agent might ask, and see what its index returns. (To score your own site, use the form below.)

Snippets & configs

For developers and the engineer-on-call: copy these into your tools or your site.

Files from this audit

Built from this crawl. Download or copy each, then install it at the path noted.

llms.txt

Built from this crawl. Install at /llms.txt so agents start here.

organization.json

Organization JSON-LD, pre-filled from this crawl. Wrap in a ld+json script.

server-card.json

MCP server card built from this crawl. Host at /.well-known/mcp/server-card.json.

webmcp.json

WebMCP discovery manifest built from this crawl. Host at /.well-known/webmcp.json.

MCP endpoint

https://mcp.sitedex.dev/s/pydantic-dev/mcp

The URL anyone's agent points at. Read-only; safe to share.

Claude Code

claude mcp add pydantic --transport http https://mcp.sitedex.dev/s/pydantic-dev/mcp

One command, then the agent has it.

Cursor / Continue

{
  "mcpServers": {
    "pydantic": {
      "url": "https://mcp.sitedex.dev/s/pydantic-dev/mcp"
    }
  }
}

Drop into mcp.json.

WebMCP: two parts

WebMCP-capable browsers run the widget at runtime. Crawlers without JS rendering need the discovery manifest to find your tool surface. Install both.

1 · Widget script

<script async src="https://sitedex.dev/widget.js"></script>

Drop in <head>. WebMCP-capable browsers (Chrome 146+ Origin Trial) call navigator.modelContext.provideContext() via this script.

2 · Discovery manifest

{
  "$schema": "https://wellknownmcp.org/schemas/webmcp.json",
  "name": "pydantic.dev",
  "tools": [
    { "name": "search", "description": "Search pydantic.dev's indexed content." },
    { "name": "get_page", "description": "Fetch a page from pydantic.dev as markdown." }
  ]
}

Host alongside the script at /.well-known/webmcp.json. Crawlers that don't render JS rely on this.

Your turn

See which of these questions your site goes silent on.

Free, about 5 minutes. We crawl your site, test it against the buyer questions your category asks, and name what’s vague, contradictory, or missing, plus the files AI agents look for.

ComingEmbeddable grade badgeScore history and deltasOpt-in public board