Sitedex

I'm building a batch processing pipeline — what are the specific rate limit headers your API returns so I can implement proper backoff without hitting hard limits?

Every rate-limited response carries four standard headers: `X-RateLimit-Limit` (total requests allowed in the current window), `X-RateLimit-Remaining` (requests remaining, clamped at 0), `X-RateLimit-Reset` (Unix timestamp in seconds when the window resets), and `Retry-After` (seconds to wait before retrying, RFC 6585 §4, set only on 429 responses). For backoff, the docs recommend preferring `Retry-After` on a 429 if your client supports it; otherwise compute the wait from `X-RateLimit-Reset`. For sustained throughput, watch `X-RateLimit-Remaining` on every 2xx and slow down before the cap.

Found on /openapi.json › info › Rate limits

Before I commit to a paid plan, what can I actually do on the free tier — is it limited to one audit, read-only access to public site data, or something else entirely?

Something else entirely. The free tier is not limited to one audit, nor is it read-only. Without paying you can: search every indexed site, read any page as markdown, view full diagnoses (every score and fix) for any site, use the hosted MCP, REST API, and CLI, and index new sites without even creating an account. The first crawl of any site is free and unlimited—'the first crawl of any site is on us.' The single credit you receive on signup is specifically for re-crawls ('your first re-crawl is on us'). Free crawls are rate-limited (10 audits per hour and 30 per day per IP) and stop at 500 pages per crawl.

Found on /pricing › Most of it’s free. › Free with an account

Your MCP endpoint is hosted — if you update the protocol version, do you maintain backward compatibility or will my agent integration break without warning?

The hosted MCP endpoint is backward compatible. Sitedex adds capabilities without breaking the ones you already use, and they won't remove or rename a tool out from under you, so an integration you ship today keeps working as they evolve.

Found on /docs › Documentation › Hosted MCP › Versioning & stability

If I buy the Starter credit pack, how many site audits does that actually translate to — is it 1 credit per audit or does the scoring depth affect the cost?

The Starter pack costs $50 and includes 15 credits, which translates to 15 site audits. Each audit costs exactly 1 credit, and scoring depth does not affect the cost. The site states that “same product everyone gets” and “Bigger packs cost less per audit. That’s the only difference,” meaning the only variable is the per-credit price based on pack size.

Found on /llms.txt › Sitedex › Pricing

Can I create API keys with restricted scopes — like read-only access to audit results without the ability to trigger new crawls or manage billing?

No. Sitedex's documentation explicitly states that read-only and scope-limited keys are not available yet: "A key carries your organization's full access — the same as signing in. Read-only and scope-limited keys aren't available yet, so treat every `sdx_…` key like a password: keep it server-side, and rotate it if it leaks."

Found on /docs › Documentation › REST API › Authentication

When you show me where my site ranks against competitors, are you comparing against all indexed sites or only those in my selected category — and how do you handle sites with no declared category?

Indexed sites appear on a global leaderboard — top sites by score — and on per-category boards for any category with enough indexed sites to compare fairly. If your site doesn't fall into a clear category yet, it still ranks globally; it just won't show in a category board until there's a cohort to rank it against.

Found on /docs › Documentation › How scoring works › Where you rank

I need to track how my site's AI-readiness changes weekly — is there a hard limit on how often I can re-audit the same domain, or can I schedule automatic recrawls?

There is no hard limit or cooldown on re-auditing the same domain—each re-audit costs one credit. Sitedex does not auto-schedule re-crawls itself, but the API lets you wire up automatic recrawls on whatever cadence you like, including weekly.

Found on / › Buyers ask AI about you now.Your site sends it to dead ends. › Common questions › Where's the documentation?

You grade sites on AI-readiness, but what exactly weights the score — is it just schema markup coverage, or do you factor in actual LLM retrieval success rates from your agent testing?

The AI-readiness score is weighted 70% to content and 30% to protocol, so it is not just schema markup coverage. The content score (70%) comes from 15 category-specific buyer questions that a real LLM answers from the site’s indexed content; a judge grades each answer on a diagnosis taxonomy (answered, vague claim, collapsed content, missing page, contradictory pages, genuinely absent). This functions as actual LLM retrieval-success testing. The protocol score (30%) checks 14 technical signals—including `llms.txt`, sitemap, robots-for-AI-crawlers, MCP card, WebMCP, and JSON-LD—that determine whether agents can read the site at all. The composite formula is `0.7 × content + 0.3 × protocol`, capped so protocol cannot more than double a weak content score.

Found on /llms.txt › Sitedex › How it works

My site is a single-page application — does your crawler execute JavaScript to index dynamically rendered content, or do I need to pre-render for Sitedex to see my full content?

Yes. Sitedex renders every page in a real browser before reading it, so JavaScript-heavy sites and SPAs get indexed the way a visitor sees them, “not as an empty shell.” No pre-rendering is required: “Nothing to pre-render, nothing to change.”

Found on / › Buyers ask AI about you now.Your site sends it to dead ends. › Common questions › I run a single-page app — can Sitedex actually see my content?

For the SEO optimization scoring, do you require specific schema.org types like Organization and WebSite, or do you also evaluate custom JSON-LD that might be more relevant to my industry?

Sitedex requires a valid Organization JSON-LD block with a name on your homepage. It does not require WebSite or industry-specific schema types, and extra custom JSON-LD does not move the score.

Found on /docs › Documentation › How scoring works › Where you rank

robots.txt

sitemap.xml

llms.txt

AI crawler access

Content signal

Pages load cleanly

Server-rendered text

MCP server card

OpenAPI spec

WebMCP

Canonical URL

Meta description

Page language

Organization schema

Sitemap dates