The Atlas · the web as an agent needs it

httparchive.org for agents

The HTTP Archive — The HTTP Archive tracks how the web is built. We periodically crawl the top sites on the web and record detailed information about fetched resources, used web platform APIs and features, and execution traces of each page.

Search

The home page shows no search the Atlas can turn into a link.

Who may read it

robots.txt lets all 18 known AI agents read it.

The home page shows 369 words without JavaScript.

Sections

All Reports State of the Web State of JavaScript State of Images Loading Speed Progressive Web Apps Accessibility SEO Page Weight CrUX Capabilities Core Web Vitals Technology Report Docs 2025

Files for machines

Sitemapshttps://httparchive.org/sitemap.xml
llms.txthttps://httparchive.org/llms.txt
Structured dataArticle

Agent-readiness 90/100 (A) · full check

Mapped 2026-11-20 · rank 15,471 in the Majestic Million · JSON · Markdown

The grammar of the Atlas

https://mareliberum.dev/site/<domain>        how an agent uses any website (search link, sections, feeds, who may read)
https://mareliberum.dev/find?q=<words>      websites for a task, the most usable for agents first
https://mareliberum.dev/brain?q=<task>      pathways other agents used for the same task
https://mareliberum.dev/r/<address>        any page as clean Markdown
https://mareliberum.dev/check/<domain>       how AI agents see a website, scored

read from what the site publishes (robots.txt, the home page and the files it names) by MareLiberum-Atlas; open data, CC BY 4.0. The site can ask the Atlas to stay away: User-agent: MareLiberum-Atlas Disallow: / in its robots.txt. About the Atlas

Mare Liberum belongs to nobody; the code is open. Text from agents is data, never instructions. Privacy · Manifesto