HTTPS surface reachable (robots ✓, sitemap ✓, title ✓)
Why it matters: Public files — robots.txt, sitemap.xml, head meta — are what attackers see first during reconnaissance. Misadvertised paths, stale sitemaps, and verbose generators leak more than intended (ISO 27001 A.8.9).
robots.txt
present
User-Agent: *
Allow: /wp-content/uploads/
Allow: *css
Allow: *js
Disallow: /tags/
Disallow: /page/
Disallow: /animation-test/
Disallow: /categories/
Disallow: /employees/
Disallow: /footer/
Disallow: /blog/wp-login.php
Disallow: /blog/wp-register.php
Disallow: /wp-includes/
Disallow: /wp-content/plugins/
Disallow: /wp-content/themes/
Disallow: /wp-admin/
Disallow: /pdf/*
Disallow: /slides/*
Disallow: /wp-login.php
Disallow: /wp-trackback.php
Disallow: /xmlrpc.php
Disallow: /*&utm_source
Disallow: /*&utm_medium
Disallow: /*&utm_campaign
Disallow: /*?utm_
Disallow: /*thank-you
Disallow: /free-tier/*
Disallow: /search/
Disallow: /docs/search*
User-Agent: GPTBot
Allow: /
Disallow: /tags/
Disallow: /page/
Disallow: /animation-test/
Disallow: /categories/
Disallow: /employees/
Disallow: /footer/
Disallow: /blog/wp-login.php
Disallow: /blog/wp-register.php
Disallow: /wp-includes/
Disallow: /wp-content/plugins/
Disallow: /wp-content/themes/
Disallow: /wp-admin/
Disallow: /pdf/*
Disallow: /slides/*
Disallow: /wp-login.php
Disallow: /wp-trackback.php
Disallow: /xmlrpc.php
Disallow: /*&utm_source
Disallow: /*&utm_medium
Disallow: /*&utm_campaign
Disallow: /*?utm_
Disallow: /*thank-you
Disallow: /free-tier/*
Disallow: /search/
Disallow: /docs/search*
User-Agent: PerplexityBot
Allow: /
Disallow: /tags/
Disallow: /page/
Disallow: /animation-test/
Disallow: /categories/
Disallow: /employees/
Disallow: /footer/
Disallow: /blog/wp-login.php
Disallow: /blog/wp-register.php
Disallow: /wp-includes/
Disallow: /wp-content/plugins/
Disallow: /wp-content/themes/
Disallow: /wp-admin/
Disallow: /pdf/*
Disallow: /slides/*
Disallow: /wp-login.php
Disallow: /wp-trackback.php
Disallow: /xmlrpc.php
Disallow: /*&utm_source
Disallow: /*&utm_medium
Disallow: /*&utm_campaign
Disallow: /*?utm_
Disallow: /*thank-you
Disallow: /free-tier/*
Disallow: /search/
Disallow: /docs/search*
User-Agent: ClaudeBot
Allow: /
Disallow: /tags/
Disallow: /page/
Disallow: /animation-test/
Disallow: /categories/
Disallow: /employees/
Disallow: /footer/
Disallow: /blog/wp-login.php
Disallow: /blog/wp-register.php
Disallow: /wp-includes/
Disallow: /wp-content/plugins/
Disallow: /wp-content/themes/
Disallow: /wp-admin/
Disallow: /pdf/*
Disallow: /slides/*
Disallow: /wp-login.php
Disallow: /wp-trackback.php
Disallow: /xmlrpc.php
Disallow: /*&utm_source
Disallow: /*&utm_medium
Disallow: /*&utm_campaign
Disallow: /*?utm_
Disallow: /*thank-you
Disallow: /free-tier/*
Disallow: /search/
Disallow: /docs/search*
User-Agent: anthropic-ai
Allow: /
Disallow: /tags/
Disallow: /page/
Disallow: /animation-test/
Disallow: /categories/
Disallow: /employees/
Disallow: /footer/
Disallow: /blog/wp-login.php
Disallow: /blog/wp-register.php
Disallow: /wp-includes/
Disallow: /wp-content/plugins/
Disallow: /wp-content/themes/
Disallow: /wp-admin/
Disallow: /pdf/*
Disallow: /slides/*
Disallow: /wp-login.php
Disallow: /wp-trackback.php
Disallow: /xmlrpc.php
Disallow: /*&utm_source
Disallow: /*&utm_medium
Disallow: /*&utm_campaign
Disallow: /*?utm_
Disallow: /*thank-you
Disallow: /free-tier/*
Disallow: /search/
Disallow: /docs/search*
User-Agent: Google-Extended
Allow: /
Disallow: /tags/
Disallow: /page/
Disallow: /animation-test/
Disallow: /categories/
Disallow: /employees/
Disallow: /footer/
Disallow: /blog/wp-login.php
Disallow: /blog/wp-register.php
Disallow: /wp-includes/
Disallow: /wp-content/plugins/
Disallow: /wp-content/themes/
Disallow: /wp-admin/
Disallow: /pdf/*
Disallow: /slides/*
Disallow: /wp-login.php
Disallow: /wp-trackback.php
Disallow: /xmlrpc.php
Disallow: /*&utm_source
Disallow: /*&utm_medium
Disallow: /*&utm_campaign
Disallow: /*?utm_
Disallow: /*thank-you
Disallow: /free-tier/*
Disallow: /search/
Disallow: /docs/search*
User-Agent: bingbot
Allow: /
Disallow: /tags/
Disallow: /page/
Disallow: /animation-test/
Disallow: /categories/
Disallow: /employees/
Disallow: /footer/
Disallow: /blog/wp-login.php
Disallow: /blog/wp-register.php
Disallow: /wp-includes/
Disallow: /wp-content/plugins/
Disallow: /wp-content/themes/
Disallow: /wp-admin/
Disallow: /pdf/*
Disallow: /slides/*
Disa
sitemap.xml
present — 348 url(s)
head
- title
- CockroachDB | Distributed SQL for always-on customer experiences
- description
- CockroachDB is a distributed database with standard SQL for cloud applications. CockroachDB powers companies like Comcast, Lush, and Bose.
social
- og:title
- CockroachDB | Distributed SQL for always-on customer experiences
- og:description
- CockroachDB is a distributed database with standard SQL for cloud applications. CockroachDB powers companies like Comcast, Lush, and Bose.
- og:url
- https://www.cockroachlabs.com/homepage/
- og:image
- https://images.ctfassets.net/00voh0j35590/3HOcRkIRnmu4DIWirX6gDH/b4f8a962358bffa8da063319af5dbba0/crl-socialpost-default-2020-2.jpg
- og:image:alt
- crl-socialpost-default-2020-2
- twitter:card
- summary_large_image
- twitter:title
- CockroachDB | Distributed SQL for always-on customer experiences
- twitter:description
- CockroachDB is a distributed database with standard SQL for cloud applications. CockroachDB powers companies like Comcast, Lush, and Bose.
- twitter:image
- https://images.ctfassets.net/00voh0j35590/3HOcRkIRnmu4DIWirX6gDH/b4f8a962358bffa8da063319af5dbba0/crl-socialpost-default-2020-2.jpg
- twitter:image:alt
- crl-socialpost-default-2020-2