HTTPS surface reachable (robots ✓, sitemap ✓, title ✓)
Why it matters: Public files — robots.txt, sitemap.xml, head meta — are what attackers see first during reconnaissance. Misadvertised paths, stale sitemaps, and verbose generators leak more than intended (ISO 27001 A.8.9).
robots.txt
present
# Allowlist your bot: https://help.pinterest.com/bot-submission-form
#
# Pinterest is hiring!
#
# Learn about the SEO work that we're doing at https://medium.com/@Pinterest_Engineering/demystifying-seo-with-experiments-a183b325cf4c
#
# Check out some of our available positions at https://www.pinterestcareers.com/
User-agent: 2checkout
User-agent: AASA-Bot
User-agent: AccessibleWebBot
User-agent: Adagiobot
User-agent: adbeat_bot
User-agent: ADmantX
User-agent: AdometryBot
User-agent: Adsense-Snapshot-Google
User-agent: AdsTxtCrawler
User-agent: Adyen
User-agent: AhrefsBot
User-agent: AhrefsSiteAudit
User-agent: Akismet
User-agent: Algolia Crawler
User-agent: AmazonProductDiscovery
User-agent: Aolbot-News
User-agent: APIs-Google
User-agent: AppInsights
User-agent: Applebot
User-agent: AppleNewsBot
User-agent: Archive-It
User-agent: archive.org_bot
User-agent: ArchiveBot
User-agent: ArchiveTeam-ArchiveBot
User-agent: AspiegelBot
User-agent: Audisto Crawler
User-agent: Awario
User-agent: Baidu-ADS
User-agent: Baiduspider
User-agent: Barkrowler
User-agent: bingbot
User-agent: BingPreview
User-agent: bitlybot
User-agent: bl.uk_ldfc_bot
User-agent: BLP_bbot
User-agent: bordermanager
User-agent: botify
User-agent: BrandVerity
User-agent: Bravebot
User-agent: BrightEdge Crawler
User-agent: BrightEdge-Crawler
User-agent: BUbiNG
User-agent: CaliperBot
User-agent: CapitalOneBot
User-agent: Catchpoint
User-agent: cf-qualys-scanner
User-agent: Chrome-Lighthouse
User-agent: CJBot
User-agent: ClarityBot
User-agent: Clickagy Intelligence Bot v2
User-agent: Cloudflare-AI-Search
User-agent: CloudflareBrowserRenderingCrawler
User-agent: coccocbot
User-agent: ContextualBot
User-agent: CookieReports
User-agent: Coveobot
User-agent: CriteoBot
User-agent: curly-
User-agent: cXensebot
User-agent: Datadog
User-agent: DataForSeoBot
User-agent: Daum
User-agent: demandbase
User-agent: Discordbot
User-agent: DotBot
User-agent: DoubleVerifyBot
User-agent: Doximity-Pipeline
User-agent: Dragonbot
User-agent: Dratabot
User-agent: DuckAssistBot
User-agent: DuckDuckBot
User-agent: DuckDuckGo-Favicons-Bot
User-agent: DVBot
User-agent: eContext
User-agent: ecurly Webhook
User-agent: EngageBDR
User-agent: europarchive.org
User-agent: evidon
User-agent: ExaleadCloudview
User-agent: FacebookBot
User-agent: facebookcatalog
User-agent: facebookexternalhit
User-agent: Facebot
User-agent: FastDAST
User-agent: FeedBurner
User-agent: Feedfetcher-Google
User-agent: FlipboardProxy
User-agent: FlippBot
User-agent: Freshping
User-agent: GeedoProductSearch
User-agent: Google Favicon
User-agent: Google News
User-agent: Google Page Speed Insights
User-agent: Google Wireless Transcoder
User-agent: Google-Apps-Script
User-agent: Google-Certificates-Bridge
User-agent: Google-CWS
User-agent: Google-Display-Ads-Bot
User-agent: Google-Firebase
User-agent: Google-HotelAdsVerifier
User-agent: Google-InspectionTool
User-agent: Google-NotebookLM
User-agent: Google-PageRenderer
User-agent: Google-PhysicalWeb
User-agent: Google-Pinpoint
User-agent: Google-Read-Aloud
User-agent: Google-Safety
User-agent: Google-Site-Verification
User-agent: Google-speakr
User-agent: Google-Structured-Data-Testing
User-agent: Google-Trust-Services
User-agent: google_partner_monitoring
User-agent: GoogleAssociationService
User-agent: GoogleImageProxy
User-agent: GoogleMessages
User-agent: GoogleProducer
User-agent: GoogleSecurityScanner
User-agent: grafana/synthetic-monitoring
User-agent: GroovinaAdsbot
User-agent: GuestpostsBot
User-agent: heritrix
User-agent: Hotjar
User-agent: HubSpot
User-agent: ia_archiver
User-agent: IABot
User-agent: IAS-Crawler
User-agent: ias_crawler
User-agent: ias_wombles
User-agent: INA dlweb
User-agent: infolink
User-agent: InteractiveAdvertisingBureau
User-agent: ip-label
User-agent: J-BRW
User-agent: J-DLC
User-agent: James BOT
User-agent: JobRapido
User-agent: Jooblebot
User-agent: Jumio
User-agent: kakaotalk-scrap
User-agent: KargoBot-Artemis
User-agent: Leikibot
User-agent: LiftoffDroid
User-agent: line-poker
User-agent: Linguee Bot
User-agent: linkchecker.pro
User-agen
sitemap.xml
present — 0 url(s)
head
- title
- Pinterest
- description
- Discover recipes, home ideas, style inspiration and other ideas to try.
social
no OpenGraph or Twitter meta tags found