HTTPS surface reachable (robots ✓, sitemap ✗, title ✓)
Why it matters: Public files — robots.txt, sitemap.xml, head meta — are what attackers see first during reconnaissance. Misadvertised paths, stale sitemaps, and verbose generators leak more than intended (ISO 27001 A.8.9).
robots.txt
present
# robots.txt for https://abcnews.com/
#
Sitemap: https://abcnews.com/xmap
Sitemap: https://abcnews.com/xmlLatestStories
Sitemap: https://abcnews.com/xmlLatestVideos
User-agent: claritybot
Disallow: /
User-agent: GPTBot
Disallow: /
User-agent: Google-Extended
Disallow: /
User-agent: CCBot
Disallow: /
User-agent: ChatGPT-User
Disallow: /
User-agent: anthropic-ai
Disallow: /
User-agent: Omgilibot
Disallow: /
User-agent: Omgili
Disallow: /
User-agent: FacebookBot
Disallow: /
User-agent: Bytespider
Disallow: /
User-agent: *
Disallow: /search?searchtext=*
Disallow: /disneyid/*
Disallow: /assets/static/ads/*
Disallow: /cgi
Disallow: /xls
Disallow: /imp
Disallow: /kmail
Disallow: /map
Disallow: /log
Disallow: /gif
Disallow: /panel
Disallow: /0/
Disallow: /promo/
Disallow: /abclinks/
Disallow: /houseads/
# Disallow: /xmldata/
Allow: /xmldata/mrss
Allow: /xmldata/rss
Allow: /xmldata/xmlPodcast
Allow: /xmldata/config
Allow: /xmldata/feed
Disallow: /sendtofriend/
Allow: /meta/sitemap
Disallow: /meta/
Disallow: /staging/
Disallow: /test/
Disallow: /swen/
Disallow: /intro/
Disallow: /go/
Disallow: /news/go/
Disallow: /widgets/
Disallow: /vp2/
Disallow: /Video/*playerIndex
Disallow: /*carousel/
Disallow: /*videoLogin?
Disallow: /video/browse/
Disallow: /*popup?
Disallow: /alerts-v1/
Disallow: /not-allowed/
Disallow: /beta-story-container/*
Disallow: /video/embed/*
Disallow: /video/amp/embed/*
Disallow: /responder
Disallow: /video/itemfeed*
Disallow: /video/caption*
Disallow: /content/news/liveBlog/*
Disallow: /configuration/*
Disallow: /rsidxfeed/*
Disallow: /xmldata/*
Disallow: /gma/api/*
Disallow: /proxy/*
User-agent: Applebot
Allow: /
Disallow: /private/
head
- title
- ABC News - Breaking News, Latest News and Videos
- description
- Your trusted source for breaking news, analysis, exclusive interviews, headlines, and videos at ABCNews.com
social
- og:locale
- en_US
- og:site_name
- ABC News
- og:url
- https://abcnews.com
- og:title
- ABC News - Breaking News, Latest News and Videos
- og:description
- Your trusted source for breaking news, analysis, exclusive interviews, headlines, and videos at ABCNews.com
- og:image
- https://s.abcnews.com/images/US/abc_news_default_2000x2000_update_16x9_992.jpg
- og:type
- website
- twitter:app:name:iphone
- ABC News
- twitter:app:id:iphone
- 300255638
- twitter:app:url:iphone
- abcnewsiphone://link/home,HomePage
- twitter:app:name:ipad
- ABC News
- twitter:app:id:ipad
- 306934135
- twitter:app:url:ipad
- abcnewsipad://link/home,HomePage
- twitter:app:name:googleplay
- ABC News
- twitter:app:id:googleplay
- com.abc.abcnews
- twitter:app:url:googleplay
- https://abcnews.com
