fetch_page_metadata

io.github.razikallayi/razi-tools · Verify this server

Read a web page's metadata without downloading any images. Returns JSON { url, finalUrl, title, description, siteName, canonical, lang, themeColor, author, generator, feeds, openGraph, twitter, icons, cached }, where `feeds` is an array of RSS/Atom URLs, `openGraph` and `twitter` are the complete tag sets as string maps, and any tag the page does not declare is simply omitted. `icons` is always an empty array here because icon resolution is switched off — call fetch_site_logo when the caller wants a logo, favicon or icon. `finalUrl` is the address after redirects (up to 3 hops are followed), which is how you resolve where a domain actually points. This is the fast, cheap counterpart to fetch_site_logo: one page fetch and no image work. It reads the served HTML only — it runs no JavaScript, so a client-rendered page may expose little, and it does not capture how the page looks (use screenshot_url for that). The crawler identifies itself as RaziMetadataBot, and HTTP status is not checked, so a 404 or bot-challenge page that serves HTML is parsed as though it were the page you asked for. Errors: 400 for a URL that is refused as unsafe, redirects too many times, does not return HTML, or exceeds the 5MB page cap; 504 if the page does not answer within 10 seconds; 502 otherwise. Results are cached for 24 hours and replayed with cached: true; 20 calls per minute per IP, then 429.