web_to_markdown

click.ciromaciel/tools · Verify this server

Fetches a public web page and returns its content as clean Markdown. WHEN TO USE: you need to read a page — documentation, an article, a product page — to answer, summarize or quote it; or the user wants a page as Markdown. HOW IT WORKS: - url: a public http(s) address. Private networks, localhost and addresses with credentials are refused. - Reads the HTML the server sends (like a simple browser, without running JavaScript). Pages that build their content with JavaScript have no text to convert — the error says so. - onlyMainContent (default true) drops menus, headers, footers, sidebars, cookie banners and ads. Set false to keep the whole page. - pages above 1 also follows links on the same site (same folder and below), breadth-first, up to that many pages (max 20), respecting robots.txt. Slower: a few seconds per page. - Limits per page: 15 seconds and 5 MB. PDFs and non-HTML files are not converted. RETURNS: for each page, its Markdown with absolute links and images, plus title, description and language when the page has them; pages that failed are listed with the reason. link: this tool's web page with the same request. ERRORS say why the page could not be read and whether retrying makes sense.