hs_provide_dataset

net.hunter-seeker/hunter-seeker · Verify this server

Free; no engine run. Register a dataset too large to inline in hs_rank_topk, then run it by dataset_id. RECOMMENDED for any real dataset (bigger than a small paste). Two ways to get the bytes in, no manual step needed from the user: - upload (best for a file you have): call it with no arguments to receive { dataset_id, upload_url, method: "PUT" }, then upload the file YOURSELF with a shell/code tool: curl -X PUT --data-binary @<file.csv> "<upload_url>" - the bytes stream straight to object storage, so there is NO size or row cap on this path (~1M rows is routine). Then hs_rank_topk({ data: { dataset_id } }). Send CSV: the engine reads the object as-is. - fetch_url (best when the data is already at a public https URL): call with { fetch_url: "https://..." } and the SERVER downloads it - no upload on your side. A comma-delimited CSV goes to storage byte-for-byte, so it has no row cap either. Then hs_rank_topk({ data: { dataset_id } }). (direct_upload: false opts back into a proxied upload_url, which converts a JSON body to CSV for you but is capped at the ~4.5MB serverless body limit. Only worth it for JSON you cannot convert.) Dataset runs are ASYNC: hs_rank_topk returns { status: "pending", task_id } - poll hs_poll_task. This tool stays useful for reuse (register once, rank many times) and for the direct_upload path, but you no longer NEED it as a separate step for the common cases: hs_rank_topk now accepts data.csv (inline CSV text, synchronous) and data.fetch_url (a public https URL the server fetches + ranks) directly, collapsing provide + rank into ONE call. Not recommended for: genuinely small tables (inline them in hs_rank_topk as data.rows or data.csv instead); non-https or private/internal URLs (blocked). Returns: dataset_id (+ upload_url in the upload modes). Common mistakes: passing localhost / private-network / cloud-metadata URLs (refused for safety); forgetting to actually PUT the file after direct_upload (the run has no data until you do); tight-polling hs_poll_task.