catalog_search

md.mostlyright/datasets · Verify this server

Search a sealed snapshot of public data sources for feeds that might answer a question — the first move when you need a source and do not already know one. Example: {"question": "county unemployment rate monthly", "limit": 10, "format": "csv"}. READ THIS BEFORE YOU TRUST A RESULT. The snapshot indexes ONE provider, Data.gov, and only part of it: about 22,000 records were catalogued out of the ~550,000 Data.gov lists, and only about a thousand of those record which data formats they publish. So a miss is NOT evidence that no such source exists — go and look yourself — and a hit is a lead to open and read, never a source anyone has verified. Returns {status, question, requirement, generation, candidates, work}. `generation` says which sealed snapshot answered and when it was published; cite its store digest. `work` reports what the search did — ranges_scanned, ranges_pruned, candidates_examined, member_bytes_read — and no wall-clock time. Every ranked candidate comes back whatever its `disposition` — admitted, human_escalation_required or refused — because refused overwhelmingly means the entry does not record its data formats, not that the source is unusable. `format` takes ONE lowercase token and withholds nothing: it states the format your question requires. Every ranked entry still comes back, and one that does not declare it arrives with `filters_match: false` and `disposition: refused`. Naming a format changes the judgement beside each entry, not which entries you see. There is no way to ask for two — the catalogue admits one format filter per question — so send `format` once or leave it out. `title`, `publisher`, `description` and `rights` may each be null, and `spatial_scope` is a LIST of bounding-box strings or null: the catalogue does not record these for every entry. `provider_record_url` is the entry's own identifier, and only when that identifier is itself an https URL — about one entry in four is identified by an opaque id and has no page to open, so a null means search the provider's own site rather than building an address out of `provider_record_id`. There are no relevance scores anywhere: the order is the ranking, and the facts are the evidence. This tool checks `question`, `limit` and `format` against the shapes above BEFORE it calls anything, so a malformed argument is refused here and never reaches the catalogue; were one to get through, a bad `limit` or a missing `question` can surface as the service's own 422 rather than as `catalog_search_invalid`. Needs a workspace credential, and the same search is `mr-data catalog search` on the command line.