> ## Documentation Index
> Fetch the complete documentation index at: https://docs.rhizomeai.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Search

> Keyword search across datasets, ranked and grouped into documents

Searches one or more datasets and returns documents, best first, each with its best matching pages. Every request runs the whole search: nothing is saved between calls (see [Pagination](#pagination)).

How a search runs:

1. In each dataset, find the pages that match `keywords` and the dataset's `filters`. A dataset where no page matches every word of any item loosens the keywords: it retries its items of more than one word with at least 40% of their words, then with any of them.
2. Keep the best 500 pages across datasets. Each dataset that matches every word of an item gets a share of the 500, larger for datasets with more such matches. A loosened dataset gets 5 pages, plus any the others can't use. When no dataset matches every word, the loosened datasets split the 500 evenly.
3. Rank those pages by meaning against `query`, or against the keywords when there is no `query`. A search of one dataset with no `query` keeps the keyword ranking.
4. Group the pages into documents. A document ranks by its best page, and documents from loosened datasets come after all the others.
5. Return documents `offset` to `offset + limit`.

<Tip>
  Send `query` with the question you're answering. In our tests on real
  questions, ranking against the question beat ranking against the keywords.
</Tip>

## Request body

<ParamField body="keywords" type="string[]" required>
  Words to match. All words in one item must match the page; items are
  alternatives. `["sterility aseptic", "media fill"]` finds pages with both
  "sterility" and "aseptic", or both "media" and "fill". 1 to 20 items, each up
  to 300 characters. In a dataset where no page matches every word of any item,
  items of more than one word are loosened: first to pages with at least 40% of
  the item's words, then to pages with any of them. That dataset's status says
  `loosened: true`, and its documents come after all the others.
</ParamField>

<ParamField body="query" type="string">
  The question in plain language, up to 2,000 characters. Used only to rank the
  matching pages, never to find them. Recommended.
</ParamField>

<ParamField body="datasets" type="string[]">
  Dataset ids to search, from [`GET
      /api/v3/datasets`](/api-reference/v3/datasets) or the
  [Datasets](/datasets/index) reference. Omit to search every dataset your plan
  can access. An unknown id is a 400. A dataset the API can't search yet, or one
  your plan can't access, is reported in `datasets` and not searched.
</ParamField>

<ParamField body="filters" type="object">
  Filters per dataset, keyed by dataset id:
  `{"fda_warning_letters": {"company": "Example Pharma"}}`. Each dataset has its
  own filters, listed on its page in [Datasets](/datasets/index). They are never
  shared: `company` means different things in different datasets. A filter the
  dataset doesn't list, a value of the wrong type, or filters for a dataset the
  search doesn't cover is a 400.
</ParamField>

<ParamField body="date_from" type="string (date)">
  `YYYY-MM-DD`, inclusive. Applies to every dataset, on that dataset's own date
  field: the approval date in one, the event date in another. Each dataset's
  page names its date field. Datasets with no date field are skipped and
  reported as `skipped`. Sending the same dataset's date filter in `filters` as
  well is a 400.
</ParamField>

<ParamField body="date_to" type="string (date)">
  `YYYY-MM-DD`, inclusive. Same rules as `date_from`.
</ParamField>

<ParamField body="limit" type="integer" default={20}>
  Documents per response, 1 to 50.
</ParamField>

<ParamField body="offset" type="integer" default={0}>
  Documents to skip, 0 to 500. See [Pagination](#pagination).
</ParamField>

## Response

<ResponseField name="results" type="array">
  Documents, best first. Documents from loosened datasets come after all the
  others, so `score` rises again where they start.

  <Expandable title="Result object">
    <ResponseField name="document_id" type="string">
      Opaque document id. Pass it to
      [`GET /api/v3/documents/{document_id}`](/api-reference/v3/documents) for the
      full text, and use it to drop duplicates across pages.
    </ResponseField>

    <ResponseField name="dataset" type="string">
      The dataset id.
    </ResponseField>

    <ResponseField name="title" type="string">
      The document's title.
    </ResponseField>

    <ResponseField name="date" type="string | null">
      The value of the dataset's date field. `null` for datasets with no date
      field.
    </ResponseField>

    <ResponseField name="url" type="string">
      Link to the source document.
    </ResponseField>

    <ResponseField name="fields" type="object">
      The dataset's fields for this document. Each dataset's page in
      [Datasets](/datasets/index) lists them.
    </ResponseField>

    <ResponseField name="pages" type="array">
      Up to 3 of the document's matching pages, best first.

      <Expandable title="Page object">
        <ResponseField name="page_num" type="integer">
          Page number. The first page is 1. Send it as `page_from` to
          [read the document](/api-reference/v3/documents) from this page.
        </ResponseField>

        <ResponseField name="score" type="number">
          The page's score. `ranking` says which kind.
        </ResponseField>

        <ResponseField name="text" type="string">
          The full text of the page.
        </ResponseField>
      </Expandable>
    </ResponseField>

    <ResponseField name="matched_pages" type="integer">
      How many of the document's pages were among the pages ranked. A lower bound
      for long documents: read the rest with
      [`GET /api/v3/documents/{document_id}`](/api-reference/v3/documents).
    </ResponseField>
  </Expandable>
</ResponseField>

<ResponseField name="datasets" type="object">
  What happened in each dataset searched or requested, keyed by dataset id.

  <Expandable title="Dataset status">
    <ResponseField name="status" type="string">
      One of the statuses below.
    </ResponseField>

    <ResponseField name="matches" type="integer">
      `ok` only. Pages that match every word of a keyword item, counted up to
      10,000. `0` when the dataset was loosened.
    </ResponseField>

    <ResponseField name="pages" type="integer">
      `ok` only. The dataset's pages among the 500 ranked.
    </ResponseField>

    <ResponseField name="loosened" type="boolean">
      `ok` only, present when `true`. No page matched every word of an item, so
      the pages came from the loosened keywords. The dataset's documents come
      after all the others.
    </ResponseField>

    <ResponseField name="reason" type="string">
      `skipped` only. Why, e.g. `no date field`.
    </ResponseField>
  </Expandable>
</ResponseField>

<ResponseField name="total" type="integer">
  Documents in the ranked list. At most the documents of the 500 pages ranked.
</ResponseField>

<ResponseField name="next_offset" type="integer | null">
  The `offset` of the next page. `null` on the last page.
</ResponseField>

<ResponseField name="ranking" type="string">
  What `score` is. `similarity`: how close the page is in meaning to `query` (or
  the keywords), comparable across datasets. `bm25`: the keyword relevance
  score, used when the search covers one dataset and has no `query`.
</ResponseField>

### Dataset statuses

| Status | Meaning |
| - | - |
| `ok` | Searched. `matches` and `pages` say how much it contributed, and `loosened` says whether only the loosened keywords matched. |
| `no_match` | No page matched, loosened keywords included. |
| `skipped` | The search has a date and the dataset has no date field. |
| `not_available` | The API can't search this dataset yet. |
| `forbidden` | The dataset needs an enterprise plan. |
| `timeout` | The dataset didn't answer in time. The other datasets still return. |
| `error` | The dataset's query failed. The other datasets still return. |

A requested dataset is always reported, never dropped. With no `datasets` in the request, the block lists every dataset searched.

```json theme={null}
"datasets": {
  "fda_483": { "status": "ok", "matches": 1840, "pages": 495 },
  "fda_crl": { "status": "ok", "matches": 0, "pages": 5, "loosened": true },
  "fda_recalls": { "status": "no_match" },
  "fda_product_classification": { "status": "skipped", "reason": "no date field" },
  "fda_faers": { "status": "forbidden" }
}
```

## Pagination

Every request reruns the whole search and returns one slice of the ranked documents. Nothing is saved between calls.

* Send `offset: 0` for the first page. The response has `total` and `next_offset`.
* For the next page, send the same body with `offset` set to `next_offset`. Stop when `next_offset` is `null`.
* Results can shift slightly between calls, because the search runs again each time. A document may appear on two pages, or be missed. Drop duplicates by `document_id`.
* Depth is capped. A search ranks at most 500 pages, so `total` is at most the documents of those pages, and an `offset` past `total` returns no results.
* Each page costs a full search, and counts against your [rate limit](/rate-limits). Ask for up to 50 documents per page with `limit`.

```python Python theme={null}
import requests

body = {
    "keywords": ["sterility aseptic", "media fill"],
    "query": "What sterility failures did FDA cite at fill-finish sites?",
    "datasets": ["fda_483", "fda_warning_letters"],
    "limit": 50,
    "offset": 0,
}
seen, documents = set(), []
while body["offset"] is not None:
    response = requests.post(
        "https://rhizomeai.com/api/v3/search",
        headers={"x-api-key": "YOUR_API_KEY"},
        json=body,
    )
    response.raise_for_status()
    data = response.json()
    for result in data["results"]:
        if result["document_id"] not in seen:
            seen.add(result["document_id"])
            documents.append(result)
    body["offset"] = data["next_offset"]

print(f"{len(documents)} documents")
```

## Limits

* A search takes up to about 30 seconds. A dataset still running then reports `timeout`, and the rest return.
* One search at a time per account, across all its API keys. A second search sent while the first runs waits up to 2 seconds, then answers 429 `concurrent_search` with a `Retry-After` header. Send searches one after another.
* When too many searches are running on the server, the API answers 503 `busy` with a `Retry-After` header. Wait that many seconds and retry.
* 50 requests per minute per API key, across all endpoints. See [Rate limits](/rate-limits).

## Errors

Errors return `{"error": {"code", "message", "field", "request_id"}}`; `field` names the value at fault when there is one. See [Errors](/errors).

| Status | Code | Description |
| - | - | - |
| 400 | `invalid_request` | No `keywords`, a value of the wrong type, `limit` or `offset` out of range, `date_from` after `date_to`, or a body that isn't JSON |
| 400 | `unknown_dataset` | An id in `datasets` doesn't exist |
| 400 | `invalid_filter` | A filter the dataset doesn't list, a value of the wrong type, filters for a dataset the search doesn't cover, or a repeated date |
| 401 | `missing_api_key`, `invalid_api_key` | Missing or invalid API key |
| 429 | `rate_limited` | Rate limit exceeded (50 requests/minute). Retry after `Retry-After` seconds. |
| 429 | `concurrent_search` | Another search from your account is still running. Retry after `Retry-After` seconds. |
| 503 | `busy` | Too many searches running on the server. Retry after `Retry-After` seconds. |
| 503 | `upstream_error`, `timeout` | Ranking failed or ran out of time. Retry after `Retry-After` seconds. |
| 504 | `timeout` | The search timed out |

<RequestExample>
  ```bash cURL theme={null}
  curl -X POST https://rhizomeai.com/api/v3/search \
    -H "Content-Type: application/json" \
    -H "x-api-key: YOUR_API_KEY" \
    -d '{
      "keywords": ["sterility aseptic", "media fill"],
      "query": "What sterility failures did FDA cite at fill-finish sites?",
      "datasets": ["fda_483", "fda_warning_letters"],
      "filters": {
        "fda_warning_letters": { "company": "Example Pharma" }
      },
      "date_from": "2023-01-01",
      "limit": 20
    }'
  ```

  ```python Python theme={null}
  import requests

  response = requests.post(
      "https://rhizomeai.com/api/v3/search",
      headers={"x-api-key": "YOUR_API_KEY"},
      json={
          "keywords": ["sterility aseptic", "media fill"],
          "query": "What sterility failures did FDA cite at fill-finish sites?",
          "datasets": ["fda_483", "fda_warning_letters"],
          "filters": {"fda_warning_letters": {"company": "Example Pharma"}},
          "date_from": "2023-01-01",
          "limit": 20,
      },
  )
  response.raise_for_status()
  data = response.json()
  print(f"{data['total']} documents")
  for result in data["results"]:
      print(result["dataset"], result["title"], result["url"])
  ```

  ```javascript JavaScript theme={null}
  const response = await fetch("https://rhizomeai.com/api/v3/search", {
    method: "POST",
    headers: {
      "Content-Type": "application/json",
      "x-api-key": "YOUR_API_KEY",
    },
    body: JSON.stringify({
      keywords: ["sterility aseptic", "media fill"],
      query: "What sterility failures did FDA cite at fill-finish sites?",
      datasets: ["fda_483", "fda_warning_letters"],
      filters: { fda_warning_letters: { company: "Example Pharma" } },
      date_from: "2023-01-01",
      limit: 20,
    }),
  });

  const data = await response.json();
  console.log(`${data.total} documents`);
  ```
</RequestExample>

<ResponseExample>
  ```json Response theme={null}
  {
    "results": [
      {
        "document_id": "WyJmZGFfNDgzIixbImh0dHBzOi8vd3d3LmZkYS5nb3YvbWVkaWEvMDAwMDAwL2Rvd25sb2FkIl1d",
        "dataset": "fda_483",
        "title": "Example Pharma Inc.",
        "date": "2024-03-12",
        "url": "https://www.fda.gov/media/000000/download",
        "fields": {
          "facility_name": "Example Pharma Inc.",
          "fei_number": "0000000000",
          "record_date": "2024-03-12",
          "record_id": "000000",
          "title": "Example Pharma Inc.",
          "url": "https://www.fda.gov/media/000000/download"
        },
        "pages": [
          {
            "page_num": 3,
            "score": 0.6241,
            "text": "OBSERVATION 2. Procedures designed to prevent microbiological contamination of drug products purporting to be sterile..."
          },
          {
            "page_num": 4,
            "score": 0.5873,
            "text": "OBSERVATION 3. Aseptic processing areas are deficient regarding the system for monitoring environmental conditions..."
          },
          {
            "page_num": 2,
            "score": 0.5402,
            "text": "OBSERVATION 1. There is a failure to thoroughly review the failure of a batch to meet its specifications. Sterility test..."
          }
        ],
        "matched_pages": 5
      }
    ],
    "datasets": {
      "fda_483": { "status": "ok", "matches": 1840, "pages": 317 },
      "fda_warning_letters": { "status": "ok", "matches": 612, "pages": 183 }
    },
    "total": 146,
    "next_offset": 20,
    "ranking": "similarity"
  }
  ```
</ResponseExample>


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.