Search Over Groups

POST

api

chunk_group

group_oriented_search

curl --request POST \
  --url https://api.trieve.ai/api/chunk_group/group_oriented_search \
  --header 'Authorization: <api-key>' \
  --header 'Content-Type: application/json' \
  --header 'TR-Dataset: <tr-dataset>' \
  --data '{
  "filters": {
    "must": [
      {
        "field": "tag_set",
        "match_all": [
          "A",
          "B"
        ]
      },
      {
        "field": "num_value",
        "range": {
          "gte": 10,
          "lte": 25
        }
      }
    ]
  },
  "get_total_pages": true,
  "group_size": 1,
  "highlight_options": {
    "highlight_delimiters": [
      "<string>"
    ],
    "highlight_max_length": 1,
    "highlight_max_num": 1,
    "highlight_results": true,
    "highlight_strategy": "exactmatch",
    "highlight_threshold": 123,
    "highlight_window": 1,
    "post_tag": "<string>",
    "pre_tag": "<string>"
  },
  "metadata": "<any>",
  "page": 1,
  "page_size": 1,
  "query": {
    "image_url": "<string>",
    "llm_prompt": "<string>"
  },
  "remove_stop_words": true,
  "score_threshold": 123,
  "search_type": "fulltext",
  "slim_chunks": true,
  "sort_options": {
    "location_bias": {
      "bias": 123,
      "location": {
        "lat": 123,
        "lon": 123
      }
    },
    "mmr": {
      "mmr_lambda": 123,
      "use_mmr": true
    },
    "recency_bias": 123,
    "sort_by": {
      "direction": "desc",
      "field": "<string>",
      "prefetch_amount": 1
    },
    "tag_weights": {},
    "use_weights": true
  },
  "typo_options": {
    "correct_typos": true,
    "disable_on_word": [
      "<string>"
    ],
    "one_typo_word_range": {
      "max": 1,
      "min": 1
    },
    "prioritize_domain_specifc_words": true,
    "two_typo_word_range": {
      "max": 1,
      "min": 1
    }
  },
  "use_quote_negated_terms": true,
  "user_id": "<string>"
}'

{
  "corrected_query": "<string>",
  "id": "3c90c3cc-0d44-4b50-8888-8dd25736052a",
  "results": [
    {
      "chunks": [
        {
          "chunk": {
            "chunk_html": "<p>Some HTML content</p>",
            "content": "Some content",
            "id": "d290f1ee-6c54-4b01-90e6-d701748f0851",
            "link": "https://example.com",
            "metadata": {
              "key1": "value1",
              "key2": "value2"
            },
            "time_stamp": "2021-01-01 00:00:00.000",
            "weight": 0.5
          },
          "highlights": [
            "highlight is two tokens: high, light",
            "whereas hello is only one token: hello"
          ],
          "score": 0.5
        }
      ],
      "file_id": "3c90c3cc-0d44-4b50-8888-8dd25736052a",
      "group": {
        "created_at": "2021-01-01 00:00:00.000",
        "dataset_id": "e3e3e3e3-e3e3-e3e3-e3e3-e3e3e3e3e3e3",
        "description": "All versions and colorways of the oversized t-shirt",
        "metadata": {
          "foo": "bar"
        },
        "name": "Versions of Oversized T-Shirt",
        "tag_set": [
          "tshirt",
          "oversized",
          "clothing"
        ],
        "tracking_id": "SNOVERSIZEDTSHIRT",
        "updated_at": "2021-01-01 00:00:00.000"
      }
    }
  ],
  "total_pages": 123
}

Authorizations

Authorization

string

header

required

Headers

TR-Dataset

string

required

The dataset id or tracking_id to use for the request. We assume you intend to use an id if the value is a valid uuid.

X-API-Version

enum<string>

The API version to use for this request. Defaults to V2 for orgs created after July 12, 2024 and V1 otherwise.

Available options:

V1,

V2

Body

application/json

JSON request payload to semantically search over groups

query

required

Query is the search query. This can be any string. The query will be used to create an embedding vector and/or SPLADE vector which will be used to find the result set. You can either provide one query, or multiple with weights. Multi-query only works with Semantic Search and is not compatible with cross encoder re-ranking or highlights.

search_type

enum<string>

required

Available options:

fulltext,

semantic,

hybrid,

bm25

filters

object

ChunkFilter is a JSON object which can be used to filter chunks. This is useful for when you want to filter chunks by arbitrary metadata. Unlike with tag filtering, there is a performance hit for filtering on metadata.

filters.must

object[] | null

All of these field conditions have to match for the chunk to be included in the result set.

Filters can be constructed using either fields on the chunk objects, ids or tracking ids of chunks, and finally ids or tracking ids of groups.

filters.must.field

string

required

Field is the name of the field to filter on. Commonly used fields are timestamp, link, tag_set, location, num_value, group_ids, and group_tracking_ids. The field value will be used to check for an exact substring match on the metadata values for each existing chunk. This is useful for when you want to filter chunks by arbitrary metadata. To access fields inside of the metadata that you provide with the card, prefix the field name with metadata..

filters.must.boolean

boolean | null

Boolean is a true false value for a field. This only works for boolean fields. You can specify this if you want values to be true or false.

filters.must.date_range

object

DateRange is a JSON object which can be used to filter chunks by a range of dates. This leverages the time_stamp field on chunks in your dataset. You can specify this if you want values in a certain range. You must provide ISO 8601 combined date and time without timezone.

Example:

{
  "gt": "2021-01-01 00:00:00.000",
  "gte": "2021-01-01 00:00:00.000",
  "lt": "2021-01-01 00:00:00.000",
  "lte": "2021-01-01 00:00:00.000"
}

filters.must.geo_bounding_box

object

filters.must.geo_polygon

object

filters.must.geo_radius

object

filters.must.match_all

(string | integer | number)[] | null

Match all lets you pass in an array of values that will return results if all of the items match. The match value will be used to check for an exact substring match on the metadata values for each existing chunk. If both match_all and match_any are provided, the match_any condition will be used.

filters.must.match_any

(string | integer | number)[] | null

Match any lets you pass in an array of values that will return results if any of the items match. The match value will be used to check for an exact substring match on the metadata values for each existing chunk. If both match_all and match_any are provided, the match_any condition will be used.

filters.must.range

object

Example:

{ "gt": 0, "gte": 0, "lt": 1, "lte": 1 }

filters.must_not

object[] | null

None of these field conditions can match for the chunk to be included in the result set.

Filters can be constructed using either fields on the chunk objects, ids or tracking ids of chunks, and finally ids or tracking ids of groups.

filters.must_not.field

string

required

filters.must_not.boolean

boolean | null

Boolean is a true false value for a field. This only works for boolean fields. You can specify this if you want values to be true or false.

filters.must_not.date_range

object

Example:

{
  "gt": "2021-01-01 00:00:00.000",
  "gte": "2021-01-01 00:00:00.000",
  "lt": "2021-01-01 00:00:00.000",
  "lte": "2021-01-01 00:00:00.000"
}

filters.must_not.geo_bounding_box

object

filters.must_not.geo_polygon

object

filters.must_not.geo_radius

object

filters.must_not.match_all

(string | integer | number)[] | null

filters.must_not.match_any

(string | integer | number)[] | null

filters.must_not.range

object

Example:

{ "gt": 0, "gte": 0, "lt": 1, "lte": 1 }

filters.should

object[] | null

Only one of these field conditions has to match for the chunk to be included in the result set.

Filters can be constructed using either fields on the chunk objects, ids or tracking ids of chunks, and finally ids or tracking ids of groups.

filters.should.field

string

required

filters.should.boolean

boolean | null

Boolean is a true false value for a field. This only works for boolean fields. You can specify this if you want values to be true or false.

filters.should.date_range

object

Example:

{
  "gt": "2021-01-01 00:00:00.000",
  "gte": "2021-01-01 00:00:00.000",
  "lt": "2021-01-01 00:00:00.000",
  "lte": "2021-01-01 00:00:00.000"
}

filters.should.geo_bounding_box

object

filters.should.geo_polygon

object

filters.should.geo_radius

object

filters.should.match_all

(string | integer | number)[] | null

filters.should.match_any

(string | integer | number)[] | null

filters.should.range

object

Example:

{ "gt": 0, "gte": 0, "lt": 1, "lte": 1 }

Example:

{
  "must": [
    {
      "field": "tag_set",
      "match_all": ["A", "B"]
    },
    {
      "field": "num_value",
      "range": { "gte": 10, "lte": 25 }
    }
  ]
}

get_total_pages

boolean | null

Get total page count for the query accounting for the applied filters. Defaults to false, but can be set to true when the latency penalty is acceptable (typically 50-200ms).

group_size

integer | null

Group_size is the number of chunks to fetch for each group. The default is 3. If a group has less than group_size chunks, all chunks will be returned. If this is set to a large number, we recommend setting slim_chunks to true to avoid returning the content and chunk_html of the chunks so as to lower the amount of time required for content download and serialization.

Required range: x >= 0

highlight_options

object

Highlight Options lets you specify different methods to highlight the chunks in the result set. If not specified, this defaults to the score of the chunks.

highlight_options.highlight_delimiters

string[] | null

Set highlight_delimiters to a list of strings to use as delimiters for highlighting. If not specified, this defaults to ["?", ",", ".", "!"]. These are the characters that will be used to split the chunk_html into splits for highlighting. These are the characters that will be used to split the chunk_html into splits for highlighting.

highlight_options.highlight_max_length

integer | null

Set highlight_max_length to control the maximum number of tokens (typically whitespace separated strings, but sometimes also word stems) which can be present within a single highlight. If not specified, this defaults to 8. This is useful to shorten large splits which may have low scores due to length compared to the query. Set to something very large like 100 to highlight entire splits.

Required range: x >= 0

highlight_options.highlight_max_num

integer | null

Set highlight_max_num to control the maximum number of highlights per chunk. If not specified, this defaults to 3. It may be less than 3 if no snippets score above the highlight_threshold.

Required range: x >= 0

highlight_options.highlight_results

boolean | null

Set highlight_results to false for a slight latency improvement (1-10ms). If not specified, this defaults to true. This will add  tags to the chunk_html of the chunks to highlight matching splits and return the highlights on each scored chunk in the response.

highlight_options.highlight_strategy

enum<string>

Available options:

exactmatch,

v1

highlight_options.highlight_threshold

number | null

Set highlight_threshold to a lower or higher value to adjust the sensitivity of the highlights applied to the chunk html. If not specified, this defaults to 0.8. The range is 0.0 to 1.0.

highlight_options.highlight_window

integer | null

Set highlight_window to a number to control the amount of words that are returned around the matched phrases. If not specified, this defaults to 0. This is useful for when you want to show more context around the matched words. When specified, window/2 whitespace separated words are added before and after each highlight in the response's highlights array. If an extended highlight overlaps with another highlight, the overlapping words are only included once. This parameter can be overriden to respect the highlight_max_length param.

Required range: x >= 0

highlight_options.post_tag

string | null

Custom html tag which should appear after highlights. If not specified, this defaults to ''.

highlight_options.pre_tag

string | null

Custom html tag which should appear before highlights. If not specified, this defaults to ''.

metadata

any | null

Metadata is any metadata you want to associate w/ the event that is created from this request

page

integer | null

Page of group results to fetch. Page is 1-indexed.

Required range: x >= 0

page_size

integer | null

Page size is the number of group results to fetch. The default is 10.

Required range: x >= 0

remove_stop_words

boolean | null

If true, stop words (specified in server/src/stop-words.txt in the git repo) will be removed. Queries that are entirely stop words will be preserved.

score_threshold

number | null

Set score_threshold to a float to filter out chunks with a score below the threshold. This threshold applies before weight and bias modifications. If not specified, this defaults to 0.0.

slim_chunks

boolean | null

Set slim_chunks to true to avoid returning the content and chunk_html of the chunks. This is useful for when you want to reduce amount of data over the wire for latency improvement (typicall 10-50ms). Default is false.

sort_options

object

Sort Options lets you specify different methods to rerank the chunks in the result set. If not specified, this defaults to the score of the chunks.

sort_options.location_bias

object

Location bias lets you rank your results by distance from a location. If not specified, this has no effect. Bias allows you to determine how much of an effect the location of chunks will have on the search results. If not specified, this defaults to 0.0. We recommend setting this to 1.0 for a gentle reranking of the results, >3.0 for a strong reranking of the results.

sort_options.mmr

object

MMR Options lets you specify different methods to rerank the chunks in the result set using Maximal Marginal Relevance. If not specified, this defaults to the score of the chunks.

sort_options.recency_bias

number | null

Recency Bias lets you determine how much of an effect the recency of chunks will have on the search results. If not specified, this defaults to 0.0. We recommend setting this to 1.0 for a gentle reranking of the results, >3.0 for a strong reranking of the results.

sort_options.sort_by

object

Sort by lets you specify a method to sort the results by. If not specified, this defaults to the score of the chunks. If specified, this can be any key in the chunk metadata. This key must be a numeric value within the payload.

sort_options.tag_weights

object | null

Tag weights is a JSON object which can be used to boost the ranking of chunks with certain tags. This is useful for when you want to be able to bias towards chunks with a certain tag on the fly. The keys are the tag names and the values are the weights.

sort_options.use_weights

boolean | null

Set use_weights to true to use the weights of the chunks in the result set in order to sort them. If not specified, this defaults to true.

typo_options

object

Typo Options lets you specify different methods to correct typos in the query. If not specified, typos will not be corrected.

use_quote_negated_terms

boolean | null

If true, quoted and - prefixed words will be parsed from the queries and used as required and negated words respectively. Default is false.

user_id

string | null

The user_id is the id of the user who is making the request. This is used to track user interactions with the search results.

Response

200

application/json

Group chunks which are similar to the embedding vector of the search query

string

required

results

object[]

required

results.chunks

object[]

required

results.group

object

required

Example:

{
  "created_at": "2021-01-01 00:00:00.000",
  "dataset_id": "e3e3e3e3-e3e3-e3e3-e3e3-e3e3e3e3e3e3",
  "description": "All versions and colorways of the oversized t-shirt",
  "metadata": { "foo": "bar" },
  "name": "Versions of Oversized T-Shirt",
  "tag_set": ["tshirt", "oversized", "clothing"],
  "tracking_id": "SNOVERSIZEDTSHIRT",
  "updated_at": "2021-01-01 00:00:00.000"
}

results.file_id

string | null

total_pages

integer

required

corrected_query

string | null

string

required

results

object[]

required

results.chunks

object[]

required

results.group

object

required

Example:

{
  "created_at": "2021-01-01 00:00:00.000",
  "dataset_id": "e3e3e3e3-e3e3-e3e3-e3e3-e3e3e3e3e3e3",
  "description": "All versions and colorways of the oversized t-shirt",
  "metadata": { "foo": "bar" },
  "name": "Versions of Oversized T-Shirt",
  "tag_set": ["tshirt", "oversized", "clothing"],
  "tracking_id": "SNOVERSIZEDTSHIRT",
  "updated_at": "2021-01-01 00:00:00.000"
}

results.file_id

string | null

total_pages

integer

required

corrected_query

string | null

group_chunks

object[]

required

total_chunk_pages

integer

required

corrected_query

string | null

Was this page helpful?

Raise issue

Create or Upsert Group or Groups Search Within Group

curl --request POST \
  --url https://api.trieve.ai/api/chunk_group/group_oriented_search \
  --header 'Authorization: <api-key>' \
  --header 'Content-Type: application/json' \
  --header 'TR-Dataset: <tr-dataset>' \
  --data '{
  "filters": {
    "must": [
      {
        "field": "tag_set",
        "match_all": [
          "A",
          "B"
        ]
      },
      {
        "field": "num_value",
        "range": {
          "gte": 10,
          "lte": 25
        }
      }
    ]
  },
  "get_total_pages": true,
  "group_size": 1,
  "highlight_options": {
    "highlight_delimiters": [
      "<string>"
    ],
    "highlight_max_length": 1,
    "highlight_max_num": 1,
    "highlight_results": true,
    "highlight_strategy": "exactmatch",
    "highlight_threshold": 123,
    "highlight_window": 1,
    "post_tag": "<string>",
    "pre_tag": "<string>"
  },
  "metadata": "<any>",
  "page": 1,
  "page_size": 1,
  "query": {
    "image_url": "<string>",
    "llm_prompt": "<string>"
  },
  "remove_stop_words": true,
  "score_threshold": 123,
  "search_type": "fulltext",
  "slim_chunks": true,
  "sort_options": {
    "location_bias": {
      "bias": 123,
      "location": {
        "lat": 123,
        "lon": 123
      }
    },
    "mmr": {
      "mmr_lambda": 123,
      "use_mmr": true
    },
    "recency_bias": 123,
    "sort_by": {
      "direction": "desc",
      "field": "<string>",
      "prefetch_amount": 1
    },
    "tag_weights": {},
    "use_weights": true
  },
  "typo_options": {
    "correct_typos": true,
    "disable_on_word": [
      "<string>"
    ],
    "one_typo_word_range": {
      "max": 1,
      "min": 1
    },
    "prioritize_domain_specifc_words": true,
    "two_typo_word_range": {
      "max": 1,
      "min": 1
    }
  },
  "use_quote_negated_terms": true,
  "user_id": "<string>"
}'

{
  "corrected_query": "<string>",
  "id": "3c90c3cc-0d44-4b50-8888-8dd25736052a",
  "results": [
    {
      "chunks": [
        {
          "chunk": {
            "chunk_html": "<p>Some HTML content</p>",
            "content": "Some content",
            "id": "d290f1ee-6c54-4b01-90e6-d701748f0851",
            "link": "https://example.com",
            "metadata": {
              "key1": "value1",
              "key2": "value2"
            },
            "time_stamp": "2021-01-01 00:00:00.000",
            "weight": 0.5
          },
          "highlights": [
            "highlight is two tokens: high, light",
            "whereas hello is only one token: hello"
          ],
          "score": 0.5
        }
      ],
      "file_id": "3c90c3cc-0d44-4b50-8888-8dd25736052a",
      "group": {
        "created_at": "2021-01-01 00:00:00.000",
        "dataset_id": "e3e3e3e3-e3e3-e3e3-e3e3-e3e3e3e3e3e3",
        "description": "All versions and colorways of the oversized t-shirt",
        "metadata": {
          "foo": "bar"
        },
        "name": "Versions of Oversized T-Shirt",
        "tag_set": [
          "tshirt",
          "oversized",
          "clothing"
        ],
        "tracking_id": "SNOVERSIZEDTSHIRT",
        "updated_at": "2021-01-01 00:00:00.000"
      }
    }
  ],
  "total_pages": 123
}

Chunk

Chunk Group

Topic

Message

Crawl

File

Analytics

Dataset

Organization

User

Auth

Health

Invitation

Stripe

Metrics

Search Over Groups

Authorizations

Headers

Body

Response