Inspiration

Performance Difference

See more

TypeScript SDK

Python SDK

Github

Community

Blog

Trieve

API Reference

Vector Inference

Site Search

Support

Dashboard

Trieve Vector Inference is an on-prem solution for fast vector inference

Introduction

Trieve is an API for building search, recommendations, and RAG experiences.

Quickly start building search, recommendations, and RAG for your application with Trieve

Quickstart

Basic terms and concepts used commonly within the Trieve ecosystem.

Trieve Primitives

Screenshots of Trieve in action across various use cases.

Screenshots

The pricing design Trieve Vector Inference

Pricing

Breakdown of all the services installed within Trieve Vector Inference

Architecture Diagram

Learn how to use the Trieve SiteSearch React Component

Using the Sitesearch Component

Learn how to self-host Trieve with Docker Compose

Docker Compose Setup

Learn how to self-host Trieve on local Kubernetes

Local Kubernetes Setup

AWS Self Hosting

Learn how to self-host Trieve on Google Cloud Platform

GCP Self Hosting

Install Trieve Vector Inference in your own AWS account

AWS Installation

Troubleshooting

For any updates to Trieve Vector Inference, this is how you should upgrade:

Upgrading your Instance

Learn how to add the Search Component using the Docusaurus plugin

Docusaurus Plugin Theme

Learn how to upload your chunks to Trieve

Uploading Chunks to Trieve

Uploading Files to Trieve

Learn how to upload bulk structured data to Trieve

Uploading CSV and JSONL Files to Trieve

Learn how to search over your data with Trieve

Searching with Trieve

Learn how to recommend content with Trieve

Recommending with Trieve

Learn how to chat with your data with Trieve

RAG with Trieve

Learn how to get started with Trieve Analytics

Analytics with Trieve

Learn how to create and manage organizations and datasets with Trieve

Creating Organizations and Datasets with Trieve

Learn how to create and use groups with Trieve

Using Groups with Trieve

Learn how to create and use safe, access controlled api keys with Trieve

Using Exposed API keys with Trieve

Working with Reranker

Working with SPLADE v2

How to use gated or private models hosted on Hugging Face

Using Custom Models

How to integrate TVI with existing OpenAI compatible endpoints

Using OpenAI SDK

Get embeddings. Returns a 424 status code if the model is not an embedding model

POST /embed

Create Embedding

POST /embed_all

Get sparse embeddings. Returns a 424 status code if the model is not a SPLADE embedding model

POST /embed_sparse

Create Sparse Embedding

Runs Reranker. Returns a 424 status code if the model is not a Reranker model.

POST /rerank

Get Ranks

OpenAI compatible route. Returns a 424 status code if the model is not an embedding model.

POST /v1/embeddings

OpenAI compatible embeddings route

GET /health

Health Check

Learn how to build a job board with Trieve

Build Search for a Job Board

Build Search for Ecommerce

Create new chunk(s). If the chunk has the same tracking_id as an existing chunk, the request will fail. Once a chunk is created, it can be searched for using the search endpoint.
If uploading in bulk, the maximum amount of chunks that can be uploaded at once is 120 chunks. Auth'ed user or api key must have an admin or owner role for the specified dataset's organization.

Create or Upsert Chunk or Chunks

This route provides the primary search functionality for the API. It can be used to search for chunks by semantic similarity, full-text similarity, or a combination of both. Results' `chunk_html` values will be modified with `<mark><b>` or custom specified tags for sub-sentence highlighting.

Search

This route provides the primary autocomplete functionality for the API. This prioritize prefix matching with semantic or full-text search.

Autocomplete

Get recommendations of chunks similar to the positive samples in the request and dissimilar to the negative.

Get Recommended Chunks

Get paginated chunks from your dataset with filters and custom sorting. If sort by is not specified, the results will sort by the id's of the chunks in ascending order. Sort by and offset_chunk_id cannot be used together; if you want to scroll with a sort by then you need to use a must_not filter with the ids you have already seen. There is a limit of 1000 id's in a must_not filter at a time.

Scroll Chunks

This route can be used to determine the number of chunk results that match a search query including score threshold and filters. It may be high latency for large limits. There is a dataset configuration imposed restriction on the maximum limit value (default 10,000) which is used to prevent DDOS attacks. Auth'ed user or api key must have an admin or owner role for the specified dataset's organization.

Count chunks above threshold

This endpoint will generate 3 suggested queries based off a hybrid search using RAG with the query provided in the request body and return them as a JSON object.

Generate suggested queries

This endpoint exists as an alternative to the topic+message resource pattern where our Trieve handles chat memory. With this endpoint, the user is responsible for providing the context window and the prompt and the conversation is ephemeral.

RAG on Specified Chunks

Update a chunk. If you try to change the tracking_id of the chunk to have the same tracking_id as an existing chunk, the request will fail. Auth'ed user or api key must have an admin or owner role for the specified dataset's organization.

Update Chunk

Update a chunk by tracking_id. This is useful for when you are coordinating with an external system and want to use the tracking_id to identify the chunk. Auth'ed user or api key must have an admin or owner role for the specified dataset's organization.

Update Chunk By Tracking Id

Get Chunk By Id

Get a singular chunk by tracking_id. This is useful for when you are coordinating with an external system and want to use your own id as the primary reference for a chunk.

Get Chunk By Tracking Id

Get Chunks By Tracking Ids

Get Chunks By Ids

Delete a chunk by its id. Auth'ed user or api key must have an admin or owner role for the specified dataset's organization.

Delete Chunk

Delete a chunk by tracking_id. This is useful for when you are coordinating with an external system and want to use the tracking_id to identify the chunk. Auth'ed user or api key must have an admin or owner role for the specified dataset's organization.

Delete Chunk By Tracking Id

Delete multiple chunks using a filter. Auth'ed user or api key must have an admin or owner role for the specified dataset's organization.

Bulk Delete Chunks

This endpoint receives a single html string and splits it into chunks based on the headings and
body content. The headings are split based on heading html tags. chunk_html has a maximum size
of 256Kb.

Split HTML Content into Chunks

Create new chunk_group(s). This is a way to group chunks together. If you try to create a chunk_group with the same tracking_id as an existing chunk_group, this operation will fail. Only 1000 chunk groups can be created at a time. Auth'ed user or api key must have an admin or owner role for the specified dataset's organization.

Create or Upsert Group or Groups

This route allows you to get groups as results instead of chunks. Each group returned will have the matching chunks sorted by similarity within the group. This is useful for when you want to get groups of chunks which are similar to the search query. If choosing hybrid search, the top chunk of each group will be re-ranked using scores from a cross encoder model. Compatible with semantic, fulltext, or hybrid search modes.

Search Over Groups

This route allows you to search only within a group. This is useful for when you only want search results to contain chunks which are members of a specific group. If choosing hybrid search, the results will be re-ranked using scores from a cross encoder model.

Search Within Group

Route to get recommended groups. This route will return groups which are similar to the groups in the request body. You must provide at least one positive group id or group tracking id.

Get Recommended Groups

Route to add a chunk to a group. One of chunk_id or chunk_tracking_id must be provided. Auth'ed user or api key must have an admin or owner role for the specified dataset's organization.

Add Chunk to Group

Route to add a chunk to a group by tracking id. One of chunk_id or chunk_tracking_id must be provided. Auth'ed user or api key must have an admin or owner role for the specified dataset's organization.

Add Chunk to Group by Tracking ID

Route to get the groups that a chunk is in.

Get Groups for Chunks

Route to get all chunks for a group. The response is paginated, with each page containing 10 chunks. Support for custom page size is coming soon. Page is 1-indexed.

Get Chunks in Group by Tracking ID

Fetch the group with the given tracking id.
get_group_by_tracking_id

Get Group by Tracking ID

Update a chunk_group. One of group_id or tracking_id must be provided. If you try to change the tracking_id to one that already exists, this operation will fail. Auth'ed user or api key must have an admin or owner role for the specified dataset's organization.

Update Group

Route to remove a chunk from a group. Auth'ed user or api key must be an admin or owner of the dataset's organization to remove a chunk from a group.

Remove Chunk from Group

Delete a chunk_group with the given tracking id. Auth'ed user or api key must have an admin or owner role for the specified dataset's organization.

Delete Group by Tracking ID

This will delete a chunk_group. If you set delete_chunks to true, it will also delete the chunks within the group. Auth'ed user or api key must have an admin or owner role for the specified dataset's organization.

Delete Group

Get Group

Route to get all chunks for a group. The response is paginated, with each page containing 10 chunks. Page is 1-indexed.

Get Chunks in Group

Fetch the groups which belong to a dataset specified by its id.

Get Groups for Dataset

Create a new chat topic. Topics are attached to a owner_id's and act as a coordinator for conversation message history of gen-AI chat sessions. Auth'ed user or api key must have an admin or owner role for the specified dataset's organization.

Create Topic

Create a new chat topic from a `topic_id`. The new topic will be attched to the owner_id and act as a coordinator for conversation message history of gen-AI chat sessions. Auth'ed user or api key must have an admin or owner role for the specified dataset's organization.

Clone Topic

Update an existing chat topic. Currently, only the name of the topic can be updated. Auth'ed user or api key must have an admin or owner role for the specified dataset's organization.

Update Topic

Get all topics belonging to an arbitary owner_id. This is useful for managing message history and chat sessions. It is common to use a browser fingerprint or your user's id as the owner_id. Auth'ed user or api key must have an admin or owner role for the specified dataset's organization.

Get All Topics for Owner ID

Delete an existing chat topic. When a topic is deleted, all associated chat messages are also deleted. Auth'ed user or api key must have an admin or owner role for the specified dataset's organization.

Delete Topic

Create message. Messages are attached to topics in order to coordinate memory of gen-AI chat sessions.Auth'ed user or api key must have an admin or owner role for the specified dataset's organization.

Create message

Edit message which exists within the topic's chat history. This will delete the message and replace it with a new message. The new message will be generated by the AI based on the new content provided in the request body. The response will include Chunks first on the stream if the topic is using RAG. The structure will look like `[chunks]||mesage`. See docs.trieve.ai for more information. Auth'ed user or api key must have an admin or owner role for the specified dataset's organization.

Edit message

Regenerate the assistant response to the last user message of a topic. This will delete the last message and replace it with a new message. The response will include Chunks first on the stream if the topic is using RAG. The structure will look like `[chunks]||mesage`. See docs.trieve.ai for more information. Auth'ed user or api key must have an admin or owner role for the specified dataset's organization.

Regenerate message

Quickly get the full object for a given message. From the message, you can get the topic and all messages which exist on that topic.

Get a message by its ID

If the topic is a RAG topic then the response will include Chunks first on each message. The structure will look like `[chunks]||mesage`. See docs.trieve.ai for more information.

Get all messages for a given topic

Upload a file to S3 bucket attached to your dataset. You can select between a naive chunking strategy where the text is extracted with Apache Tika and split into segments with a target number of segments per chunk OR you can use a vision LLM to convert the file to markdown and create chunks per page. Auth'ed user must be an admin or owner of the dataset's organization to upload a file.

Upload File

This route is useful for uploading very large CSV or JSONL files. Once you have completed the upload, chunks will be automatically created from the file for each line in the CSV or JSONL file. The chunks will be indexed and searchable. Auth'ed user must be an admin or owner of the dataset's organization to upload a file.

Create Presigned CSV/JSONL S3 PUT URL

Get all files which belong to a given dataset specified by the dataset_id parameter. 10 files are returned per page.

Get Files for Dataset

Get a signed s3 url corresponding to the file_id requested such that you can download the file.

Get File Signed URL

Delete a file from S3 attached to the server based on its id. This will disassociate chunks from the file, but only delete them all together if you specify delete_chunks to be true. Auth'ed user or api key must have an admin or owner role for the specified dataset's organization.

Delete File

This route allows you to view all user events.

Get All User Events

This route allows you to view the CTR analytics for a dataset.

Get CTR Analytics

This route allows you to view the RAG analytics for a dataset.

Get RAG Analytics

This route allows you to view the recommendation analytics for a dataset.

Get Recommendation Analytics

This route allows you to view the search analytics for a dataset.

Get Search Analytics

This route allows you to view the cluster analytics for a dataset.

Get Cluster Analytics

This route allows you to view the top datasets for a given type.

Get Top Datasets

This route allows you to send user event data to the system.

Send User Event Data

This route allows you to send clickstream data to the system. Clickstream data is used to fine-tune the re-ranking of search results and recommendations.

Send CTR Data

This route allows you to Rate a RAG query.

	OPENAI Cloud	JINA AI Cloud*	JINA (SageMaker)**	TVI Jina	TVI BGE-M3	TVI Nomic
P50 Latency	193.15 ms	179.33 ms	185.21 ms	19.06 ms	14.69 ms	21.36 ms
P90 Latency	261.25 ms	271.87 ms	296.19 ms	23.09 ms	16.90 ms	29.81 ms
P99 Latency	621.05 ms	402.43 ms	306.94 ms	24.27 ms	18.80 ms	30.29 ms
Requests Made	324	324	324	324	324	324
Requests Failed	0	0	3	0	0	0

Get Started

Self Hosting

Guides

API Reference

Introduction

Inspiration

Performance Difference

See more

AWS On-Prem Installation

Creating Embeddings

Custom Embedding Models

SPLADE V2

Get Started

Self Hosting

Guides

API Reference

​Inspiration

​Performance Difference

​See more

AWS On-Prem Installation

Creating Embeddings

Custom Embedding Models

SPLADE V2

Inspiration

Performance Difference

See more