Upload File

Authorizations

Authorization

string

header

required

Headers

TR-Dataset

string<uuid>

required

The dataset id or tracking_id to use for the request. We assume you intend to use an id if the value is a valid uuid.

Body

application/json

JSON request payload to upload a file

base64_file

string

required

Base64 encoded file.

file_name

string

required

Name of the file being uploaded, including the extension.

chunkr_create_task_req_payload

object

Will use chunkr.ai to process the file when this object is defined. See docs.chunkr.ai/api-references/task/create-task for detailed information about what each field on this request payload does.

Show child attributes

chunkr_create_task_req_payload.chunk_processing

object

Controls the setting for the chunking and post-processing of each chunk.

Show child attributes

chunkr_create_task_req_payload.chunk_processing.ignore_headers_and_footers

boolean

default:true

Whether to ignore headers and footers in the chunking process. This is recommended as headers and footers break reading order across pages.

chunkr_create_task_req_payload.chunk_processing.target_length

integer<int32>

default:512

The target number of words in each chunk. If 0, each chunk will contain a single segment.

Required range: x >= 0

chunkr_create_task_req_payload.chunk_processing.tokenizer

object

Specifies which tokenizer to use for the chunking process.

This type supports two ways of specifying a tokenizer:

Using a predefined tokenizer from the Tokenizer enum
Using any Hugging Face tokenizer by providing its model ID as a string (e.g. "facebook/bart-large", "Qwen/Qwen-tokenizer", etc.)

When using a string, any valid Hugging Face tokenizer ID can be specified, which will be loaded using the Hugging Face tokenizers library.

Option 1
Option 2

Show child attributes

chunkr_create_task_req_payload.chunk_processing.tokenizer.Enum

enum<string>

required

Common tokenizers used for text processing.

These values represent standard tokenization approaches and popular pre-trained tokenizers from the Hugging Face ecosystem.

Available options:

Word,

Cl100kBase,

XlmRobertaBase,

BertBaseUncased

chunkr_create_task_req_payload.error_handling

enum<string>

Controls how errors are handled during processing:

Fail: Stops processing and fails the task when any error occurs
Continue: Attempts to continue processing despite non-critical errors (eg. LLM refusals etc.)

Available options:

Fail,

Continue

chunkr_create_task_req_payload.expires_in

integer<int32> | null

The number of seconds until task is deleted. Expried tasks can not be updated, polled or accessed via web interface.

chunkr_create_task_req_payload.high_resolution

boolean | null

default:false

Whether to use high-resolution images for cropping and post-processing. (Latency penalty: ~7 seconds per page)

chunkr_create_task_req_payload.llm_processing

object

Controls the LLM used for the task.

Show child attributes

chunkr_create_task_req_payload.llm_processing.fallback_strategy

Specifies the fallback strategy for LLM processing

This can be:

None - No fallback will be used
Default - The system default fallback model will be used
Model - A specific model ID will be used as fallback (check the documentation for the models.)

Available options:

None

chunkr_create_task_req_payload.llm_processing.max_completion_tokens

integer<int32> | null

The maximum number of tokens to generate.

Required range: x >= 0

chunkr_create_task_req_payload.llm_processing.model_id

string | null

The ID of the model to use for the task. If not provided, the default model will be used. Please check the documentation for the model you want to use.

chunkr_create_task_req_payload.llm_processing.temperature

number<float>

The temperature to use for the LLM.

chunkr_create_task_req_payload.ocr_strategy

enum<string>

default:All

Controls the Optical Character Recognition (OCR) strategy.

All: Processes all pages with OCR. (Latency penalty: ~0.5 seconds per page)
Auto: Selectively applies OCR only to pages with missing or low-quality text. When text layer is present the bounding boxes from the text layer are used.

Available options:

All,

Auto

chunkr_create_task_req_payload.pipeline

enum<string>

default:Azure

Available options:

Azure,

Chunkr

chunkr_create_task_req_payload.segment_processing

object

Controls the post-processing of each segment type.

Allows you to generate HTML and Markdown from chunkr models for each segment type. By default, the HTML and Markdown are generated manually using the segmentation information except for Table, Formula and Picture. You can optionally configure custom LLM prompts and models to generate an additional llm field with LLM-processed content for each segment type.

The configuration of which content sources (HTML, Markdown, LLM, Content) of the segment should be included in the chunk's embed field and counted towards the chunk length can be configured through the embed_sources setting.

Show child attributes

chunkr_create_task_req_payload.segment_processing.Caption

object

Controls the processing and generation for the segment.

crop_image controls whether to crop the file's images to the segment's bounding box. The cropped image will be stored in the segment's image field. Use All to always crop, or Auto to only crop when needed for post-processing.
html is the HTML output for the segment, generated either through huerstics (Auto) or using Chunkr fine-tuned models (LLM)
llm is the LLM-generated output for the segment, this uses off-the-shelf models to generate a custom output for the segment
markdown is the Markdown output for the segment, generated either through huerstics (Auto) or using Chunkr fine-tuned models (LLM)
embed_sources defines which content sources will be included in the chunk's embed field and counted towards the chunk length. The array's order determines the sequence in which content appears in the embed field (e.g., [Markdown, LLM] means Markdown content is followed by LLM content). This directly affects what content is available for embedding and retrieval.

Show child attributes

chunkr_create_task_req_payload.segment_processing.Caption.crop_image

enum<string>

default:Auto

Controls the cropping strategy for an item (e.g. segment, chunk, etc.)

All crops all images in the item
Auto crops images only if required for post-processing

Available options:

All,

Auto

chunkr_create_task_req_payload.segment_processing.Caption.embed_sources

enum<string>[]

Available options:

HTML,

Markdown,

LLM,

Content

chunkr_create_task_req_payload.segment_processing.Caption.html

enum<string>

default:Auto

Available options:

LLM,

Auto

chunkr_create_task_req_payload.segment_processing.Caption.llm

string | null

Prompt for the LLM mode

chunkr_create_task_req_payload.segment_processing.Caption.markdown

enum<string>

default:Auto

Available options:

LLM,

Auto

chunkr_create_task_req_payload.segment_processing.Footnote

object

Controls the processing and generation for the segment.

crop_image controls whether to crop the file's images to the segment's bounding box. The cropped image will be stored in the segment's image field. Use All to always crop, or Auto to only crop when needed for post-processing.
html is the HTML output for the segment, generated either through huerstics (Auto) or using Chunkr fine-tuned models (LLM)
llm is the LLM-generated output for the segment, this uses off-the-shelf models to generate a custom output for the segment
markdown is the Markdown output for the segment, generated either through huerstics (Auto) or using Chunkr fine-tuned models (LLM)
embed_sources defines which content sources will be included in the chunk's embed field and counted towards the chunk length. The array's order determines the sequence in which content appears in the embed field (e.g., [Markdown, LLM] means Markdown content is followed by LLM content). This directly affects what content is available for embedding and retrieval.

Show child attributes

chunkr_create_task_req_payload.segment_processing.Footnote.crop_image

enum<string>

default:Auto

Controls the cropping strategy for an item (e.g. segment, chunk, etc.)

All crops all images in the item
Auto crops images only if required for post-processing

Available options:

All,

Auto

chunkr_create_task_req_payload.segment_processing.Footnote.embed_sources

enum<string>[]

Available options:

HTML,

Markdown,

LLM,

Content

chunkr_create_task_req_payload.segment_processing.Footnote.html

enum<string>

default:Auto

Available options:

LLM,

Auto

chunkr_create_task_req_payload.segment_processing.Footnote.llm

string | null

Prompt for the LLM mode

chunkr_create_task_req_payload.segment_processing.Footnote.markdown

enum<string>

default:Auto

Available options:

LLM,

Auto

chunkr_create_task_req_payload.segment_processing.Formula

object

Controls the processing and generation for the segment.

crop_image controls whether to crop the file's images to the segment's bounding box. The cropped image will be stored in the segment's image field. Use All to always crop, or Auto to only crop when needed for post-processing.
html is the HTML output for the segment, generated either through huerstics (Auto) or using Chunkr fine-tuned models (LLM)
llm is the LLM-generated output for the segment, this uses off-the-shelf models to generate a custom output for the segment
markdown is the Markdown output for the segment, generated either through huerstics (Auto) or using Chunkr fine-tuned models (LLM)
embed_sources defines which content sources will be included in the chunk's embed field and counted towards the chunk length. The array's order determines the sequence in which content appears in the embed field (e.g., [Markdown, LLM] means Markdown content is followed by LLM content). This directly affects what content is available for embedding and retrieval.

Show child attributes

chunkr_create_task_req_payload.segment_processing.Formula.crop_image

enum<string>

default:Auto

Controls the cropping strategy for an item (e.g. segment, chunk, etc.)

All crops all images in the item
Auto crops images only if required for post-processing

Available options:

All,

Auto

chunkr_create_task_req_payload.segment_processing.Formula.embed_sources

enum<string>[]

Available options:

HTML,

Markdown,

LLM,

Content

chunkr_create_task_req_payload.segment_processing.Formula.html

enum<string>

default:LLM

Available options:

LLM,

Auto

chunkr_create_task_req_payload.segment_processing.Formula.llm

string | null

Prompt for the LLM model

chunkr_create_task_req_payload.segment_processing.Formula.markdown

enum<string>

default:LLM

Available options:

LLM,

Auto

chunkr_create_task_req_payload.segment_processing.ListItem

object

Controls the processing and generation for the segment.

crop_image controls whether to crop the file's images to the segment's bounding box. The cropped image will be stored in the segment's image field. Use All to always crop, or Auto to only crop when needed for post-processing.
html is the HTML output for the segment, generated either through huerstics (Auto) or using Chunkr fine-tuned models (LLM)
llm is the LLM-generated output for the segment, this uses off-the-shelf models to generate a custom output for the segment
markdown is the Markdown output for the segment, generated either through huerstics (Auto) or using Chunkr fine-tuned models (LLM)
embed_sources defines which content sources will be included in the chunk's embed field and counted towards the chunk length. The array's order determines the sequence in which content appears in the embed field (e.g., [Markdown, LLM] means Markdown content is followed by LLM content). This directly affects what content is available for embedding and retrieval.

Show child attributes

chunkr_create_task_req_payload.segment_processing.ListItem.crop_image

enum<string>

default:Auto

Controls the cropping strategy for an item (e.g. segment, chunk, etc.)

All crops all images in the item
Auto crops images only if required for post-processing

Available options:

All,

Auto

chunkr_create_task_req_payload.segment_processing.ListItem.embed_sources

enum<string>[]

Available options:

HTML,

Markdown,

LLM,

Content

chunkr_create_task_req_payload.segment_processing.ListItem.html

enum<string>

default:Auto

Available options:

LLM,

Auto

chunkr_create_task_req_payload.segment_processing.ListItem.llm

string | null

Prompt for the LLM mode

chunkr_create_task_req_payload.segment_processing.ListItem.markdown

enum<string>

default:Auto

Available options:

LLM,

Auto

chunkr_create_task_req_payload.segment_processing.Page

object

Controls the processing and generation for the segment.

crop_image controls whether to crop the file's images to the segment's bounding box. The cropped image will be stored in the segment's image field. Use All to always crop, or Auto to only crop when needed for post-processing.
html is the HTML output for the segment, generated either through huerstics (Auto) or using Chunkr fine-tuned models (LLM)
llm is the LLM-generated output for the segment, this uses off-the-shelf models to generate a custom output for the segment
markdown is the Markdown output for the segment, generated either through huerstics (Auto) or using Chunkr fine-tuned models (LLM)
embed_sources defines which content sources will be included in the chunk's embed field and counted towards the chunk length. The array's order determines the sequence in which content appears in the embed field (e.g., [Markdown, LLM] means Markdown content is followed by LLM content). This directly affects what content is available for embedding and retrieval.

Show child attributes

chunkr_create_task_req_payload.segment_processing.Page.crop_image

enum<string>

default:Auto

Controls the cropping strategy for an item (e.g. segment, chunk, etc.)

All crops all images in the item
Auto crops images only if required for post-processing

Available options:

All,

Auto

chunkr_create_task_req_payload.segment_processing.Page.embed_sources

enum<string>[]

Available options:

HTML,

Markdown,

LLM,

Content

chunkr_create_task_req_payload.segment_processing.Page.html

enum<string>

default:LLM

Available options:

LLM,

Auto

chunkr_create_task_req_payload.segment_processing.Page.llm

string | null

Prompt for the LLM model

chunkr_create_task_req_payload.segment_processing.Page.markdown

enum<string>

default:LLM

Available options:

LLM,

Auto

chunkr_create_task_req_payload.segment_processing.PageFooter

object

Controls the processing and generation for the segment.

crop_image controls whether to crop the file's images to the segment's bounding box. The cropped image will be stored in the segment's image field. Use All to always crop, or Auto to only crop when needed for post-processing.
html is the HTML output for the segment, generated either through huerstics (Auto) or using Chunkr fine-tuned models (LLM)
llm is the LLM-generated output for the segment, this uses off-the-shelf models to generate a custom output for the segment
markdown is the Markdown output for the segment, generated either through huerstics (Auto) or using Chunkr fine-tuned models (LLM)
embed_sources defines which content sources will be included in the chunk's embed field and counted towards the chunk length. The array's order determines the sequence in which content appears in the embed field (e.g., [Markdown, LLM] means Markdown content is followed by LLM content). This directly affects what content is available for embedding and retrieval.

Show child attributes

chunkr_create_task_req_payload.segment_processing.PageFooter.crop_image

enum<string>

default:Auto

Controls the cropping strategy for an item (e.g. segment, chunk, etc.)

All crops all images in the item
Auto crops images only if required for post-processing

Available options:

All,

Auto

chunkr_create_task_req_payload.segment_processing.PageFooter.embed_sources

enum<string>[]

Available options:

HTML,

Markdown,

LLM,

Content

chunkr_create_task_req_payload.segment_processing.PageFooter.html

enum<string>

default:Auto

Available options:

LLM,

Auto

chunkr_create_task_req_payload.segment_processing.PageFooter.llm

string | null

Prompt for the LLM mode

chunkr_create_task_req_payload.segment_processing.PageFooter.markdown

enum<string>

default:Auto

Available options:

LLM,

Auto

chunkr_create_task_req_payload.segment_processing.PageHeader

object

Controls the processing and generation for the segment.

crop_image controls whether to crop the file's images to the segment's bounding box. The cropped image will be stored in the segment's image field. Use All to always crop, or Auto to only crop when needed for post-processing.
html is the HTML output for the segment, generated either through huerstics (Auto) or using Chunkr fine-tuned models (LLM)
llm is the LLM-generated output for the segment, this uses off-the-shelf models to generate a custom output for the segment
markdown is the Markdown output for the segment, generated either through huerstics (Auto) or using Chunkr fine-tuned models (LLM)
embed_sources defines which content sources will be included in the chunk's embed field and counted towards the chunk length. The array's order determines the sequence in which content appears in the embed field (e.g., [Markdown, LLM] means Markdown content is followed by LLM content). This directly affects what content is available for embedding and retrieval.

Show child attributes

chunkr_create_task_req_payload.segment_processing.PageHeader.crop_image

enum<string>

default:Auto

Controls the cropping strategy for an item (e.g. segment, chunk, etc.)

All crops all images in the item
Auto crops images only if required for post-processing

Available options:

All,

Auto

chunkr_create_task_req_payload.segment_processing.PageHeader.embed_sources

enum<string>[]

Available options:

HTML,

Markdown,

LLM,

Content

chunkr_create_task_req_payload.segment_processing.PageHeader.html

enum<string>

default:Auto

Available options:

LLM,

Auto

chunkr_create_task_req_payload.segment_processing.PageHeader.llm

string | null

Prompt for the LLM mode

chunkr_create_task_req_payload.segment_processing.PageHeader.markdown

enum<string>

default:Auto

Available options:

LLM,

Auto

chunkr_create_task_req_payload.segment_processing.Picture

object

Controls the processing and generation for the segment.

crop_image controls whether to crop the file's images to the segment's bounding box. The cropped image will be stored in the segment's image field. Use All to always crop, or Auto to only crop when needed for post-processing.
html is the HTML output for the segment, generated either through huerstics (Auto) or using Chunkr fine-tuned models (LLM)
llm is the LLM-generated output for the segment, this uses off-the-shelf models to generate a custom output for the segment
markdown is the Markdown output for the segment, generated either through huerstics (Auto) or using Chunkr fine-tuned models (LLM)
embed_sources defines which content sources will be included in the chunk's embed field and counted towards the chunk length. The array's order determines the sequence in which content appears in the embed field (e.g., [Markdown, LLM] means Markdown content is followed by LLM content). This directly affects what content is available for embedding and retrieval.

Show child attributes

chunkr_create_task_req_payload.segment_processing.Picture.crop_image

enum<string>

default:All

Controls the cropping strategy for an item (e.g. segment, chunk, etc.)

All crops all images in the item
Auto crops images only if required for post-processing

Available options:

All,

Auto

chunkr_create_task_req_payload.segment_processing.Picture.embed_sources

enum<string>[]

Available options:

HTML,

Markdown,

LLM,

Content

chunkr_create_task_req_payload.segment_processing.Picture.html

enum<string>

default:LLM

Available options:

LLM,

Auto

chunkr_create_task_req_payload.segment_processing.Picture.llm

string | null

Prompt for the LLM model

chunkr_create_task_req_payload.segment_processing.Picture.markdown

enum<string>

default:LLM

Available options:

LLM,

Auto

chunkr_create_task_req_payload.segment_processing.SectionHeader

object

Controls the processing and generation for the segment.

crop_image controls whether to crop the file's images to the segment's bounding box. The cropped image will be stored in the segment's image field. Use All to always crop, or Auto to only crop when needed for post-processing.
html is the HTML output for the segment, generated either through huerstics (Auto) or using Chunkr fine-tuned models (LLM)
llm is the LLM-generated output for the segment, this uses off-the-shelf models to generate a custom output for the segment
markdown is the Markdown output for the segment, generated either through huerstics (Auto) or using Chunkr fine-tuned models (LLM)
embed_sources defines which content sources will be included in the chunk's embed field and counted towards the chunk length. The array's order determines the sequence in which content appears in the embed field (e.g., [Markdown, LLM] means Markdown content is followed by LLM content). This directly affects what content is available for embedding and retrieval.

Show child attributes

chunkr_create_task_req_payload.segment_processing.SectionHeader.crop_image

enum<string>

default:Auto

Controls the cropping strategy for an item (e.g. segment, chunk, etc.)

All crops all images in the item
Auto crops images only if required for post-processing

Available options:

All,

Auto

chunkr_create_task_req_payload.segment_processing.SectionHeader.embed_sources

enum<string>[]

Available options:

HTML,

Markdown,

LLM,

Content

chunkr_create_task_req_payload.segment_processing.SectionHeader.html

enum<string>

default:Auto

Available options:

LLM,

Auto

chunkr_create_task_req_payload.segment_processing.SectionHeader.llm

string | null

Prompt for the LLM mode

chunkr_create_task_req_payload.segment_processing.SectionHeader.markdown

enum<string>

default:Auto

Available options:

LLM,

Auto

chunkr_create_task_req_payload.segment_processing.Table

object

Controls the processing and generation for the segment.

crop_image controls whether to crop the file's images to the segment's bounding box. The cropped image will be stored in the segment's image field. Use All to always crop, or Auto to only crop when needed for post-processing.
html is the HTML output for the segment, generated either through huerstics (Auto) or using Chunkr fine-tuned models (LLM)
llm is the LLM-generated output for the segment, this uses off-the-shelf models to generate a custom output for the segment
markdown is the Markdown output for the segment, generated either through huerstics (Auto) or using Chunkr fine-tuned models (LLM)
embed_sources defines which content sources will be included in the chunk's embed field and counted towards the chunk length. The array's order determines the sequence in which content appears in the embed field (e.g., [Markdown, LLM] means Markdown content is followed by LLM content). This directly affects what content is available for embedding and retrieval.

Show child attributes

chunkr_create_task_req_payload.segment_processing.Table.crop_image

enum<string>

default:Auto

Controls the cropping strategy for an item (e.g. segment, chunk, etc.)

All crops all images in the item
Auto crops images only if required for post-processing

Available options:

All,

Auto

chunkr_create_task_req_payload.segment_processing.Table.embed_sources

enum<string>[]

Available options:

HTML,

Markdown,

LLM,

Content

chunkr_create_task_req_payload.segment_processing.Table.html

enum<string>

default:LLM

Available options:

LLM,

Auto

chunkr_create_task_req_payload.segment_processing.Table.llm

string | null

Prompt for the LLM model

chunkr_create_task_req_payload.segment_processing.Table.markdown

enum<string>

default:LLM

Available options:

LLM,

Auto

chunkr_create_task_req_payload.segment_processing.Text

object

Controls the processing and generation for the segment.

crop_image controls whether to crop the file's images to the segment's bounding box. The cropped image will be stored in the segment's image field. Use All to always crop, or Auto to only crop when needed for post-processing.
html is the HTML output for the segment, generated either through huerstics (Auto) or using Chunkr fine-tuned models (LLM)
llm is the LLM-generated output for the segment, this uses off-the-shelf models to generate a custom output for the segment
markdown is the Markdown output for the segment, generated either through huerstics (Auto) or using Chunkr fine-tuned models (LLM)
embed_sources defines which content sources will be included in the chunk's embed field and counted towards the chunk length. The array's order determines the sequence in which content appears in the embed field (e.g., [Markdown, LLM] means Markdown content is followed by LLM content). This directly affects what content is available for embedding and retrieval.

Show child attributes

chunkr_create_task_req_payload.segment_processing.Text.crop_image

enum<string>

default:Auto

Controls the cropping strategy for an item (e.g. segment, chunk, etc.)

All crops all images in the item
Auto crops images only if required for post-processing

Available options:

All,

Auto

chunkr_create_task_req_payload.segment_processing.Text.embed_sources

enum<string>[]

Available options:

HTML,

Markdown,

LLM,

Content

chunkr_create_task_req_payload.segment_processing.Text.html

enum<string>

default:Auto

Available options:

LLM,

Auto

chunkr_create_task_req_payload.segment_processing.Text.llm

string | null

Prompt for the LLM mode

chunkr_create_task_req_payload.segment_processing.Text.markdown

enum<string>

default:Auto

Available options:

LLM,

Auto

chunkr_create_task_req_payload.segment_processing.Title

object

Controls the processing and generation for the segment.

crop_image controls whether to crop the file's images to the segment's bounding box. The cropped image will be stored in the segment's image field. Use All to always crop, or Auto to only crop when needed for post-processing.
html is the HTML output for the segment, generated either through huerstics (Auto) or using Chunkr fine-tuned models (LLM)
llm is the LLM-generated output for the segment, this uses off-the-shelf models to generate a custom output for the segment
markdown is the Markdown output for the segment, generated either through huerstics (Auto) or using Chunkr fine-tuned models (LLM)
embed_sources defines which content sources will be included in the chunk's embed field and counted towards the chunk length. The array's order determines the sequence in which content appears in the embed field (e.g., [Markdown, LLM] means Markdown content is followed by LLM content). This directly affects what content is available for embedding and retrieval.

Show child attributes

chunkr_create_task_req_payload.segment_processing.Title.crop_image

enum<string>

default:Auto

Controls the cropping strategy for an item (e.g. segment, chunk, etc.)

All crops all images in the item
Auto crops images only if required for post-processing

Available options:

All,

Auto

chunkr_create_task_req_payload.segment_processing.Title.embed_sources

enum<string>[]

Available options:

HTML,

Markdown,

LLM,

Content

chunkr_create_task_req_payload.segment_processing.Title.html

enum<string>

default:Auto

Available options:

LLM,

Auto

chunkr_create_task_req_payload.segment_processing.Title.llm

string | null

Prompt for the LLM mode

chunkr_create_task_req_payload.segment_processing.Title.markdown

enum<string>

default:Auto

Available options:

LLM,

Auto

chunkr_create_task_req_payload.segmentation_strategy

enum<string>

default:LayoutAnalysis

Controls the segmentation strategy:

LayoutAnalysis: Analyzes pages for layout elements (e.g., Table, Picture, Formula, etc.) using bounding boxes. Provides fine-grained segmentation and better chunking. (Latency penalty: ~TBD seconds per page).
Page: Treats each page as a single segment. Faster processing, but without layout element detection and only simple chunking.

Available options:

LayoutAnalysis,

Page

create_chunks

boolean | null

Create chunks is a boolean which determines whether or not to create chunks from the file. If false, you can manually chunk the file and send the chunks to the create_chunk endpoint with the file_id to associate chunks with the file. Meant mostly for advanced users.

description

string | null

Description is an optional convience field so you do not have to remember what the file contains or is about. It will be included on the group resulting from the file which will hold its chunk.

group_tracking_id

string | null

Group tracking id is an optional field which allows you to specify the tracking id of the group that is created from the file. Chunks created will be created with the tracking id of group_tracking_id|<index of chunk>

link

string | null

Link to the file. This can also be any string. This can be used to filter when searching for the file's resulting chunks. The link value will not affect embedding creation.

metadata

any | null

Metadata is a JSON object which can be used to filter chunks. This is useful for when you want to filter chunks by arbitrary metadata. Unlike with tag filtering, there is a performance hit for filtering on metadata. Will be passed down to the file's chunks.

pdf2md_options

object

We plan to deprecate pdf2md in favor of chunkr.ai. This is a legacy option for using a vision LLM to convert a given file into markdown and then ingest it.

Show child attributes

pdf2md_options.use_pdf2md_ocr

boolean

required

Parameter to use pdf2md_ocr. If true, the file will be converted to markdown using gpt-4o. Default is false.

pdf2md_options.split_headings

boolean | null

Split headings is an optional field which allows you to specify whether or not to split headings into separate chunks. Default is false.

pdf2md_options.system_prompt

string | null

Prompt to use for the gpt-4o model. Default is None.

rebalance_chunks

boolean | null

Rebalance chunks is an optional field which allows you to specify whether or not to rebalance the chunks created from the file. If not specified, the default true is used. If true, Trieve will evenly distribute remainder splits across chunks such that 66 splits with a target_splits_per_chunk of 20 will result in 3 chunks with 22 splits each.

split_avg

boolean | null

Split average will automatically split your file into multiple chunks and average all of the resulting vectors into a single output chunk. Default is false. Explicitly enabling this will cause each file to only produce a single chunk.

split_delimiters

string[] | null

Split delimiters is an optional field which allows you to specify the delimiters to use when splitting the file before chunking the text. If not specified, the default [.!?\n] are used to split into sentences. However, you may want to use spaces or other delimiters.

tag_set

string[] | null

Tag set is a comma separated list of tags which will be passed down to the chunks made from the file. Tags are used to filter chunks when searching. HNSW indices are created for each tag such that there is no performance loss when filtering on them.

target_splits_per_chunk

integer | null

Target splits per chunk. This is an optional field which allows you to specify the number of splits you want per chunk. If not specified, the default 20 is used. However, you may want to use a different number.

Required range: x >= 0

time_stamp

string | null

Time stamp should be an ISO 8601 combined date and time without timezone. Time_stamp is used for time window filtering and recency-biasing search results. Will be passed down to the file's chunks.

webhook_url

string | null

Optional webhook URL to receive notifications for each page processed.

Response

Confirmation that the file is uploading

file_metadata

object

required

Show child attributes

file_metadata.created_at

string<date-time>

required

file_metadata.dataset_id

string<uuid>

required

file_metadata.file_name

string

required

file_metadata.id

string<uuid>

required

file_metadata.size

integer<int64>

required

file_metadata.updated_at

string<date-time>

required

file_metadata.link

string | null

file_metadata.metadata

unknown

file_metadata.tag_set

(string | null)[] | null

file_metadata.time_stamp

string<date-time> | null

Example:

{
  "created_at": "2021-01-01 00:00:00.000",
  "dataset_id": "e3e3e3e3-e3e3-e3e3-e3e3-e3e3e3e3e3e3",
  "file_name": "file.txt",
  "id": "e3e3e3e3-e3e3-e3e3-e3e3-e3e3e3e3e3e3",
  "link": "https://trieve.ai",
  "metadata": { "key": "value" },
  "size": 1000,
  "tag_set": "tag1,tag2",
  "time_stamp": "2021-01-01 00:00:00.000",
  "updated_at": "2021-01-01 00:00:00.000"
}

Chunk

Chunk Group

Topic

Message

Crawl

File

Analytics

Experiments

Dataset

Organization

User

Auth

Health

Invitation

Stripe

Metrics

Public

Authorizations

Headers

Body

Response