Build a product inventory with Trieve
In this guide we are going to use the Amazon Berkeley objects dataset (https://amazon-berkeley-objects.s3.amazonaws.com/index.html) to build an inventory to search with TrieveCloning the example repository
Getting the data
CSV Metadata
Image CSV Data
images.csv
is formatted as the following.
Running the ingest script
To run the ingest script you first need to add or modify a .env file withTypes
The first step is to define all the typesImage Hashmap
Since images and json bodies are in separate files, we parse the images into a hashmapProcessing a Singular Chunk
For each row of json we construct aChunk
object, each of which can be pushed into Trieve separately or in bulk. We use the imageHashMap
object constructed from before to get the image_url
for each.
Iterating and sending chunks to Trieve
The last step is we iterate through all the json files, process each line into a Chunk object. We use the/chunk
endpoint to create chunks, in this case we are sending chunks in a batch size of 120 which is the max for this route.