Using Custom Models

Custom or fine tuned models in Trieve Vector Inference

The open source text models on Hugging Face may not be what you always want,

Update embedding_models.yaml

To use a private or custom model with Trieve Vector Inference, you will need to update your embedding_models.yaml file.If the model is a private or gated Hugging Face model, you will need to include your Hugging Face API token.

embedding_models.yaml

...
models:
  ...
  my-custom-model:
    replicas: 1
    revision: main
    modelName: trieve/private-model-example
	hfToken: "hf_**********************************"
...

Update your TVI cluster

Update TVI to include your models

helm upgrade -i vector-inference \
    oci://709825985650.dkr.ecr.us-east-1.amazonaws.com/trieve/trieve-embeddings  \
    -f embedding_models.yaml

Get embeddings endpoint

kubectl get ing

Working with SPLADE v2 Using OpenAI SDK

​Custom or fine tuned models in Trieve Vector Inference

Custom or fine tuned models in Trieve Vector Inference