​
Custom or fine tuned models in Trieve Vector Inference

The open source text models on hugging face may not be what you always want,

1

Update embedding_models.yaml

To use a private or custom model with Trieve Vector Inference, you will need to update your embedding_models.yaml file.

If the model is a private or gated hugging face model, you will need to include your huggingface api token

embedding_models.yaml
...
models:
  ...
  my-custom-model:
    replicas: 1
    revision: main
    modelName: trieve/private-model-example
	hfToken: "hf_**********************************"
...
2

Update your TVI cluster

Update TVI to include your models

helm upgrade -i vector-inference \
    oci://registry-1.docker.io/trieve/embeddings-helm \
    -f embedding_models.yaml
3

Get embeddings endpoint

kubectl get ing
Working with Splade v2Using OpenAI SDK