​
Custom or fine tuned models in Trieve Vector Inference

The open source text models on Hugging Face may not be what you always want,

1

Update embedding_models.yaml

To use a private or custom model with Trieve Vector Inference, you will need to update your embedding_models.yaml file.

If the model is a private or gated Hugging Face model, you will need to include your Hugging Face API token.

embedding_models.yaml
...
models:
  ...
  my-custom-model:
    replicas: 1
    revision: main
    modelName: trieve/private-model-example
	hfToken: "hf_**********************************"
...
2

Update your TVI cluster

Update TVI to include your models

helm upgrade -i vector-inference \
    oci://709825985650.dkr.ecr.us-east-1.amazonaws.com/trieve/trieve-embeddings  \
    -f embedding_models.yaml
3

Get embeddings endpoint

kubectl get ing
Working with SPLADE v2Using OpenAI SDK