Model Delivery

Guide · Open source · Apache 2.0

Run your own mirror of Hugging Face models with HF_ENDPOINT

Model Delivery is an open-source, self-hosted registry for AI model files: one program, modelctl, that you run on a machine of your own. This page shows how to copy models from Hugging Face into it, and how to make huggingface_hub and the hf command line download from your copy by setting HF_ENDPOINT.

Why mirror the models you depend on

A build, a training job or an inference server that loads a model by name downloads it from huggingface.co. That works until the machine has no route to the internet, the download is rate limited, or the repository is changed or removed. Each of those stops work that has nothing to do with the model itself.

A mirror takes the dependency away. You copy the model once to a server you run, and your tools download from that server from then on. A mirrored model is served with no connection to huggingface.co.

Mirror a model and download it from your own server

Three steps. The quick start in the registry's README has the same flow with Docker.

1. Start the registry

modelctl serve

# On first start it prints an admin token. Client commands read it from here:
export MODELCTL_TOKEN=md_...

The registry keeps its database and the model files in one data directory, ./data unless you say otherwise. It listens on port 8080.

2. Copy a model from Hugging Face

modelctl mirror hf-internal-testing/tiny-random-bert

# A particular branch, tag or revision id:
modelctl mirror openai-community/gpt2 --revision <branch, tag or 40-character revision id>

Every file is checked against the digest Hugging Face publishes for it before it is stored. If any file fails, nothing is recorded: the model does not appear half copied. The revision keeps the id it has on Hugging Face.

3. Point your tools at the registry

export HF_ENDPOINT=http://localhost:8080

hf download hf-internal-testing/tiny-random-bert
python -c "from huggingface_hub import snapshot_download as d; d('hf-internal-testing/tiny-random-bert')"

The files now come from your registry. huggingface_hub reads HF_ENDPOINT when it is imported, so set the variable before starting Python. To go back to Hugging Face, remove it.

Which clients work

The registry implements the part of the Hugging Face API that clients use to download and upload models, not the whole of it. These calls are run against the registry with the real huggingface_hub library in the project's automated tests: snapshot_download, hf_hub_download, hf download, model_info, list_repo_files, list_repo_tree and list_repo_refs, by branch, tag and revision id.

Libraries that download through huggingface_hub, such as transformers and diffusers, use those same calls, so they are expected to work. They are not tested individually yet. The compatibility page lists every call and route that is covered, and the ones that are not.

For a mirrored model the revision id is the upstream commit id, and each file's ETag is the one Hugging Face reports. Pinning a revision id therefore gives the same files from the registry as from Hugging Face.

Datasets

A dataset repository is mirrored the same way, with --repo-type dataset. A model and a dataset of the same name are two repositories, and push, pull, inspect, list and delete take the same flag.

modelctl mirror --repo-type dataset stanfordnlp/imdb

hf download stanfordnlp/imdb --repo-type dataset
python -c "from datasets import load_dataset; load_dataset('stanfordnlp/imdb')"

load_dataset is tested for repositories of csv, jsonl or parquet files. Other layouts are expected to work but are not tested. The Hub's dataset viewer is not implemented, so the library falls back to the repository's own files. It also sends one request, with the user's token, to datasets-server. followed by the registry's host name; the operations guide explains what to do about that name.

Private and gated models

Pass a Hugging Face token that has access to the model. modelctl mirror also reads it from MODELCTL_UPSTREAM_TOKEN or HF_TOKEN. The token is sent with that one request and is not stored.

modelctl mirror meta-llama/Llama-3.2-1B --upstream-token "$HF_TOKEN"

Once mirrored, a gated or private model is an ordinary model in your registry: who can read it is decided by your registry's settings, not by Hugging Face. Honour the model's licence and access terms. See mirroring private and gated models.

Who can download from your mirror

By default anyone who can reach the registry can download and browse, and nobody can write without a token. That suits a home or lab network. Start the server with --require-auth to require a token for downloads too; clients then send a registry token the usual way.

modelctl token create laptop --role read

export HF_ENDPOINT=https://models.example.com
export HF_TOKEN=md_...

The registry serves plain HTTP. Put a reverse proxy in front of it for HTTPS before it is reachable from anywhere beyond a network you trust. The operations guide covers access and HTTPS.

What a mirror does not do

The README lists what is not supported yet in full.

Related guides