Model Delivery

Open source · Apache 2.0 · Your models. Your infrastructure. Your rules.

A self-hosted registry and Hugging Face mirror for AI models

Model Delivery mirrors models and datasets from Hugging Face once, then serves them from your own machines to the tools you already use. Point HF_ENDPOINT at it and huggingface_hub, the hf command line and load_dataset download as before.

Get started View on GitHub

Mirror a Hugging Face model in four commands

# 1. Run the registry (one binary; prints an admin token on first start)
modelctl serve

# 2. Copy a model from Hugging Face, verified file by file
modelctl mirror hf-internal-testing/tiny-random-bert

# 3. Point your existing tools at your registry
export HF_ENDPOINT=http://localhost:8080

# 4. Download exactly as before. This never contacts huggingface.co.
python -c "from huggingface_hub import snapshot_download as d; d('hf-internal-testing/tiny-random-bert')"

Anything built on huggingface_hub reads HF_ENDPOINT, so the same switch works for hf download and for libraries that download through it, such as transformers and diffusers.

Why run your own model registry

It keeps working

Builds, training jobs and servers fetch models from inside your network. Once a model is mirrored, an outage, a rate limit or a removed repository somewhere else does not stop you. A mirrored model is served with no connection to huggingface.co, so the registry also works on a private, offline or air-gapped network.

The same bytes every time

Every revision is immutable and keeps the id it has upstream. Pin a revision and you get exactly those files, this year and next.

Nothing to rewrite

It speaks the download API Hugging Face clients already use. Switching is one environment variable, and switching back is removing it.

What it does

Mirror
Copy a repository from Hugging Face at a branch, tag or revision id. Private and gated models work with your own Hugging Face token.
Datasets
Everything here works for dataset repositories too: mirror them with modelctl mirror --repo-type dataset, cache them on demand, push, pull and upload them, and serve them to snapshot_download(repo_type="dataset"), hf download --repo-type dataset and load_dataset.
Pull-through cache
Optionally, run it as a caching proxy for Hugging Face: the registry fetches each file the first time someone downloads it, sending it to the client while it arrives, then serves it locally from then on. Ask for one file of a large repository and it fetches one file.
Verify
Every file is checked against the digest the hub publishes before it is stored. A mirror that fails verification records nothing, and modelctl verify re-checks everything in storage later, to find damage.
Serve
Existing clients download through the compatible API, including range requests.
Upload
Store your own models too, with hf upload and the Hugging Face libraries, or with modelctl push.
Deduplicate
Files are stored by the SHA-256 of their content, so identical files are kept once across revisions and repositories.
Store anywhere
Local disk out of the box, or an S3-compatible bucket such as Cloudflare R2, with optional direct downloads from the bucket.
Control access
Run it as a private registry: tokens with read, write and admin roles. Downloads can be open on your network or require a token.
Monitor
GET /metrics is a Prometheus endpoint: cache hit ratio, upstream failures, what is stored and pending, and the age of the last backup. No metric names a repository or a file.
Browse
A small web interface for repositories, revisions, files and storage use.

Guides

Who uses it

Otium runs batch AI inference on spare cloud GPUs, and every worker it starts has to load model weights before it can do any work. Otium uses Model Delivery as the origin for those weights: its caches fill from the registry, each file checked against its digest, and fall back to Hugging Face if the registry cannot be reached. Otium is built by Spectrum Labs, the company behind Model Delivery, and is the reason the registry exists.

Running it yourself? Open an issue on the repository and say how; that is what decides what gets built next.

One binary, on a Raspberry Pi or a server

Think of a Docker registry or Artifactory, built for model files. One binary with SQLite inside. No database server, queue or cluster to operate. It runs on a Raspberry Pi, a homelab box or a server, on Linux, macOS and Windows.

docker run -d -p 8080:8080 -v registry-data:/data ghcr.io/model-delivery/registry

What it does not do yet

It is an early release. Better to read this here than to find out later.

The compatibility page lists exactly which Hugging Face client calls are tested, and the roadmap says what comes next.

Documentation