Model Delivery

Guide · Open source · Apache 2.0

Serve Hugging Face models on an offline or air-gapped network

Model Delivery is an open-source, self-hosted registry for AI model files: one program, modelctl, that you run on a machine of your own. This page shows how to use it where there is no route to huggingface.co: what the server connects to, how to get models across, and how the tools inside download them.

The problem on a closed network

Tools built on huggingface_hub load a model by name and download it from huggingface.co. On a private, offline or air-gapped network that request has nowhere to go. Copying model directories around by hand works, but every script then needs a local path instead of a model name, and nothing checks that the files are the ones that were published.

A registry inside the network keeps the names. Clients set HF_ENDPOINT to it and download as they would anywhere else, and every file was checked against its digest when it was stored.

What the server connects to, and when

The server makes outbound connections in two cases only:

It makes no other outbound connection. There is no telemetry and no update check, and the web interface loads nothing from elsewhere. A model that has been mirrored is served with no connection to huggingface.co; mirroring a model, cutting the connection and downloading with a standard client is a flow the project runs as an automated test.

Way one: mirror outside, carry the data directory in

Run a registry on a machine that can reach Hugging Face, and mirror what you need. Every file is checked against the digest Hugging Face publishes for it.

# On the connected machine
modelctl serve --data ./data
export MODELCTL_TOKEN=md_...        # the admin token printed on first start
modelctl mirror openai-community/gpt2
modelctl mirror meta-llama/Llama-3.2-1B --upstream-token "$HF_TOKEN"   # a gated model

The data directory is the whole registry: registry.db, which records which files make up which revision and holds the tokens, and blobs/, the model files. Stop the server, copy the directory to whatever you carry across, and start a registry on it inside.

# On the inside machine, with the copied directory
modelctl serve --data /srv/registry/data

Mirrored revisions keep the ids they have on Hugging Face, so a script that pins a revision id works unchanged inside. The tokens come across with the database. Run the same version of modelctl inside, or a newer one: a server refuses to open a data directory written by a newer version than itself.

Carrying the directory across again later replaces the inside registry's database with the connected one's. If you update this way, treat the connected registry as the source and do not push models into the inside one.

Way two: pull outside, push inside

When a registry already runs inside and you want to add one model to it, carry the model's files instead. modelctl pull downloads a revision from a registry and verifies it; modelctl push uploads a directory as a new revision.

# On the connected machine, from the registry you mirrored into
modelctl pull openai-community/gpt2 --dir ./gpt2

# Inside, after carrying ./gpt2 across
export MODELCTL_ENDPOINT=http://models.internal:8080
export MODELCTL_TOKEN=md_...        # a token with the write role
modelctl push ./gpt2 openai-community/gpt2

One difference matters. A pushed revision gets a new id, derived from its content, parent, message and time. It does not keep the id the model has on Hugging Face, so clients inside that pin an upstream revision id will not find it. Download by branch or tag, or use way one, which keeps the ids.

If the inside network can reach one machine that can reach the outside, there is a third way: set --upstream on the inside registry to a Model Delivery registry on that machine, and modelctl mirror on the inside registry then copies from it. See mirroring from somewhere other than Hugging Face.

Download inside the network

export HF_ENDPOINT=http://models.internal:8080

hf download openai-community/gpt2
python -c "from huggingface_hub import snapshot_download as d; d('openai-community/gpt2')"

huggingface_hub reads HF_ENDPOINT when it is imported, so set it before starting Python. Libraries that download through huggingface_hub, such as transformers and diffusers, are expected to work the same way; they are not tested individually yet. The compatibility page lists what is tested.

By default anyone who can reach the registry can download. Start it with --require-auth to require a token, and put a reverse proxy in front for HTTPS: the registry itself serves plain HTTP.

What it does not do

The README has the statement on offline and air-gapped networks under Configuration, and the operations guide covers copying and backing up the data directory.

Related guides