Guide · Open source · Apache 2.0
Serve Hugging Face models on an offline or air-gapped network
Model Delivery is an open-source, self-hosted registry for AI model files: one program, modelctl, that you run on a machine of your own. This page shows how to use it where there is no route to huggingface.co: what the server connects to, how to get models across, and how the tools inside download them.
The problem on a closed network
Tools built on huggingface_hub load a model by name and download it from huggingface.co. On a private, offline or air-gapped network that request has nowhere to go. Copying model directories around by hand works, but every script then needs a local path instead of a model name, and nothing checks that the files are the ones that were published.
A registry inside the network keeps the names. Clients set HF_ENDPOINT to it and download as they would anywhere else, and every file was checked against its digest when it was stored.
What the server connects to, and when
The server makes outbound connections in two cases only:
- The upstream hub, Hugging Face unless you set another, and only when you run
modelctl mirroror turn on pull-through caching. Pull-through is off by default. - Its storage bucket, if you configured S3 storage. With the default storage, files are on local disk.
It makes no other outbound connection. There is no telemetry and no update check, and the web interface loads nothing from elsewhere. A model that has been mirrored is served with no connection to huggingface.co; mirroring a model, cutting the connection and downloading with a standard client is a flow the project runs as an automated test.
Way one: mirror outside, carry the data directory in
Run a registry on a machine that can reach Hugging Face, and mirror what you need. Every file is checked against the digest Hugging Face publishes for it.
# On the connected machine
modelctl serve --data ./data
export MODELCTL_TOKEN=md_... # the admin token printed on first start
modelctl mirror openai-community/gpt2
modelctl mirror meta-llama/Llama-3.2-1B --upstream-token "$HF_TOKEN" # a gated model
The data directory is the whole registry: registry.db, which records which files make up which revision and holds the tokens, and blobs/, the model files. Stop the server, copy the directory to whatever you carry across, and start a registry on it inside.
# On the inside machine, with the copied directory
modelctl serve --data /srv/registry/data
Mirrored revisions keep the ids they have on Hugging Face, so a script that pins a revision id works unchanged inside. The tokens come across with the database. Run the same version of modelctl inside, or a newer one: a server refuses to open a data directory written by a newer version than itself.
Carrying the directory across again later replaces the inside registry's database with the connected one's. If you update this way, treat the connected registry as the source and do not push models into the inside one.
Way two: pull outside, push inside
When a registry already runs inside and you want to add one model to it, carry the model's files instead. modelctl pull downloads a revision from a registry and verifies it; modelctl push uploads a directory as a new revision.
# On the connected machine, from the registry you mirrored into
modelctl pull openai-community/gpt2 --dir ./gpt2
# Inside, after carrying ./gpt2 across
export MODELCTL_ENDPOINT=http://models.internal:8080
export MODELCTL_TOKEN=md_... # a token with the write role
modelctl push ./gpt2 openai-community/gpt2
One difference matters. A pushed revision gets a new id, derived from its content, parent, message and time. It does not keep the id the model has on Hugging Face, so clients inside that pin an upstream revision id will not find it. Download by branch or tag, or use way one, which keeps the ids.
If the inside network can reach one machine that can reach the outside, there is a third way: set --upstream on the inside registry to a Model Delivery registry on that machine, and modelctl mirror on the inside registry then copies from it. See mirroring from somewhere other than Hugging Face.
Download inside the network
export HF_ENDPOINT=http://models.internal:8080
hf download openai-community/gpt2
python -c "from huggingface_hub import snapshot_download as d; d('openai-community/gpt2')"
huggingface_hub reads HF_ENDPOINT when it is imported, so set it before starting Python. Libraries that download through huggingface_hub, such as transformers and diffusers, are expected to work the same way; they are not tested individually yet. The compatibility page lists what is tested.
By default anyone who can reach the registry can download. Start it with --require-auth to require a token, and put a reverse proxy in front for HTTPS: the registry itself serves plain HTTP.
What it does not do
- It does not update itself. A mirror is a copy made at one moment. A newer revision reaches the inside network when you mirror it outside and carry it across again.
- Pull-through caching is no use without a route out. It fetches a file from the upstream the first time the file is downloaded, so a file nobody downloaded while connected is not there. Use
modelctl mirror, which copies whole revisions. - Pushed models get new revision ids. Only the data directory route keeps the ids from Hugging Face.
- Large uploads from Hugging Face tools need one setting. If people inside upload their own models with
hf upload, files over 10 MiB needHF_HUB_DISABLE_XET=1on the client.modelctl pushneeds nothing. - No Spaces. Models and datasets work the same way, with
--repo-type dataseton themodelctlcommands; Spaces are not supported. - One server. There is no replication or high availability.
The README has the statement on offline and air-gapped networks under Configuration, and the operations guide covers copying and backing up the data directory.
Related guides
- Run your own Hugging Face mirrorThe mirror commands in full, with private and gated models.
- Pull-through cacheFor networks that do have a route out: fetch each file the first time it is asked for.
- S3 and Cloudflare R2 storageKeep the model files in a bucket instead of on the server's disk.
- Model DeliveryWhat the registry is, and everything else it does.