Deployment Bundle Generator

Open model → running endpoint, one script.

Paste any Hugging Face link — LaunchKit reads the real architecture off the Hub, recommends an engine + tensor-parallel config, checks the VRAM math, and generates a bundle you copy to any bare GPU box over SSH. The script preflights the machine (docker or a python venv with uv — it handles both), downloads weights, starts an OpenAI-compatible server, and smoke-tests it. Text and multimodal models supported.

⛨ 100% client-side — model lookups go straight from your browser to huggingface.co; your HF token is only ever asked for on your own machine, never here

Model

Paste a Hugging Face link or repo id

…or pick a curated model

Weight precision

Hardware

GPU

Tensor parallel (GPU count)

TP ranks must sit on NVLink for decent inter-token latency — check nvidia-smi topo -m shows NV# between them.

Engine & runtime

Runtime on the GPU machine

auto = use docker if present, else install into a python venv with uv. TensorRT-LLM requires docker.

Serving shape

Max context length

Target concurrency

Port

KV cache dtype

GPU memory fraction: 0.90

Advanced (optional — production hygiene)

Pin weights revision (commit sha)

Served model name (alias)

Trust remote code

Engine metrics

Chunked prefill (tokens)

Tool-call parser

Reasoning parser

Flags are mapped per engine (e.g. chunked prefill → vLLM --max-num-batched-tokens, SGLang --chunked-prefill-size, TRT-LLM --max_num_tokens). Revision pinning and parsers apply to vLLM/SGLang; TRT-LLM ignores what it doesn't support.

Feasibility — live VRAM math

checking…