Flash, now the Nix way: run the Flash CLI straight from GitHub

Flash ships a first-class Nix flake, so you can run the Flash CLI straight from GitHub with nix run github:runpod/flash, no clone and no virtualenv. The flake carries its own flake.lock and gives you a pinned dev shell. Nix pins your local toolchain; Flash runs the workload on a Runpod GPU.

In a rush? The short version:

• Run Flash from GitHub: nix run github:runpod/flash
• Get a reproducible dev shell with Flash ready to import and hermetic checks, with no clone or virtualenv
• Built for the NixOS and Flox communities

When we built Flash, the pitch was simple: you write a Python function, decorate it, and run it on a GPU. No Docker. No container orchestration. Just Python.

Now there’s a Nix-native way to get there, too.

Get started

Try it in one line. You’ll need Nix with flakes enabled:

nix run github:runpod/flash -- --help

Or install Flash the classic way:

pip install runpod-flash

Requires Python 3.10 through 3.14, worker images target 3.10 through 3.13. On macOS, the system python3 is 3.9, so install a newer Python with Homebrew or uv first.

Then authenticate. Flash login opens your browser, and after you approve the request your API key is saved locally. You can also set RUNPOD_API_KEY directly. You will need a Runpod account with a verified email address.

flash login
flash init my-project

The Flash source and Nix flake live at github.com/runpod/flash. If you want to go deeper on the Nix side, the Nix guide covers the dev shell helpers, hermetic checks, and how to pin a specific tag or commit. For the Flash SDK itself, including @Endpoint, Flash apps, and deploying to production, start with the Flash docs.

Nix and Flash: each great, and great together

Runpod is the AI Developer Cloud, and part of that job is meeting developers in the toolchain they already trust. For a large and devoted community, that toolchain is Nix.

For a lot of developers, “just Python” is exactly right. But if you live in Nix, there was still one wrinkle: getting Flash into an environment you trust, with the right Python and every local dependency pinned.

So we closed that gap. Flash now ships a first-class Nix flake, and you can run the CLI straight from GitHub:

nix run github:runpod/flash -- --help

No clone. No virtualenv. No wrangling Python versions. The command resolves Flash and its Nix dependencies through the flake and runs the CLI.

If you don’t have flakes enabled globally, enable them for this command:

nix --extra-experimental-features 'nix-command flakes' \
  run github:runpod/flash -- --help

The first run may build Flash-specific packages locally while fetching available upstream packages from the Nix binary cache. Subsequent runs reuse identical results already present in your Nix store.

CI builds and checks the flake is verified on Linux (x86_64 and ARM) on every change. Apple Silicon was verified manually for this blog post.

What is Nix and how do I get it?

Nix is a package manager and build system that treats software as the result of explicitly declared inputs. Packages are stored under unique paths in /nix/store, allowing different versions of the same library to coexist without interfering with each other.

That matters for AI developers, whose projects often depend on a fragile combination of Python versions, framework releases, and system libraries.

With flakes, those inputs are declared in flake.nix and resolved in flake.lock. Commit both alongside your code, and nix develop can recreate the same local development environment later or on another machine.

Flash runs your GPU workload remotely on Runpod, so your local Nix environment does not need a CUDA toolkit or GPU drivers. Flash runs natively on macOS and Linux; Windows via WSL2.

On macOS or Linux, install Nix with the official multi-user installer in daemon mode:

bash <(curl -L https://nixos.org/nix/install --daemon

The daemon install creates build users 30001-30032 and group 30000).

Once it finishes, open a new terminal and verify the installation:

nix-shell -p hello --run hello

To enable flakes globally:

test -d /etc/nix || sudo mkdir /etc/nix
echo 'experimental-features = nix-command flakes' | sudo tee -a /etc/nix/nix.conf

What’s new

The flake gives you three ways in, all from the same GitHub repository:

nix run github:runpod/flash -- --help     # run the CLI
nix profile install github:runpod/flash   # put flash on your PATH
nix develop github:runpod/flash           # enter the Flash dev shell

The Flash repository carries its own flake.lock, so its Nix dependencies are pinned. You can also create your own flake that takes Flash as an input and commit your project’s flake.lock.

The Nix guide covers the dev shell helpers and hermetic checks. And pip install runpod-flash still works as before, with the Flash SDK supporting Python 3.10 through 3.14, and worker images target 3.10 through 3.13.

Why this matters to the Nix community

Nix people care about reproducibility. Nix tracks package inputs precisely, so your development environment doesn’t have to depend on whatever happens to be installed globally on a particular machine.

That’s a natural fit for Flash. Nix can pin the environment you use to write and deploy your application; Flash takes care of turning that application into a remote GPU workload.

The flake carries its own flake.lock, so nix run github:runpod/flash doesn’t depend on whatever nixpkgs channel you happen to have configured locally. CI builds and checks the flake on Linux as the project changes.

We wanted to open Flash up to the whole Nix ecosystem — the NixOS community and the growing number of teams using Flox, which brings declarative, reproducible environments to teams without requiring everyone to write Nix by hand.

How it works

The flake is intentionally modular: a thin flake.nix at the repository root delegates to focused files under nix/, grouped by concern: the Python environment, packaged CLI, dev shell, and checks.

Because the Nix inputs are captured in flake.lock, contributors can work against the same package graph instead of depending on globally installed Python packages.

The full guide — installing Nix, enabling flakes, using the dev-shell helpers, and understanding the checks — lives in the Nix guide in the repository.

Nix and Flash demo

Here’s a small project that captions an image on a Runpod GPU.

It also demonstrates the boundary between the two tools: Nix pins the local development environment; Flash packages and deploys the code that runs remotely on the GPU.

Your laptop doesn’t need CUDA, GPU drivers, or a local PyTorch installation.

The project has two files.

First, flake.nix, which pulls Flash directly from GitHub:

{
  description = "Image captioning on Runpod GPUs, with a pinned local toolchain";

  inputs = {
    # Pin Flash to the release this example was tested against.
    flash.url = "github:runpod/flash/v1.20.1";

    # Use the same nixpkgs and flake-utils inputs as Flash.
    nixpkgs.follows = "flash/nixpkgs";
    flake-utils.follows = "flash/flake-utils";
  };

  outputs =
    { nixpkgs, flash, flake-utils, ... }:
    flake-utils.lib.eachDefaultSystem (
      system:
      let
        pkgs = nixpkgs.legacyPackages.${system};

        flashPkg =
          pkgs.python314.pkgs.toPythonModule
            flash.packages.${system}.default;

        python = pkgs.python314.withPackages (ps: [
          flashPkg
          ps.pillow
          ps.pip
        ]);
      in
      {
        devShells.default = pkgs.mkShell {
          packages = [
            python
            pkgs.ruff
            pkgs.git
          ];
        };
      }
    );
}

Second, caption.py. The @Endpoint function runs remotely on an RTX 4090 and loads the BLIP captioning model there. load_thumbnail runs locally using the Pillow version supplied by the Nix environment:

import asyncio
import base64
import sys

from PIL import Image
from runpod_flash import Endpoint, GpuType


@Endpoint(
    name="captioner",
    gpu=GpuType.NVIDIA_GEFORCE_RTX_4090,
    dependencies=[
        "torch==2.14.1",
        "transformers==4.57.6",
        "pillow==12.3.0",
    ],
)
def caption(image_b64: str) -> dict:
    import base64
    import io

    import torch
    from PIL import Image
    from transformers import BlipForConditionalGeneration, BlipProcessor

    model_id = "Salesforce/blip-image-captioning-base"
    model_revision = "82a37760796d32b1411fe092ab5d4e227313294b"

    processor = BlipProcessor.from_pretrained(
        model_id,
        revision=model_revision,
    )

    model = BlipForConditionalGeneration.from_pretrained(
        model_id,
        revision=model_revision,
    ).to("cuda")

    image = Image.open(
        io.BytesIO(base64.b64decode(image_b64))
    ).convert("RGB")

    inputs = processor(
        images=image,
        return_tensors="pt",
    ).to("cuda")

    output = model.generate(
        **inputs,
        max_new_tokens=30,
    )

    text = processor.decode(
        output[0],
        skip_special_tokens=True,
    )

    return {
        "caption": text,
        "gpu": torch.cuda.get_device_name(0),
    }


def load_thumbnail(path: str) -> str:
    import io

    image = Image.open(path)
    image.thumbnail((512, 512))

    buf = io.BytesIO()
    image.convert("RGB").save(buf, format="JPEG")

    return base64.b64encode(
        buf.getvalue()
    ).decode("ascii")


async def main() -> None:
    image_b64 = load_thumbnail(sys.argv[1])
    result = await caption(image_b64)

    print(
        f"{result['caption']}  "
        f"(on {result['gpu']})"
    )


if __name__ == "__main__":
    asyncio.run(main())

Create a new project directory and save the two examples above as flake.nix and caption.py. Run the following commands from that directory.

git init
git add flake.nix caption.py

nix develop

The first resolution creates your project's flake.lock. Add that too:

git add flake.lock

The first nix develop resolves Flash and its dependencies and records their exact revisions in flake.lock. Commit that file alongside your project to reproduce the same development environment later. Run nix flake update when you want to pick up newer versions.

Now authenticate and deploy:

flash login
flash deploy --python-version 3.12

The Nix development shell uses Python 3.14, while this example targets Python 3.12 on the Runpod worker. The local and remote Python versions do not need to match.

Now caption an image. This one is a COCO sample photo of two cats:

curl -fL -o cats.jpg \
  https://images.cocodataset.org/val2017/000000039769.jpg

python caption.py cats.jpg

You should get output similar to:

two cats sleeping on a pink couch with remotes  (on NVIDIA GeForce RTX 4090)

Cold start on Runpod Serverless typically runs 10-60 seconds depending on GPU availability and image size. Add the one-time BLIP download on top of that for the first call.

When you're finished, remove the deployment:

flash undeploy captioner

What is actually reproducible here?

There are two environments in this example.

1. On your machine, flake.lock pins the Nix inputs that produce your development environment. That includes the Flash version, Python environment, and local packages.
2. On the worker, Flash deploys the GPU workload with the dependencies declared by the endpoint.

Those aren't quite the same guarantee.

Pinning direct Python package versions improves worker repeatability, but a list of pip requirements isn't a complete lockfile: transitive dependencies can still change unless they're pinned too. Model repositories can also change unless you specify an immutable revision.

For this demo, pinning the Flash release, important worker package versions, and the model revision removes the largest sources of drift. For a workload where exact reconstruction matters, use a fully resolved dependency set for the worker as well.

To keep the example short, the function loads the model during execution. For a production workload, you'd normally avoid repeatedly initializing the same model.

Where Nix support fits

Flash went from beta to generally available around one idea: make running serverless GPU and CPU workloads feel like writing local Python.

Nix support doesn't change what Flash does. It gives developers who already rely on Nix a native way to install Flash, create a reproducible local development environment, and keep that environment alongside the rest of their project configuration.

The division is simple: Nix controls the environment on your machine. Flash takes your Python workload from there to Runpod infrastructure.

Try it

Try Flash directly from GitHub:

nix run github:runpod/flash -- --help

Or install it with pip:

pip install runpod-flash

Then:

flash login
flash init my-project

You can find the source code and Nix flake at github.com/runpod/flash, or read the full documentation at docs.runpod.io/flash. To get started, sign up at runpod.io/console, and join our community on Discord to connect with other developers.

Give it a try, and tell us what you build.