Get started with Flash - Runpod Documentation

Flash is now part of the Runpod SDK. This page documents the standalone runpod-flash package, which stops getting fixes and updates after its deprecation date. For new projects, use the Runpod SDK quickstart. To see how Flash commands and imports map to the SDK, see Migrating from Flash.

This quickstart gets you running GPU workloads on Runpod in minutes. You’ll execute a function on a remote GPU and see the results immediately.

Requirements

Runpod account with a verified email address.
An API key with All access permissions to your Runpod account.
Python 3.10 to 3.14 installed. Workers run Python 3.12 by default, whatever your local version.
uv installed.

Step 1: Install Flash

Flash runs natively on macOS and Linux. Windows users can run Flash through WSL2.

Create a virtual environment and install Flash using uv:

uv venv
source .venv/bin/activate
uv pip install runpod-flash

When you create a project with flash init, Flash generates AGENTS.md (and CLAUDE.md as a symlink) with CLI-first rules for AI coding assistants. These files tell AI tools like Claude Code, Cursor, and Codex to use Flash commands instead of raw Runpod API calls. See flash init for details.

Optional: Install coding agent integration

If you’re using an AI coding agent like Claude Code, Cline, or Cursor, you can install the Flash skill package to give your agent detailed context about the Flash SDK:

npx skills add runpod/runpod-plugins-official

This enables your coding agent to provide more accurate Flash code suggestions and troubleshooting help.

Step 2: Authenticate with Runpod

flash login

# If using uv:
uv run flash login

This opens your browser to authorize Flash. After you approve, your credentials are saved, allowing you to run Flash commands and scripts.

Alternatively, you can set the RUNPOD_API_KEY environment variable or add it to a .env file. See flash login for details.

Step 3: Copy this code

Create a file called gpu_demo.py and paste this code into it:

import asyncio
from runpod_flash import Endpoint, GpuGroup

@Endpoint(
    name="flash-quickstart",
    gpu=GpuGroup.ANY, # Use any available GPU
    workers=3,
    idle_timeout=300, # Keep worker running for 5 minutes
    dependencies=["torch"]
)
def gpu_matrix_multiply(size):
    # IMPORTANT: Import packages INSIDE the function
    import torch

    # Get GPU name
    device_name = torch.cuda.get_device_name(0)

    # Create random matrices on the GPU
    A = torch.rand(size, size, device="cuda")
    B = torch.rand(size, size, device="cuda")

    # Multiply matrices on the GPU
    C = A @ B

    return {
        "matrix_size": size,
        "result_mean": C.mean().item(),
        "gpu": device_name
    }

# Call the function
async def main():
    print("Running matrix multiplication on Runpod GPU...")
    result = await gpu_matrix_multiply(1000)

    print(f"\n✓ Matrix size: {result['matrix_size']}x{result['matrix_size']}")
    print(f"✓ Result mean: {result['result_mean']:.4f}")
    print(f"✓ GPU used: {result['gpu']}")

if __name__ == "__main__":
    asyncio.run(main())

Make sure you activate your virtual environment in the same directory where you created the gpu_demo.py file. If you open a new terminal, run source .venv/bin/activate before executing the script.

Step 4: Deploy the endpoint

From the directory that contains gpu_demo.py, deploy the endpoint:

flash deploy

# If using uv:
uv run flash deploy

Flash builds your project on your machine, uploads it, and creates the flash-quickstart endpoint. The app is named after the current directory, and the endpoint is deployed to the production environment:

✓ built quickstart  1 files, 0 deps, 0.2 MB
No app 'quickstart' found. Creating app and 'production' environment...

✓ uploaded  0.2 MB  1.3s
✓ deployed to production  2.3s

  flash-quickstart  https://api.runpod.ai/v2/abc123xyz/runsync

curl -X POST https://api.runpod.ai/v2/abc123xyz/runsync \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $RUNPOD_API_KEY" \
  -d '{"input": {}}'

The URL contains your endpoint ID (abc123xyz here). You’ll use it in Step 9 to call the endpoint over HTTP. Don’t run the printed curl example as is in 1.21.0. Step 9 explains why.

Every flash command and script prints a deprecation notice (a FutureWarning in scripts) because runpod-flash is moving into the Runpod SDK. The notice doesn’t affect your deployment.

flash deploy installs your endpoint’s dependencies into a local build directory before uploading. Flash leaves torch, torchvision, torchaudio, and triton out of the upload because the worker image already provides them, but a dependency that requires torch (such as accelerate) still downloads it during the local build. On Linux, that can be several GB. If the build runs out of disk space, remove that dependency, or exclude it with --exclude <package>. --no-deps also skips transitive dependencies, but it skips them for every package, so use it only if you list every dependency your code needs yourself. See flash deploy for details.

Step 5: Run it

Execute the script from the same directory:

python gpu_demo.py

Flash sends the call to your deployed endpoint and prints the result:

Running matrix multiplication on Runpod GPU...

✓ Matrix size: 1000x1000
✓ Result mean: 249.8286
✓ GPU used: NVIDIA RTX A5000

The first run takes about a minute while a worker starts. Later runs return in a few seconds while the worker is still warm.

The first request on a new worker can take longer than the 90 seconds runpod-flash 1.21.0 waits for a result, especially if the worker has to wait for a free GPU. If the script fails with timed out after 90s. the endpoint may not be deployed or the worker is still starting or KeyError: 'matrix_size', the endpoint is deployed and the worker is still starting. Run the script again. See First run fails after 90 seconds.

If you see endpoint 'flash-quickstart' not found in app '...' environment 'production', the endpoint isn’t deployed for this directory. Run flash deploy from the directory that contains gpu_demo.py, then run the script again from that same directory.

If you’re having authorization issues, you can set your API key directly in your terminal:

export RUNPOD_API_KEY="your_key"

Replace your_key with your actual API key from the Runpod console.

Step 6: Update and run again

With your endpoint running, change an input and run the script again:

Open gpu_demo.py and change the matrix size from 1000 to 2000:
result = await gpu_matrix_multiply(2000)
Run the script again:
python gpu_demo.py

This time, the result should appear in a few seconds because the worker is already warm. The argument is sent with the request, so you don’t need to redeploy. If you change the code inside gpu_matrix_multiply or its @Endpoint configuration, run flash deploy again so the endpoint picks up the new version. Workers that are already warm can keep serving the previous version for a few minutes while new workers replace them. If a result doesn’t reflect your change, or fails with unexpected keyword argument after you added a parameter, wait a minute and run the script again.

Step 7: Understand what you just did

Let’s break down the code you just ran:

Imports and setup

import asyncio
from runpod_flash import Endpoint, GpuGroup

asyncio: Enables asynchronous execution (endpoint functions run async).
Endpoint: The class that marks functions for remote execution.
GpuGroup: Enum for selecting GPU types or groups of GPUs.

Flash automatically loads your credentials from flash login or the RUNPOD_API_KEY environment variable.

The `@Endpoint` decorator

@Endpoint(
    name="flash-quickstart",
    gpu=GpuGroup.ANY,
    workers=3,
    idle_timeout=300,
    dependencies=["torch"]
)
def gpu_matrix_multiply(size):
    import torch

    # Get GPU name
    device_name = torch.cuda.get_device_name(0)

    # Create random matrices on the GPU
    A = torch.rand(size, size, device="cuda")
    B = torch.rand(size, size, device="cuda")

    # Multiply matrices on the GPU
    C = A @ B

    return {
        "matrix_size": size,
        "result_mean": C.mean().item(),
        "gpu": device_name
    }

The @Endpoint decorator configures everything in one place:

name: Identifies your endpoint in the Runpod console.
gpu: Which GPU to use (GpuGroup.ANY accepts any available GPU for faster provisioning).
workers: Maximum parallel workers (allows 3 concurrent executions).
idle_timeout: Seconds a worker stays active after completing a request before scaling down. Setting this to 300 (5 minutes) gives you more time to iterate on your code while the worker remains warm.
dependencies: Python packages your function needs. torch is already in the worker image, so Flash leaves it out of the upload.
Function body: Runs on the remote worker, not your local machine. device="cuda" creates the matrices in GPU memory, so the multiplication runs on the GPU.
Return value: The result is returned to your local machine as a Python dictionary.

See GPU types for available GPUs or endpoint functions for all configuration options.

You must import packages inside the function body, not at the top of your file. These imports need to happen on the remote worker.

Calling the function

async def main():
    print("Running matrix multiplication on Runpod GPU...")
    result = await gpu_matrix_multiply(1000)

    print(f"\n✓ Matrix size: {result['matrix_size']}x{result['matrix_size']}")
    print(f"✓ Result mean: {result['result_mean']:.4f}")
    print(f"✓ GPU used: {result['gpu']}")

if __name__ == "__main__":
    asyncio.run(main())

Here’s what happens when you call an @Endpoint decorated function:

Flash looks up the endpoint by its name in your app (the current directory name) and environment (production by default). flash deploy must have created it first.
Flash sends the function’s arguments to the endpoint.
A GPU worker runs the deployed function with those arguments.
The result is returned to your local machine as a Python dictionary, where it’s printed in your terminal.

Everything outside the @Endpoint function (all the print statements, etc.) runs locally on your machine. Only the decorated function runs remotely.

Step 8: Run multiple operations in parallel

Flash makes it easy to run multiple GPU operations concurrently. Replace your main() function with the code below, and keep the if __name__ == "__main__": block at the end of the file. Without it, the script exits without printing anything.

async def main():
    print("Running 3 matrix operations in parallel...")

    # Run all three operations at once
    results = await asyncio.gather(
        gpu_matrix_multiply(500),
        gpu_matrix_multiply(1000),
        gpu_matrix_multiply(2000)
    )

    # Print results
    for i, result in enumerate(results, 1):
        print(f"\n{i}. Size: {result['matrix_size']}x{result['matrix_size']}")
        print(f"   Mean: {result['result_mean']:.4f}")
        print(f"   GPU: {result['gpu']}")

if __name__ == "__main__":
    asyncio.run(main())

You only changed code outside the @Endpoint function, so you don’t need to redeploy. Run the script again:

python gpu_demo.py

All three requests are sent at the same time. They run in parallel when several workers are up (workers=3), or one after another on a single warm worker:

Running 3 matrix operations in parallel...

1. Size: 500x500
   Mean: 125.3097
   GPU: NVIDIA RTX A5000

2. Size: 1000x1000
   Mean: 249.9442
   GPU: NVIDIA RTX A5000

3. Size: 2000x2000
   Mean: 500.1321
   GPU: NVIDIA RTX A5000

Step 9: Call the endpoint from any language

Your endpoint is a standard Serverless endpoint, so any language that can send HTTP requests can call it. The JSON input object holds the function’s arguments by name. flash login stores your API key for Flash, but curl needs it in an environment variable. Export it first, replacing your_key with your API key from the Runpod console:

export RUNPOD_API_KEY="your_key"

Submit a job with /run, using the endpoint ID from Step 4:

curl -X POST https://api.runpod.ai/v2/ENDPOINT_ID/run \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $RUNPOD_API_KEY" \
  -d '{"input": {"size": 500}}'

{"id": "50e2248c-55c3-4f61-b633-35e447a4d82d-u2", "status": "IN_QUEUE"}

Then poll /status with the job ID until status is COMPLETED:

curl https://api.runpod.ai/v2/ENDPOINT_ID/status/JOB_ID \
  -H "Authorization: Bearer $RUNPOD_API_KEY"

{
  "id": "50e2248c-55c3-4f61-b633-35e447a4d82d-u2",
  "status": "COMPLETED",
  "output": {"gpu": "NVIDIA RTX A5000", "matrix_size": 500, "result_mean": 124.87},
  "delayTime": 15,
  "executionTime": 189
}

Use /run and /status for long jobs and cold starts. /runsync returns the result in one request but waits at most 90 seconds by default, then returns IN_QUEUE or IN_PROGRESS and you poll /status anyway.

Always send at least one key in input. Runpod drops a job whose input is empty ({"input": {}}), and after about 40 seconds it fails with job timed out after 1 retries. The curl example that flash deploy prints in runpod-flash 1.21.0 and earlier uses an empty input, so replace it with your function’s arguments.

Clean up

When you’re done testing, clean up the endpoints:

# List all endpoints
flash undeploy list

# Remove the quickstart endpoint
flash undeploy flash-quickstart

# Or remove all endpoints
flash undeploy --all

# If using uv:
uv run flash undeploy list
uv run flash undeploy flash-quickstart
uv run flash undeploy --all

Next steps

You’ve successfully run GPU code on Runpod! Now you’re ready to learn more about Flash:

Generate images with Flash

Use Stable Diffusion XL to generate images from text prompts.

Endpoint functions

Learn how to configure and optimize endpoint functions.

Build Flash apps

Deploy production APIs.

Explore Flash examples

Browse example Flash scripts and apps on GitHub.

Troubleshooting

Endpoint not found

RuntimeError: endpoint 'flash-quickstart' not found in app 'quickstart' environment 'production'. deploy it first with 'flash deploy'.

Solution: Calling an @Endpoint function sends the request to an endpoint you’ve already deployed. Run flash deploy from the directory that contains your script, then run the script from that same directory. Flash uses the directory name as the app name, so running from a different directory looks up a different app.

First run fails after 90 seconds

Issue: The first run fails with one of these errors:

RuntimeError: request to endpoint 'flash-quickstart' timed out after 90s. the endpoint may not be deployed or the worker is still starting. deploy with 'flash deploy' or increase timeout with FLASH_SENTINEL_TIMEOUT env var.

KeyError: 'matrix_size'

Cause: runpod-flash 1.21.0 waits up to 90 seconds for a result. The first request on a new worker can take longer: the worker may wait for a free GPU, then it starts and runs your function. The endpoint is deployed, and the request keeps running after the error. Solution: Run the script again. The worker keeps starting after the error, so the next run is faster. Raising FLASH_SENTINEL_TIMEOUT above 90 doesn’t help in 1.21.0: the request returns at 90 seconds with the job still running, and the script fails with the KeyError instead. To see whether a worker is up, check the endpoint’s health with its ID from flash deploy. curl reads your API key from the RUNPOD_API_KEY environment variable, which flash login doesn’t set, so export it first:

export RUNPOD_API_KEY="your_key"
curl https://api.runpod.ai/v2/ENDPOINT_ID/health \
  -H "Authorization: Bearer $RUNPOD_API_KEY"

A non-zero workers.throttled means the worker is waiting for a GPU. A non-zero workers.idle, workers.ready, or workers.running means a worker is up.

Job timed out after 1 retries

RuntimeError: remote execution failed: job timed out after 1 retries

Solution: The worker stopped responding while it held the job. Two common causes:

Empty input over HTTP: a request with {"input": {}} never reaches your function. Send at least one argument, as shown in Step 9.
Redeploying during a request: flash deploy replaces running workers, so a request that’s in progress fails. Wait for running requests to finish before you redeploy, or run the request again.

Authentication error

flash commands print:

No RunPod API key found. Set one with:

Scripts that call an endpoint fail with:

httpx.HTTPStatusError: Client error '401 Unauthorized' for url 'https://api.runpod.ai/v2/flash/runsync'

Solution: Run flash login to authenticate with your Runpod account:

flash login

# If using uv:
uv run flash login

Alternatively, set the RUNPOD_API_KEY environment variable:

export RUNPOD_API_KEY="your_key"

Template name conflict

Error: endpoint template names must be unique

Solution: Each endpoint needs a unique name. If you’ve deployed an endpoint before with the same name, either:

Use a different name for your new endpoint
Undeploy the existing endpoint with flash undeploy <name> --force

Job stuck in queue

Requests wait for minutes, and the endpoint’s health check shows "throttled": 1 with no idle or running workers. Solution: No GPU of the types you selected is free. GpuGroup.ANY (used in this quickstart) accepts any available GPU. If you narrowed gpu to specific types, list several so Flash can fall back:

from runpod_flash import Endpoint, GpuType

@Endpoint(
    name="flash-quickstart",
    gpu=[
        GpuType.NVIDIA_GEFORCE_RTX_4090,
        GpuType.NVIDIA_RTX_A5000,
        GpuType.NVIDIA_RTX_A6000
    ],
    dependencies=["torch"]
)
def gpu_matrix_multiply(size):
    ...

You can also check GPU availability in the console.

Import errors

ModuleNotFoundError: No module named 'numpy'

Solution: Move imports inside the @Endpoint function:

@Endpoint(name="compute", gpu=GpuGroup.ANY, dependencies=["numpy"])
def my_function():
    import numpy as np  # Import here, not at top of file
    # ...

See the execution model for more troubleshooting.

​Requirements

​Step 1: Install Flash

​Optional: Install coding agent integration

​Step 2: Authenticate with Runpod

​Step 3: Copy this code

​Step 4: Deploy the endpoint

​Step 5: Run it

​Step 6: Update and run again

​Step 7: Understand what you just did

​Imports and setup

​The @Endpoint decorator

​Calling the function

​Step 8: Run multiple operations in parallel

​Step 9: Call the endpoint from any language

​Clean up

​Next steps

Generate images with Flash

Endpoint functions

Build Flash apps

Explore Flash examples

​Troubleshooting

​Endpoint not found

​First run fails after 90 seconds

​Job timed out after 1 retries

​Authentication error

​Template name conflict

​Job stuck in queue

​Import errors

Requirements

Step 1: Install Flash

Optional: Install coding agent integration

Step 2: Authenticate with Runpod

Step 3: Copy this code

Step 4: Deploy the endpoint

Step 5: Run it

Step 6: Update and run again

Step 7: Understand what you just did

Imports and setup

The `@Endpoint` decorator

Calling the function

Step 8: Run multiple operations in parallel

Step 9: Call the endpoint from any language

Clean up

Next steps

Troubleshooting

Endpoint not found

First run fails after 90 seconds

Job timed out after 1 retries

Authentication error

Template name conflict

Job stuck in queue

Import errors