Guides

/

AI agents

Run Local AI Models on a Cloud GPU

TL;DR

If a model does not fit your laptop, run it on a cloud GPU desktop. VRAM decides what fits, so match the model to the card first, keep your weights on persistent storage, and shut the machine down when you finish. For an always-on endpoint, a dedicated server or hosted API is the better fit.

Updated

·

5 chapters

·

8 min read

Apps

If you want to run a language model, an image model or a notebook yourself and your laptop cannot hold it, this guide is for you. It covers why a cloud GPU desktop helps, how to size a model to a card, and how to set up Ollama, a local LLM, DeepSeek-R1, ComfyUI, Stable Diffusion and Jupyter. It ends with an honest section on when a per-minute desktop is the wrong tool. Everything here comes from Vagon's own posts, so the sizing numbers are rough guides, not benchmarks.

Why run a model on a cloud GPU

Vagon's local LLM guide makes a point of the wording. The point of a "local" model was never the word local. It is control. You pick the model, you keep your prompts and data on a machine you command, there is no per-token bill, and a provider cannot change the model under you. In the cloud you get a real NVIDIA card to do it with.

The other reason is memory. The guides keep returning to VRAM: it decides what fits, before speed matters. A model that crawls on a laptop, or does not load, can run properly on a card with enough memory. The Ollama post frames the moment well: you run a model bigger than your laptop can handle, the fan spins up, and tokens crawl out.

A cloud desktop adds two things a headless GPU pod does not. You can open a browser and a chat UI or node editor on the machine itself, and each Vagon computer is an isolated VM, so experiments and custom code stay contained and you can reset. Drivers and CUDA come preconfigured in these posts' setups, so you skip the part that usually breaks on a fresh machine.

Read next

Large language models: Ollama, runtimes and DeepSeek-R1

Start with Ollama. The Ollama guide is the simplest path: launch a GPU Ubuntu desktop, check the card with nvidia-smi, install with the one-line script, pull a model and run it. The post says a cloud GPU lets you run 34B and 70B models at usable speed, and that you can put a web chat UI in front of it because the machine is a real desktop.

The local LLM guide widens the choice of runtime. Ollama is the easiest start, a web UI gives a richer interface, and vLLM or a dedicated inference server gives throughput. Its sizing table is the most useful part of this guide. For quantized models it gives roughly 5 to 8 GB of VRAM for 7B to 8B, 10 to 12 GB for 13B, 20 to 24 GB for 34B, and 40 GB or more for 70B. Quantization (a tag such as q4) shrinks memory use for a small loss in quality, and long context windows need extra headroom. Start with a model comfortably inside your card.

Safety matters because the machine can reach things. The same post says to use scoped, throwaway credentials, keep the endpoint private, treat model-driven actions with suspicion, and reset when in doubt. For the agent side of that, see Cloud computers for AI agents.

DeepSeek-R1 has its own older walkthrough. The DeepSeek-R1 guide uses llama.cpp and a quantized model, with an optional web interface, and compares serverless GPUs, Vagon desktops and local machines. It also lists local requirements: a high-performance GPU, 16 GB of RAM or more, and 10 GB to 50 GB of disk for a 4-bit model depending on its size. It is older than the Ubuntu series, so check model names before you follow it.

Read next

Image generation: ComfyUI and Stable Diffusion

ComfyUI is the heaviest of these on VRAM. How to Run ComfyUI in the Cloud says SDXL, Flux and video models overwhelm most laptop GPUs, and that the node editor suits a real desktop better than a headless pod. It lists other reasons for the cloud: a Mac or machine without NVIDIA, a disposable setup for custom nodes, and needing the GPU only sometimes.

Its rough GPU guide is: a T4 for older or small models and basic generation, an A10G for SDXL at standard resolutions, A10G or higher for SDXL with upscaling and for Flux, and higher plans for video generation. Storage matters here, because checkpoints are large and re-downloading them every session is painful.

Run Stable Diffusion on a low-end device is the older, simpler version. Pick a GPU-backed performance, start the computer, and use a GUI. That GUI works only with NVIDIA GPUs, per the post. You can switch performance later and keep your files.

Read next

Notebooks and training with Jupyter

How to Run Jupyter on a Cloud GPU Linux Desktop is for the cell that kills your kernel: training, large datasets and big-model inference. On a GPU Ubuntu desktop, torch.cuda.is_available() returns True without version wrangling, because drivers and CUDA are set up. You open Jupyter in a real browser, manage files normally and can watch the GPU work. The advice carries over from the other guides: match model and batch size to your VRAM, keep notebooks and datasets on persistent storage, and use the desktop for bursty work rather than as an always-on notebook server.

Read next

Let Claude or Codex drive ComfyUI

Once ComfyUI runs, an agent can help you iterate. The Claude guide and the Codex guide are twins. Through Comfy MCP the assistant can inspect available nodes and models, validate workflows and manage execution. Computer use covers what MCP does not: navigating the canvas and reviewing images.

The two posts agree on method. Start with a workflow that already runs, so you know ComfyUI, the models and your hardware can execute it. Change specified parameters instead of asking for endless generations. Preserve the settings behind images you keep. The Codex guide adds a product-photo problem: a label that changes or a bottle that gets wider between generations, which is why a workflow beats a vague prompt.

To decide whether the agent itself needs a GPU, read When Does Your AI Agent Workflow Need a GPU?. The model answering requests and the tools running locally are two different loads.

Read next

What to run, and on what

What you want to run

Rough fit (from the posts)

Watch for

7B to 8B LLM (quantized)

About 5 to 8 GB VRAM; T4 and up

Long context needs headroom

13B LLM

About 10 to 12 GB; tight on T4, comfortable on A10G

Quantization trades a little quality

34B LLM

About 20 to 24 GB; A10G

Check the fit before downloading

70B LLM

40 GB or more; higher GPU plans

Larger plans needed

ComfyUI, older or small models

T4

Keep checkpoints on persistent storage

ComfyUI, SDXL and Flux

A10G or higher

Start comfortable, step up on out-of-memory errors

ComfyUI, video models

Higher GPU plans

Heavy on VRAM

Jupyter training or big-model inference

Match model and batch to VRAM

Use persistent storage for data

Always-on endpoint

Not a desktop

Use a dedicated server or hosted API

Doing this on Vagon

Running models locally is mostly a VRAM problem, and a Vagon Cloud Computer lets you rent the VRAM instead of buying it. Create a Linux (Ubuntu) computer with a GPU performance. T4 tiers have 16 GB of GPU memory and A10G tiers 24 GB, so check them against the VRAM guide above. Switch performance when a model needs more, and shut down when you're done; billing stops with it.

Add a storage plan so weights and checkpoints survive between sessions (pricing). This suits focused sessions. If you generate or serve all day, every day, owning a GPU or renting a monthly server will likely cost less.

FAQ

Can I run Ollama on a cloud GPU?

Yes. Vagon's guide launches a GPU Ubuntu desktop, checks the GPU with nvidia-smi, installs Ollama with its one-line script, then pulls a model. A cloud GPU lets you run larger models than a laptop can handle.

How much VRAM do I need for a local LLM?

Vagon's local-LLM guide gives a rough quantized guide: about 5 to 8 GB for 7B to 8B models, 10 to 12 GB for 13B, 20 to 24 GB for 34B and 40 GB or more for 70B. Long context windows use extra VRAM, so leave headroom.

Is a cloud GPU desktop good for an always-on model endpoint?

No. Vagon's guides say a per-minute desktop is built for bursts of building, testing or generating. For a model that runs around the clock, a dedicated monthly GPU server or a hosted API is cheaper and simpler.

Can I run ComfyUI on a Mac or a laptop without an NVIDIA GPU?

You can run it on a cloud desktop instead. Vagon's ComfyUI guide lists being on a Mac or a machine without NVIDIA as a reason to use the cloud, with drivers and CUDA already configured.

Do I need to re-download models every session?

Not if you use persistent storage. The Ollama and ComfyUI guides both recommend it because model weights and checkpoints are large.

Can Claude or Codex run ComfyUI workflows?

Through Comfy MCP they can inspect nodes and models, validate workflows and manage execution, per Vagon's guides. Both advise starting from a workflow that already runs and reviewing the results.

Vagon Computer

Run it on a cloud computer

Open it from any browser, Mac, iPad or Chromebook. No local hardware needed.

Read next

All posts in this guide

All guides →

3D & Animation

Guide

Use ComfyUI with Codex via MCP and Computer Use

·

15 min read

AI Agents

Guide

Use ComfyUI with Claude via MCP and Computer Use

·

15 min read

Cloud PCs & Performance

Guide

Run Jupyter on a Cloud GPU Linux Desktop

·

20 min read

AI Agents

Guide

Run a Local LLM on a Cloud Ubuntu GPU Desktop

·

19 min read

AI Agents

Guide

Run ComfyUI on a Cloud GPU Ubuntu Desktop

·

20 min read

AI Agents

Guide

Run Ollama on a Cloud GPU Ubuntu Desktop

·

19 min read

AI Agents

Guide

Run DeepSeek-R1 on Low-End Devices and Cloud

·

10 min read

3D & Animation

Guide

Run Stable Diffusion on a Low-End Device

·

4 min read

Vagon Computer

Run this guide on a cloud computer.

Open it from any browser, Mac, iPad or Chromebook. No local hardware needed.

Vagon Computer

Run this guide on a cloud computer.

Open it from any browser, Mac, iPad or Chromebook. No local hardware needed.