Local Models for AI RPGs (2026): What It Takes, What It Costs, and Whether You Should
A local model is a language model running on your own machine, on weights you downloaded, with nothing leaving your computer. No subscription, no usage meter, no filter, and no company that can change the terms next quarter. That is the pitch, and the pitch is real. What is usually left out is what it costs — in hardware, in setup, and in the one thing hosted platforms are quietly much better at.
This page is the map. It explains what “running it locally” actually involves, what your hardware can honestly handle, which of the two pieces you need does what, and where the trade sits against paying somebody else. Where a step deserves its own walkthrough, this routes you to it rather than repeating it badly.
It is worth saying up front who this is for. Not everyone who plays AI RPGs wants to install anything, and there is no shame in that — the platform directory exists precisely so you can sign in and play tonight. But a real share of this hobby ends up here eventually, usually for one of three reasons: the bill, the filter, or the discomfort of a campaign living on someone else’s server. If you are one of them, this is the honest version.
The Thing Nobody Explains First: It Is Two Pieces
The single biggest source of confusion is that “running a local model” is not one program. It is two, and they do completely different jobs.
The backend loads the model file and generates text. It is not something you play in — most of the time it is a window with a log in it, or nothing at all. Ollama, KoboldCpp and LM Studio are backends.
The front-end is the thing you actually roleplay in: character cards, chat history, world info, prompt formatting, all the machinery that turns a raw text generator into a game. SillyTavern is the front-end almost everyone lands on.
You need one of each, and they talk to each other over a local address on your own machine. Once you have seen that split, most local-setup guides stop being confusing — they are describing one half and assuming you know about the other.
There is a partial exception worth knowing: some backends ship a usable chat interface of their own, so you can type at a model within minutes of installing. That is a fine way to check your hardware works. It is not a roleplay setup, because none of the character, memory and prompt machinery is there.
Should You Do This At All?
The honest decision, before any downloads.
Reasons that hold up. You want content nobody filters, and you have found that hosted platforms either refuse or quietly steer — our uncensored guide covers that whole landscape including the non-local routes. You want a campaign that cannot be deleted, rate-limited or repriced by a company. You are running enough sessions that a subscription has started to feel like a bad deal. Or you simply want to know how the thing works, which is a good enough reason on its own.
Reasons that do not. “It will be better” — mostly it will not, on the axes hosted platforms compete on. “It will be cheaper” is true only after the GPU, which is the expensive part. “It will be private” is true and worth having, but check whether the platform you are leaving actually does anything with your data first; several are better on this than their reputation suggests, and our data-custody audit went through all of it field by field.
The reason people actually stay. In practice the ones who stick with local setups are not the ones chasing quality — they are the ones who wanted control, found it, and discovered they liked tinkering. If tinkering sounds like a cost rather than a feature, weigh that heavily.
Hardware: What You Can Honestly Run
Video memory is the constraint. Not your CPU, not your system RAM, not your disk. A model has to fit in VRAM to run at conversational speed, and everything else is a rounding error next to that.
| VRAM | What runs comfortably | What it feels like |
|---|---|---|
| ~8GB | 7B–8B models | Genuinely good prose, short memory. The entry point. |
| 12GB+ | 13B-class with headroom | The sweet spot for roleplay quality per pound spent. |
| 24GB+ | 30B+ , or mid-size at gentler quantization | Closest thing to hosted quality you will get at home. |
Below roughly 8GB you can still get a model running, but you will be compressing it hard enough that the writing degrades noticeably, and at that point a free hosted tier is usually the better experience even though it is the worse deal. Running on CPU and system RAM alone works and is far too slow for a scene to hold together — a reply that takes minutes breaks the thing you came for.
Two numbers people forget. Context costs VRAM too, on top of the model itself, so the long campaign you are planning eats into the budget that decides which model you can load. And quantization is the dial that makes all of this negotiable: the same model can be squeezed to fit smaller cards, with quality falling as you squeeze. Both setup guides below handle the specifics.
Picking a Backend
All three run the same model files and connect to the same front-ends. The difference is how much machinery you want to see.
Ollama — the easy on-ramp. Installs like a normal application, pulls models with a single command, and chooses sensible quantization for you. If you want to be playing tonight rather than reading about flags, start here. Our full walkthrough covers the install, the SillyTavern connection, and which model to pull first.
KoboldCpp — the control option. A single file, no install, and it exposes the loading and sampling settings Ollama hides. Worth it if you are coaxing awkward hardware, or if you have opinions about samplers. Our walkthrough covers the same ground with the settings visible.
LM Studio — the graphical middle. A desktop app with model browsing and loading in a proper interface, which suits people who would rather click than type commands. Broadly Ollama’s audience with a heavier front door.
The real answer for most people is: pick Ollama, get something working end to end, and only move to KoboldCpp if you hit a wall you can name. A working setup you understand beats a theoretically better one you do not.
Picking a Model
This is where local play stops being a technical problem and starts being a taste problem, and where the advice ages fastest — the open-weight world moves quickly enough that any specific checkpoint recommendation has a short shelf life.
What holds steady is the shape of the choice. Llama, Qwen and Mistral are the families most roleplay-tuned models are built on, and community fine-tunes of them are what most people actually run — a base model is trained to be a helpful assistant, and roleplay tunes undo that helpfulness in the direction you want. Size beats cleverness for prose, up to what your VRAM allows. And a model’s advertised context window is a ceiling, not a promise: filling it slows generation and eats memory.
Best LLM for roleplay compares the local families against the hosted frontier models directly, which is the comparison worth reading before you commit a weekend. If you would rather rent frontier models by the token than run your own, OpenRouter is the route — one key, most models, no subscription and no GPU.
What You Give Up, Measured
Here is the part this site can answer that a general local-LLM guide cannot.
We have scored 25 platforms on seven axes from first-hand play. Not one of them runs on hardware you own. Three let you supply the model: Janitor AI takes your own API key, WyrdTale runs inside your existing Claude or ChatGPT subscription over a connector, and Chub AI accepts a key or a router. All three still send your text to somebody’s servers — they hand you control over which model and what it costs, not over where it runs.
So the local route is not a platform choice at all. It is a decision to leave the directory and assemble the thing yourself, and the scoreboard says what that costs:
| Highest-scored platform that accepts your own model | 3.0 (Janitor AI, WyrdTale) |
| Highest-scored platform overall | 4.4 (Voyage) |
| Of the top five, how many let you choose the model | none |
The entire top of our board is closed, and three of the top five will not even name the model they run. That is the trade in one line: the further you go toward owning the model, the further you get from the best-scoring products in the category.
It is worth being precise about why, because it is not that closed platforms have better models. It is that a score on our axes rewards the things a platform builds around the model — a rules engine, state held outside the context window, memory that survives a long campaign, characters that stay themselves. A local setup gives you a model and a front-end. Everything the top of the board is scoring for is machinery somebody else wrote, and running your own weights does not get you any of it.
What You Gain
Set against that, three things the board cannot score.
Nothing is filtered. Not “fewer refusals” — nothing between you and the model. This is the single most common reason people end up here, and no hosted platform can match it, because a hosted platform has a company attached to it.
Nothing is metered. After the hardware, a session costs electricity. Long sessions, retries, rerolls, a campaign you play every night for a year — the marginal cost never changes. What an AI RPG actually costs covers the five ways hosted platforms meter play, and the shape of the meter is usually what pushes people out. One thing local does not win is efficiency per reply: a home card serves one conversation where a data centre serves thousands, and the arithmetic is in what a night of AI roleplay costs in water and energy.
Nothing can be taken away. The weights are on your disk. Models are not deprecated out from under you, prices do not change, and terms of service do not get rewritten. For a campaign measured in months, that is worth more than it sounds.
Where Local Setups Actually Break
Four failure modes, in the order people hit them.
- Memory, not prose. The first disappointment is almost never the writing — it is the model forgetting. Smaller context plus no state held outside the conversation means the wall arrives sooner than on a paid hosted tier. Why campaigns fall apart explains the mechanism, and the campaign memory tool is the practical fix on any platform.
- The setup becomes the hobby. Time spent tuning samplers is time not spent playing. Set a budget for fiddling.
- Repetition loops. Smaller models fall into repeating phrasings more readily, and it is usually a settings problem rather than a model problem — breaking the repetition loop covers the fixes.
- No one to ask. When a hosted platform breaks, someone else fixes it. When yours breaks, you are the support team.
Where to Go Next
To get something running tonight: SillyTavern + Ollama is the shortest path from nothing to playing. If you would rather see the settings, SillyTavern + KoboldCpp covers the same ground with the machinery visible.
To understand the front-end first: what SillyTavern is explains the piece you will spend all your time in, and whether it suits you.
To compare before committing: best LLM for roleplay sets the local families against the hosted models. If the pull is the filter rather than the hardware, uncensored AI roleplay covers every route including the ones that need no GPU. If it is the bill, OpenRouter rents frontier models by the token.
To see what you would be leaving: the benchmark reports what we measured across the whole hosted field, and the compare table sorts all 25 platforms by any axis — including the three that will take your own key.
Frequently Asked Questions
What is a local model in AI roleplay? A language model whose weights are downloaded to your own machine and run on your own GPU, instead of being called over the internet from a company’s servers. Nothing you type leaves your computer, there is no subscription and no per-message cost, and the model keeps working if the company that made it disappears. The trade is that you supply the hardware, you do the setup, and the models you can realistically run at home are smaller than the ones the big platforms rent you.
What hardware do I need to run an AI RPG locally? Video memory is the constraint that matters, not system RAM and not the CPU. Roughly 8GB of VRAM runs a comfortable 7B–8B model, 12GB or more moves you to 13B-class models with real headroom, and 24GB or more opens up 30B-plus models or mid-size models at gentler quantization. Below about 8GB you can still run something, but you will be trading quality hard enough that a free hosted tier is usually the better experience.
Do any AI RPG platforms run on a local model? No. Across the 25 platforms we have scored, not one runs on hardware you own. Three — Janitor AI, WyrdTale and Chub AI — let you supply your own API key or your own Claude or ChatGPT subscription, which gives you control over which model answers and what it costs, but the model still runs on somebody else’s servers. Genuinely local play means leaving the platform directory and running a front-end like SillyTavern against a backend on your own machine.
Is a local model good enough for roleplay? For prose and character voice, a well-chosen 13B-class model at home is closer to a frontier hosted model than most people expect, and for uncensored content it is in a different category entirely because nothing is filtering it. Where local models lose is long-campaign coherence: you are usually running a smaller context window than a paid hosted tier gives you, so the thing that breaks first is the model remembering what happened forty turns ago.
Ollama or KoboldCpp — which should I use? Ollama if you want it working tonight. It handles model downloads and quantization choices for you, and the setup is close to installing any other application. KoboldCpp if you want control over how the model is loaded and sampled, or if you are running on hardware that needs coaxing. Both connect to the same front-ends and both run the same model files, so the choice is about how much of the machinery you want to see, not about what you end up playing.
Is running a model locally actually free? There is no subscription and no per-token cost, so after setup the marginal cost of a session is electricity. The cost you have already paid is the GPU, and the cost you keep paying is your own time — models need choosing, settings need tuning, and when something breaks there is no support team. Free in money, not free in effort.
Can I run an AI RPG locally without a good GPU? You can run one badly. A model can run on CPU and system RAM alone, but generation speed drops to the point where a single reply takes long enough to break the flow of a scene. If your machine cannot spare the video memory, the honest options are a hosted platform’s free tier, or renting model access by the token through a router rather than a subscription.