KoboldCpp vs Ollama for SillyTavern (2026): Which Local Backend Should You Run?
KoboldCpp and Ollama are two free programs that run an AI model on your own computer and let SillyTavern talk to it. They are closer than their reputations suggest. SillyTavern’s documentation describes both as built on llama.cpp, and both run GGUF model files.
The real difference is what each one decides for you, and what each one lets SillyTavern decide. Ollama hides most choices. KoboldCpp shows them. And SillyTavern, it turns out, does not send the same set of settings to both.
We read Ollama’s documentation, KoboldCpp’s source code (version 1.122.1) and wiki, and the part of SillyTavern’s code (version 1.19.0) that builds the request for each backend, all on 10 October 2026. We did not benchmark either program. There is no speed table on this page, and we explain below why we would not trust one.
Setup guides for each: SillyTavern + KoboldCpp and SillyTavern + Ollama. If you have not decided to run a model locally at all, start with local models for AI RPGs.
The Short Answer
- Choose Ollama if you want the shortest route to a first reply, you like downloading models by name, and the standard samplers are enough for you.
- Choose KoboldCpp if you want to load any GGUF file you find, use every sampler SillyTavern offers, and see exactly how the model is placed on your card.
- Either one is fine for a first local setup. They do not conflict, and you can keep both.
- The model matters more than the backend. The same GGUF file writes the same way in both, if the settings that reach it are the same.
Side by Side
| KoboldCpp | Ollama | |
|---|---|---|
| What it is | One file that you run. No installer | An installed app with a background service |
| How you get a model | Download a GGUF file yourself and open it | Pull a model by name from Ollama’s library |
| A GGUF file you already have | Opens directly | Needs a Modelfile and an import step |
| Where you pick the model | In KoboldCpp’s launcher, before it starts | In SillyTavern’s connection panel |
| Default context size | 16,384 tokens (version 1.122.1) | 4k, 32k or 256k, by card memory |
| Who sets the context in a SillyTavern chat | KoboldCpp, when the model loads | SillyTavern, with every request |
| SillyTavern samplers that reach the model | The full Text Completion set | Fifteen named settings |
| Reads the model’s prompt template for SillyTavern | Yes | No |
| Default port | 5001 | 11434 |
| Who can reach it by default | Any device on your network | Only the same PC |
| Unloads the model when idle | Not by default | After 5 minutes, unless told otherwise |
| Its own chat page | Yes, KoboldAI Lite | The Ollama app |
| Images, voice, speech input | Built in, each with its own model | Not part of what we read |
The rest of this page explains the rows that change how a roleplay session goes.
Getting a Model
This is the difference people notice first.
Ollama has a library, which SillyTavern’s documentation calls “a nifty catalog of models available for one-click download”. You type a model’s name and Ollama downloads it, in a version it has chosen. You do not handle a file.
KoboldCpp has no library. You find a GGUF file, usually on Hugging Face, choose the size that fits your card, download it, and open it in the launcher. The same documentation describes KoboldCpp as something that “runs from a single binary file on Windows, Mac, and Linux” and “supports GGUF models”.
For roleplay this matters more than it sounds. Many of the models that players recommend to each other are community finetunes, published as GGUF files by the people who made them. With KoboldCpp, any of those files works the moment it is on your disk.
With Ollama it also works, with a step in between. Ollama’s documentation describes it: create a text file named Modelfile that contains a line starting with FROM and the path to the GGUF file, and then create a model from it. The documentation adds a detail worth knowing: “Ollama does not quantize GGUF models during import.” You get the file as it is. SillyTavern’s page says the same thing in fewer words: Ollama “supports GGUF models wrapped in Ollama’s own format”.
So the fair summary is this. Ollama is easier when the model you want is in its library. KoboldCpp is easier when it is not.
Which model to choose is a separate question. See the best model for SillyTavern.
Context Size: Who Decides
Context size is how much of the conversation the model can see at once. It is the setting that most affects a long campaign, and the two programs handle it in opposite ways.
The defaults
KoboldCpp. The default in version 1.122.1 is 16,384 tokens, for every PC. It is set in the launcher and fixed when the model loads.
Ollama. The default depends on your card. From its documentation:
| Card memory | Ollama’s default context |
|---|---|
| Less than 24 GiB | 4k |
| 24 to 48 GiB | 32k |
| 48 GiB or more | 256k |
Most gaming cards have less than 24 GiB. So on a typical PC, Ollama used alone starts at 4k tokens, a quarter of KoboldCpp’s default. For a chat with a long character card and a lorebook, 4k is small.
With SillyTavern, the picture changes
Here is the part that is easy to miss. SillyTavern’s code puts its own context size into every request it sends to Ollama, in the field Ollama calls num_ctx. The value comes from the Context slider in SillyTavern.
So in a SillyTavern chat, Ollama’s 4k default does not apply. The SillyTavern slider does. That has two consequences.
- Good: you do not have to configure Ollama’s context at all. Set it in SillyTavern.
- Careful: a high slider value is a real request for memory. If you set 32,000 in SillyTavern and the card cannot hold that next to the model, part of the model moves to the processor, and the chat gets slow. Ollama’s documentation gives the command to check,
ollama ps, which shows aPROCESSORcolumn such as “100% GPU” and aCONTEXTcolumn with the size that was really allocated.
With KoboldCpp it is the other way around. KoboldCpp fixes the context at load time, and SillyTavern’s slider is only a limit on how much SillyTavern sends. To keep the two in step, SillyTavern has a checkbox, Derive context size from backend. In the 1.19.0 interface that checkbox is shown for KoboldCpp and llama.cpp. It is not shown for Ollama, which does not need it for the reason above.
Our KoboldCpp settings guide covers how the context size trades against speed.
The Settings SillyTavern Actually Sends
This is the finding we did not expect, and it is the strongest reason to prefer one backend over the other for roleplay.
When you move a slider in SillyTavern’s sampler panel, you assume the model receives it. For a local backend, that depends on which backend.
For Ollama, SillyTavern’s server builds the request from a fixed list. In version 1.19.0 the list has fifteen names:
| Sent to Ollama | What it is |
|---|---|
num_predict | Response length |
num_ctx | Context size |
num_batch | Batch size, only if set in SillyTavern’s config file |
stop | Stop strings |
temperature | Temperature |
top_k, top_p, min_p | The three common cut-off samplers |
typical_p, tfs_z | Two older samplers |
repeat_penalty, repeat_last_n | Repetition penalty and its range |
presence_penalty, frequency_penalty | The two penalties by count |
seed | Seed |
Everything else in the request is left out. The code picks these keys and no others.
For KoboldCpp, the request carries the full Text Completion set. The code adds KoboldCpp’s sampler order, the DRY sequence breakers and a grammar string on top.
What this means in practice:
- DRY and XTC, the two samplers many roleplay presets rely on to stop repeated phrases and tired word choices, are not in the Ollama list.
- Dynamic temperature, mirostat and a custom sampler order are not in it.
- Banned tokens, logit bias and grammar are not in it.
So a SillyTavern preset written for KoboldCpp can behave differently on Ollama. Nothing is wrong with the preset. Some of its settings never arrive. If your problem is a model that repeats itself, our guide to repetition loops explains what those samplers do. On Ollama, repetition penalty is the tool that reaches the model.
We want to be exact about the limit of this claim. It describes what SillyTavern 1.19.0 sends. It is not a statement about what Ollama can do by other routes, and the list can change in a later version.
The Prompt Template
A local model expects its prompt in a particular format, with particular markers around each turn. If the format is wrong, the writing gets worse or the model talks as you.
In both cases SillyTavern builds the prompt itself, using the instruct template you choose. For Ollama the code says so directly: it sends the request with raw set to true, which tells Ollama not to apply its own template.
The difference is in finding the right template. SillyTavern has options to derive the instruct and context templates from the backend, by reading the template stored in the model. In the 1.19.0 code, the check for whether a backend supports this names two: KoboldCpp and llama.cpp. Ollama is not in it.
So with KoboldCpp, SillyTavern can pick the template for you. With Ollama, you pick it by hand from the model’s page. It is a small job, and it is one more thing to get wrong.
Memory and the Graphics Card
Both programs decide automatically how much of the model goes on the card. They differ in how much you can see and change.
KoboldCpp shows the whole process in its console and gives you a control for each part: the number of GPU layers, an automatic fit that has been the default since February 2026, the batch size, the type of context cache, memory mapping, and settings for several cards. All of them are in the launcher and can be saved to a file.
Ollama makes the decision and reports the result. ollama ps shows how the model was split between card and processor. What can be changed is changed through environment variables for the whole service. The documentation lists, for example, OLLAMA_CONTEXT_LENGTH for the default context and OLLAMA_KV_CACHE_TYPE for the precision of the context cache, with the note that the second “is a global option - meaning all models will run with the specified quantization type.”
Two features exist in both:
| Feature | KoboldCpp | Ollama |
|---|---|---|
| Flash attention | On by default since January 2026 | Used “automatically when the selected backend and devices support it” |
| Quantized context cache | --quantkv, default f16, per launch | OLLAMA_KV_CACHE_TYPE, default f16, for all models |
If your model fits on your card with room to spare, you will not touch any of this in either program. If you are fitting a model that is a little too big, KoboldCpp gives you more to work with.
Why there is no speed table
Searches for this comparison often end in “speed” or “performance”. We did not test it, and we would be careful with anyone’s numbers, for three reasons that come from the documentation itself.
- The two programs start from different context sizes, and context size changes how much of the model fits on the card.
- Both choose how many layers go on the card automatically, with their own methods and reserves.
- SillyTavern sends them different sampler sets.
A test that does not fix all three is measuring the defaults, not the engines. If speed is your question, load the same file in both with the same context size, check that both report the whole model on the card, and time it on your own PC.
Staying Loaded
Ollama unloads a model when it has not been used for a while. Its documentation: “By default models are kept in memory for 5 minutes before being unloaded.” The next message after that has to load the model again, which takes time.
SillyTavern changes this as well. Its config file has an ollama.keepAlive setting, and the shipped value is -1, which the file’s own comment explains as “Keep the model loaded indefinitely”. SillyTavern sends that with each request. So once you have chatted through SillyTavern, the model stays in your card’s memory until you stop Ollama or load something else. If a game then runs badly, this is why.
KoboldCpp loads the model when it starts and keeps it until you close the window. That is simpler to understand: if the window is open, the memory is in use.
Network Defaults
The two programs start with opposite defaults.
| KoboldCpp | Ollama | |
|---|---|---|
| Listens on | All network interfaces | 127.0.0.1 only |
| Port | 5001 | 11434 |
| To change | --host | The OLLAMA_HOST environment variable |
| Password | Optional, --password, for text endpoints | We found none in the pages we read |
For KoboldCpp, the help text for the host option says: “If this flag is not set, all routable interfaces are accepted.” For Ollama, the documentation says: “Ollama binds 127.0.0.1 port 11434 by default.”
What this means for a player:
- Playing from your phone at home. KoboldCpp can be reached from another device on your network as it comes. Ollama has to be told to listen first. But note that you normally connect your phone to SillyTavern, not to the backend, and SillyTavern talks to the backend on the same PC. In that setup neither default matters. See SillyTavern on mobile.
- On a shared network. Ollama’s default is the more private one. With KoboldCpp, set
--host 127.0.0.1or a password if you do not want other devices to reach it. - From outside your home. KoboldCpp has a built-in flag,
--remotetunnel, that creates a public address. Ollama’s documentation describes doing the same with separate tools.
What Else Each One Does
KoboldCpp is several programs in one file. Besides text, it can load an image model, a voice model and a speech recognition model, each from its own tab in the launcher, and it serves its own chat page, KoboldAI Lite, with story, adventure and chat modes. For a SillyTavern player that means one program can supply text, images and voice, if the card has room for all of them. It can also be the whole setup with no SillyTavern at all, which is one of the routes in our SillyTavern alternatives guide.
Ollama is a model service with an app. Its strength is that many other tools connect to it. Its documentation covers an OpenAI-compatible API and mentions cloud models next to local ones. We did not read further into those for this page, because they are not what a SillyTavern roleplay setup uses.
One bridge between the two: KoboldCpp can pretend to be Ollama. Its wiki says there is “a simple Ollama compatible API” for tools that only support Ollama, and adds, in capitals, that for normal usage this “is NOT recommended”. Use it for a tool that gives you no other choice. For SillyTavern, use the KoboldCpp connection type.
Changing Models
With Ollama, SillyTavern’s connection panel lists the models Ollama has, and you choose one there. SillyTavern’s server also has a route that asks Ollama to download a model. You can move between models without leaving SillyTavern.
With KoboldCpp, the model is chosen in the launcher. To change it you close KoboldCpp and start it with another file, or use its admin mode, which the wiki describes for switching configurations while it runs.
If you like trying a different model every evening, Ollama’s way is more comfortable. If you settle on one model for a campaign, it makes no difference.
What About LM Studio and Oobabooga?
They come up in the same searches, so here is what we can say from the sources we read. We did not read LM Studio’s or oobabooga’s own documentation for this page, and we do not compare them in detail.
LM Studio. It is not one of the named Text Completion types in SillyTavern 1.19.0. The types are a fixed list, which includes KoboldCpp, llama.cpp, Ollama, oobabooga and TabbyAPI, plus a generic option. LM Studio appears in the code once, in a comment beside some alternative setting names: “In case someone wants to use LM Studio as Text Completion API”. So it connects through a general OpenAI-compatible route, not through a connection made for it.
Oobabooga (Text Generation WebUI). It is a named type. SillyTavern’s documentation describes it as having the “broadest support” for model formats, lists several formats beyond GGUF, and adds a caution: “Regular updates, which can sometimes break compatibility with SillyTavern”.
llama.cpp itself. SillyTavern’s documentation calls it “the original source from which KoboldCpp and Ollama were forked”, with a “lightweight CLI interface”. It is a named type, and it shares KoboldCpp’s ability to give SillyTavern the model’s template.
For most players choosing a first backend, the choice is still between the two on this page.
Which One Should You Run?
You are setting up a local model for the first time. Start with Ollama. One install, one model name, and SillyTavern sets the context for you. Our Ollama guide has the steps. Keep in mind that the instruct template is yours to choose.
You follow roleplay model recommendations from the community. Use KoboldCpp. Those models arrive as GGUF files, and the presets shared with them often use samplers that SillyTavern does not send to Ollama.
Your model repeats itself and you want every tool against it. Use KoboldCpp, for the same reason.
Your card is a little too small for the model you want. Use KoboldCpp. The controls for fitting a model are in front of you. See KoboldCpp settings.
You also use coding tools or other apps that expect Ollama. Keep Ollama for those, and decide separately for roleplay. Running both is normal.
You want images and voice from the same program. KoboldCpp.
You do not want to think about any of this. Ollama, or no local model at all. A cloud model through an API key is less work than either. See the best LLM for roleplay.
If something goes wrong after you choose, see KoboldCpp not working or the troubleshooting section of the Ollama guide.
Frequently Asked Questions
Is KoboldCpp or Ollama better for SillyTavern?
KoboldCpp is the better fit if you want every SillyTavern sampler to reach the model and you want to load any GGUF file directly. Ollama is the better fit if you want the shortest path to a working setup and you like choosing and downloading models by name. Both run the same kind of model file, and SillyTavern can connect to both.
Is KoboldCpp faster than Ollama?
We did not measure it, and we do not publish a speed claim we have not tested. SillyTavern’s documentation describes both as based on llama.cpp. In practice, speed depends mostly on how much of the model fits on your graphics card and on the context size, and both programs let you control those.
Does SillyTavern send all its sampler settings to Ollama?
No. In SillyTavern 1.19.0 the server keeps fifteen settings for an Ollama request, such as temperature, top K, top P, min P, repetition penalty and the context size, and drops the rest. Samplers outside that list, including DRY, XTC, dynamic temperature and a custom sampler order, are not sent to Ollama. KoboldCpp receives them.
What is Ollama’s default context size?
Ollama’s documentation says the default depends on your graphics card’s memory: 4k tokens with less than 24 GiB, 32k with 24 to 48 GiB, and 256k with 48 GiB or more. When you use SillyTavern, the context size in SillyTavern is sent to Ollama with every request, so the SillyTavern slider is the value that applies.
Can Ollama run a GGUF file I downloaded?
Yes, with one extra step. Ollama’s documentation says to create a small text file called a Modelfile that points at the GGUF file, and then to create a model from it. KoboldCpp opens a GGUF file directly, with no import step.
Can I use KoboldCpp and Ollama on the same PC?
Yes. They use different ports, 5001 for KoboldCpp and 11434 for Ollama, and SillyTavern stores a separate connection for each. Do not load large models in both at once, because they share the same graphics card memory.
Does KoboldCpp work with tools made for Ollama?
Partly. KoboldCpp has an Ollama-compatible API so that tools which only support Ollama can connect to it. The project’s own wiki says this is not recommended for normal use and exists for compatibility. For SillyTavern, use the KoboldCpp connection type.
Ollama’s default context sizes, its network default, the five-minute unload, the environment variables, flash attention and the GGUF import steps are from Ollama’s documentation, read on 10 October 2026. KoboldCpp’s defaults and help texts are from its source code on the concedo branch (version 1.122.1), and the quoted explanations from its wiki. The list of settings sent to Ollama, the raw flag, the keep-alive value, the template derivation check, the context checkbox and the list of connection types are from SillyTavern’s code and shipped config file on the release branch (version 1.19.0). The one-line descriptions of each backend are from SillyTavern’s documentation. We did not benchmark either program, we did not compare how the same model writes on each, and we did not test LM Studio or oobabooga. The recommendations by kind of player are our reading.
KoboldCpp, Ollama, SillyTavern, llama.cpp, LM Studio and Text Generation WebUI are separate projects. Arcanum is not affiliated with any of them. Product names and trademarks are the property of their respective owners.