KoboldCpp Settings (2026): GPU Layers, Context Size, Threads and What the Defaults Are Now
KoboldCpp’s settings are the options you choose in its launcher before the model loads: how much of the model goes on the graphics card, how much context it can hold, and which acceleration it uses. Everything about the writing itself, such as temperature, is set later in the front end.
Most advice about these settings is out of date. In the last fifteen months KoboldCpp raised its default context size three times, turned flash attention on, renamed its main GPU flag, removed two backends and replaced manual GPU layers with an automatic fit. A guide written in 2024 describes a different launcher.
We read the argument definitions and the launcher code in KoboldCpp’s source on the concedo branch, which was version 1.122.1 on 10 October 2026, then the project’s wiki and its release notes back to 2023. Every default below is from the code. Where the wiki says something different from the code, we say so.
This page is about the launcher. To connect KoboldCpp to SillyTavern for the first time, see the KoboldCpp setup guide. If it will not start or will not use your graphics card, see KoboldCpp not working. If you are still choosing a backend, see KoboldCpp vs Ollama.
The Short Answer
- Leave GPU Layers at -1. That is the default, and since February 2026 it means autofit.
- Choose the context size on purpose. The default is now 16,384 tokens. On a card with little memory, a smaller context is the easiest way to get more speed.
- Set the same context size in your front end. In SillyTavern there is a checkbox that reads it from KoboldCpp.
- Leave threads on automatic.
- Flash attention, context shifting and fast forwarding are already on. Do not turn them off without a reason.
- Do not tick Low VRAM. The code’s own help text says it will be slow.
- Press Save Config. Then you do not have to do this again.
- Samplers are not here. Temperature and the rest belong to the front end.
Every Default, From the Code
This is what version 1.122.1 uses when you change nothing. The flag is the command line name. The launcher has the same options on its tabs: Quick Launch, Hardware, Context, Network and others.
| Setting | Flag | Default in 1.122.1 | What it controls |
|---|---|---|---|
| GPU layers | --gpulayers | -1 (autofit) | How much of the model is placed on the graphics card |
| Context size | --contextsize | 16384 | The largest prompt the loaded model accepts, in tokens |
| Threads | --threads | Calculated from your processor | Processor threads used for generation |
| Batch size | --batchsize | 512 | Tokens processed at once when reading the prompt |
| Physical batch size | --ubatchsize | -1 (same as batch size) | Added in 1.122.1 |
| Flash attention | --noflashattention | On | A more memory-efficient way to compute attention |
| KV cache type | --quantkv | f16 (not quantized) | The precision of the context cache |
| Context shifting | --noshift | On | Dropping old tokens without reading the prompt again |
| Fast forwarding | --nofastforward | On | Skipping the part of the prompt that did not change |
| SWA | --noswa | On for models that support it | A smaller context cache for some model families |
| Smart cache | --smartcache | Off | Snapshots of the context cache kept in RAM |
| mmap | --usemmap | Off | Reading the model from disk on demand |
| mlock | --usemlock | Off | Stopping the system from paging the model out |
| Low VRAM | --lowvram | Off | Keeping the context cache off the graphics card |
| High priority | --highpriority | Off | Raising the process priority. Marked experimental |
| Default amount to generate | --defaultgenamt | 2048 | Reply length when the front end does not send one |
| Chat template | --chatcompletionsadapter | AutoGuess | The instruct format for the chat completions endpoint |
| Port | --port | 5001 | Where the server listens |
| Host | --host | All routable interfaces | Which network addresses can reach it |
| Password | --password | None | An API key for the text endpoints |
| Request queue | --multiuser | 10 | How many requests may wait in line |
Two rows deserve a comment before the detail.
Host. The help text says that if the flag is not set, “all routable interfaces are accepted”. So a default KoboldCpp is reachable from other devices on your network, not only from your own PC. That is useful for playing from a phone. It also means that on a shared network you may want a password. See the network section below.
Context size. The number is high compared with what older guides assume, and it has moved. That has a cost, and it is the next thing to understand.
GPU Layers and Autofit
A model is a stack of layers. Each layer that is placed on the graphics card runs fast. Each layer that stays in system memory runs on the processor and is slow. “GPU layers” is the count of layers on the card.
For years this was a number you found by trial and error. That is no longer the default path.
- Version 1.104 (20 December 2025) added
--autofit, taken from the same feature in llama.cpp. The release note warned that it “overwrites all your manual layer configs and tensor overrides and is not guaranteed to work”. - Version 1.108.2 (18 February 2026) made it the default: “Autofit is now triggered if
-1gpulayers (default) is selected and tensor splits or tensor overrides are not set. Setting your own GPU layers overrides this behavior”.
The wiki describes what autofit does: it “will set the appropriate layers, MoE tensor overrides and tensor splits to try and use an optimal amount of memory.”
So the three values of the field mean this:
| GPU Layers | What happens |
|---|---|
-1 | Autofit. KoboldCpp decides, leaving a reserve of 1,024 MB by default (--autofitpadding) |
0 | No offload. The model runs on the processor |
| Any other number | Exactly that many layers go on the card. Autofit is off |
On a Mac, -1 is handled differently: the code prints “MacOS detected: Auto GPU layers set to maximum” and offloads everything.
How to see what it chose. Read the console window while the model loads. With the default you will see the line “GPU layers is default: Will enable AutoFit for increased estimation accuracy.” If the line is instead “Unable to detect VRAM, but autofit may still be used if applicable.” or “No GPU backend found, or could not automatically determine GPU layers”, KoboldCpp did not find your card, and the model may be running on the processor. That case is covered in KoboldCpp not working.
When to set the number yourself. Autofit is not compatible with a manual tensor split, manual tensor overrides, or the option that keeps Mixture of Experts weights on the processor. If you use any of those, autofit is off and you choose the layers. You may also want a manual number if autofit leaves a lot of card memory unused, or if the load fails and you want to step down.
The manual method has not changed, and SillyTavern’s own KoboldCpp page still explains it well. Load the model with a small number of layers. Read how much card memory those layers took. Divide to get the cost of one layer. Then divide your remaining memory by that cost. In the page’s own example, 9 layers used about 7 GB, so one layer cost about 778 MB, and 12 layers fitted a 12 GB card.
Be careful with the rest of that page. It describes KoboldCpp 1.58, which is from February 2024. It tells you to select “Use CuBLAS” and to think about “QuantMatMul”. The CUDA option has been renamed since, and the layer field it calls populated “when you loaded your model” now starts at -1.
Context Size
Context size is the largest prompt the loaded model will accept, counted in tokens. It is fixed when the model loads. To change it you load the model again.
The default has moved three times
| Version | Date | What the release note says |
|---|---|---|
| 1.96.2 | 17 July 2025 | ”Default context size is now 8k (up from 4k) to better represent modern models. This may affect your memory usage. Existing kcpps configs are unaffected.” |
| 1.99.4 | 21 September 2025 | ”Increased default context size and amount generated” |
| 1.122.1 | 26 September 2026 | ”Increased default contextsize.” |
The value in the 1.122.1 code is 16,384. The allowed range is 256 to 524,288.
This matters for a practical reason. The context cache lives in the same memory as the model. A bigger context takes more of the card, and what it takes is not available for layers. So when the default goes up and you update, autofit may place fewer layers on the card than before, and the same model on the same PC gets slower. Nothing is broken. The program is now reserving room for a longer conversation.
The 1.96.2 note also tells you who is not affected: anyone who loads a saved .kcpps file keeps the context size that is written in it.
Choosing a value
- If the whole model fits on your card with room to spare, the default is fine, and you can go higher if your model supports it.
- If only part of the model fits, try 8,192. For roleplay, 8,192 tokens is still a long stretch of conversation, and the memory you get back goes to layers.
- Do not go above what the model was trained for unless you know why. The launcher’s tooltip says the size is “Model specific”. KoboldCpp will stretch a model past its trained length with a technique called RoPE scaling, and the wiki says that comes with “minor to moderate quality degradation”.
Our guide to why LLMs forget explains what the context window does to a long campaign, and why a bigger number is not always a better story.
Make the front end agree
KoboldCpp and the front end each have a context setting, and they do not have to match. If SillyTavern is set to 8,192 and KoboldCpp was loaded with 16,384, you paid for memory you are not using. If SillyTavern is set higher than KoboldCpp, the prompt is cut.
SillyTavern has a checkbox for this, Derive context size from backend. In the 1.19.0 interface it is shown only for KoboldCpp and llama.cpp. Tick it and SillyTavern asks KoboldCpp what it was loaded with. KoboldCpp answers on an endpoint made for the purpose, /api/extra/true_max_context_length.
Where the wiki is behind
The wiki’s context section still talks about raising the context “above 2048”, a slider that “only goes up to 2048”, and a maximum of “32k for GGUF models”. The code’s default is 16,384 and its limit is 524,288. Trust the launcher over the wiki on this point.
Threads
Threads are the processor threads used for generation. The default is not a fixed number. The code calculates it, and the calculation is short:
- Take the number of logical processors and halve it. That is the guess for physical cores.
- If the result is 3 or less, use it. If it is more, subtract one, but never go below 3.
- On an Intel processor, never use more than 8. The comment in the code says why: “this helps avoid e-cores”.
- Never use more than 64.
| Logical processors | Threads on AMD | Threads on Intel |
|---|---|---|
| 4 | 2 | 2 |
| 8 | 3 | 3 |
| 12 | 5 | 5 |
| 16 | 7 | 7 |
| 24 | 11 | 8 |
| 32 | 15 | 8 |
The wiki’s advice agrees with the calculation: “The optimal number of threads to use is usually approximately equal to the number of physical CPU cores your system has.” It adds one line that is useful for players with a good card: “If running with full GPU offload, then setting 1 thread may be enough.”
BLAS threads (--blasthreads) is a second number, used while the prompt is being read. Left alone, it takes the same value as threads.
More threads than physical cores does not help and can make things slower. If your thread count looks low, that is on purpose.
Batch Size
Batch size is the number of prompt tokens processed at once. The default is 512. The allowed values are fixed steps from 16 to 4,096, and -1 turns batching off.
The wiki’s guidance is to leave it: “Usually, you do not need to change this value”. If you are short of memory, it suggests “lower values such as 128 for devices with less memory, at the expense of lower prompt processing speeds”. It is also the wiki’s first suggestion when you see an error about a failed allocation: reduce the context, then lower the batch size to 128 or 64.
Version 1.122.1 added a second value, the physical batch size (--ubatchsize). Its default of -1 means “the same as batch size”. Leave it.
Flash Attention and the KV Cache
Flash attention is a way of doing the same calculation with less memory, most noticeable at large context sizes. It had a slow start in KoboldCpp:
- Added as an “experimental flag” in version 1.64.1 (1 May 2024).
- Turned on by default in 1.99 (September 2025), then turned off again in the 1.99.1 hotfix, in the words of the note, “by popular demand”.
- On by default in the launcher from 1.106.2 (17 January 2026) and on the command line from 1.107.3 (31 January 2026).
So today it is on. The old advice to tick it is finished, and so is the old advice to avoid it on Vulkan: the limits and warnings for Vulkan were removed in version 1.91. The flag that exists now is the opposite one, --noflashattention.
KV cache quantization stores the context cache at lower precision to save memory. The option is --quantkv, and the values are f16 (the default, full size), bf16, q8_0, q5_1 and q4_0. Two facts from the code and the notes:
- It needs flash attention for the full effect. Without it, the help text says, “only K cache is quantized”.
- The values changed. Before version 1.112.2 (20 April 2026) you wrote a digit, such as
--quantkv 1. The note for that version calls the digits “deprecated, avoid using them”. Old command lines still work, but new ones should use the names.
If your card is almost big enough, q8_0 is the first step to try. It makes the context cache smaller, and that room goes to model layers. We did not measure the effect on writing quality and we will not guess at it.
Context Shifting, Fast Forwarding and SWA
These three decide whether KoboldCpp has to read your whole prompt again on every turn. Reading the prompt again is the long pause before the first word appears. In a long chat it is the difference between an answer in two seconds and an answer in forty.
Fast forwarding is on by default. The wiki: it “allows the AI to skip reused tokens in the context that have already been processed in the previous turn.”
Context shifting is on by default. When the chat is longer than the context, old tokens have to leave. Context shifting removes them from the cache and adds the new ones “without requiring any reprocessing”.
The wiki states the condition under which this works, and it is the important sentence for roleplay:
“So long as memory is not changed or edited and you don’t use world info, you should be able to avoid almost all reprocessing between consecutive generations even at max context.”
In SillyTavern terms: the early part of your prompt has to stay the same from one turn to the next. A lorebook entry that switches on and off near the top of the prompt changes it. So does an edited character description, a summary that is rewritten, or anything else inserted before the chat history. Each change means the prompt is read again from the point where it changed. This is not a fault in KoboldCpp. It is the price of dynamic prompts on a local model.
SWA is the setting that changed under people
SWA (sliding window attention) is a feature of some model families. It makes the context cache much smaller. The release notes name Gemma 4 as the main case.
- Until May 2026 it was off unless you asked for it. The developers explained the choice in the notes for version 1.111: “We give you the freedom to have it disabled by default so that Context Shift can work.”
- Version 1.114.1 (30 May 2026) reversed that, and marked it as a breaking change: “SWA is now enabled by default on all models that support it (was off by default previously).”
The two features cannot be used together. The help text for the flag says SWA “saves memory but cannot be used with context shifting.”
So if you run a model that supports SWA, and your chats began to pause for a full reprocess after an update in mid-2026, this is the likely cause. You have two choices:
- Keep SWA. You save memory, and long chats reprocess more often.
--swapaddingextends the window if it is too small. - Turn SWA off with
--noswa, or untick Allow SWA on the Context tab. Context shifting works again, and the context cache takes more memory.
Models without SWA are not affected. The wiki still says to use --useswa to turn it on, which describes the behaviour before 1.114.1.
Smart cache (--smartcache) is a different tool for a related problem. It keeps snapshots of the context cache in system memory, so when you go back to an earlier conversation the snapshot is loaded and the prompt is not read again. It is off by default and uses RAM for each slot. It is worth a look if you switch between characters often.
Smart context is the old method that context shifting replaced. The help text calls it “Outdated. Not recommended.”
Memory Options: mmap, mlock and Low VRAM
mmap reads parts of the model file from disk when they are needed. It used to be on. Since version 1.82 (18 January 2025) it is off: “mmap is no longer the default option.” Turn it on with --usemmap if the model is larger than your free memory.
mlock forces the model to stay in RAM so the system does not page it out. The help text says “Not usually recommended.” Since version 1.120 it can be combined with mmap.
Direct I/O (--usedirectio) arrived in 1.120 (29 August 2026). The help text says it “may improve cold-load times on some storage.” It cannot be used together with mmap.
Low VRAM keeps the context cache in system memory and off the card. It sounds like the right box for a small card. The help text says otherwise: “Not recommended, will be slow.” SillyTavern’s KoboldCpp page is more direct: “Do not tick Low VRAM, even if you have low VRAM.” Use a smaller context, a quantized KV cache or fewer layers first.
The Backend: CUDA, Vulkan or CPU
The backend is the acceleration method. There are three now.
| Backend | Flag | For |
|---|---|---|
| CUDA | --usecuda | NVIDIA cards |
| Vulkan | --usevulkan | AMD, Intel and NVIDIA cards |
| CPU | --usecpu | No graphics card |
If you choose none, KoboldCpp picks one. That has been true on the command line since version 1.75 (September 2024).
What changed:
--usecublasbecame--usecudain version 1.96.2. The old name still works.- OpenBLAS was removed in version 1.75. It became the single CPU option.
- CLBlast was removed. The 1.107.3 notes describe a new Vulkan option for old PCs that “replaces the removed CLBlast options”.
- Row split for multiple GPUs was removed in 1.118.1. The choices are layer split and tensor split.
The backend also depends on which file you downloaded. The release page describes them:
| File | For |
|---|---|
koboldcpp.exe / koboldcpp-linux-x64 | NVIDIA cards |
oldpc | An older processor or an older NVIDIA card (CUDA 11 and AVX1) |
nocuda | No NVIDIA card. Smaller, with Vulkan |
koboldcpp-mac-arm64 | Apple M-series Macs |
For AMD cards, the release notes recommend “trying the Vulkan option in the nocuda build first”. There is also a ROCm build for Linux.
Saving Your Settings
The launcher has two buttons, Save Config and Load Config. A saved configuration is a file that ends in .kcpps.
- Load Config reads a saved file back into the launcher.
--config file.kcppsstarts KoboldCpp from the file on the command line. The help text has a warning: “Other arguments will be ignored”. If you add flags next to--config, they do nothing.--showguiopens the launcher with the file’s settings filled in, so you can change one thing before starting.--exportconfig file.kcppswrites your current command line to a file.
A second kind of file ends in .kcppt. It is a template made to be shared. The wiki says templates “will not include device specific settings like the GPU to use”. If you download a template from someone else, your own PC still decides the hardware part.
If your settings are gone every time you open the launcher, this is the answer: save a config and open that. SillyTavern’s page puts it simply: “Click Save so you don’t have to configure KoboldCpp on every launch.”
One more reason to save. As the context section showed, a saved file protects you from a changed default.
Settings That Are Not in the Launcher
Samplers. Temperature, top P, repetition penalty and the others are sent by the front end with each request. In SillyTavern they are part of a preset. The wiki gives a set of safe starting values, “Top-P=0.92, RepPen=1.1, Temperature=0.7 and a sampler order of [6,0,1,3,4,2,5]”. SillyTavern’s code carries the same sampler order as its KoboldCpp default, so you do not need to set the order by hand.
The model. No setting here makes a model write better. That is the model’s job. See the best model for SillyTavern.
Default parameters. The launcher has one field, --gendefaults, for values to use when a front end sends nothing. It matters for simple tools that connect to KoboldCpp. SillyTavern sends its own values.
Thinking. For models that reason before they answer, two launcher options exist. --jinjathink, with the values default, true or false, switches thinking in the model’s own chat template, and that template is used only on the chat completions endpoint. --reasoningeffort, added in version 1.115.2, sets a default reasoning effort for requests that do not send one. If you connect SillyTavern through Text Completion, SillyTavern builds the prompt itself, so look at its instruct template and reasoning settings first.
Network Settings
- Port. 5001. Change it with
--portif something else uses it. - Host. All interfaces, as described above. To limit KoboldCpp to your own PC, set
--host 127.0.0.1. - Password.
--passwordsets an API key. The help text is precise about its reach: “This key will be required for all text endpoints. Image endpoints are not secured.” - Remote tunnel.
--remotetunnelcreates a Cloudflare tunnel and prints an address you can open from outside your network. Anyone with the address can use your model, so set a password when you use it.
Download It From the Right Place
One warning that is not about settings. The current release notes open with a notice from the developers about a fake website. In a post dated 26 September 2026 they say the address koboldcpp.com is held by someone else, that the site copies the project’s look, and that it tries “to mislead visitors into downloading malware”. They add that it sometimes ranks above the real project in search results.
We did not visit or test that site. We report what the project says. The real downloads are on the project’s GitHub releases page, under the account LostRuins.
A Sensible Starting Point
For a roleplay setup with SillyTavern, on one graphics card:
- Download the right file for your card from the GitHub releases page.
- Choose the model file on the Quick Launch tab.
- Leave GPU Layers at
-1. - Set Context Size to 8,192 if only part of the model fits your card, or leave 16,384 if all of it fits.
- Leave threads, batch size and flash attention alone.
- Press Save Config.
- Press Launch, and read the console for the autofit line and for how much memory was found.
- In SillyTavern, tick Derive context size from backend.
Then change one thing at a time, and only to solve a problem you can name:
| The problem | The setting to try first |
|---|---|
| The model is slow and the card is not full | Check the console found your card. Then set GPU layers by hand |
| The load fails or the PC runs out of memory | Lower the context size. Then --quantkv q8_0 |
| A long pause before every reply in a long chat | Check for SWA. Check what changes at the top of your prompt |
| It got slower after an update | Compare the context size with your old one |
| The model is bigger than your memory | --usemmap |
| Settings are lost each time | Save Config |
Frequently Asked Questions
What are the best settings for KoboldCpp?
For most people the best settings are the defaults, with two decisions of your own. Leave GPU Layers at -1 so that autofit decides how much of the model goes on the graphics card, and choose a context size that your card can hold and that matches the context size in your front end. Flash attention, context shifting and fast forwarding are already on.
How many GPU layers should I use in KoboldCpp?
Leave the field at -1. Since version 1.108.2, released on 18 February 2026, -1 turns on autofit, which works out the number of layers, the tensor split and the placement of the weights from your free memory. Set a number yourself only if autofit loads too little or the load fails. Zero means no GPU offload at all.
What context size should I use in KoboldCpp?
The default in version 1.122.1 is 16,384 tokens. A larger context uses more memory, so on a small graphics card a lower value such as 8,192 can let more of the model fit on the card and run faster. Whatever you choose, set the same number in SillyTavern or tick its option to derive the context size from the backend.
How many threads should KoboldCpp use?
Leave it on automatic unless you have a reason. The code takes half of your logical processors as the number of physical cores, subtracts one when that number is above three, and never uses more than eight on an Intel processor. The wiki says the best value is about the number of physical cores, and that one thread may be enough when the whole model is on the graphics card.
Does KoboldCpp save my settings?
It saves them when you ask. The launcher has Save Config and Load Config buttons, and the settings go into a file that ends in .kcpps. You can open that file directly, or start KoboldCpp with the config flag and the file name. When a config file is loaded, the other command line arguments are ignored.
Should I enable Low VRAM in KoboldCpp?
Usually not. The help text in the code says the option is not recommended and will be slow, because it keeps the context cache off the graphics card. SillyTavern’s own KoboldCpp page gives the same advice, even for cards with little memory. Lower the context size or the number of GPU layers first.
Is flash attention on by default in KoboldCpp?
Yes. It has been on by default in the launcher since version 1.106.2 of 17 January 2026 and on the command line since version 1.107.3 of 31 January 2026. You turn it off with the noflashattention flag or with the checkbox on the Hardware tab.
Where do I set temperature and other sampler settings for KoboldCpp?
Not in the launcher. Samplers such as temperature, top P and repetition penalty are sent by the front end with every request, so in a SillyTavern setup you set them in SillyTavern. The launcher only has a field for default parameters, which apply when a front end sends nothing.
The defaults, the allowed values, the thread calculation, the autofit conditions, the console messages and the help texts were read from koboldcpp.py on KoboldCpp’s concedo branch (version 1.122.1) on 10 October 2026. Dated changes are from the project’s GitHub release notes. Quoted explanations are from the project’s wiki, and we note the three places where the wiki is behind the code. The SillyTavern details are from its documentation page for KoboldCpp and from the 1.19.0 code. The fake-site notice is the developers’ own statement and we did not test it. We did not run benchmarks for this guide: we make no claim about how much faster or slower any setting is on a given PC. The starting point and the table of first things to try are our reading.
KoboldCpp is an open-source project. Arcanum is not affiliated with KoboldCpp, SillyTavern, NVIDIA, AMD, Intel or any project named here. Product names and trademarks are the property of their respective owners.