Canonical: https://krasis.ai/docs/configuration/ [Documentation](https://krasis.ai/docs/) Start here [Overview](https://krasis.ai/docs/)[Getting started](https://krasis.ai/docs/getting-started/) Everyday guides [Choosing a model](https://krasis.ai/docs/models/)[Connecting your apps](https://krasis.ai/docs/clients/)[Memory and speed](https://krasis.ai/docs/memory/)[Troubleshooting](https://krasis.ai/docs/troubleshooting/) Technical reference [Configuration](https://krasis.ai/docs/configuration/)[API and authentication](https://krasis.ai/docs/api/) For Krasis 1.0.22 [Release history](https://krasis.ai/releases/) Krasis documentation # Configuration reference Save a working profile, override settings and understand the main server controls. On this page [Start from a saved profile](https://krasis.ai/docs/configuration/#start-from-a-saved-profile)[Server and GPU selection](https://krasis.ai/docs/configuration/#server-and-gpu-selection)[Common saved keys](https://krasis.ai/docs/configuration/#common-saved-keys)[Precision controls](https://krasis.ai/docs/configuration/#precision-controls)[Context and memory](https://krasis.ai/docs/configuration/#context-and-memory)[Conversation-prefix caching](https://krasis.ai/docs/configuration/#conversation-prefix-caching)[Remote access and diagnostics](https://krasis.ai/docs/configuration/#remote-access-and-diagnostics) ## Start from a saved profile Use the launcher or Manager to create a working profile first. Launcher profiles live under `~/.krasis/config` (or `%USERPROFILE%\.krasis\config` on Windows). Reusing a generated profile preserves model-specific settings that a minimal hand-written file may miss. Config files use `KEY=VALUE` lines. Start one with: ``` krasis --config /path/to/profile.conf ``` Command-line flags override values in that config. For example, to change only the port: ``` krasis --config /path/to/profile.conf --port 8013 ``` The tables below cover common controls. For the complete catalogue and specialised modes, use the [1.0.22 advanced reference](https://github.com/brontoguana/krasis/blob/v1.0.22/ADVANCED.md). ## Server and GPU selection | Flag | Purpose | | ---------------------- | ----------------------------------------------------------------------------------- | | \--model-path PATH | Local Hugging Face checkpoint directory with safetensors and config files. | | \--selected-gpus 0,2 | Select visible GPU indices; verify the inventory on the machine running the server. | | \--num-gpus N | Limit how many GPUs to use. | | \--multi-gpu-mode auto | Measure and choose a topology supported by the loaded execution graph. | | \--pp-partition 24,24 | Explicit layer partition for a compatible multi-GPU setup. | | \--host ADDRESS | Bind address; default 0.0.0.0. Use 127.0.0.1 for local-only access. | | \--port N | Inference server port; default 8012. | | \--api-keys-file PATH | Custom authentication file; see [API keys](https://krasis.ai/docs/api/#api-keys). | A model’s architecture determines which multi-GPU topologies are available. Selecting `auto` does not imply that every architecture supports expert-serving peers. ## Common saved keys These are examples of individual lines to edit in an existing generated profile, not a complete model configuration: ``` CFG_HOST="127.0.0.1" CFG_PORT="8012" CFG_MAX_CONTEXT_TOKENS="8192" CFG_API_KEYS_FILE="/absolute/path/to/keys.json" ``` An empty or unset `CFG_API_KEYS_FILE` selects the default key-file location. A custom file must already exist and be valid at startup. Relative paths resolve from the server’s working directory. ## Precision controls | Flag | Choices and meaning | | ---------------------- | ------------------------------------------------------------------------------------------------- | | \--gpu-expert-bits | 4 or 8; default 4. | | \--cpu-expert-bits | 4 or 8; default 4. | | \--attention-quant | HQQ attention modes or bf16. Launcher suggestions differ from direct-server defaults. | | \--kv-dtype | Conventional k6v6, k4v4 or bf16 where supported; architecture-specific state has its own choices. | | \--shared-expert-quant | int8 or bf16; default int8. | | \--dense-mlp-quant | int8 or bf16; default int8. | | \--lm-head-quant | int8 or bf16; default int8. | | \--vision-quant | int4 or validation bf16; vision capability must be assessed separately. | The launcher offers HQQ4 and HQQ6 plus mixed promotion presets. Use its generated profile to preserve the exact representation of a mixed preset. Direct-server attention defaults to BF16, while fresh launcher selections normally use HQQ6\. Old profiles can retain older choices. AWQ attention and Polar4 cache are deprecated and disabled for new runs. BF16-heavy modes are validation profiles. See [Memory and speed](https://krasis.ai/docs/memory/) before changing precision. ## Context and memory | Flag | Meaning | | ----------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------- | | \--max-context-tokens N | Cap the effective context; 0 uses the model-declared limit. Actual memory must still support it. | | \--kv-cache-mb N | Cache-memory budget; documented default 1000 MB. | | \--vram-safety-margin N | Reserved VRAM safety margin in MB; documented default 600. | | \--stream-attention | Stream attention weights from CPU for large models. | | \--hcs-host-cache-mode source | RAM-saving host backing; mirror duplicates data for faster reloads, auto chooses according to available RAM. | | \--dynamic-hcs-tail-blocks N | Recency-adaptive expert-cache tail; valid 1..5, default 2. | | \--approved-heatmap-mode MODE | auto uses a compatible approved heatmap with local fallback; off uses local startup capture; require fails without a compatible approved artifact. | The corresponding saved keys include `CFG_MAX_CONTEXT_TOKENS`, `CFG_KV_CACHE_MB`, `CFG_KV_DTYPE`, `CFG_HCS` and `CFG_HCS_HOST_CACHE_MODE`. HCS expert residency is separate from conversation-prefix caching. ## Conversation-prefix caching To enable RAM-backed prefix-state caching in a saved profile: ``` CFG_PREFIX_CACHE="1" CFG_PREFIX_CACHE_RAM_FRACTION="0.25" ``` The fraction must be greater than zero and at most one. The budget follows available RAM rather than reserving a fixed amount. Cache hits require an exact compatible prefix and runtime signature. Image-input, speculative-decode and multi-GPU continuation requests have additional unsupported modes in this release. Fresh launcher selections and the direct server parser enable this feature by default. Existing saved profiles can override it; set it explicitly when reproducibility matters. Statistics are available at `GET /v1/session-cache/stats`, with an API key when authentication is enabled. ## Remote access and diagnostics `--ssh-tunnel user@host` sets up a reverse SSH tunnel; an optional `:ssh_port` selects the SSH port. `--ssh-key-path PATH` selects a key. The remote loopback port forwards to the local inference server. Use an existing authorised SSH destination. `--benchmark-only` runs a benchmark and exits. `--timing` adds per-layer instrumentation and overhead; leave it off for normal speed measurements. Experimental pruning, CPU-tail and speculative settings need their own quality and compatibility checks. See the [full reference](https://github.com/brontoguana/krasis/blob/v1.0.22/ADVANCED.md) before enabling them.