Canonical: https://krasis.ai/releases/

 Stable releases

# What’s changed.

Features, fixes and known limits for each published stable release, newest first.

These are the original notes for each version. Older installation instructions, performance results and support claims describe that release. Use the [current installation guide](https://krasis.ai/install/) for setup, or visit [prerelease downloads](https://krasis.ai/prerelease/) for builds in testing.

v1.0.22 2026-10-25 

[Original release notes and assets](https://github.com/brontoguana/krasis/releases/tag/v1.0.22)

Krasis 1.0.22 adds automatic JSON API-key authentication to the inference server, including TUI launches.

* Create `~/.krasis/api-keys.json` containing client names mapped to secret keys. Existing installations remain unauthenticated until this file exists.
* Clients use standard `Authorization: Bearer <key>` headers. Additions and removals take effect on new requests without restarting.
* Empty mappings deny access. Invalid, unreadable or removed files fail closed once authentication is enabled. Explicit paths must exist at startup.
* Override the path with `CFG_API_KEYS_FILE` or `--api-keys-file`; settings survive TUI and Manager handoffs.
* Requests are authenticated before body processing or model queue admission. Health checks and CORS preflight remain public.

Setup details: <https://github.com/brontoguana/krasis/blob/v1.0.22/ADVANCED.md#api-keys>

This release contains the API-key changes on the 1.0.21 stable baseline. The ongoing MiniMax M3.1 implementation and qualification work is not included. Existing model qualification limits remain unchanged.

Linux Python 3.10–3.13 wheels and the native Windows x64 installer are built by the release workflows and appear as each build completes. NVIDIA Ampere or newer is required; macOS is unsupported.

v1.0.21 2026-10-22 

[Original release notes and assets](https://github.com/brontoguana/krasis/releases/tag/v1.0.21)

Krasis 1.0.21 is available for Linux x86-64 and native Windows x64\. Download and setup: <https://krasis.ai/install/>

* Native Windows installer includes a private Python runtime, Start Menu launcher and Krasis Manager. Exact-commit CI validates packaging and the install/reinstall/uninstall lifecycle.
* Linux wheels support Python 3.10–3.13 with vendored CUDA sidecars. The installer selects the matching Linux wheel even when Windows assets share the release.
* Includes the verified GLM conversation/cache runtime and CUDA ownership/stream fixes, pinned model downloads, terminal navigation/editing fixes, saved precision controls and reliable Q-then-Enter shutdown.
* Adds experimental single-GPU Qwen3.5 dense text support. The rc.19 runtime passed a fresh GPU Podman install and actual TUI launch of Qwen3.5-27B, answered 703 then 714 in conversation, preserved 58 settings and shut down cleanly. Stable changes from rc.19 are version/documentation only.

Requirements: NVIDIA Ampere or newer, current NVIDIA drivers, and sufficient system RAM, VRAM and disk for the chosen model. macOS is unsupported. Windows GPU inference has not been hardware-qualified by the hosted CI lifecycle tests.

Model-specific limits remain: dense Qwen3.5 is experimental single-GPU text with a retained strict numerical-reference outlier; vision is unqualified. Its default 27B BF16 profile needs more than 32 GB VRAM. GLM qualification applies only to the exact tested scales-only, thinking-enabled text configuration and retains its numerical warning. Default INT4 profiles do not inherit that qualification. DeepSeek V4.1 Flash 1017 and Flint are not supported.

Install Linux using install.sh, or Windows using KrasisSetup-1.0.21-win64.exe. Checksums and exact-source Windows provenance accompany the release.

v1.0.20 2026-08-04 

[Original release notes and assets](https://github.com/brontoguana/krasis/releases/tag/v1.0.20)

# Krasis v1.0.20

This patch release restores the validated Windows release path for Krasis.

## Fixed

* Fixed Windows Rust test-target compilation for the CPU-tail affinity query. Linux continues to use the process affinity mask; platforms without the implemented native affinity path report no eligible CPUs, and calibrated CPU-tail affinity still fails visibly rather than silently degrading.
* The fix unblocks the mandatory Windows server response-contract gate. It does not change model execution, CUDA kernels, VRAM calibration, quantization, or Linux performance.

## Windows validation

The Windows installer was built from the exact v1.0.20 source and passed:

* Windows wheel and CUDA sidecar compilation.
* Rust HTTP/SSE response contracts.
* Native launcher build and tests.
* Private-runtime payload verification.
* Clean install, isolated launch, legacy repair, and safe uninstall lifecycle tests.
* Exact-commit provenance and SHA-256 verification before release promotion.

Windows users can download `KrasisSetup-1.0.20-win64.exe` from the release assets. A CPython 3.12 Windows wheel is also published for the installer/runtime workflow.

All model support, tool-use improvements, performance results, and known limitations documented for v1.0.19 remain unchanged.

v1.0.19 2026-08-04 

[Original release notes and assets](https://github.com/brontoguana/krasis/releases/tag/v1.0.19)

# Krasis v1.0.19

This release fixes latent GPU memory corruption in the shared split-dense prefill path and brings the Rust chat-template renderer closer to Hugging Face/Jinja behavior. Users of v1.0.18 should upgrade: the scratch bug could overrun mapped GPU memory without failing immediately, including during startup calibration.

## Critical runtime fix

* Fixed split-dense MLP activation reusing routed-MoE scratch that was too small for dense-dominant model geometries.
* A captured GLM-4.7 request required 48,117,760 BF16 activation elements but had capacity for 35,162,112, overrunning by 25,911,296 bytes. The 39,920-token startup calibration overran by 314,441,728 bytes and survived only because adjacent GPU allocations happened to be mapped.
* Allocation, exact request admission, and the linear pre-budget now use the larger runtime-derived requirement:  
`max(prompt_tokens × dense_intermediate_size, fused_sorted_rows × moe_intermediate_size)`
* Split-dense launches now fail closed if the shared buffer is undersized. There is no GLM-specific exception, request cap, fixed hardware value, or fallback.
* Temporary fault-localization synchronization was removed after diagnosis, so the fix adds no diagnostic environment lookup to the normal per-layer path.

## Chat-template and tool-use fixes

* Matched Hugging Face/Jinja template behavior for `trim_blocks`, `lstrip_blocks`, Python boolean/`None`/container rendering through `string`, `tojson(ensure_ascii=...)` and indentation, and both `from_json` and `fromjson`.
* Nemotron’s pinned tool prompt now renders byte-for-byte identically in Krasis and Hugging Face without a model-name special case.
* Hardened GLM parsing for an observed complete doubled `<tool_call>` wrapper. Only the balanced duplicate form is normalized; malformed or truncated output remains visible text.
* Added an isolated, pinned Opencode Podman harness for repeatable real-client streaming tool tests.

### Tool-use verification status

| Family                      | Krasis grammar support                      | Live client evidence                                                                                              |
| --------------------------- | ------------------------------------------- | ----------------------------------------------------------------------------------------------------------------- |
| DeepSeek-V4-Flash           | DSML                                        | Opencode streaming round trip passed                                                                              |
| Qwen3-Coder-Next            | Function XML                                | Opencode round trip passed in the v1.0.18 release gate                                                            |
| Ornith                      | Function XML                                | Ornith-397B streaming and non-streaming round trips passed                                                        |
| Step-3.7-Flash              | Function XML                                | Opencode round trip passed with a concise agent prompt                                                            |
| Gemma 4                     | Native call grammar                         | Opencode and direct non-streaming round trips passed                                                              |
| GLM-4.7 / GLM-5.2           | Argument XML                                | GLM-4.7 Opencode plus direct streaming/non-streaming passed; runtime remains preview. GLM-5.2 was not live-tested |
| Qwen3 / Qwen3-235B          | JSON                                        | Grammar/unit-tested only                                                                                          |
| Qwen3.5 / Qwen3.6           | Function XML                                | Grammar/unit-tested only                                                                                          |
| MiniMax M2.x                | Native MiniMax grammar                      | Grammar/unit-tested only; no complete local M2.7 checkpoint was available                                         |
| Nemotron-3 Nano / Super     | Function XML                                | Parser/unit-tested; Super was live-tested but did not reliably emit a native call                                 |
| DeepSeek-V2 / V2-Lite / VL2 | No grammar in the shipped fallback template | Not supported; no invented fallback                                                                               |

Grammar transport support does not guarantee that every quantized checkpoint will choose a tool reliably.

## Reference gate correction

* Fixed a false-PASS path where one strong sampled prefill position could hide a failed multi-token autoregressive continuation.
* Multi-token artifacts now have to satisfy both the established prefill and decode criteria. Single-token and decode-only artifact behavior is preserved.
* `./dev server-test` now covers response parsing, chat-template rendering, and reference-verdict contracts.

## Developer command cleanup

* Removed general `./dev` config shortcuts that resolved to rejected `fp8_e4m3` or `polar4` KV configurations: `q235`, `v2lite`, `deepseek-vl`/`vl2`, `qwen35`, and `minimax`.
* Supported general shortcuts are now `qcn`, `dsv4`, and `gemma`. Other models can still be launched with an explicit validated `.conf` path. Witness/reference selectors remain separate.

## Known limitations

* Nemotron-3-Super’s current INT4/HQQ4 production path does not reliably emit tool calls and shows material autoregressive quality degradation on a difficult llama-witness sequence. It matched 0/50 generated tokens and retained the witness token in its top 10 for 4/50 positions, despite teacher-forced agreement of 44/50 top-1 and 50/50 top-10 on the witness path. This is a model-quality issue, not a prompt-render or parser issue.
* MiniMax-M2.7 tool grammar is unit-tested, but the local checkpoint is incomplete. The roughly 214 GiB current checkpoint was not downloaded for this release, so no live MiniMax round trip is claimed.
* Large agent system policies and many schemas count against model context and can degrade tool selection. Opencode’s full Build policy plus six tools exceeded Gemma 4’s validated context and caused Step-3.7 to repeat prompt fragments; both completed structured round trips with the concise verification agent.
* GLM-4.7 remains a preview runtime even though its live tool round trip and the captured six-tool stress request pass after the scratch fix. GLM-5.2 remains preview and was not live-tested in this campaign.

## Verification

Fresh final-source gates passed:

* Rust server parser: 18/18
* Chat templates: 12/12
* Reference verdict contracts: 5/5
* Split-dense scratch geometries: 2/2
* Model/config matrix: 17/17
* Launcher matrix: 24/24
* DeepSeek-V4 math matrix: 20/20
* Locked Cargo metadata, exact-source build, 1.0.19 CPython 3.11 wheel, and import-origin verification

The tested dependency graph is unchanged from v1.0.18; only Krasis’s own package version changed in `Cargo.lock`.

v1.0.18 2026-08-04 

[Original release notes and assets](https://github.com/brontoguana/krasis/releases/tag/v1.0.18)

# Krasis v1.0.18

This release fixes structured tool calling across the shipped model families. DeepSeek-V4-Flash previously received tool definitions correctly but returned its native DSML invocation as plain assistant text, so clients such as Opencode could not execute it. Krasis now translates each supported template’s native grammar to OpenAI-compatible `tool_calls` in both streaming and non-streaming responses, with `finish_reason: "tool_calls"`.

## Tool-use support

| Model family                | Native grammar                              | Status                                   |
| --------------------------- | ------------------------------------------- | ---------------------------------------- |
| DeepSeek-V4-Flash           | Typed DSML                                  | Supported                                |
| Qwen3 base / Qwen3-235B     | JSON in <tool\_call>                        | Supported                                |
| Qwen3-Coder-Next            | Function/parameter XML                      | Supported                                |
| Qwen3.5 / Qwen3.6           | Function/parameter XML                      | Supported                                |
| Ornith 35B / 397B           | Function/parameter XML                      | Supported                                |
| Step-3.7-Flash              | Function/parameter XML                      | Supported                                |
| Nemotron-3 Nano / Super     | Function/parameter XML                      | Supported                                |
| GLM-4.7 / GLM-5.2           | arg\_key / arg\_value XML                   | Supported; model runtime remains preview |
| Gemma 4                     | Native call:name{...}                       | Supported                                |
| MiniMax M2.x                | Native minimax:tool\_call                   | Supported                                |
| DeepSeek-V2 / V2-Lite / VL2 | No grammar in the shipped fallback template | Not supported                            |

Additional fixes and safeguards:

* Typed parameters retain strings, numbers, booleans, objects, arrays, and null.
* Multiple tool calls, assistant tool-call history, and tool-role results work.
* Streaming buffers native markup and emits structured OpenAI name/argument chunks.
* DeepSeek’s observed abbreviated inner DSML tags are handled within a canonical DSML container.
* Gemma grammar delimiters are selectively preserved without leaking unrelated special tokens.
* Unsupported templates fail visibly; malformed or truncated blocks remain visible text.
* The `qcn` development alias now resolves the validated INT4/HQQ4/k4v4 config.

## Verification

* Rust server contracts: 17/17.
* DeepSeek math: 20/20.
* Model/config matrix: 17/17.
* Launcher matrix: 24/24.
* Live structured call plus tool-result round trip: DeepSeek-V4-Flash, Qwen3-Coder-Next, and Gemma 4.
* DeepSeek streaming verified with structured name and argument chunks, `finish_reason: "tool_calls"`, and no DSML leakage.
* Approved heatmaps loaded without quick rebuilds in all live tests.

No model math, quantization, quality baseline, or benchmark path changed in this release.

v1.0.17 2026-08-03 

[Original release notes and assets](https://github.com/brontoguana/krasis/releases/tag/v1.0.17)

# Krasis v1.0.17

Krasis v1.0.17 adds release-grade DeepSeek-V4-Flash-0731 support, improves long-context prefill and Ornith-397B decode, and strengthens measured VRAM calibration.

## DeepSeek-V4-Flash-0731

* Added the model to the curated downloader and launcher as `dsv4` / `deepseek-v4`.
* Added native Rust/CUDA support for its compressed sparse-attention, learned-index, hyper-connection, and MoE runtime paths.
* Published a matching 80-prompt approved HCS heatmap for fresh installations.
* Quality gates passed: llama-witness 4/4, full 281-window WikiText-2 PPL 4.8214 versus the frozen scalar INT4 anchor of 4.8159 (+0.114%), DeepSeek math 20/20, model/config 17/17, and launcher 24/24.
* Timing-disabled prefill on one RTX PRO 6000 96 GB: 152.2 tok/s at 1K, 320.6 at 2K, 906.3 at 8.6K, 1,328.2 at 23K, and 1,204.3 at 62K.
* Internal decode: 29.38 / 28.20 / 28.49 tok/s for 50/100/250-token generations at approximately 1K context, and 19.41 tok/s at 62K context.
* The long-context GEMM path has a stated short-context tradeoff: 1K prefill is lower than the preceding scalar-index runtime (152.2 versus 193.0 tok/s), while 8.6K–62K prefill improves substantially.

## Runtime and calibration

* Startup VRAM monitoring now samples calibration transients at 1 ms rather than allowing short allocation lows to escape a coarse monitor.
* Long-prompt calibration measures a successive prompt-length curve through the configured maximum instead of extrapolating from a short probe.
* Request admission and HCS reload continue to use the loaded model’s measured transient requirements and the configured 600 MiB safety margin; no model- or GPU-specific headroom constants were added.
* cuBLAS workspaces used by the new prefill paths are bound to preallocated runtime storage so the first request cannot trigger an unbudgeted lazy allocation.

## Ornith-1.0-397B

* Runtime-derived route-preparation and deterministic parallel top-k improvements raised timing-disabled 50/100/250-token decode from 23.64/21.91/20.27 to 26.23/24.16/22.27 tok/s, with unchanged prefill/VRAM behavior.
* The full Rust perplexity path scored 297,053 tokens in 291 windows at PPL 3.7727.

## GLM preview work

* This release includes experimental GLM-4.7/GLM-5.2 sparse DSA, compact MLA, exact-prefix, tokenizer-backend, and CPU-tail foundations.
* GLM remains preview/experimental in v1.0.17\. Its current witness and benchmark evidence is useful for development but has not completed the same full release-quality validation battery as DeepSeek-V4-Flash-0731.

Krasis continues to target Ampere and newer NVIDIA GPUs. The production hot paths remain Rust/CUDA, with runtime-measured VRAM/HCS decisions and full-GPU prefill.

v1.0.16 2026-07-27 

[Original release notes and assets](https://github.com/brontoguana/krasis/releases/tag/v1.0.16)

Krasis v1.0.16 is the first stable release of the current Rust/CUDA-focused runtime line.

Highlights:

* Native Windows installer with sealed private runtime and maximized Krasis Start Menu launcher
* Full Windows Marlin, FlashAttention, and FLA sidecar support for Ampere and newer NVIDIA architectures
* Standards-complete OpenAI-compatible JSON serialization, including Windows model paths
* Fast full-GPU prefill and Rust/CUDA decode with measured HCS/VRAM management
* Expanded Qwen, Ornith, Step, Gemma, and Nemotron MoE support
* Content-addressed Windows sidecar reuse and exact preflight-tested installer promotion

Windows updates currently use manually downloaded installers from this release page. The temporary rc.10 updater shortcuts are removed automatically when installing v1.0.16.

v1.0.15 2026-06-05 

[Original release notes and assets](https://github.com/brontoguana/krasis/releases/tag/v1.0.15)

Krasis v1.0.15 hardens VRAM pressure recovery and Typhon stability while preserving measured safety-floor behavior.

Highlights:

* Startup/server-ready now drains and resamples HCS pressure before publishing ready.
* CUDA illegal-address faults in prefill/scratch/source-HCS sync now exit immediately instead of continuing with a poisoned context.
* Request-time pressure recovery now drains before chat prefill, uses request token counts for HCS prefill eviction, and preserves measured post-scratch safety bands for optional pinning.
* Q122B prefill scratch and chunk planning improvements remain included.

Validation:

* ./dev build passed.
* ./dev release-test QCN passed all 3 configs.
* llama-witness validation passed 4/4 on each config.
* Health scan clean for CUDA errors, VRAM monitor warnings, hard-floor exits, and nonzero HCS copy failures.

Release-test speeds:

* INT4 k4v4 HQQ4: 6433.2 prefill, 90.36 decode, 158.41 HTTP.
* INT4 k6v6 HQQ6+8: 6377.3 prefill, 88.96 decode, 144.13 HTTP.
* INT8 k6v6 HQQ6+8 multi-GPU: 5000.9 prefill, 40.30 decode, 68.01 HTTP.
v1.0.14 2026-06-04 

[Original release notes and assets](https://github.com/brontoguana/krasis/releases/tag/v1.0.14)

Krasis v1.0.14

Changes:

* Fixed VRAM pressure handling after Typhon cleanup-time lows below the safety margin could leave CUDA in an unsafe state.
* Fixed ./dev release-test model alias resolution so documented aliases such as QCN resolve before invoking the guarded release-test runner.
* Includes recent main fixes for scoped benchmark GPU cleanup, disabled FP8 KV cache modes, and SSH tunnel key-path support.

Validation:

* ./dev build passed and built krasis-1.0.14-cp311-cp311-linux\_x86\_64.whl locally.
* ./dev release-test QCN passed all 3 configs.
* QCN speed test after the Typhon pressure fix completed cleanly with no CUDA errors, VRAM monitor warnings, hard-floor exits, or HCS copy failures.
v1.0.13 2026-05-30 

[Original release notes and assets](https://github.com/brontoguana/krasis/releases/tag/v1.0.13)

* Stable release of the Witsy/Qwen vision compatibility and dynamic HCS safety fixes from the `1.0.13` prerelease line.
* When a loaded model supports Qwen image inputs, `/models` now advertises only the Witsy/OpenAI-compatible `-vision` model id instead of listing separate text and vision entries. Use the same `-vision` id for both text and image requests; text-only requests still follow the normal text path.
* Chat completions now accept OpenAI’s `max_completion_tokens` field as an alias for `max_tokens`, matching current OpenAI SDK clients such as Witsy.
* The Hugging Face downloader now includes `preprocessor_config.json` and `processor_config.json`, which are required for Qwen image preprocessing.
* Pillow is now declared as a package dependency because the Qwen image preprocessing path requires `PIL` in clean installs and containers.
* Quieted transient socket read timeouts / `EAGAIN` (`os error 11`) during HTTP request parsing; incomplete probe connections are ignored without printing a scary error, while malformed requests still return `400`.
* Fixed source-mode dynamic HCS VRAM-pressure eviction synchronization. When Krasis trims soft HCS chunks after a below-safety VRAM event, it now synchronizes the CUDA context before freeing source-backed dynamic HCS chunks, matching the decode/prefill boundary safety used by normal prefill eviction.
* Fixed dynamic HCS promotion ordering during graph decode. Promoted expert slot contents and the GPU expert pointer table are now updated on the same CUDA replay stream, preventing later graph/prefill work from seeing stale or incompletely ordered promoted pointers after image-heavy requests on tight VRAM systems.
v1.0.12 2026-05-29 

[Original release notes and assets](https://github.com/brontoguana/krasis/releases/tag/v1.0.12)

* Improved OpenAI-compatible model discovery for clients such as Witsy. The model-list endpoint now accepts `/models` as well as `/v1/models`, tolerates trailing slashes/query strings, and includes the standard `created` field in the returned model object.
* Also accepts root-base OpenAI chat paths (`/chat/completions`) in addition to `/v1/chat/completions`, so clients work whether their base URL is configured as `http://host:port` or `http://host:port/v1`.
* The server-ready banner now prints client setup details for OpenAI-compatible apps: base URL, chat endpoint, models endpoint, API key value, and model name.
v1.0.11 2026-05-29 

[Original release notes and assets](https://github.com/brontoguana/krasis/releases/tag/v1.0.11)

Krasis v1.0.11

* Adds experimental Qwen image support for OpenAI-style image chat requests.
* Uses lazy BF16 Qwen3VL vision staging for image setup, then releases the tower before decode.
* Keeps the text-only Rust/CUDA prefill/decode path separate; final QCN speed-test: 5925.2 pp / 91.09 tg / 150.62 HTTP.
* Adds structured insufficient\_vram errors for constrained image requests.
* Fixes QCN release-test witness selection and first-token witness prefix validation.
* Fixes aux HQQ multi-GPU decode setup and measured aux HCS VRAM budgeting.
* Full QCN release-test passed all 3 configs.
v1.0.10 2026-05-28 

[Original release notes and assets](https://github.com/brontoguana/krasis/releases/tag/v1.0.10)

* Fix source-mode dynamic HCS decode/prefill boundary ordering that could trigger CUDA illegal-address failures on 16 GB GPUs.
* Order dynamic HCS promotions before cold staging buffer reuse and synchronize source-mode HCS eviction before freeing soft chunks.
* Validated on Typhon with Qwen3.6-35B in RAM saver/source mode; benchmark completed without CUDA errors.
* Release gate speed-test passed on QCN INT4 HQQ4 k4v4 with thinking off and RAM saver off/mirror.
v1.0.9 2026-05-27 

[Original release notes and assets](https://github.com/brontoguana/krasis/releases/tag/v1.0.9)

* Add HCS RAM saver source mode and make it the default, reloading soft HCS chunks from the Marlin host cache instead of duplicating the soft host mirror.\\n- Keep RAM saver Off as `--hcs-host-cache-mode mirror`, preserving the prior mirrored HCS host-cache path for fastest reloads.\\n- Add opt-in `KRASIS_RAM_LEDGER=1` system RAM ledger checkpoints for model load, calibration, HCS init, and server-ready residency.\\n- Correct `./dev speed-test` to the current release-gate QCN surface: INT4, HQQ4 attention, k4v4 KV, thinking off, RAM saver off/mirror.\\n- Archive the 2026-05-27 QCN HQQ4/k4v4 mirror speed-test logs in `benchmarks/`.
v1.0.8 2026-05-27 

[Original release notes and assets](https://github.com/brontoguana/krasis/releases/tag/v1.0.8)

* Fix Typhon post-prefill hard exits during HCS reload before decode by using bounded synchronous reload on immediate-wait request, benchmark, and reference paths.
* Preserve live VRAM pressure checks at the decode boundary instead of queuing all soft-HCS reload DMA before synchronization.
* Validated with local ./dev build, Typhon ./dev build, an instrumented 8870-token streaming request, and a clean Typhon benchmark with no hard exit.
v1.0.7 2026-05-27 

[Original release notes and assets](https://github.com/brontoguana/krasis/releases/tag/v1.0.7)

* Fix Typhon RTX 5080 benchmark hard exit during timed prefill by reserving measured scratch allocation overhead in optional prefill pinning.
* Verify live free VRAM after allocating the optional pinning pool and skip it if the runtime floor would be violated.
* Bind CUDA device before HCS prefill eviction memory queries.
* Restore HCS after benchmark prefill-only requests so subsequent decode benchmarks are not run with HCS drained.
* Add KRASIS\_DEV\_\* overrides to ./dev for remote validation environments.
v1.0.6 2026-05-27 

[Original release notes and assets](https://github.com/brontoguana/krasis/releases/tag/v1.0.6)

* Release redundant GPU execution source tensors after Rust setup to recover VRAM on constrained GPUs.\\n- Add Zephyrus RTX 3070 Laptop 8 GB Qwen3.5-35B-A3B benchmark logs and README row.\\n- Bump package version to 1.0.6.\\n\\n**Full Changelog**: <https://github.com/brontoguana/krasis/compare/v1.0.5...v1.0.6>
v1.0.5 2026-05-23 

[Original release notes and assets](https://github.com/brontoguana/krasis/releases/tag/v1.0.5)

**Full Changelog**: <https://github.com/brontoguana/krasis/compare/v1.0.4...v1.0.5>

v1.0.4 2026-05-23 

[Original release notes and assets](https://github.com/brontoguana/krasis/releases/tag/v1.0.4)

**Full Changelog**: <https://github.com/brontoguana/krasis/compare/v1.0.3...v1.0.4>

v1.0.3 2026-05-23 

[Original release notes and assets](https://github.com/brontoguana/krasis/releases/tag/v1.0.3)

**Full Changelog**: <https://github.com/brontoguana/krasis/compare/v1.0.2...v1.0.3>

v1.0.2 2026-05-20 

[Original release notes and assets](https://github.com/brontoguana/krasis/releases/tag/v1.0.2)

**Full Changelog**: <https://github.com/brontoguana/krasis/compare/v1.0.1...v1.0.2>

v1.0.1 2026-05-19 

[Original release notes and assets](https://github.com/brontoguana/krasis/releases/tag/v1.0.1)

**Full Changelog**: <https://github.com/brontoguana/krasis/compare/v1.0.0...v1.0.1>

v1.0.0 2026-05-19 

[Original release notes and assets](https://github.com/brontoguana/krasis/releases/tag/v1.0.0)

**Full Changelog**: <https://github.com/brontoguana/krasis/compare/v0.1.64...v1.0.0>

v0.1.64 2026-03-19 

[Original release notes and assets](https://github.com/brontoguana/krasis/releases/tag/v0.1.64)

## Changes since v0.1.63

* **VRAM budget fix:** Dense MLP workspace now included in prefill budget (max of MoE vs dense intermediates)
* **KV cache auto-cap:** Clamps KV cache to available VRAM after weight loading (downward only, warns when capped)
* **HCS pool fix:** hard\_budget\_mb=0 detection now correct
* **CPU cache building disabled:** No longer builds unused CPU expert caches; deleted 159 GB of stale CPU cache files
* **Release test report improved:** TOC with links, compact benchmark summaries with model/config info, full sanity responses for manual verification
* **GLM-4.7 graph capture segfault fixed**
* **AWQ attention template added for GLM-4.7**

### Release test results

**QCN (Qwen3-Coder-Next) on RTX 5090:**

* INT4/4 BF16: 71.4 tok/s decode, 3518 prefill, 60.6% HCS
* INT4/4 AWQ: 65.9 tok/s decode, 3536 prefill, 66.5% HCS
* INT8/8 BF16: 38.0 tok/s decode, 3368 prefill, 30.7% HCS
* INT8/8 AWQ: 35.7 tok/s decode, 3430 prefill, 33.5% HCS
* Multi-GPU INT4 BF16: 41.0 tok/s decode, 3544 prefill, 82.1% HCS

**Q3.5-35B on RTX 5090:**

* INT4/4 BF16: 110.7 tok/s decode, 4495 prefill, 100% HCS
* INT4/4 AWQ: 95.1 tok/s decode, 4407 prefill
* INT8/8 BF16: 60.9 tok/s decode, 4391 prefill
* INT8/8 AWQ: 57.1 tok/s decode, 4282 prefill
* Multi-GPU INT4 BF16: 49.7 tok/s decode, 4402 prefill

All sanity tests (14 prompts, multi-turn) passed across all configs.

v0.1.63 2026-03-17 

[Original release notes and assets](https://github.com/brontoguana/krasis/releases/tag/v0.1.63)

## v0.1.63

First stable release after 13 release candidates.

### What’s new since v0.1.62

* **Thinking budget safety valve (rc13):** Force-injects `</think>` when 4096-token thinking budget exhausted, preventing degenerate infinite-thinking loops. Applied to all 4 decode paths.
* **Multi-turn chat fix (rc12):** Fixed conversation context being lost between turns, CUDA graph reuse across requests.
* **Tool use support (rc10):** OpenAI-compatible tool/function calling.
* **Request logging:** All requests logged with timing data.
* **Silent fallback elimination:** Removed fake timing estimation fallbacks — if server doesn’t send real timing data, it says so instead of fabricating numbers.
* **INT8 Marlin fix:** Corrected INT8 quantisation path.
* **AWQ VRAM-safe loading:** Attention quantisation loads within VRAM budget.

### Performance (1x RTX 5090 32GB, PCIe 4.0)

| Model             | Params | Prefill    | Decode    | vs llama.cpp       |
| ----------------- | ------ | ---------- | --------- | ------------------ |
| Qwen3.5-35B-A3B   | 35B    | 4475 tok/s | 109 tok/s | 4.6x pp / 3.6x tg  |
| Qwen3-Coder-Next  | 80B    | 3560 tok/s | 70 tok/s  | 6.9x pp / 3.8x tg  |
| Qwen3.5-122B-A10B | 122B   | 2897 tok/s | 28 tok/s  | —                  |
| Qwen3-235B-A22B   | 235B   | 2124 tok/s | 9.3 tok/s | 26.7x pp / 2.9x tg |

### Install

```
curl -fsSL https://raw.githubusercontent.com/brontoguana/krasis/main/install.sh | bash
```

v0.1.62 2026-02-27 

[Original release notes and assets](https://github.com/brontoguana/krasis/releases/tag/v0.1.62)

## Changes

* **Fix server SIGINT deadlock** — Ctrl-C now works reliably. Python signal handlers only dispatch between bytecodes, but the accept loop runs in native Rust (py.allow\_threads). Installed a raw libc SIGINT handler that atomically sets the running flag, so the server exits cleanly on the next 10ms poll.
* Updated Qwen3.5 benchmark log
v0.1.61 2026-02-27 

[Original release notes and assets](https://github.com/brontoguana/krasis/releases/tag/v0.1.61)

* Revert double-buffered DMA from CUDA events back to torch.cuda.synchronize() — flushes PyTorch memory allocator between group cycles, fixing potential VRAM regression on constrained GPUs (e.g. RTX 5080)
* Fix OpenAI SSE timing event: wrap in valid chunk structure so clients like OpenCode don’t error on malformed JSON
v0.1.60 2026-02-27 

[Original release notes and assets](https://github.com/brontoguana/krasis/releases/tag/v0.1.60)

### FP16 CPU KV cache replaces FP8 LUT gather

* **14% decode speed recovery**: 12.70 → 14.45 tok/s (engine), 15.84 tok/s (network)
* Replaced slow FP8 gather instruction (\~12 cycles on Zen 2) with native F16C `_mm256_cvtph_ps` (\~3-5 cycles)
* Memory: 2 bytes/element (half of f32, double FP8) — negligible tradeoff on high-RAM systems
* Accuracy: FP16 (10-bit mantissa) vs FP8 E4M3 (3-bit) — essentially lossless
* F16C available on all AVX2-capable CPUs (Intel Ivy Bridge+, AMD Piledriver+)
* Default max\_tokens raised to 2048 (from previous commit)
v0.1.59 2026-02-27 

[Original release notes and assets](https://github.com/brontoguana/krasis/releases/tag/v0.1.59)

## Changes

* **Double-buffered DMA with CUDA events**: Replace `cuda.synchronize()` with CUDA events for pinned buffer protection. Two buffer sets (A/B ping-pong) allow CPU memcpy into buffer B while GPU DMA reads from buffer A, eliminating CPU stalls between layer groups.
* **Unbuffered Python output**: Dev script sets `PYTHONUNBUFFERED=1` so benchmark/test output streams in real time instead of buffering until process exit.
* **Benchmark timeout**: Increased from 10 to 20 minutes to accommodate large model benchmarks (QCN full suite).
* Updated QCN benchmark log with latest numbers (1016 tok/s prefill at 20K, 12.70 tok/s decode).
v0.1.58 2026-02-27 

[Original release notes and assets](https://github.com/brontoguana/krasis/releases/tag/v0.1.58)

## Changes since v0.1.57

* **Unified benchmark paths**: engine and network benchmarks now use identical code (server\_prefill + Rust decode + server\_cleanup). Only difference is HTTP transport layer. Fixed 33% decode measurement discrepancy.
* **Ring buffer SSE streaming**: decode loop pushes tokens to mpsc channel, writer thread flushes every 100ms + on first token (accurate client TTFT). Decouples decode from network I/O.
* **Rust HTTP server is now the only server**: removed Python FastAPI/uvicorn server entirely.
* **Enable-thinking config**: configurable via .conf files, passed through to model.
* **Dev tooling**: ./dev script for build/run/test workflows, DEV.md documenting the dev environment.
* **Qwen3.5-35B-A3B test config**: testconfigs/qwen35-4-4.conf added.
* **Benchmark results (QCN)**: prefill overhead 7.1% at 20K, decode overhead 6.5% — both reasonable HTTP/SSE costs.
v0.1.57 2026-02-26 

[Original release notes and assets](https://github.com/brontoguana/krasis/releases/tag/v0.1.57)

* Redesigned launcher TUI: VRAM and System RAM tables side by side
* Host/Port merged into single config line
* Controls on one line (Load/Save shortened)
* System RAM in GB, removed duplicate expert lines
* Includes all v0.1.56 fixes (CUDA toolkit detection, error suppression)
v0.1.56 2026-02-26 

[Original release notes and assets](https://github.com/brontoguana/krasis/releases/tag/v0.1.56)

* Fix CUDA toolkit detection: reads driver version from nvidia-smi header instead of mapping from GPU compute capability
* Fallback to major CUDA meta-package if exact version not available
* Suppress scary red errors when CUDA toolkit install fails but nvcc already available
* Install script wrapped in function to survive curl-pipe execution
* Auto-install Python 3 when missing
* Podman container scripts for isolated testing
v0.1.55 2026-02-26 

[Original release notes and assets](https://github.com/brontoguana/krasis/releases/tag/v0.1.55)

## What’s new

* **FP8 E4M3 CPU KV cache**: Stores KV in FP8 on CPU (matching GPU format), saving \~18 GB RAM on Qwen3-235B. AVX2 256-entry compile-time LUT for fast decode reads. GPU→CPU transfer is now a straight memcpy with no conversion.
* **VRAM budget accuracy**: Fixed estimator to account for DMA pipeline doubling (2x expert buffers), permanent gate weights (all layers, BF16 + F32), FlashInfer workspace (128 MB), and prefill MoE intermediates. Qwen3-235B estimate: 10 GB → 12.7 GB (matches observed \~14 GB).
* **OOM fix during loading**: Layer streaming now loads one layer at a time to GPU, extracts attention to CPU, then frees. Peak during load drops from 17 GB to \~4.6 GB.
* **Client disconnect cancellation**: Server detects when chat client disconnects (TCP close) and cancels in-flight generation within \~600ms via AtomicBool in the Rust decode loop.
* **Launcher TUI improvements**: Removed vestigial PP partition field. Added inline VRAM (cyan) and RAM (green) size columns for every configurable component. Model list sorted case-insensitively.
* **Sanity test framework**: `--sanitytest` flag for krasis-chat with standardized prompts. Results for Qwen3.5-35B-A3B (7/7 PASS), DeepSeek-V2-Lite, DeepSeek-VL2.
* **New perplexity results**: Qwen3.5-35B-A3B WikiText-2, Qwen3-Coder-Next C4 + WikiText-2.
v0.1.54 2026-02-26 

[Original release notes and assets](https://github.com/brontoguana/krasis/releases/tag/v0.1.54)

* Fix inference\_mode on \_copy\_kv\_cache and \_copy\_recurrent\_state\_from\_gpu (same bug class as v0.1.48)
* Add Test 6: multi-turn sequential conversation test (5 turns, exercises KV cache copy path)
* KV cache steps in 100 MB increments in launcher TUI
v0.1.53 2026-02-26 

[Original release notes and assets](https://github.com/brontoguana/krasis/releases/tag/v0.1.53)

Install script: show ‘restart terminal’ as first step before krasis-setup

v0.1.52 2026-02-26 

[Original release notes and assets](https://github.com/brontoguana/krasis/releases/tag/v0.1.52)

Revert KV cache auto-sizing, add configurable KV cache to launcher TUI and CLI with max-context estimate

v0.1.51 2026-02-25 

[Original release notes and assets](https://github.com/brontoguana/krasis/releases/tag/v0.1.51)

Fix venv creation: check ensurepip early + auto-install retry on failure

v0.1.50 2026-02-25 

[Original release notes and assets](https://github.com/brontoguana/krasis/releases/tag/v0.1.50)

* Install script auto-installs python3-venv (no more manual apt install)
* Supports apt (Debian/Ubuntu) and dnf (Fedora/RHEL)
v0.1.49 2026-02-25 

[Original release notes and assets](https://github.com/brontoguana/krasis/releases/tag/v0.1.49)

* Auto-size KV cache to available VRAM (no more manual –kv-cache-mb needed)
* Chat prompt length check: Cancel/Truncate/Send when prompt exceeds cache
* Server 413 error for oversized prompts, /health reports max\_context\_tokens
* Fix: inference\_mode crash on large prompts (\_zero\_kv\_cache)
* Network tests: added 100K token test, all response text shown for review
* Updated benchmarks (QCN: 15.81 tok/s decode, PPL 7.23)
v0.1.48 2026-02-25 

[Original release notes and assets](https://github.com/brontoguana/krasis/releases/tag/v0.1.48)

* Hard-fail with clear error when no NVIDIA GPU detected (no more silent CPU fallback)
* Hard-fail on missing GPU dependencies (Triton CPU fallback, missing Python.h)
* krasis-setup now installs python3.X-dev headers alongside ninja
* All failures point user to run krasis-setup
v0.1.47 2026-02-25 

[Original release notes and assets](https://github.com/brontoguana/krasis/releases/tag/v0.1.47)

Bundle benchmark prompt files in wheel package. Rename prefill\_prompt\_10k → prefill\_prompt. Remove dead code.

v0.1.46 2026-02-25 

[Original release notes and assets](https://github.com/brontoguana/krasis/releases/tag/v0.1.46)

GitHub-only release — wheels attached as release assets. Install with: curl -sSf <https://raw.githubusercontent.com/brontoguana/krasis/main/install.sh> | bash

v0.1.45 2026-02-25 

[Original release notes and assets](https://github.com/brontoguana/krasis/releases/tag/v0.1.45)

NUMA-aware decode, MLA AVX2 optimizations, default 40 threads

* NUMA detection and interleaved mmap allocation (spreads pages across all memory controllers)
* Rayon thread pinning to NUMA nodes via sched\_setaffinity
* AVX2 MLA kernels: mla\_rope (-51%), mla\_attn (-77%), mla\_o\_proj (-40%)
* Default thread count raised to min(physical\_cores, 40)
* QCN: 97.9ms -> 53.4ms (1.83x speedup)
* V2-Lite MLA: 57.5ms -> 35.5ms (1.62x speedup)
v0.1.44 2026-02-23 

[Original release notes and assets](https://github.com/brontoguana/krasis/releases/tag/v0.1.44)

## Changes

* **–config flag**: Server and launcher accept `--config path/to/file.conf` to load Krasis CFG\_ format config files. CLI args override config values.
* **Launcher config overhaul**: No more auto-load/save of `~/.krasis/config`. TUI starts with hardcoded defaults. New \[L\] Load Config and \[S\] Save Config on the config screen.
* **HCS removed from UI**: Removed from launcher, server CLI, and config handling. Underlying code preserved but disabled.
* **Fix 3 GPU DMA prefill race conditions**: Streaming attention DMA race, cross-group prefetch in multi-chunk, missing sync before empty\_cache. These caused garbage output for prompts above \~2500 tokens.
* **Press Q to quit** the running server.
* **CUDA graph capture safety**: Failed captures now properly clean up instead of leaving streams in capture mode.
* **Network validation tests**: `tests/test_network.py` with 12 tests (9 multi-prompt + 3 large-prompt).

Install: `pip install krasis==0.1.44`

v0.1.43 2026-02-23 

[Original release notes and assets](https://github.com/brontoguana/krasis/releases/tag/v0.1.43)

## INT8 attention removed as an option

Per-channel symmetric INT8 quantization is too lossy for attention projections (Q/K/V/O). Causes PPL \~5600 vs \~10 — pure garbage output on all models.

### Changes

* **Removed INT8 from all choice menus**: launcher TUI, `--attention-quant` CLI, suite, stress\_test
* **Safety guard in weight\_loader**: forces bf16 with warning if stale config somehow passes int8
* **Launcher config migration**: prints warning and overrides stale `CFG_ATTENTION_QUANT=int8` from old `~/.krasis/config` files

### Still working fine with INT8

* shared\_expert, dense\_mlp, lm\_head — W8A8 `int8_linear` unaffected for these components

### Future

INT8 attention will need GPTQ-style per-group quantization to be viable. Tracked for future work.

v0.1.42 2026-02-23 

[Original release notes and assets](https://github.com/brontoguana/krasis/releases/tag/v0.1.42)

## Critical Fix

**INT8 attention default was not fully fixed in v0.1.40.** The launcher, suite, vram\_budget, and stress\_test still defaulted to `int8` for attention quantization, which causes catastrophic perplexity (\~5600 vs \~10.6) and garbage output.

### What happened

* v0.1.40 fixed `config.py` and `server.py` but missed 5 other locations
* The launcher TUI wrote `CFG_ATTENTION_QUANT=int8` to `~/.krasis/config`
* Anyone who launched via the TUI got INT8 attention = garbage

### Action required

If you have an existing `~/.krasis/config` file, either:

* **Delete it** (`rm ~/.krasis/config`) and re-run the launcher
* **Edit it** and change `CFG_ATTENTION_QUANT="int8"` to `CFG_ATTENTION_QUANT="bf16"`
v0.1.41 2026-02-23 

[Original release notes and assets](https://github.com/brontoguana/krasis/releases/tag/v0.1.41)

## Changes

* Benchmark now tests prefill at 3 different lengths (20K/35K/50K tokens) instead of one fixed length
* Reports best tok/s across runs instead of average
* Updated QCN 1-GPU benchmark log (943.6 tok/s at 20K, 7.09 tok/s decode)
v0.1.40 2026-02-23 

[Original release notes and assets](https://github.com/brontoguana/krasis/releases/tag/v0.1.40)

## Fix INT8 attention default + QCN perplexity results

### INT8 Attention Fix

* W8A8 `int8_linear` produces catastrophic PPL on all models when used for attention weights (\~5600 vs \~10.6 on QCN WikiText-2)
* Root cause: per-token symmetric activation quantization too destructive for attention projections
* Changed attention default from `int8` to `bf16` in config, server, and measure\_ppl
* W8A8 remains correct and fast for shared\_expert/lm\_head/dense\_mlp

### Perplexity Results

* **Qwen3-Coder-Next WikiText-2: PPL = 10.64** (BPC = 3.41)
* 298,938 tokens scored, 292 windows, 121 tok/s GPU prefill throughput
* Config: 1 GPU, INT4 experts, BF16 attention, FP8 KV, layer\_group\_size=2
v0.1.39 2026-02-23 

[Original release notes and assets](https://github.com/brontoguana/krasis/releases/tag/v0.1.39)

## Fix GPU prefill perplexity corruption during DMA pipelining

### Bug

GPU prefill with DMA pipelining produced wildly wrong outputs (PPL \~2000 instead of \~4.5 on WikiText-2). The `_forward_engine` fallback path was unaffected.

### Root Cause

`free_layer_group(clear_cache=False)` skipped `torch.cuda.empty_cache()` during pipelining to save \~30ms/group. Without it, PyTorch’s caching allocator reused freed GPU memory blocks while `fused_marlin_moe` kernels were still reading from them, corrupting expert weight data mid-computation.

### Fix

Always call `empty_cache()` after freeing expert weights (\~8ms/group overhead, negligible at 10K+ tokens). Verified: QCN WikiText-2 PPL 2169 → 4.55.

### Other Changes

* `KRASIS_NO_PIPELINE=1` env var to disable DMA pipelining for debugging
* `KRASIS_SKIP_PERSISTENT=1` / `KRASIS_DEBUG_PERSISTENT=1` debug env vars
* Fix benchmark decode prompt text to use actual timed prompt
* Fix PyPI publish: version in pyproject.toml was stuck at 0.1.35
v0.1.38 2026-02-23 

[Original release notes and assets](https://github.com/brontoguana/krasis/releases/tag/v0.1.38)

## Fix GPU prefill perplexity corruption during DMA pipelining

### Bug

GPU prefill with DMA pipelining produced wildly wrong outputs (PPL \~2000 instead of \~4.5 on WikiText-2). The `_forward_engine` fallback path was unaffected.

### Root Cause

`free_layer_group(clear_cache=False)` skipped `torch.cuda.empty_cache()` during pipelining to save \~30ms/group. Without it, PyTorch’s caching allocator reused freed GPU memory blocks while `fused_marlin_moe` kernels were still reading from them, corrupting expert weight data mid-computation.

### Fix

Always call `empty_cache()` after freeing expert weights (\~8ms/group overhead, negligible at 10K+ tokens). Verified: QCN WikiText-2 PPL 2169 → 4.55.

### Other Changes

* `KRASIS_NO_PIPELINE=1` env var to disable DMA pipelining for debugging
* `KRASIS_SKIP_PERSISTENT=1` / `KRASIS_DEBUG_PERSISTENT=1` debug env vars
* Fix benchmark decode prompt text to use actual timed prompt
v0.1.37 2026-02-23 

[Original release notes and assets](https://github.com/brontoguana/krasis/releases/tag/v0.1.37)

## Fix GPU prefill perplexity corruption during DMA pipelining

### Bug

GPU prefill with DMA pipelining produced wildly wrong outputs (PPL \~2000 instead of \~4.5 on WikiText-2). The `_forward_engine` fallback path was unaffected.

### Root Cause

`free_layer_group(clear_cache=False)` skipped `torch.cuda.empty_cache()` during pipelining to save \~30ms/group. Without it, PyTorch’s caching allocator reused freed GPU memory blocks while `fused_marlin_moe` kernels were still reading from them, corrupting expert weight data mid-computation.

### Fix

Always call `empty_cache()` after freeing expert weights. Verified: QCN WikiText-2 PPL 2169 → 4.55 (matches engine reference: 4.52).

### Other Changes

* `KRASIS_NO_PIPELINE=1` env var to disable DMA pipelining for debugging
* `KRASIS_SKIP_PERSISTENT=1` / `KRASIS_DEBUG_PERSISTENT=1` debug env vars
* Fix benchmark decode prompt text to use actual timed prompt
v0.1.36 2026-02-23 

[Original release notes and assets](https://github.com/brontoguana/krasis/releases/tag/v0.1.36)

## What’s New

### Cache-Defeated Benchmarks

* 6 unique Project Gutenberg prefill prompts (\~250K chars / \~50K tokens each): Kafka, Wells, Dickens (x2), Brontë, Homer
* Different prompts for warmup vs timed runs — defeats CPU/GPU/expert caches
* 6 different decode prompts covering distinct topics
* Auto-truncates prompts to KV cache capacity (with 200 token margin)

### Token Scaling Analysis

* New script (`tests/token_scaling_bench.py`) measures prefill speed at 100–25K tokens
* Fixed cost per prefill: \~3.6s (PCIe DMA overhead)
* Marginal cost: 0.63ms/token (1591 tok/s throughput)
* Superlinear scaling observed at 25K+ tokens

### Perplexity Evaluation

* Added WikiText-103 dataset alongside WikiText-2
* Dataset registry with `load_dataset_text()`, `list_datasets()`, `run_perplexity()`
* `--perplexity` flag for interactive launcher integration

### Other Changes

* GPU prefill threshold raised from 300 → 500 tokens
* Updated 1-GPU and 2-GPU QCN benchmark logs with 62K token results (694 tok/s prefill, 7.0 tok/s decode)
v0.1.35 2026-02-22 

[Original release notes and assets](https://github.com/brontoguana/krasis/releases/tag/v0.1.35)

* Add perplexity measurement system (`perplexity/measure_ppl.py`)
* Add `return_all_logits` support in forward/prefill paths
* Fix version sync (Cargo.toml + pyproject.toml)
v0.1.34 2026-02-22 

[Original release notes and assets](https://github.com/brontoguana/krasis/releases/tag/v0.1.34)

## Perplexity considerations, maintenance

* Add perplexity-considerations.md — methodology for validating Krasis quantization quality
* Sync Cargo.lock version to 0.1.32
* Updated QCN benchmark log with latest run
* Simplified V2-Lite NaN debug script
v0.1.33 2026-02-22 

[Original release notes and assets](https://github.com/brontoguana/krasis/releases/tag/v0.1.33)

## Fix cross-stream memory safety in swap\_prefetch()

* Fix use-after-free race in GPU prefill pipelining — `torch.cuda.synchronize()` before `free_layer_group()`
* Fix cross-stream memory ordering in `swap_prefetch()` — replace host-side `stream.synchronize()` with GPU-side `record_event()` \+ `wait_event()`
* Add `record_stream()` on all prefetched tensors to prevent allocator recycling
* Resolves V2-Lite NaN at 4K+ tokens and QCN corruption at 500+ tokens during pipelined GPU prefill
v0.1.32 2026-02-22 

[Original release notes and assets](https://github.com/brontoguana/krasis/releases/tag/v0.1.32)

* **Long input stress test**: Add \~60K word (\~80K token) prompt to stress test to verify context length robustness (linear attention numerical stability, KV cache, RoPE)
* **Auto-scaling timeout**: Timeout scales with prompt length so long prefills don’t false-fail
v0.1.31 2026-02-22 

[Original release notes and assets](https://github.com/brontoguana/krasis/releases/tag/v0.1.31)

* Benchmark warmup now runs a full pass (3x prefill + 3x decode) before the timed runs, ensuring all kernel variants are compiled
v0.1.30 2026-02-22 

[Original release notes and assets](https://github.com/brontoguana/krasis/releases/tag/v0.1.30)

* Run 3 warmup prefills before benchmark to fully compile all FlashInfer JIT kernel variants
v0.1.29 2026-02-22 

[Original release notes and assets](https://github.com/brontoguana/krasis/releases/tag/v0.1.29)

* Cached weight loading now shows “this loads the full model into RAM, may take a minute” and elapsed time on completion
v0.1.28 2026-02-22 

[Original release notes and assets](https://github.com/brontoguana/krasis/releases/tag/v0.1.28)

* **Clear progress messages during loading**:  
  * First run: “Building GPU INT4 Marlin + CPU INT4 expert cache (one-time, may take several minutes)” with cache path and completion time
  * Subsequent runs: “Loading expert weights from cache” (instant)
* **Benchmark warmup timing**: Shows FlashInfer JIT compilation time during warmup prefill
v0.1.27 2026-02-22 

[Original release notes and assets](https://github.com/brontoguana/krasis/releases/tag/v0.1.27)

* **Fix benchmark warmup**: Add full-length warmup prefill to trigger FlashInfer JIT compilation before timed runs. Previously run 1 included minutes of JIT overhead (e.g., 205s vs 6s on RTX 5080).
v0.1.26 2026-02-22 

[Original release notes and assets](https://github.com/brontoguana/krasis/releases/tag/v0.1.26)

* **Add sglang dependency**: Required for Marlin MoE GPU prefill kernels — `krasis-setup` now installs it alongside FlashInfer and sgl-kernel
* **Suppress TF32 warning**: Set `torch.set_float32_matmul_precision('high')`
* **Fix CPU-only torch**: `--force-reinstall` when existing torch lacks CUDA support
v0.1.25 2026-02-22 

[Original release notes and assets](https://github.com/brontoguana/krasis/releases/tag/v0.1.25)

* **Fix FlashInfer JIT cache**: Clear stale cache that references old `/usr/bin/nvcc` after CUDA toolkit upgrade — this was causing `/usr/bin/nvcc: not found` errors
* **Always set CUDA\_HOME**: Ensures FlashInfer/PyTorch JIT finds the correct toolkit even if env was stale
* **Ubuntu 24.04 support**: Setup script detects Ubuntu version for correct NVIDIA repo URL
* **RHEL/Fedora**: Setup now configures PATH and CUDA\_HOME after install (was missing)
* **CPU-only torch fix**: `--force-reinstall` when existing torch lacks CUDA
* **Consistent nvcc search**: Both launcher and setup search 7 common locations
v0.1.24 2026-02-22 

[Original release notes and assets](https://github.com/brontoguana/krasis/releases/tag/v0.1.24)

* Launcher searches multiple nvcc paths (/usr/local/cuda-12.x/bin, /usr/local/cuda/bin, /usr/bin) — no longer requires nvcc on PATH
* Sets CUDA\_HOME env var so FlashInfer/PyTorch JIT compilation finds the correct CUDA toolkit
* krasis-setup writes CUDA\_HOME to \~/.bashrc alongside PATH
v0.1.23 2026-02-22 

[Original release notes and assets](https://github.com/brontoguana/krasis/releases/tag/v0.1.23)

## Changes

* **krasis-setup writes CUDA PATH to \~/.bashrc** — no more manual PATH config after installing CUDA toolkit
* **Launcher auto-adds /usr/local/cuda/bin to PATH** — finds nvcc even before shell restart
* **Auto-detect FP8 support** — GPUs with SM < 8.9 (e.g. RTX 3090) automatically use BF16 KV cache instead of FP8
* **CUDA toolkit installed from NVIDIA repo** — gets proper nvcc version that supports your GPU (fixes Blackwell/5080 JIT compilation)

## Upgrade

```
pipx upgrade krasis
```

v0.1.22 2026-02-22 

[Original release notes and assets](https://github.com/brontoguana/krasis/releases/tag/v0.1.22)

## Changes

* **krasis-setup: install proper CUDA toolkit from NVIDIA repo** — the Ubuntu `nvidia-cuda-toolkit` apt package ships nvcc 12.0 which can’t compile for Blackwell (RTX 5080/5090) or other newer GPUs. krasis-setup now:  
  * Detects your GPU’s compute capability
  * Determines the minimum CUDA toolkit version needed
  * Detects if existing nvcc is too old and upgrades it
  * Installs from NVIDIA’s official apt repository (supports WSL too)

## Upgrade & Fix

```
pipx upgrade krasis
krasis-setup    # will detect old nvcc and upgrade it
```

v0.1.21 2026-02-22 

[Original release notes and assets](https://github.com/brontoguana/krasis/releases/tag/v0.1.21)

## Changes

* **`krasis-setup` now installs ninja** — FlashInfer JIT needs both `nvcc` and `ninja`. Setup installs both in one step.
* **Launcher no longer installs packages** — just does quick checks and points to `krasis-setup` if anything is missing.

## Upgrade

```
pipx upgrade krasis
krasis-setup    # will install ninja if missing
```

v0.1.20 2026-02-22 

[Original release notes and assets](https://github.com/brontoguana/krasis/releases/tag/v0.1.20)

## Changes

* **`krasis-setup` now handles sudo internally** — just run `krasis-setup` and it will prompt for your password when installing system packages. No need for `sudo krasis-setup`.

## First-time Setup

```
pipx install krasis
krasis-setup    # handles everything (prompts for password if needed)
krasis          # launch
```

## Upgrade

```
pipx upgrade krasis
```

v0.1.19 2026-02-22 

[Original release notes and assets](https://github.com/brontoguana/krasis/releases/tag/v0.1.19)

## Changes

* **New `krasis-setup` command** — one-command GPU environment setup  
  * Installs CUDA toolkit (nvcc) needed for FlashInfer JIT compilation
  * Installs CUDA-enabled PyTorch
  * Installs FlashInfer + sgl-kernel
  * Usage: `sudo krasis-setup` (for nvcc), then `krasis-setup` (for Python packages)
* **Launcher warns if nvcc is missing** and points to `krasis-setup`

## First-time Setup

```
pipx install krasis
sudo krasis-setup    # install CUDA toolkit (needs sudo)
krasis-setup         # install Python GPU packages
krasis               # launch
```

## Upgrade

```
pipx upgrade krasis
```

v0.1.18 2026-02-22 

[Original release notes and assets](https://github.com/brontoguana/krasis/releases/tag/v0.1.18)

## Changes

* **Fix transformers v5 BatchEncoding handling** — BatchEncoding doesn’t pass `isinstance(dict)` and iterating yields key names. Now uses direct `["input_ids"]` access which works for all container types.

## Upgrade

```
pipx upgrade krasis
```

v0.1.17 2026-02-22 

[Original release notes and assets](https://github.com/brontoguana/krasis/releases/tag/v0.1.17)

## Changes

* **Fix transformers v5 compatibility** — `apply_chat_template(tokenize=True)` in transformers v5 returns a `BatchEncoding` dict instead of `List[int]`. Now handled correctly everywhere.

## Upgrade

```
pipx upgrade krasis
```

v0.1.16 2026-02-22 

[Original release notes and assets](https://github.com/brontoguana/krasis/releases/tag/v0.1.16)

## Changes

* **Force token IDs to int** — aggressively coerce tokenizer output to `List[int]` regardless of transformers version behavior (handles str, list-of-str, numpy arrays). Secondary guard in `model.generate()`.

## Upgrade

```
pipx upgrade krasis
```

v0.1.15 2026-02-22 

[Original release notes and assets](https://github.com/brontoguana/krasis/releases/tag/v0.1.15)

## Changes

* **Fix benchmark crash with string tokens** — guard in both tokenizer and model.generate() for transformers versions where apply\_chat\_template returns a string instead of token IDs

## Upgrade

```
pipx upgrade krasis
```

v0.1.14 2026-02-22 

[Original release notes and assets](https://github.com/brontoguana/krasis/releases/tag/v0.1.14)

## Changes

* **Fix benchmark crash on some transformers versions** — `apply_chat_template` returning a string instead of token IDs

## Upgrade

```
pipx upgrade krasis
```

v0.1.13 2026-02-22 

[Original release notes and assets](https://github.com/brontoguana/krasis/releases/tag/v0.1.13)

## Changes

* **Fix multi-prompt repetition bug** — linear attention recurrent state now properly reset between HTTP requests; previously conv\_state and recurrent\_state persisted across requests causing progressive output corruption (repetition/garbage after several prompts)
* **Package benchmark prompts in wheel** — benchmark prompt files now included in the pip package so `--benchmark` works when installed via pipx

## Upgrade

```
pipx upgrade krasis
```

v0.1.12 2026-02-22 

[Original release notes and assets](https://github.com/brontoguana/krasis/releases/tag/v0.1.12)

## New Features

* **Stress test**: 84 diverse prompts (short, long, code, math, unicode, adversarial, multi-language). Validates token IDs, logits (no NaN/inf), CUDA stability, and decoder output. Tested 84/84 pass on Qwen3-Coder-Next.  
  * Launcher TUI: “Stress Test” option
  * Server flag: `--stress-test`
  * Standalone: `python -m krasis.stress_test --model-path /path/to/model`

## Fixes

* **INT8 matmul alignment**: Fix `CUBLAS_STATUS_NOT_SUPPORTED` when token count is not a multiple of 8
* **Ctrl-C after benchmark**: Reliable signal handling in “Benchmark and Launch” mode
* **Faster install**: torch removed from hard pip deps (launcher auto-installs CUDA torch)
* **Benchmark and Launch**: Now correctly starts the server after benchmark (was exiting)

**Full Changelog**: <https://github.com/brontoguana/krasis/compare/v0.1.10...v0.1.12>

v0.1.11 2026-02-22 

[Original release notes and assets](https://github.com/brontoguana/krasis/releases/tag/v0.1.11)

## Fixes

* **INT8 matmul alignment**: Fix `CUBLAS_STATUS_NOT_SUPPORTED` crash when token count is not a multiple of 8 (e.g. 25 tokens). `torch._int_mm` on SM89 requires 8-aligned dimensions.
* **Ctrl-C after benchmark**: Server now registers signal handlers directly so Ctrl-C works reliably when using “Benchmark and Launch” mode.
* **Faster install**: Removed torch from hard pip dependencies (was causing 2GB+ CPU torch download). The launcher auto-setup installs CUDA torch on first run instead.

**Full Changelog**: <https://github.com/brontoguana/krasis/compare/v0.1.10...v0.1.11>

v0.1.10 2026-02-22 

[Original release notes and assets](https://github.com/brontoguana/krasis/releases/tag/v0.1.10)

## Fixes

* **Benchmark and Launch**: `--benchmark` now runs the benchmark then starts the server (previously it exited after benchmark). Added `--benchmark-only` for the old exit-after-benchmark behavior.
* **Launcher TUI**: Split into three options — “Benchmark and Launch”, “Benchmark Only”, and “Launch”.

## Auto-setup (from v0.1.9)

First run on a CUDA machine automatically installs CUDA-enabled PyTorch + GPU packages.

**Full Changelog**: <https://github.com/brontoguana/krasis/compare/v0.1.9...v0.1.10>

v0.1.9 2026-02-22 

[Original release notes and assets](https://github.com/brontoguana/krasis/releases/tag/v0.1.9)

## New Features

* **Auto-install CUDA torch on first run**: The launcher now detects NVIDIA GPUs, checks if torch has CUDA support, and automatically installs the correct CUDA-enabled PyTorch + flashinfer + sgl-kernel. Detects driver version to pick the right CUDA toolkit (11.8, 12.1, 12.4, or 12.6). Subsequent launches skip the check.
* **torch added to package dependencies**: `pip install krasis` now pulls in torch, transformers, and other required packages automatically.

## Install

```
pipx install krasis
krasis  # first run auto-installs CUDA torch if needed
```

**Full Changelog**: <https://github.com/brontoguana/krasis/compare/v0.1.8...v0.1.9>

v0.1.8 2026-02-22 

[Original release notes and assets](https://github.com/brontoguana/krasis/releases/tag/v0.1.8)

## Improvements

* **Async pinned bounce for cross-GPU EP transfer**: When P2P is unavailable, cross-GPU expert parallel transfers now use pinned CPU bounce buffers with a dedicated copy stream. Eliminates host-thread blocking during D2H/H2D copies via event-based sync. QCN 2GPU decode improved 7.21 → 7.81 tok/s.

## Other Changes

* Added 1GPU benchmark log and benchmark TOML configs
* Removed old benchmark suite logs
* Added utility scripts (dump\_trace.sh, setup\_pcie.sh)

**Full Changelog**: <https://github.com/brontoguana/krasis/compare/v0.1.7...v0.1.8>

v0.1.7 2026-02-16 

[Original release notes and assets](https://github.com/brontoguana/krasis/releases/tag/v0.1.7)

## Bug Fixes

* **FP8 KV cache race condition**: Fix CUDA illegal memory access on second request. Temporary FP8→BF16 upcast tensors were freed while FlashInfer’s MLA kernel was still reading them asynchronously. Validated 95/95 on V2-Lite + 95/95 on Qwen3-Coder-Next.

## New Features

* **Server registry** (`~/.krasis/servers/`): Servers write a JSON entry on startup, cleaned up on shutdown. Chat client uses registry-based discovery instead of port scanning.
* **Default port**: Changed from 8080 → 8012 to avoid common HTTP server conflicts.
v0.1.6 2026-02-15 

[Original release notes and assets](https://github.com/brontoguana/krasis/releases/tag/v0.1.6)

## What’s New

### Chat Client Improvements

* **Channel filter for GPT OSS**: Strips `<|channel|>analysis` internal reasoning, shows only `<|channel|>final` response to user. Shows “(thinking…)” indicator during hidden channels. Non-channel models pass through unchanged.
* **Streaming fix**: `select.select()` replaces `sock.settimeout()` which permanently corrupted `http.client`’s chunked transfer reader after first timeout during slow prefill.
* **Conversation history**: Full message history correctly retained and sent with each request.
* **`krasis-chat` CLI entry point**: Now available as standalone command after `pipx install`.

### Install

```
pipx install git+https://github.com/brontoguana/krasis.git
```

Then run:

```
krasis-chat              # auto-discover local servers
krasis-chat --port 8012  # connect to specific port
```

v0.1.5 2026-02-15 

[Original release notes and assets](https://github.com/brontoguana/krasis/releases/tag/v0.1.5)

## Changes

* **Multi-line paste**: Chat client now uses bracketed paste mode — pasted text with newlines is captured as multi-line input instead of submitting on first newline
* Enter still submits normally when typing
* Backspace, Ctrl-C, Ctrl-D all work

Install/upgrade: `pipx install krasis==0.1.5` or `pipx upgrade krasis`

v0.1.4 2026-02-15 

[Original release notes and assets](https://github.com/brontoguana/krasis/releases/tag/v0.1.4)

## Changes

* **Server warmup**: Runs a short generation on startup before accepting requests, eliminating \~80s cold-start on first request
* **Chat Ctrl-C fix**: Ctrl-C now exits cleanly during streaming (was previously swallowed by blocking I/O)

Install: `pipx install krasis` or `pip install krasis==0.1.4`

v0.1.3 2026-02-15 

[Original release notes and assets](https://github.com/brontoguana/krasis/releases/tag/v0.1.3)

Fixes:

* Auto-optimise now tests GPU prefill strategies first, then decode strategies (was testing both together, leading to suboptimal results)
* –benchmark flag forces fresh auto-optimise (never uses cache)
* All previous fixes included (libnuma optional, Python 3.10-3.13 wheels, \~/.krasis home directory)
v0.1.2 2026-02-15 

[Original release notes and assets](https://github.com/brontoguana/krasis/releases/tag/v0.1.2)

Fix CI: builds wheels for Python 3.10, 3.11, 3.12, 3.13 on manylinux x86\_64.

* libnuma made optional at build time (graceful fallback)
* Fixed Python version targeting (was accidentally including 3.14 which PyO3 doesn’t support)
v0.1.1 2026-02-15 

[Original release notes and assets](https://github.com/brontoguana/krasis/releases/tag/v0.1.1)

Fix: v0.1.0 only shipped a Python 3.8 wheel. This release builds wheels for Python 3.10, 3.11, 3.12, and 3.13.

Changes:

* libnuma made optional at build time (CI compatibility)
* NUMA support gracefully disabled when library not available
* Launcher uses \~/.krasis/ for config and models (pip install compatible)
* Updated README with pip install instructions
v0.1.0 2026-02-15 

[Original release notes and assets](https://github.com/brontoguana/krasis/releases/tag/v0.1.0)

First PyPI release of Krasis — hybrid LLM runtime for MoE models.

* Rust+PyO3 MoE engine with AVX2 INT4/INT8 kernels
* GPU prefill via fused Marlin MoE (INT4)
* CPU decode with NUMA-aware memory allocation
* Dual-format weight cache (GPU Marlin + CPU-optimized)
* Layer-grouped, HCS, LRU prefill strategies
* Auto-optimiser for strategy selection
* OpenAI-compatible API server