Canonical: https://krasis.ai/adq/ How we test # Can it still do _the work?_ ADQ stands for **Agentic Development Qualification**. It is Krasis’s extensive acceptance programme for checking whether a model running in a specific configuration can complete complex software-development work. Loading a model, generating fluent text and reporting fast tokens per second are useful checks. They do not establish that it can repair code, use tools correctly or keep track of decisions across a long working session. Quantisation, attention, cache handling and runtime defects can affect those abilities even when a short answer looks convincing. ADQ combines real development tasks with numerical comparisons, long-context checks and operational testing. We run the tests and retain the evidence; this is a Northloom qualification programme, not external certification. ## What we test ### 1\. The actual configuration We check the settings that reach inference through the real client: templates, sampling, thinking controls, token budgets and tool parsing. A setting shown as enabled must demonstrably work. Reasoning, final answers and tool arguments must reach the correct place. ### 2\. Repeatable runtime execution Frozen greedy requests exercise cold starts, warmed execution, repeated requests and fresh server lifecycles. We compare token IDs and retained numerical outputs exactly. Unexplained differences block new task scoring; repeatability alone still does not prove correctness. ### 3\. Numerical fidelity Identical token inputs are compared with an independent source-precision reference where supported. We examine logits, token probabilities, rankings and continuations at increasing lengths, normally through 131,072 tokens. Quantisation differences, localised outliers and missing reference coverage are reported explicitly. ### 4\. Long-context behaviour Frozen real content places facts and code relationships near the beginning, middle and end. Tests require retrieval, distinguishing similar facts and combining distant information. Independent runtimes receive the same cases and objective checks; provider agreement is not itself proof of correctness. ### 5\. Productive development work A tool-using agent works in isolated, disposable repositories. Tasks cover diagnosis, repairs, regression tests, configuration and scheduling logic, and decisions buried in long project history. Completed changes face public and hidden tests, protected-file checks and instruction checks. A plausible plan without a working implementation cannot pass. ### 6\. Operational endurance Sustained client/server testing covers growing conversations, tool exchanges, save/restore, unrelated-session isolation, cancellation, recovery and supported request handling. We inspect CUDA, expert-streaming and cache health, token accounting and minimum free VRAM. Throughput is measured separately without diagnostic tracing. ## Long context means exercised context **ADQ-128K** exercises 131,072 input tokens; **ADQ-256K** exercises 262,144; **ADQ-500K** exercises 500,000\. Counts use the checkpoint’s tokenizer, not character estimates or repeated filler. A 500K development task must actually put that history into the model’s context, with room left for tools and answers. Allocating a large cache does not earn a tier. Detailed numerical comparison normally stops at 131K; behavioural and productive checks extend to the declared tier. Passing a 500K task does not imply a 500K numerical-reference comparison. ## How a result earns its meaning The checkpoint revision, runtime build, weight and cache precisions, hardware, thinking/sampling settings, context tier, client and task suite define the tested identity. Prompts, acceptance rules and budgets are frozen for scoring. First outcomes, interruptions, retries and incomplete runs stay visible; a later success does not erase an earlier failure. We report **productive tasks, functional configuration, determinism, numerical fidelity, runtime integrity and VRAM safety separately**. A task pass can coexist with a numerical warning or a safety failure. One must not conceal the other. The **ADQ · scoped** badge means a productive qualification is recorded for a particular tested profile. Follow it to the scope below. It does not automatically qualify the default download settings, your GPU, vision, every context length or future runtime changes. No badge means no qualifying result is listed here; it does not mean the model failed. ## Recorded qualifications Summaries reviewed on 22 October 2026 from retained campaign reports and terminal verdicts. These are bounded task results, not estimates of reliability across all coding work. 21 October 2026 ### GLM-5.3-Flash Scoped productive qualification · ADQ-500K · 4/4 tasks **Tested scope:** Text, thinking enabled, greedy generation; scales-only routed INT4, HQQ6 attention, K6V6 cache, INT8 dense/shared/head, RTX PRO 6000. **Separate findings:** Configuration, frozen determinism controls, runtime integrity, operational testing and VRAM safety passed. Numerical fidelity retains a localised outlier warning. Short determinism controls do not establish 500K numerical identity across server lifecycles. **Limits:** Requires a separately prepared scales-only expert cache: ordinary launcher INT4 defaults do not inherit this qualification. History passed on a disclosed clean interruption retry after an unfinished attempt; earlier task exposure and retrospective configuration checks are recorded. Scored streaming consumed IDs were not separately captured; supplementary native-ID probes are separate evidence. Vision, unseen tasks, other GPUs and other configurations are outside this result. Evidence record GLM current-main campaign, recovery-20261021-0844/SCOPED\_REPORT.md and FINAL\_RECONCILIATION.json. These retained internal engineering records are summarised here; they are not public report downloads. 14 September 2026 ### DeepSeek-V4-Flash-Vision-Exp Scoped productive qualification · ADQ-500K · 4/4 tasks **Tested scope:** Accepted September r12 sweep: routed INT4, mixed-precision treatments and INT8 controls on frozen campaign identities. This is evidence for those measured profiles, not every launcher configuration. **Separate findings:** All six treatments passed the four productive tasks, exact replay checks and 12/12 operational-soak checks. INT4 and mixed treatments retain numerical outlier warnings; INT8 was within the recorded envelope. Every treatment failed the separate 600 MiB VRAM-reserve requirement. **Limits:** Productive qualification does not constitute a clean overall safety pass. Vision was 4/4 semantically but 3/4 under the unchanged literal checker (center versus centre). Numerical-envelope provenance limitations remain disclosed. These once-only results do not establish repeated-task reliability or qualify a mixed production default. Evidence record Accepted r12 campaign, ADQ\_VERDICTS.md interpretation addendum and EVIDENCE\_AUDIT\_R13.md. These retained internal engineering records are summarised here; they are not public report downloads. ## Why it matters when choosing a model The hardware checker answers whether a model may fit. ADQ adds evidence about useful work in a tested setup. Read both: memory fit, productive capability and safe operation are separate requirements. [Check your hardware →](https://krasis.ai/install/)[Read the research](https://krasis.ai/research/white-paper/) Our living white paper explains the methodology and earlier failures. Its 19 October edition predates the scoped GLM result above; historical findings remain dated rather than being rewritten as later successes.