CyberSecQwen‑4B releases as a Locally Runnable 4B Model for Defensive Cybersecurity

CyberSecQwen‑4B, a 4‑billion‑parameter model released under the Apache 2.0 license, debuted on May 8, 2026 as a locally runnable tool aimed at defensive cybersecurity teams. Built for the AMD Developer Hackathon, the model is designed so sensitive incident artifacts can remain on‑premises rather than being sent to hosted APIs — a capability that matters to organizations that must limit data exfiltration and operate in air‑gapped environments.

The model is a narrow, specialized fine‑tune focused on cyber threat intelligence (CTI) tasks: CWE classification, mapping CVEs to CWEs, and structured CTI question‑and‑answer. The authors argue a well‑tuned 4B can match or outperform larger specialists on these tightly scoped tasks while fitting on a 12 GB consumer GPU, making it deployable in constrained operational environments.

In head‑to‑head tests the team evaluated CyberSecQwen‑4B against Cisco’s Foundation‑Sec‑Instruct‑8B with the CTI‑Bench protocol (n=5, temperature 0.3). On the CTI‑MCQ test (2,500 items) CyberSecQwen‑4B scored 0.5868 ± 0.0029 versus 0.4996 for the 8B baseline, an advantage of 8.7 percentage points. On CTI‑RCM (1,000 CVE→CWE items) it scored 0.6664 ± 0.0023 compared with 0.6850 for the 8B, a 1.9 point deficit; the authors note this retains 97.3% of the 8B’s RCM accuracy while outperforming on MCQ.

The post emphasizes why these numeric tradeoffs matter operationally: running models locally avoids the exfiltration risks of sending incident data to external APIs, reduces per‑call costs that accumulate across thousands of low‑confidence SOC alerts, and enables analysis inside air‑gapped or partially connected networks common in critical infrastructure and government environments.

Training and deployment were executed end‑to‑end on a single AMD Instinct MI300X with 192 GB HBM3 in the AMD Developer Cloud. The pipeline used ROCm 7, vLLM, and FlashAttention‑2, training in full bf16 at sequence length 4,096 with batch size 4. The authors supply a hardware‑agnostic train.sh and say the recipe can be adapted to other 40+ GB datacenter GPUs by removing AMD environment variables and installing the appropriate flash‑attn wheel. Cited component versions include PyTorch 2.6.0 (ROCm), flash‑attn 2.8.3, and vLLM 0.10.1.

For builders and defenders the practical takeaway is concrete: domain‑focused 4B models can achieve near‑parity or better on key CTI benchmarks while meeting operational constraints — local execution, lower API cost exposure, and air‑gap compatibility. The post also links a five‑minute walkthrough video and detailed configuration files for teams that want to reproduce the training or adapt the recipe to their hardware.