Leopo1d/OpenVul-Qwen3-4B-DPO

OpenVul-Qwen3-4B-DPO

OpenVul-Qwen3-4B-DPO, post-trained from OpenVul-Qwen3-4B-SFT-ep5, serves as an advanced vulnerability detection LLM optimized to distinguish between vulnerable code and its patched counterparts without an explicit reward model.

📚 Data Curation:

Trained on Paired CoTs sampled directly from the SFT LLM to minimize distribution shift.

💡 Key Feature:

Focuses on context-level vulnerability detection, utilizing inter-procedural contexts (global variables, type definitions, callee functions etc.) rather than isolated functions.

🔗 Related Links:

• Code: GitHub
• Data: OpenVul_Vulnerability_Preference_Dataset_for_DPO
• Paper: From SFT to RL: Demystifying the Post-Training Pipeline for LLM-based Vulnerability Detection

📄 Prompt Template (RECOMMENDED!):

We recommend to use vLLM for inference. Please set enable_thinking=True, n=8, repetition_penalty=1.0, temperature=0.6, top_p=0.95, top_k=20, min_p=0, max_tokens=32768. More details can be found in code.

• System Prompt

You are a vulnerability detection expert specializing in identifying security flaws in C/C++ code, with a focus on Common Weakness Enumeration (CWE) standards. You provide precise, evidence-based analysis without speculation, and clearly label any vulnerabilities you detect.

• User Prompt

Your task is to evaluate whether the following C/C++ code contains any security vulnerabilities.

You will be provided with two sections:
1. Context: Relevant code such as includes, type definitions, global variables, macros, and definitions of any functions called within the target function.
2. Code: The target function to analyze.

Use all available information to analyze the function step by step.
If the target function alone is insufficient to determine whether a vulnerability exists, refer to the Context section before making a judgment.
Do not assume vulnerabilities — only report what is supported by the code and context.

In your final response, list all detected vulnerabilities and CWE identifiers if applicable.
Conclude with one of the following indicators on a new line:
- HAS_VUL — if any vulnerabilities are found
- NO_VUL — if no vulnerabilities are found

```Context
{Context}
```
```Code
File: {Located File}
Method: {Function Name}
----------------------------------------
{Target Function}
```

Analyze the code now.

📎 Citation:

@misc{li2026sftrldemystifyingposttraining,
      title={From SFT to RL: Demystifying the Post-Training Pipeline for LLM-based Vulnerability Detection}, 
      author={Youpeng Li and Fuxun Yu and Xinda Wang},
      year={2026},
      eprint={2602.14012},
      archivePrefix={arXiv},
      primaryClass={cs.CR},
      url={https://arxiv.org/abs/2602.14012}, 
}