| | Quant | Size (MB) | PPL | Size (%) | Accuracy (%) | PPL error rate | | |
| | ------ | --------- | ------- | -------- | ------------ | -------------- | | |
| | IQ1_S | 301 | 32.6390 | 31.75 | 69.27 | 0.59 | | |
| | IQ1_M | 303 | 29.9721 | 31.96 | 75.43 | 0.53 | | |
| | IQ2_XXS | 306 | 28.2728 | 32.28 | 79.97 | 0.52 | | |
| | IQ2_XS | 309 | 26.5764 | 32.59 | 85.07 | 0.48 | | |
| | IQ2_S | 310 | 26.3276 | 32.70 | 85.88 | 0.48 | | |
| | IQ2_M | 313 | 25.6364 | 33.02 | 88.19 | 0.46 | | |
| | Q2_K_S | 315 | 25.6897 | 33.23 | 88.01 | 0.46 | | |
| | IQ3_XXS | 318 | 24.6070 | 33.54 | 91.88 | 0.44 | | |
| | Q3_K_S | 322 | 24.5354 | 33.97 | 92.15 | 0.44 | | |
| | IQ3_S | 322 | 24.0335 | 33.97 | 94.07 | 0.43 | | |
| | IQ3_XS | 322 | 24.0335 | 33.97 | 94.07 | 0.43 | | |
| | Q2_K | 322 | 24.4565 | 33.97 | 92.45 | 0.44 | | |
| | IQ3_M | 326 | 23.8652 | 34.39 | 94.74 | 0.43 | | |
| | IQ4_XS | 333 | 23.6261 | 35.13 | 95.70 | 0.42 | | |
| | IQ4_NL | 336 | 23.6102 | 35.44 | 95.76 | 0.42 | | |
| | Q4_0 | 336 | 23.8547 | 35.44 | 94.78 | 0.42 | | |
| | Q3_K_M | 338 | 23.5268 | 35.65 | 96.10 | 0.42 | | |
| | Q3_K_L | 352 | 23.3214 | 37.13 | 96.95 | 0.42 | | |
| | Q4_1 | 357 | 23.3396 | 37.66 | 96.87 | 0.42 | | |
| | Q4_K_S | 367 | 23.2008 | 38.71 | 97.45 | 0.41 | | |
| | Q5_0 | 379 | 23.1914 | 39.98 | 97.49 | 0.41 | | |
| | Q4_K_M | 379 | 23.1306 | 39.98 | 97.75 | 0.41 | | |
| | Q5_K_S | 393 | 22.8146 | 41.46 | 99.10 | 0.41 | | |
| | Q5_1 | 399 | 22.8381 | 42.09 | 99.00 | 0.41 | | |
| | Q5_K_M | 400 | 22.7761 | 42.19 | 99.27 | 0.41 | | |
| | Q6_K | 482 | 22.6302 | 50.84 | 99.91 | 0.40 | | |
| | Q8_0 | 506 | 22.6359 | 53.38 | 99.88 | 0.40 | | |
| | F16 | 948 | 22.6094 | 100 | 100 | 0.40 | | |