File size: 2,282 Bytes
f0ebb22 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 | | Quant | Size (MB) | PPL | Size (%) | Accuracy (%) | PPL error rate |
| ------ | --------- | ------- | -------- | ------------ | -------------- |
| IQ1_S | 301 | 32.6390 | 31.75 | 69.27 | 0.59 |
| IQ1_M | 303 | 29.9721 | 31.96 | 75.43 | 0.53 |
| IQ2_XXS | 306 | 28.2728 | 32.28 | 79.97 | 0.52 |
| IQ2_XS | 309 | 26.5764 | 32.59 | 85.07 | 0.48 |
| IQ2_S | 310 | 26.3276 | 32.70 | 85.88 | 0.48 |
| IQ2_M | 313 | 25.6364 | 33.02 | 88.19 | 0.46 |
| Q2_K_S | 315 | 25.6897 | 33.23 | 88.01 | 0.46 |
| IQ3_XXS | 318 | 24.6070 | 33.54 | 91.88 | 0.44 |
| Q3_K_S | 322 | 24.5354 | 33.97 | 92.15 | 0.44 |
| IQ3_S | 322 | 24.0335 | 33.97 | 94.07 | 0.43 |
| IQ3_XS | 322 | 24.0335 | 33.97 | 94.07 | 0.43 |
| Q2_K | 322 | 24.4565 | 33.97 | 92.45 | 0.44 |
| IQ3_M | 326 | 23.8652 | 34.39 | 94.74 | 0.43 |
| IQ4_XS | 333 | 23.6261 | 35.13 | 95.70 | 0.42 |
| IQ4_NL | 336 | 23.6102 | 35.44 | 95.76 | 0.42 |
| Q4_0 | 336 | 23.8547 | 35.44 | 94.78 | 0.42 |
| Q3_K_M | 338 | 23.5268 | 35.65 | 96.10 | 0.42 |
| Q3_K_L | 352 | 23.3214 | 37.13 | 96.95 | 0.42 |
| Q4_1 | 357 | 23.3396 | 37.66 | 96.87 | 0.42 |
| Q4_K_S | 367 | 23.2008 | 38.71 | 97.45 | 0.41 |
| Q5_0 | 379 | 23.1914 | 39.98 | 97.49 | 0.41 |
| Q4_K_M | 379 | 23.1306 | 39.98 | 97.75 | 0.41 |
| Q5_K_S | 393 | 22.8146 | 41.46 | 99.10 | 0.41 |
| Q5_1 | 399 | 22.8381 | 42.09 | 99.00 | 0.41 |
| Q5_K_M | 400 | 22.7761 | 42.19 | 99.27 | 0.41 |
| Q6_K | 482 | 22.6302 | 50.84 | 99.91 | 0.40 |
| Q8_0 | 506 | 22.6359 | 53.38 | 99.88 | 0.40 |
| F16 | 948 | 22.6094 | 100 | 100 | 0.40 |
|