David Soušek
sousekd
AI & ML interests
Running AI on-prem with agents processing confidential data. Supporting government, private sector, and all kinds of weirdos.
Recent Activity
replied to
csabakecskemeti's
post
3 days ago
Looking for some help to test an INT8 Deepseek 3.2:
SGLang supports Channel wise INT8 quants on CPUs with AMX instructions (Xeon 5 and above AFAIK)
https://lmsys.org/blog/2025-07-14-intel-xeon-optimization/
Currently uploading an INT8 version of Deepseek 3.2 Speciale:
https://huggingface.co/DevQuasar/deepseek-ai.DeepSeek-V3.2-Speciale-Channel-INT8
I cannot test this I'm on AMD
"AssertionError: W8A8Int8LinearMethod on CPU requires that CPU has AMX support"
(I assumed it can fall back to some non optimized kernel but seems not)
If anyone with the required resources (Intel Xeon 5/6 + ~768-1TB ram) can help to test this that would be awesome.
If you have hints how to make this work on AMD Threadripper 7000 Pro series please guide me.
Thanks all!
new activity
5 days ago
ubergarm/Kimi-K2-Instruct-GGUF:IQ2_KS passes the Moonshot K2 Vendor Verifier test
liked
a model
5 days ago
mistralai/Mistral-Large-3-675B-Instruct-2512-Eagle
Organizations
None yet