Masoom Alam

EmbeddingGemma looks really promising — especially the focus on efficiency. In my own experiments, I quantized DistilBERT (SST-2) into an int8 ONNX model and saw latency reductions of around 20–30% with almost no accuracy drop. I’m curious if the team has explored quantization-aware training (QAT) for EmbeddingGemma, or if post-training quantization is sufficient to maintain embedding quality? Here’s my demo repo for reference: https://www.linkedin.com/posts/dr-mm-alam-93991120b_demofirst-aichips-edgeai-activity-7381674484098883584-0Rwn/?utm_source=share&utm_medium=member_desktop&rcm=ACoAADVZuP0BheDJgKL8dWk-bNo7Yd4zhsOnNL4

Very interesting post — I appreciate how you highlight MXFP4 quantization and the performance trade-offs in Transformers. In my own experiments, I applied quantization to a mmalam786/distilbert-sst2-int8-onnx-demo and saw ~20-30% latency reduction with negligible accuracy drop. Do you have insights on how MXFP4 handles activation quantization vs weight quantization in edge scenarios? Happy to share my demo if useful: https://www.linkedin.com/posts/dr-mm-alam-93991120b_demofirst-aichips-edgeai-activity-7381674484098883584-0Rwn/?utm_source=share&utm_medium=member_desktop&rcm=ACoAADVZuP0BheDJgKL8dWk-bNo7Yd4zhsOnNL4