DuoAttention: Efficient Long-Context LLM Inference with Retrieval and Streaming Heads
Paper
•
2410.10819
•
Published
•
7
Jiaming Tang
Sakits
AI & ML interests
Efficient Machine Learning
Recent Activity
updated
a model
2 days ago
Sakits/so101_pickplace_calibration_pi05_async_shared_obs_050000
published
a model
2 days ago
Sakits/so101_pickplace_calibration_pi05_async_shared_obs_050000
updated
a model
5 days ago
Sakits/so101_pickplace_calibration_pi05_async_rnn_0.5_wopos
Organizations
authored
a
paper
about 1 year ago
authored
3
papers
over 1 year ago
AWQ: Activation-aware Weight Quantization for LLM Compression and Acceleration
Paper
•
2306.00978
•
Published
•
11
Quest: Query-Aware Sparsity for Efficient Long-Context LLM Inference
Paper
•
2406.10774
•
Published
•
4
OliVe: Accelerating Large Language Models via Hardware-friendly Outlier-Victim Pair Quantization
Paper
•
2304.07493
•
Published
•
1