Kernels
kernels-bot commited on
Commit
f0004f6
·
verified ·
1 Parent(s): c22a544

Uploaded using `kernel-builder`.

Browse files
Files changed (1) hide show
  1. README.md +21 -12
README.md CHANGED
@@ -1,23 +1,32 @@
1
  ---
 
2
  license: apache-2.0
3
- tags:
4
- - kernels
5
  ---
6
 
7
- ![Status](https://hubwebhook.dholtz.com/shield?repo=kernels-community/paged-attention)
8
 
9
- ## attention
10
 
11
- Paged attention kernels from [vLLM](https://github.com/vllm-project/) and [mistral.rs](https://github.com/EricLBuehler/mistral.rs).
 
 
12
 
 
 
13
 
14
- ### Performance
 
15
 
16
- <img class="dark:hidden border border-gray-200 dark:border-gray-700 rounded-lg" src="media/benches_light_animation.svg" />
17
- <img class="hidden dark:block border border-gray-200 dark:border-gray-700 rounded-lg" src="media/benches_dark_animation.svg" />
 
 
 
 
 
 
 
18
 
19
- <img class="dark:hidden border border-gray-200 dark:border-gray-700 rounded-lg" src="media/benches_light_latency.svg" />
20
- <img class="hidden dark:block border border-gray-200 dark:border-gray-700 rounded-lg" src="media/benches_dark_latency.svg" />
21
 
22
- <img class="dark:hidden border border-gray-200 dark:border-gray-700 rounded-lg" src="media/benches_light_throughput.svg" />
23
- <img class="hidden dark:block border border-gray-200 dark:border-gray-700 rounded-lg" src="media/benches_dark_throughput.svg" />
 
1
  ---
2
+ library_name: kernels
3
  license: apache-2.0
 
 
4
  ---
5
 
6
+ This is the repository card of kernels-community/paged-attention that has been pushed on the Hub. It was built to be used with the [`kernels` library](https://github.com/huggingface/kernels). This card was automatically generated.
7
 
8
+ ## How to use
9
 
10
+ ```python
11
+ # make sure `kernels` is installed: `pip install -U kernels`
12
+ from kernels import get_kernel
13
 
14
+ kernel_module = get_kernel("kernels-community/paged-attention")
15
+ convert_fp8 = kernel_module.convert_fp8
16
 
17
+ convert_fp8(...)
18
+ ```
19
 
20
+ ## Available functions
21
+ - `convert_fp8`
22
+ - `copy_blocks`
23
+ - `ops`
24
+ - `paged_attention_v1`
25
+ - `paged_attention_v2`
26
+ - `reshape_and_cache`
27
+ - `reshape_and_cache_flash`
28
+ - `swap_blocks`
29
 
30
+ ## Benchmarks
 
31
 
32
+ Benchmarking script is available for this kernel. Run `kernels benchmark kernels-community/paged-attention`.