jordan0811 commited on
Commit
6eca370
·
verified ·
1 Parent(s): 46195af

Create README.md

Browse files
Files changed (1) hide show
  1. README.md +61 -0
README.md ADDED
@@ -0,0 +1,61 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: apache-2.0
3
+ language:
4
+ - en
5
+ base_model:
6
+ - apple/MobileCLIP2-S4
7
+ - apple/MobileCLIP2-S2
8
+ pipeline_tag: image-text-to-text
9
+ tags:
10
+ - MobileCLIP
11
+ - MobileCLIP2
12
+ - CLIP
13
+ - Classification
14
+ ---
15
+
16
+ # MobileCLIP2
17
+
18
+ The following versions of MobileCLIP2 have been converted to run on the Axera NPU using w8a16 quantization. Compatible with Pulsar2 version: 4.2
19
+ - MobileCLIP2-S2
20
+ - MobileCLIP2-S4
21
+
22
+ If you want to know how to convert the MobileCLIP2 model into an axmodel that can run on the axera npu board, please read [this link](https://github.com/AXERA-TECH/axera.ml-mobileclip) in detail.
23
+
24
+ ## Support Platform
25
+ - AX650
26
+
27
+ ## End-of-board inference time
28
+ - MobileCLIP2-S2
29
+ | Stage | Time |
30
+ |------|------|
31
+ | image encoder | 19.146 ms |
32
+ | text encoder | 5.675 ms |
33
+
34
+ - MobileCLIP2-S4
35
+ | Stage | Time |
36
+ |------|------|
37
+ | image encoder | 65.328 ms |
38
+ | text encoder | 12.663 ms |
39
+
40
+
41
+ ## How to use
42
+
43
+ Download all files from this repository to the device
44
+
45
+ Run the following command:
46
+ ```bash
47
+ python3 run_axmodel.py -ie ./mobileclip2_s4_image_encoder.axmodel -te ./mobileclip2_s4_text_encoder.axmodel -i ./zebra.jpg -t "a zebra" "a dog" "two zebras"
48
+ ```
49
+
50
+ Model input and output examples are as follows:
51
+ 1. the image you want to input:
52
+
53
+ ![](zebra.jpg)
54
+
55
+ 3. The description of the text you want to categorize:
56
+
57
+ ["a zebra", "a dog", "two zebras"]
58
+
59
+ 4. Model output class confidence scores:
60
+
61
+ Label probs: [[6.095444e-02 5.628616e-14 9.390456e-01]]