Hugging Face's logo

qualcomm
/
Stable-Diffusion-v2.1

Unconditional Image Generation
PyTorch
generative_ai
android
Model card Files
xet
 Community
1
Stable-Diffusion-v2.1 / README.md
qaihm-bot's picture
qaihm-bot
v0.51.0
24d7785 verified
preview code
|
raw
history blame contribute delete
16.5 kB
metadata 
library_name: pytorch
license: other
tags:
  - generative_ai
  - android
pipeline_tag: unconditional-image-generation

Stable-Diffusion-v2.1: Optimized for Qualcomm Devices

Generates high resolution images from text prompts using a latent diffusion model. This model uses CLIP ViT-L/14 as text encoder, U-Net based latent denoising, and VAE based decoder to generate the final image.

This is based on the implementation of Stable-Diffusion-v2.1 found here. This repository contains pre-exported model files optimized for Qualcomm® devices. You can use the Qualcomm® AI Hub Models library to export with custom configurations. More details on model performance across various devices, can be found here.

Qualcomm AI Hub Models uses Qualcomm AI Hub Workbench to compile, profile, and evaluate this model. Sign up to run these models on a hosted Qualcomm® device.

Deploy to Snapdragon X Elite NPU

Please follow the Stable Diffusion Windows App tutorial to quantize model with custom weights.

Quantize and Deploy Your Own Fine-Tuned Stable Diffusion

Please follow the Quantize Stable Diffusion tutorial to quantize model with custom weights.

Getting Started

There are two ways to deploy this model on your device:

Option 1: Download Pre-Exported Models

Below are pre-exported model assets ready for deployment.

Runtime Precision Chipset SDK Versions Download
PRECOMPILED_QNN_ONNX w8a16 Snapdragon® 8 Elite Gen 5 Mobile QAIRT 2.42, ONNX Runtime 1.24.3 Download
PRECOMPILED_QNN_ONNX w8a16 Snapdragon® X2 Elite QAIRT 2.42, ONNX Runtime 1.24.3 Download
PRECOMPILED_QNN_ONNX w8a16 Snapdragon® X Elite QAIRT 2.42, ONNX Runtime 1.24.3 Download
PRECOMPILED_QNN_ONNX w8a16 Snapdragon® 8 Gen 3 Mobile QAIRT 2.42, ONNX Runtime 1.24.3 Download
PRECOMPILED_QNN_ONNX w8a16 Qualcomm® QCS8550 (Proxy) QAIRT 2.42, ONNX Runtime 1.24.3 Download
PRECOMPILED_QNN_ONNX w8a16 Snapdragon® 8 Elite For Galaxy Mobile QAIRT 2.42, ONNX Runtime 1.24.3 Download
PRECOMPILED_QNN_ONNX w8a16 Snapdragon® 7 Gen 4 Mobile QAIRT 2.42, ONNX Runtime 1.24.3 Download
PRECOMPILED_QNN_ONNX w8a16 Qualcomm® QCM6690 QAIRT 2.42, ONNX Runtime 1.24.3 Download
PRECOMPILED_QNN_ONNX w8a16 Qualcomm® QCS9075 QAIRT 2.42, ONNX Runtime 1.24.3 Download
QNN_CONTEXT_BINARY w8a16 Snapdragon® 8 Elite Gen 5 Mobile QAIRT 2.45 Download
QNN_CONTEXT_BINARY w8a16 Snapdragon® X2 Elite QAIRT 2.45 Download
QNN_CONTEXT_BINARY w8a16 Snapdragon® X Elite QAIRT 2.45 Download
QNN_CONTEXT_BINARY w8a16 Snapdragon® 8 Gen 3 Mobile QAIRT 2.45 Download
QNN_CONTEXT_BINARY w8a16 Qualcomm® QCS8550 (Proxy) QAIRT 2.45 Download
QNN_CONTEXT_BINARY w8a16 Qualcomm® SA8775P QAIRT 2.45 Download
QNN_CONTEXT_BINARY w8a16 Snapdragon® 8 Elite For Galaxy Mobile QAIRT 2.45 Download
QNN_CONTEXT_BINARY w8a16 Snapdragon® 7 Gen 4 Mobile QAIRT 2.45 Download
QNN_CONTEXT_BINARY w8a16 Qualcomm® SA7255P QAIRT 2.45 Download
QNN_CONTEXT_BINARY w8a16 Qualcomm® QCM6690 QAIRT 2.45 Download
QNN_CONTEXT_BINARY w8a16 Qualcomm® QCS9075 QAIRT 2.45 Download

For more device-specific assets and performance metrics, visit Stable-Diffusion-v2.1 on Qualcomm® AI Hub.

Option 2: Export with Custom Configurations

Use the Qualcomm® AI Hub Models Python library to compile and export the model with your own:

  • Custom weights (e.g., fine-tuned checkpoints)
  • Custom input shapes
  • Target device and runtime configurations

This option is ideal if you need to customize the model beyond the default configuration provided here.

See our repository for Stable-Diffusion-v2.1 on GitHub for usage instructions.

Model Details

Model Type: Model_use_case.image_generation

Model Stats:

  • Input: Text prompt to generate image

Performance Summary

Model Runtime Precision Chipset Inference Time (ms) Peak Memory Range (MB) Primary Compute Unit
text_encoder PRECOMPILED_QNN_ONNX w8a16 Snapdragon® 8 Elite Gen 5 Mobile 4.599 ms 0 - 10 MB NPU
text_encoder PRECOMPILED_QNN_ONNX w8a16 Snapdragon® X2 Elite 3.218 ms 417 - 417 MB NPU
text_encoder PRECOMPILED_QNN_ONNX w8a16 Snapdragon® X Elite 7.713 ms 418 - 418 MB NPU
text_encoder PRECOMPILED_QNN_ONNX w8a16 Snapdragon® 8 Gen 3 Mobile 6.019 ms 0 - 12 MB NPU
text_encoder PRECOMPILED_QNN_ONNX w8a16 Qualcomm® QCS8550 (Proxy) 8.04 ms 0 - 449 MB NPU
text_encoder PRECOMPILED_QNN_ONNX w8a16 Qualcomm® QCS9075 7.4 ms 0 - 3 MB NPU
text_encoder PRECOMPILED_QNN_ONNX w8a16 Qualcomm® QCM6690 82.08 ms 0 - 11 MB NPU
text_encoder PRECOMPILED_QNN_ONNX w8a16 Snapdragon® 8 Elite For Galaxy Mobile 5.06 ms 0 - 12 MB NPU
text_encoder PRECOMPILED_QNN_ONNX w8a16 Snapdragon® 7 Gen 4 Mobile 12.021 ms 0 - 10 MB NPU
text_encoder QNN_CONTEXT_BINARY w8a16 Snapdragon® 8 Elite Gen 5 Mobile 4.829 ms 0 - 10 MB NPU
text_encoder QNN_CONTEXT_BINARY w8a16 Snapdragon® X2 Elite 4.187 ms 0 - 0 MB NPU
text_encoder QNN_CONTEXT_BINARY w8a16 Snapdragon® X Elite 7.841 ms 0 - 0 MB NPU
text_encoder QNN_CONTEXT_BINARY w8a16 Snapdragon® 8 Gen 3 Mobile 5.754 ms 0 - 8 MB NPU
text_encoder QNN_CONTEXT_BINARY w8a16 Qualcomm® QCS8275 (Proxy) 17.374 ms 0 - 8 MB NPU
text_encoder QNN_CONTEXT_BINARY w8a16 Qualcomm® QCS8550 (Proxy) 8.134 ms 1 - 2 MB NPU
text_encoder QNN_CONTEXT_BINARY w8a16 Qualcomm® SA8775P 8.366 ms 0 - 8 MB NPU
text_encoder QNN_CONTEXT_BINARY w8a16 Qualcomm® QCS9075 8.045 ms 0 - 2 MB NPU
text_encoder QNN_CONTEXT_BINARY w8a16 Qualcomm® QCM6690 82.919 ms 0 - 6 MB NPU
text_encoder QNN_CONTEXT_BINARY w8a16 Qualcomm® SA7255P 17.374 ms 0 - 8 MB NPU
text_encoder QNN_CONTEXT_BINARY w8a16 Snapdragon® 8 Elite For Galaxy Mobile 4.922 ms 0 - 9 MB NPU
text_encoder QNN_CONTEXT_BINARY w8a16 Snapdragon® 7 Gen 4 Mobile 12.124 ms 1 - 7 MB NPU
unet PRECOMPILED_QNN_ONNX w8a16 Snapdragon® 8 Elite Gen 5 Mobile 70.927 ms 0 - 9 MB NPU
unet PRECOMPILED_QNN_ONNX w8a16 Snapdragon® X2 Elite 66.191 ms 845 - 845 MB NPU
unet PRECOMPILED_QNN_ONNX w8a16 Snapdragon® X Elite 157.732 ms 842 - 842 MB NPU
unet PRECOMPILED_QNN_ONNX w8a16 Snapdragon® 8 Gen 3 Mobile 111.128 ms 0 - 11 MB NPU
unet PRECOMPILED_QNN_ONNX w8a16 Qualcomm® QCS8550 (Proxy) 153.824 ms 0 - 897 MB NPU
unet PRECOMPILED_QNN_ONNX w8a16 Qualcomm® QCS9075 148.423 ms 0 - 3 MB NPU
unet PRECOMPILED_QNN_ONNX w8a16 Qualcomm® QCM6690 2412.561 ms 0 - 11 MB NPU
unet PRECOMPILED_QNN_ONNX w8a16 Snapdragon® 8 Elite For Galaxy Mobile 100.957 ms 0 - 11 MB NPU
unet PRECOMPILED_QNN_ONNX w8a16 Snapdragon® 7 Gen 4 Mobile 216.799 ms 0 - 11 MB NPU
unet QNN_CONTEXT_BINARY w8a16 Snapdragon® 8 Elite Gen 5 Mobile 64.788 ms 0 - 9 MB NPU
unet QNN_CONTEXT_BINARY w8a16 Snapdragon® X2 Elite 67.595 ms 0 - 0 MB NPU
unet QNN_CONTEXT_BINARY w8a16 Snapdragon® X Elite 157.523 ms 0 - 0 MB NPU
unet QNN_CONTEXT_BINARY w8a16 Snapdragon® 8 Gen 3 Mobile 109.836 ms 0 - 8 MB NPU
unet QNN_CONTEXT_BINARY w8a16 Qualcomm® QCS8275 (Proxy) 346.003 ms 0 - 7 MB NPU
unet QNN_CONTEXT_BINARY w8a16 Qualcomm® QCS8550 (Proxy) 152.738 ms 0 - 2 MB NPU
unet QNN_CONTEXT_BINARY w8a16 Qualcomm® SA8775P 147.856 ms 0 - 8 MB NPU
unet QNN_CONTEXT_BINARY w8a16 Qualcomm® QCS9075 147.177 ms 0 - 2 MB NPU
unet QNN_CONTEXT_BINARY w8a16 Qualcomm® QCM6690 2468.005 ms 0 - 10 MB NPU
unet QNN_CONTEXT_BINARY w8a16 Qualcomm® SA7255P 346.003 ms 0 - 7 MB NPU
unet QNN_CONTEXT_BINARY w8a16 Snapdragon® 8 Elite For Galaxy Mobile 99.957 ms 0 - 13 MB NPU
unet QNN_CONTEXT_BINARY w8a16 Snapdragon® 7 Gen 4 Mobile 213.566 ms 0 - 7 MB NPU
vae PRECOMPILED_QNN_ONNX w8a16 Snapdragon® 8 Elite Gen 5 Mobile 132.639 ms 3 - 13 MB NPU
vae PRECOMPILED_QNN_ONNX w8a16 Snapdragon® X2 Elite 144.688 ms 63 - 63 MB NPU
vae PRECOMPILED_QNN_ONNX w8a16 Snapdragon® X Elite 260.23 ms 64 - 64 MB NPU
vae PRECOMPILED_QNN_ONNX w8a16 Snapdragon® 8 Gen 3 Mobile 200.01 ms 3 - 10 MB NPU
vae PRECOMPILED_QNN_ONNX w8a16 Qualcomm® QCS8550 (Proxy) 262.127 ms 0 - 65 MB NPU
vae PRECOMPILED_QNN_ONNX w8a16 Qualcomm® QCS9075 246.199 ms 3 - 6 MB NPU
vae PRECOMPILED_QNN_ONNX w8a16 Qualcomm® QCM6690 6213.647 ms 0 - 6 MB NPU
vae PRECOMPILED_QNN_ONNX w8a16 Snapdragon® 8 Elite For Galaxy Mobile 171.805 ms 3 - 15 MB NPU
vae PRECOMPILED_QNN_ONNX w8a16 Snapdragon® 7 Gen 4 Mobile 507.911 ms 2 - 9 MB NPU
vae QNN_CONTEXT_BINARY w8a16 Snapdragon® 8 Elite Gen 5 Mobile 123.132 ms 0 - 9 MB NPU
vae QNN_CONTEXT_BINARY w8a16 Snapdragon® X2 Elite 128.959 ms 0 - 0 MB NPU
vae QNN_CONTEXT_BINARY w8a16 Snapdragon® X Elite 257.612 ms 0 - 0 MB NPU
vae QNN_CONTEXT_BINARY w8a16 Snapdragon® 8 Gen 3 Mobile 195.277 ms 0 - 7 MB NPU
vae QNN_CONTEXT_BINARY w8a16 Qualcomm® QCS8275 (Proxy) 713.296 ms 0 - 9 MB NPU
vae QNN_CONTEXT_BINARY w8a16 Qualcomm® QCS8550 (Proxy) 260.689 ms 0 - 2 MB NPU
vae QNN_CONTEXT_BINARY w8a16 Qualcomm® SA8775P 244.5 ms 0 - 9 MB NPU
vae QNN_CONTEXT_BINARY w8a16 Qualcomm® QCS9075 242.058 ms 0 - 3 MB NPU
vae QNN_CONTEXT_BINARY w8a16 Qualcomm® QCM6690 6119.655 ms 0 - 7 MB NPU
vae QNN_CONTEXT_BINARY w8a16 Qualcomm® SA7255P 713.296 ms 0 - 9 MB NPU
vae QNN_CONTEXT_BINARY w8a16 Snapdragon® 8 Elite For Galaxy Mobile 158.943 ms 0 - 9 MB NPU
vae QNN_CONTEXT_BINARY w8a16 Snapdragon® 7 Gen 4 Mobile 502.972 ms 0 - 7 MB NPU

License

  • The license for the original implementation of Stable-Diffusion-v2.1 can be found here.

References

Community

Usage and Limitations

This model may not be used for or in connection with any of the following applications:

  • Accessing essential private and public services and benefits;
  • Administration of justice and democratic processes;
  • Assessing or recognizing the emotional state of a person;
  • Biometric and biometrics-based systems, including categorization of persons based on sensitive characteristics;
  • Education and vocational training;
  • Employment and workers management;
  • Exploitation of the vulnerabilities of persons resulting in harmful behavior;
  • General purpose social scoring;
  • Law enforcement;
  • Management and operation of critical infrastructure;
  • Migration, asylum and border control management;
  • Predictive policing;
  • Real-time remote biometric identification in public spaces;
  • Recommender systems of social media platforms;
  • Scraping of facial images (from the internet or otherwise); and/or
  • Subliminal manipulation