Instructions to use mlx-community/DeepSeek-R1-Distill-Llama-8B-4bit-mlx with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- MLX
How to use mlx-community/DeepSeek-R1-Distill-Llama-8B-4bit-mlx with MLX:
# Download the model from the Hub pip install huggingface_hub[hf_xet] huggingface-cli download --local-dir DeepSeek-R1-Distill-Llama-8B-4bit-mlx mlx-community/DeepSeek-R1-Distill-Llama-8B-4bit-mlx
- Notebooks
- Google Colab
- Kaggle
- Local Apps
- LM Studio
rope_scaling must contain keys {'factor', 'type'}
#1
by sanjay-complyance - opened
rope_scaling must contain keys {'factor', 'type'}
was facing this error when I tried to quantize the unquantized model. any inputs on why i am facing this and possible solutions
ADDITIONAL INFO:
Device: M1 pro, 32 gb ram
MLX lora training example script