Instructions to use QuixiAI/Wizard-Vicuna-13B-Uncensored with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use QuixiAI/Wizard-Vicuna-13B-Uncensored with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="QuixiAI/Wizard-Vicuna-13B-Uncensored")# Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("QuixiAI/Wizard-Vicuna-13B-Uncensored") model = AutoModelForCausalLM.from_pretrained("QuixiAI/Wizard-Vicuna-13B-Uncensored") - Inference
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use QuixiAI/Wizard-Vicuna-13B-Uncensored with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "QuixiAI/Wizard-Vicuna-13B-Uncensored" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "QuixiAI/Wizard-Vicuna-13B-Uncensored", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker
docker model run hf.co/QuixiAI/Wizard-Vicuna-13B-Uncensored
- SGLang
How to use QuixiAI/Wizard-Vicuna-13B-Uncensored with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "QuixiAI/Wizard-Vicuna-13B-Uncensored" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "QuixiAI/Wizard-Vicuna-13B-Uncensored", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "QuixiAI/Wizard-Vicuna-13B-Uncensored" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "QuixiAI/Wizard-Vicuna-13B-Uncensored", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }' - Docker Model Runner
How to use QuixiAI/Wizard-Vicuna-13B-Uncensored with Docker Model Runner:
docker model run hf.co/QuixiAI/Wizard-Vicuna-13B-Uncensored
Not entirely without bias
I really appreciate these models the Bloke puts out and have been testing their boundaries. What I've found is that indeed, IF pre-prompted correctly, then you can get it to do some downright offensible things.. but NOT anything. It remains strong bias against anything racial, it bends a story towards an arc of justice or making the tory come out ok in the end. If pushed to extreme violence or personal animosity, it will bias itself into a loop where it simply repeats itself.
All that to say, of course, we don't need those things or want to engage with them (obviously lol) but it does remain a hindrance to a truly unhindered bot
This is and old model in LLM lifespans and should be considered relatively obslete but not without value. I would suggest trying out some of our newer Dolphin models.
Interesting, I thought this was still fairly new. My experience with mistral, dolphin, and vicuna has been that Vicuna handles chat the best. Now I wonder though If im just using old models. Thanks for the tip
Apolgies we updated the model cards with non expiring links to our Discord and it pushes the models to recently updated. If you liked this model I would give the Dolphin 2.2 Yi 200k a try.