Hugging Face's logo

Spaces:
tnt306
/
finetuned-llm-demo-app
Sleeping

App Files Community
1
finetuned-llm-demo-app / run_local_llm_server.py

Commit History

Use HuggingFace persisent storage
c7f1f5a

tnt306 commited on

Change the way to get cpu count
8d33281

tnt306 commited on

Fix error when starting streamlit; start original first
2cc9f75

tnt306 commited on

Add threads dynamically
3359734

tnt306 commited on

Support getting LOCAL LLM API KEY from environment
5f62773

tnt306 commited on

Support Tools
b9aed24

tnt306 commited on

remove verbose log
1cf4cc4

tnt306 commited on

Support RAG
93d7da6

tnt306 commited on

Use mlock to force system to keep model in RAM
2ee09a5

tnt306 commited on

Start Finetuned model first
38cf468

tnt306 commited on

Stable version
ef9997f

tnt306 commited on

Add print and print flush
519f144

tnt306 commited on

Temporary upload
d304f41

tnt306 commited on

Initial Version (Qwen2.5-7B-Instruct-1M-q4_k_m Original & Finetuned; Chat Web UI)
b77991a

tnt306 commited on