Upload folder using huggingface_hub

.gitattributes

@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+tokenizer.json filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,141 @@
+---
+license: apache-2.0
+base_model: Qwen/Qwen2.5-Coder-14B-Instruct
+library_name: peft
+pipeline_tag: text-generation
+tags:
+  - qlora
+  - data-science
+  - code-generation
+  - peft
+  - qwen2
+  - lora
+  - sft
+  - unsloth
+language:
+  - en
+---
+
+# DataSci-Coder-14B: Qwen2.5-Coder-14B LoRA Adapter for Data Science
+
+A QLoRA fine-tuned adapter for [Qwen2.5-Coder-14B-Instruct](https://huggingface.co/Qwen/Qwen2.5-Coder-14B-Instruct) optimized for data science code generation. The model outputs clean, runnable Python code with zero explanatory text — strictly following code-only instructions.
+
+## Key Results
+
+| Metric | DS-Tuned (FT) | Base Model | Delta |
+|--------|:---:|:---:|:---:|
+| Hard Eval (12 complex tasks) | 12/12 | 12/12 | Tie |
+| Constraint Compliance | 93.3% | 91.4% | **+1.9%** |
+| Code-Only Compliance | 10/10 | 6/10 | **+67%** |
+| Code Ratio | 100% | 87.9% | **+12.1%** |
+
+## What It Does
+
+- Generates complete, runnable Python code for data science tasks
+- Covers statistics, machine learning, deep learning, NLP, time series, and visualization
+- Follows instructions precisely — when told "no explanations," it outputs only code (base model ignores this 40% of the time)
+- Handles complex tasks: Bayesian inference, VAEs, GANs, survival analysis, stacking ensembles, SHAP, anomaly detection
+
+## Training Details
+
+| Parameter | Value |
+|-----------|-------|
+| Base Model | `unsloth/Qwen2.5-Coder-14B-Instruct-bnb-4bit` |
+| Method | QLoRA (4-bit quantization) |
+| LoRA Rank | 16 |
+| LoRA Alpha | 32 |
+| LoRA Targets | q/k/v/o_proj, gate/up/down_proj |
+| Trainable Parameters | 68.8M / 14.8B (0.46%) |
+| Training Examples | 10,795 |
+| Epochs | 1 |
+| Final Loss | 0.5933 |
+| Training Time | 1.9 hours on NVIDIA L40S |
+| Precision | bfloat16 |
+| Optimizer | Paged AdamW 8-bit |
+| Learning Rate | 3e-5 (cosine schedule) |
+| Effective Batch Size | 16 (1 x 16 grad accum) |
+
+## Training Data
+
+10,795 curated data science instruction-response pairs from:
+- 6 public HuggingFace datasets (CodeAlpaca, Evol-Instruct, etc.)
+- University coursework (statistics, ML, deep learning)
+- Data science newsletters
+- Hand-curated examples
+
+All examples filtered for Python code quality, data science relevance, and length. Categories: machine learning, deep learning, statistics, data wrangling, visualization, NLP, time series, numerical computing.
+
+## Usage
+
+```python
+from unsloth import FastLanguageModel
+import torch
+
+model, tokenizer = FastLanguageModel.from_pretrained(
+    model_name="jsmall12/DataSci-Coder-14B-LoRA",
+    max_seq_length=2048,
+    load_in_4bit=True,
+    dtype=None,
+)
+FastLanguageModel.for_inference(model)
+
+messages = [
+    {"role": "system", "content": "You are an expert data science coding assistant. Respond ONLY with clean, runnable Python code. Use inline comments for explanation. No text outside code blocks."},
+    {"role": "user", "content": "Write a function to train a logistic regression model with sklearn and print the classification report."},
+]
+
+text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
+inputs = tokenizer(text, return_tensors="pt").to("cuda")
+
+with torch.no_grad():
+    output = model.generate(
+        **inputs,
+        max_new_tokens=1024,
+        temperature=0.1,
+        do_sample=True,
+        top_p=0.9,
+        repetition_penalty=1.15,
+        use_cache=False,
+    )
+
+response = tokenizer.decode(output[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True)
+print(response)
+```
+
+## Evaluation
+
+### Hard Eval (12 Complex Tasks)
+
+All 12 tasks produced correct, complete, runnable implementations:
+
+| Category | Tasks | Score |
+|----------|-------|:---:|
+| Statistics | Bayesian A/B testing, Kaplan-Meier survival analysis, time series CV + ARIMA, VIF + Ridge/Lasso/ElasticNet | 4/4 |
+| Machine Learning | Stacking ensemble, SHAP importance, Isolation Forest, TF-IDF + SVM pipeline | 4/4 |
+| Deep Learning | LR scheduler (warmup + cosine), BiLSTM + attention, VAE, GAN | 4/4 |
+
+### Constraint Eval (10 Multi-Constraint Tests)
+
+| Test | FT | Base | Delta |
+|------|:---:|:---:|:---:|
+| C01 Multi-step data cleaning | 8/8 | 8/8 | 0 |
+| C02 Complete ML pipeline | 12/12 | 12/12 | 0 |
+| C03 Statistical hypothesis test | 9/9 | 7/9 | **+2** |
+| C04 PyTorch architecture | 9/9 | 7/9 | **+2** |
+| C05 EDA visualizations | 11/12 | 10/12 | **+1** |
+| C06 Cross-validated pipeline | 12/12 | 12/12 | 0 |
+| C07 Time series ARIMA | 9/10 | 10/10 | -1 |
+| C08 DL training function | 8/10 | 8/10 | 0 |
+| C09 Pandas method chain | 10/10 | 10/10 | 0 |
+| C10 Model evaluation | 10/13 | 12/13 | -2 |
+| **Total** | **98/105** | **96/105** | **+2** |
+
+## Hardware Requirements
+
+- **Minimum:** ~10GB VRAM (4-bit quantized)
+- **Recommended:** 24GB+ VRAM (L4, A100, etc.)
+- Tested on: NVIDIA L40S (44GB), NVIDIA T4 x2 (15GB each)
+
+## License
+
+Apache 2.0

adapter_config.json

@@ -0,0 +1,50 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": {
+    "base_model_class": "Qwen2ForCausalLM",
+    "parent_library": "transformers.models.qwen2.modeling_qwen2",
+    "unsloth_fixed": true
+  },
+  "base_model_name_or_path": "unsloth/Qwen2.5-Coder-14B-Instruct-bnb-4bit",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "peft_version": "0.18.1",
+  "qalora_group_size": 16,
+  "r": 16,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "v_proj",
+    "up_proj",
+    "gate_proj",
+    "down_proj",
+    "o_proj",
+    "q_proj",
+    "k_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}

adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:29025996a3f095ddc80f270f6ac16990cea5e5e372dce46989e6ff09352fbea1
+size 275341720

chat_template.jinja

@@ -0,0 +1,54 @@
+{%- if tools %}
+    {{- '<|im_start|>system\n' }}
+    {%- if messages[0]['role'] == 'system' %}
+        {{- messages[0]['content'] }}
+    {%- else %}
+        {{- 'You are Qwen, created by Alibaba Cloud. You are a helpful assistant.' }}
+    {%- endif %}
+    {{- "\n\n# Tools\n\nYou may call one or more functions to assist with the user query.\n\nYou are provided with function signatures within <tools></tools> XML tags:\n<tools>" }}
+    {%- for tool in tools %}
+        {{- "\n" }}
+        {{- tool | tojson }}
+    {%- endfor %}
+    {{- "\n</tools>\n\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\n<tool_call>\n{\"name\": <function-name>, \"arguments\": <args-json-object>}\n</tool_call><|im_end|>\n" }}
+{%- else %}
+    {%- if messages[0]['role'] == 'system' %}
+        {{- '<|im_start|>system\n' + messages[0]['content'] + '<|im_end|>\n' }}
+    {%- else %}
+        {{- '<|im_start|>system\nYou are Qwen, created by Alibaba Cloud. You are a helpful assistant.<|im_end|>\n' }}
+    {%- endif %}
+{%- endif %}
+{%- for message in messages %}
+    {%- if (message.role == "user") or (message.role == "system" and not loop.first) or (message.role == "assistant" and not message.tool_calls) %}
+        {{- '<|im_start|>' + message.role + '\n' + message.content + '<|im_end|>' + '\n' }}
+    {%- elif message.role == "assistant" %}
+        {{- '<|im_start|>' + message.role }}
+        {%- if message.content %}
+            {{- '\n' + message.content }}
+        {%- endif %}
+        {%- for tool_call in message.tool_calls %}
+            {%- if tool_call.function is defined %}
+                {%- set tool_call = tool_call.function %}
+            {%- endif %}
+            {{- '\n<tool_call>\n{"name": "' }}
+            {{- tool_call.name }}
+            {{- '", "arguments": ' }}
+            {{- tool_call.arguments | tojson }}
+            {{- '}\n</tool_call>' }}
+        {%- endfor %}
+        {{- '<|im_end|>\n' }}
+    {%- elif message.role == "tool" %}
+        {%- if (loop.index0 == 0) or (messages[loop.index0 - 1].role != "tool") %}
+            {{- '<|im_start|>user' }}
+        {%- endif %}
+        {{- '\n<tool_response>\n' }}
+        {{- message.content }}
+        {{- '\n</tool_response>' }}
+        {%- if loop.last or (messages[loop.index0 + 1].role != "tool") %}
+            {{- '<|im_end|>\n' }}
+        {%- endif %}
+    {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+    {{- '<|im_start|>assistant\n' }}
+{%- endif %}

tokenizer.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:94df820dd44997bc42540f6e0b8629201e4b45285972712cda88b704fbf5c640
+size 11422455

tokenizer_config.json

@@ -0,0 +1,16 @@
+{
+  "add_prefix_space": false,
+  "backend": "tokenizers",
+  "bos_token": null,
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|im_end|>",
+  "errors": "replace",
+  "extra_special_tokens": [],
+  "is_local": false,
+  "model_max_length": 32768,
+  "pad_token": "<|PAD_TOKEN|>",
+  "padding_side": "left",
+  "split_special_tokens": false,
+  "tokenizer_class": "Qwen2Tokenizer",
+  "unk_token": null
+}

training_config.json

@@ -0,0 +1,15 @@
+{
+  "base_model": "unsloth/Qwen2.5-Coder-14B-Instruct-bnb-4bit",
+  "lora_r": 16,
+  "lora_alpha": 32,
+  "seq_length": 2048,
+  "epochs": 1,
+  "batch_size": 1,
+  "grad_accum": 16,
+  "effective_batch_size": 16,
+  "lr": 3e-05,
+  "train_loss": 0.5933350541856554,
+  "train_examples": 10795,
+  "val_examples": 1195,
+  "train_time_hours": 1.87
+}