joschu0/tinker-llama-lora-test

This model is a LoRA adapter fine-tuned with Tinker from Thinking Machines Lab.

Model Details

Attribute	Value
Base Model	meta-llama/Llama-3.1-8B
Training Type	Supervised Learning (SFT)
LoRA Rank	8
LoRA Alpha	32
Target Modules	all-linear

Usage

With PEFT

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

# Load base model
base_model = AutoModelForCausalLM.from_pretrained("meta-llama/Llama-3.1-8B")
tokenizer = AutoTokenizer.from_pretrained("meta-llama/Llama-3.1-8B")

# Load LoRA adapter
model = PeftModel.from_pretrained(base_model, "joschu0/tinker-llama-lora-test")

With Tinker (for sampling/inference)

This checkpoint is also available on Tinker for high-throughput sampling:

import tinker

sc = tinker.ServiceClient()
sampling_client = sc.create_sampling_client("tinker://7f4705e7-551f-5133-b4bb-33444c0c405b:train:0/sampler_weights/test-push-to-hub")
result = sampling_client.sample(...)

Tinker path: tinker://7f4705e7-551f-5133-b4bb-33444c0c405b:train:0/sampler_weights/test-push-to-hub

Note: This is a sampler checkpoint and can only be used for inference, not for continued training.

Training

This model was trained using the Tinker API. For more information about training with Tinker, see the Tinker documentation.

---

Trained with Tinker by Thinking Machines Lab