Model Card for AbishekUzumaki/tinyllama-biomedical 🧠 Model Details
Model Description: This model is a fine-tuned version of TinyLlama (1.1B parameters) adapted for biomedical domain tasks, such as clinical question answering, biomedical text generation, and terminology understanding. It was trained on a curated biomedical dataset to enhance domain-specific understanding and terminology usage.
Developed by: Abishek Uzumaki Model type: Causal Language Model (Decoder-only Transformer) Language(s): English License: Apache 2.0 (same as base TinyLlama) Finetuned from: TinyLlama/TinyLlama-1.1B
Frameworks used: PEFT 0.13.0, Transformers, Accelerate, Datasets, PyTorch Cloud environment: Google Colab (T4 GPU)
📚 Model Sources
Repository: Hugging Face Model Page
Demo (optional): Coming soon
💡 Uses ✅ Direct Use
Biomedical text summarization
Clinical and biomedical Q&A
Research-related text generation
Domain-specific chatbot applications
⚙️ Downstream Use
Embedding or retrieval-augmented generation in biomedical applications
Integration with healthcare or research assistants
🚫 Out-of-Scope Use
Real-world medical diagnosis or treatment recommendation
Any use requiring verified clinical validation
⚠️ Bias, Risks, and Limitations
May reflect biases in biomedical literature or datasets
Not intended for direct patient-facing medical use
Model may generate inaccurate or outdated medical information
Recommendation: Always validate model outputs with trusted medical professionals or verified literature before practical application.
🧩 Training Details
Training Data: Biomedical literature, PubMed abstracts, and domain-specific text corpora (processed and cleaned for fine-tuning).
Preprocessing:
Tokenized with TinyLlama tokenizer
Cleaned for duplicates, non-English, and unrelated text
Training Procedure:
Fine-tuned using PEFT LoRA method
Optimizer: AdamW
Epochs: 3
Batch Size: 4
Learning Rate: 2e-4
Gradient Accumulation: 8
GPU Used: NVIDIA T4
Framework: Hugging Face Transformers + PEFT
📊 Evaluation
Testing Data: Biomedical Q&A and summarization benchmarks Metrics: Perplexity, domain coherence, and factual accuracy (manual inspection) Results:
Improved biomedical terminology accuracy
Reduced hallucinations in domain tasks compared to base TinyLlama
⚙️ Technical Specifications
Architecture: Transformer (Decoder-only) Parameters: ~1.1B Hardware: NVIDIA T4 GPU (Colab) Software:
Transformers v4.x
PEFT v0.13.0
Datasets v2.x
PyTorch 2.x
🌱 Environmental Impact Factor Value Hardware Google Colab (T4 GPU) Hours Used ~2–3 hours Cloud Provider Google Cloud
📬 Contact
Author: Abishek Uzumaki Email: [abishekbalamurugan858@gmail.com] Hugging Face: https://huggingface.co/Master-Abi
Framework versions
- PEFT 0.13.0
- Downloads last month
- -