my_indo_2_model

my_indo_2_model is a fine-tuned version of FacebookAI/xlm-roberta-base on a custom Indonesian question answering dataset using the SQuAD-style extractive QA format.

The model was trained to extract span-based answers from a given context in response to a question in Bahasa Indonesia. It is suitable for building QA systems or chatbot backends for Indonesian language applications.

---

🧠 Model Description

• Base model: FacebookAI/xlm-roberta-base
• Task: Extractive Question Answering
• Language: Multilingual (fokus pada Bahasa Indonesia)
• Tokenizer: xlm-roberta-base tokenizer
• Training format: SQuAD-style JSON format (question, context, answer)

---

📈 Evaluation Results

The model was evaluated on a held-out validation set with the following results:

Metric	Value
Loss	1.3813
Exact Match	43.73%
F1 Score	64.92%
Runtime	13.14 s
Eval speed	48.55 samples/sec

These results indicate that the model performs reasonably well on extractive question answering tasks in Bahasa Indonesia.

---

✅ Intended Uses & Limitations

Intended Uses:

• Membangun chatbot atau QA system berbasis teks Bahasa Indonesia
• Menjawab pertanyaan berbasis konteks dokumen
• Digunakan dalam riset atau pembelajaran NLP untuk low-resource languages

Limitations:

• Hanya dapat menjawab pertanyaan dengan tipe span-based (jawaban eksplisit di dalam konteks)
• Tidak didesain untuk pertanyaan tipe "yes/no", "unanswerable", atau open-ended
• Belum diuji pada domain di luar dataset pelatihan (e.g. kedokteran, hukum, dsb.)

---

📚 Training & Evaluation Data

• Dataset: Custom Indonesian dataset (subset SPAN only)
• Jumlah data: Tidak disebutkan (silakan update jika tersedia)
• Format: JSON dengan kolom question, context, dan answer
• Distribusi: 80% training – 20% validation

---

⚙️ Training Procedure

Hyperparameters:

• Learning rate: 2e-5
• Train batch size: 16
• Eval batch size: 16
• Epochs: 3
• Optimizer: AdamW (betas=(0.9, 0.999), epsilon=1e-8)
• LR scheduler: linear

Training Logs:

Epoch	Step	Validation Loss
1	160	2.0673
2	320	1.4765
3	480	1.3813

---

🛠 Framework Versions

• Transformers: 4.54.1
• PyTorch: 2.6.0+cu124
• Datasets: 4.0.0
• Tokenizers: 0.21.4

---

📤 How to Use

from transformers import AutoTokenizer, AutoModelForQuestionAnswering

tokenizer = AutoTokenizer.from_pretrained("your-username/my_indo_2_model")
model = AutoModelForQuestionAnswering.from_pretrained("your-username/my_indo_2_model")

question = "Siapa presiden ketiga Indonesia?"
context = "Bacharuddin Jusuf Habibie adalah Presiden ketiga Republik Indonesia yang menjabat dari 1998 hingga 1999."

inputs = tokenizer(question, context, return_tensors="pt", max_length=384, truncation=True)
outputs = model(**inputs)

start = outputs.start_logits.argmax()
end = outputs.end_logits.argmax() + 1

answer = tokenizer.decode(inputs["input_ids"][0][start:end])
print(answer)