| --- |
| license: apache-2.0 |
| language: |
| - en |
| base_model: |
| - google-t5/t5-base |
| pipeline_tag: summarization |
| --- |
| |
| **Model Name:** LoRA Fine-Tuned Model for Dialogue Summarization |
| **Model Type:** Seq2Seq with Low-Rank Adaptation (LoRA) |
| **Base Model:** `google/t5-base` |
|
|
| ## Model Details |
| - **Architecture**: T5-base |
| - **Finetuning Technique**: LoRA (Low-Rank Adaptation) |
| - **PEFT Method**: Parameter Efficient Fine-Tuning |
| - **Data**: samsumdataset |
| - **Metrics**: Evaluated using ROUGE (ROUGE-1, ROUGE-2, ROUGE-L, ROUGE-Lsum) |
|
|
| ## Intended Use |
| This model is designed for summarizing dialogues, such as conversations between individuals in a chat or messaging context. It’s suitable for applications in: |
| - **Customer Service**: Summarizing chat logs for quality monitoring or training. |
| - **Messaging Apps**: Generating conversation summaries for user convenience. |
| - **Content Creation**: Assisting writers by summarizing character dialogues. |
|
|
| ## Training Process |
|
|
| Optimizer: AdamW with learning rate 3e-5 |
|
|
| Batch Size: 4 (gradient accumulation steps of 2) |
|
|
| Training Epochs: 2 |
|
|
| Evaluation Metrics: ROUGE-1, ROUGE-2, ROUGE-L, ROUGE-Lsum |
|
|
| Hardware: Trained on a single GPU with mixed precision to optimize performance. |
|
|
| The model was trained using the Seq2SeqTrainer class from transformers, with LoRA parameters applied to selected attention layers to reduce computation without compromising accuracy. |
