kalomaze
/

MiniSymposium-Demo

Model card Files Files and versions

kalomaze commited on Nov 25, 2023

Commit

2c94f2a

·

1 Parent(s): 5395163

Update README.md

Files changed (1) hide show

README.md +1 -1

README.md CHANGED Viewed

@@ -6,7 +6,7 @@ license: apache-2.0
 MiniSymposium is an experimental QLora model that I created based on Mistral 7b. I created it attempting to achieve these goals:
 1. Demonstrate the untapped potential of using a small, focused dataset of handwritten examples instead of training on a large amount of synthetic GPT outputs
-2. Create a dataset that allows the model to explore different possible answers from multiple perspectives before reaching a conclusion.
 3. Develop a model that performs well across various prompt formats, rather than overfitting to a specific kind of format
 The current trend in QLora/Lora-based finetuning (and finetuning in general for local LLMs) is to use large synthetic datasets. These are usually GPT datasets that are trained with higher learning rates.

 MiniSymposium is an experimental QLora model that I created based on Mistral 7b. I created it attempting to achieve these goals:
 1. Demonstrate the untapped potential of using a small, focused dataset of handwritten examples instead of training on a large amount of synthetic GPT outputs
+2. Create a dataset that allows the model to explore different possible answers from multiple perspectives before reaching a conclusion
 3. Develop a model that performs well across various prompt formats, rather than overfitting to a specific kind of format
 The current trend in QLora/Lora-based finetuning (and finetuning in general for local LLMs) is to use large synthetic datasets. These are usually GPT datasets that are trained with higher learning rates.