Create README.md
Browse files
README.md
ADDED
|
@@ -0,0 +1,51 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: apache-2.0
|
| 3 |
+
---
|
| 4 |
+
## MAPO: Advancing Multilingual Reasoning through Multilingual Alignment-as-Preference Optimization
|
| 5 |
+
see our paper in https://arxiv.org/abs/2401.06838
|
| 6 |
+
|
| 7 |
+
View the Github Project:
|
| 8 |
+
https://github.com/NJUNLP/MAPO
|
| 9 |
+
|
| 10 |
+
|
| 11 |
+
## Benchmarks
|
| 12 |
+
| System | [MSVAMP](https://huggingface.co/datasets/Mathoctopus/MSVAMP) | [MGSM](https://huggingface.co/datasets/juletxara/mgsm) | [MNumGLUESub](https://huggingface.co/datasets/Mathoctopus/MSVAMP) |
|
| 13 |
+
| --- | :---: | :---: | :---: |
|
| 14 |
+
| GPT-3.5-Turbo | 46.6 | 42.2 | 49.4 |
|
| 15 |
+
| [MAmmoTH 7B](https://huggingface.co/TIGER-Lab/MAmmoTH-7B) | 26.3 | 21.3 | 24.2 |
|
| 16 |
+
| [WizardMath 7B](https://huggingface.co/WizardLM/WizardMath-7B-V1.1) | 32.5 | 23.0 | 28.7 |
|
| 17 |
+
| [MetaMath 7B](https://huggingface.co/meta-math/MetaMath-7B-V1.0) | 46.2 | 37.0 | 43.2 |
|
| 18 |
+
| [QAlign 7B](https://huggingface.co/Wenhao97/QAlign-MetaMathQA-7B) | 57.2 | 49.6 | - |
|
| 19 |
+
| [MathOctopus 7B](https://huggingface.co/Mathoctopus/Parallel_7B) | 41.2 | 39.5 | 37.1 |
|
| 20 |
+
| **[+ MAPO-DPO(ours)🔥](https://huggingface.co/kevinpro/MathOctopus-MAPO-DPO-7B)** | **57.4** | **41.6** | **50.4** |
|
| 21 |
+
| [MetaMathOctopus 7B](https://huggingface.co/kevinpro/MetaMathOctopus-7B) | 53.0 | 45.5 | 39.2 |
|
| 22 |
+
| **[+ MAPO-DPO(ours) 👑](https://huggingface.co/kevinpro/MetaMathOctopus-MAPO-DPO-7B)** | **64.7** | **51.6** | **52.9** |
|
| 23 |
+
| MistralMathOctopus 7B | 59.0 | 58.0 | 56.8 |
|
| 24 |
+
| **+ MAPO-DPO(ours) 👑**(coming soon) | **74.6** | **67.3** | **70.0** |
|
| 25 |
+
|
| 26 |
+
| System | [MSVAMP](https://huggingface.co/datasets/Mathoctopus/MSVAMP) | [MGSM](https://huggingface.co/datasets/juletxara/mgsm) | [MNumGLUESub](https://huggingface.co/datasets/Mathoctopus/MSVAMP) |
|
| 27 |
+
| ------ | :-------------------------------------------------------------: | :------------------------------------------------------: | :-------------------------------------------------------------: |
|
| 28 |
+
| GPT-3.5-Turbo | 46.6 | 42.2 | 49.4 |
|
| 29 |
+
| [MAmmoTH 13B](https://huggingface.co/TIGER-Lab/MAmmoTH-13B) | 38.6 | 28.9 | 29.5 |
|
| 30 |
+
| [WizardMath 13B](https://huggingface.co/WizardLM/WizardMath-13B-V1.1) | 35.7 | 28.3 | 29.0 |
|
| 31 |
+
| [MetaMath 13B](https://huggingface.co/meta-math/MetaMath-13B-V1.0) | 46.2 | 43.9 | 43.3 |
|
| 32 |
+
| [QAlign 13B](https://huggingface.co/Wenhao97/QAlign-MetaMathQA-13B) | 62.6 | 57.1 | - |
|
| 33 |
+
| [MathOctopus 13B](https://huggingface.co/Mathoctopus/Parallel_13B) | 51.8 | 46.0 | 40.3 |
|
| 34 |
+
| **[+ MAPO-DPO(ours)🔥](https://huggingface.co/kevinpro/MathOctopus-MAPO-DPO-13B)** | **60.1** | **48.5** | **53.8** |
|
| 35 |
+
| [MetaMathOctopus 13B](https://huggingface.co/kevinpro/MetaMathOctopus-13B) | 56.3 | 51.4 | 49.5 |
|
| 36 |
+
| **[+ MAPO-DPO(ours) 👑](https://huggingface.co/kevinpro/MetaMathOctopus-MAPO-DPO-13B)** | **67.0** | **58.0** | **59.8** |
|
| 37 |
+
|
| 38 |
+
|
| 39 |
+
|
| 40 |
+
## Citation
|
| 41 |
+
If you find this model helpful, feel free to cite our paper:
|
| 42 |
+
```
|
| 43 |
+
@misc{she2024mapo,
|
| 44 |
+
title={MAPO: Advancing Multilingual Reasoning through Multilingual Alignment-as-Preference Optimization},
|
| 45 |
+
author={Shuaijie She and Wei Zou and Shujian Huang and Wenhao Zhu and Xiang Liu and Xiang Geng and Jiajun Chen},
|
| 46 |
+
year={2024},
|
| 47 |
+
eprint={2401.06838},
|
| 48 |
+
archivePrefix={arXiv},
|
| 49 |
+
primaryClass={cs.CL}
|
| 50 |
+
}
|
| 51 |
+
```
|