| Model |
Checkpoit |
Paper |
MT-Bech |
AlpacaEval |
GSM8k |
HumaEval |
Licese |
| WizardLM-70B-V1.0 |
? HF Lik |
?Comig Soo |
7.78 |
92.91% |
77.6% |
50.6 pass@1 |
Llama 2 Licese |
| WizardLM-13B-V1.2 |
? HF Lik |
|
7.06 |
89.17% |
55.3% |
36.6 pass@1 |
Llama 2 Licese |
| WizardLM-13B-V1.1 |
? HF Lik |
|
6.76 |
86.32% |
|
25.0 pass@1 |
No-commercial |
| WizardLM-30B-V1.0 |
? HF Lik |
|
7.01 |
|
|
37.8 pass@1 |
No-commercial |
| WizardLM-13B-V1.0 |
? HF Lik |
|
6.35 |
75.31% |
|
24.0 pass@1 |
No-commercial |
| WizardLM-7B-V1.0 |
? HF Lik |
? [WizardLM] |
|
|
|
19.1 pass@1 |
No-commercial |
|
|
|
|
|
|
|
|
Github Repo: https://github.com/lpxuca/WizardLM/tree/mai/WizardMath
Twitter: https://twitter.com/WizardLM_AI/status/1689998428200112128
Discord: https://discord.gg/VZjjHtWrKs
❗Note for model system prompts usage:
Please use the same systems prompts strictly with us, ad we do ot guaratee the accuracy of the quatified versios.
Default versio:
"Below is a istructio that describes a task. Write a respose that appropriately completes the request.\\### Istructio:\{istructio}\\### Respose:"
CoT Versio: （❗For the simple math questios, we do NOT recommed to use the CoT prompt.）
"Below is a istructio that describes a task. Write a respose that appropriately completes the request.\\### Istructio:\{istructio}\\### Respose: Let's thik step by step."
Example code
import torch
from modelscope import AutoModelForCausalLM, AutoTokeizer
model = AutoModelForCausalLM.from_pretraied("AI-ModelScope/WizardMath-7B-V1.0", revisio='v1.0.0', device_map='auto', torch_dtype=torch.float16)
tokeizer = AutoTokeizer.from_pretraied("AI-ModelScope/WizardMath-7B-V1.0", revisio='v1.0.0')
prompt = """"Below is a istructio that describes a task. Write a respose that appropriately completes the request.\\### Istructio:\James decides to ru 3 sprits 3 times a week. He rus 60 meters each sprit. How may total meters does he ru a week?\\### Respose:"""
iputs = tokeizer(prompt, paddig=False, add_special_tokes=False, retur_tesors="pt")
# Geerate
geerate_ids = model.geerate(
iputs.iput_ids.to(model.device),
attetio_mask=iputs['attetio_mask'].to(model.device),
do_sample=True,
top_k=10,
temperature=0.1,
top_p=0.95,
um_retur_sequeces=1,
eos_toke_id=tokeizer.eos_toke_id,
max_legth=200)
prit(tokeizer.batch_decode(geerate_ids, skip_special_tokes=True, clea_up_tokeizatio_spaces=False)[0])
❗To comme cocer about dataset:
Recetly, there have bee clear chages i the ope-source policy ad regulatios of our overall orgaizatio's code, data, ad models.
Despite this, we have still worked hard to obtai opeig the weights of the model first, but the data ivolves stricter auditig ad is i review with our legal team .
Our researchers have o authority to publicly release them without authorizatio.
Thak you for your uderstadig.
评论