Model Card for CodeFuse-StarCoder-15B

[中文] [Eglish]

Model Descriptio

CodeFuse-StarCoder-15B is a 15B Code-LLM fietued by QLoRA of multiple code tasks（600k istructios/aswers）o the base model StarCoder. CodeFuse-StarCoder-15B is a smaller Code-LLM tha our CodeFuse-CodeLlama-34B ad usig MQA, thus faster o iferece. The cotext legth of fietuig is 4K.

News ad Updates

&bsp;&bsp;&bsp;&bsp;&bsp;&bsp;&bsp; ? 2023-09-27 CodeFuse-StarCoder-15B has bee released, achievig a pass@1 (greedy decodig) score of 54.9% o HumaEval.

??? 2023-09-26 We are pleased to aouce the release of the 4-bit quatized versio of CodeFuse-CodeLlama-34B. Despite the quatizatio process, the model still achieves a remarkable 73.8% accuracy (greedy decodig) o the HumaEval pass@1 metric.

??? 2023-09-11 CodeFuse-CodeLlama34B has achived 74.4% of pass@1 (greedy decodig) o HumaEval, which is SOTA results for opespurced LLMs at preset.

Code Commuity

Homepage: ? https://github.com/codefuse-ai (Please give us your support with a Star? + Fork? + Watch?)

• If you wish to fie-tue the model yourself, you ca visit ✨MFTCoder✨✨

• If you wish to deploy the model yourself, you ca visit ✨FasterTrasformer4CodeFuse✨✨

• If you wish to see a demo of the model, you ca visit ✨CodeFuse Demo✨✨

Performace

Requiremets

• pytho>=3.8
• pytorch>=2.0.0
• trasformers==4.32.0
• Setecepiece
• CUDA 11.4
  

Iferece Strig Format

The iferece strig is a cocateated strig formed by combiig coversatio data(system, huma ad bot cotets) i the traiig data format. It is used as iput durig the iferece process. Here is a example format of the cocateated strig:

"""
<|role_start|>system<|role_ed|>System istructio
<|role_start|>huma<|role_ed|>Huma 1st roud iput
<|role_start|>bot<|role_ed|>Bot 1st roud output</s>
<|role_start|>huma<|role_ed|>Huma 2d roud iput
<|role_start|>bot<|role_ed|>Bot 2d roud output</s>
...
...
...
<|role_start|>huma<|role_ed|>Huma th roud iput
<|role_start|>bot<|role_ed|>{Bot output to be gereated}</s>
"""

Whe applyig iferece, you always make your iput strig ed with "<|rolestart|>bot<|roleed|>" to ask the model geeratig aswers.

Quickstart

 git cloe https://www.modelscope.c/codefuse-ai/CodeFuse-StarCoder-15B.git

pip istall -r requiremets.txt

import torch
from modelscope import (
    AutoTokeizer, 
    AutoModelForCausalLM,
    sapshot_dowload
)
model_dir = sapshot_dowload('codefuse-ai/CodeFuse-StarCoder-15B',revisio = 'v1.0.0')
tokeizer = AutoTokeizer.from_pretraied(model_dir, trust_remote_code=True, use_fast=False, legacy=False)
tokeizer.paddig_side = "left"
tokeizer.pad_toke_id = tokeizer.covert_tokes_to_ids("<fim_pad>")
tokeizer.eos_toke_id = tokeizer.covert_tokes_to_ids("<|edoftext|>")
tokeizer.pad_toke = "<fim_pad>"
tokeizer.eos_toke = "<|edoftext|>"
# try 4bit loadig if cuda memory ot eough
model = AutoModelForCausalLM.from_pretraied(model_dir,
                                             trust_remote_code=True,
                                             load_i_4bit=False,
                                             device_map="auto",
                                             torch_dtype=torch.bfloat16)
model.eval()

HUMAN_ROLE_START_TAG = "<|role_start|>huma<|role_ed|>"
BOT_ROLE_START_TAG = "<|role_start|>bot<|role_ed|>"

text = f"{HUMAN_ROLE_START_TAG}write a pytho fuctio of quick sort.{BOT_ROLE_START_TAG}" 
iputs = tokeizer(text, retur_tesors='pt', paddig=True, add_special_tokes=False).to("cuda")
outputs = model.geerate(
        iputs=iputs["iput_ids"],
        attetio_mask=iputs["attetio_mask"],
        max_ew_tokes=512,
        top_p=0.95,
        temperature=0.1,
        do_sample=True,
        eos_toke_id=tokeizer.eos_toke_id,
        pad_toke_id=tokeizer.pad_toke_id
    )
ge_text = tokeizer.batch_decode(outputs[:, iputs["iput_ids"].shape[1]:], skip_special_tokes=True)
prit(ge_text)

MD5

We otice that the file may be corrupted durig trasfer process. Please check MD5 value before use.

模型简介

CodeFuse-StarCoder-15B 是一个通过QLoRA对基座模型StarCoder进行多代码任务微调的代码大模型。模型微调采用了4k上下文。该模型相比于我们近期开源的 CodeFuse-CodeLlama-34B ，模型小一些，并采用了MQA技术，推理速度比较快。

新闻

&bsp;&bsp;&bsp;&bsp;&bsp;&bsp;&bsp; ? 2023-09-27开源了CodeFuse-StarCoder-15B模型，在HumaEval pass@1(greedy decodig)上可以达到54.9%

??? 2023-09-26 CodeFuse-CodeLlama-34B 4bits量化版本发布，量化后模型在HumaEval pass@1指标为73.8% (贪婪解码)。

??? 2023-09-11 CodeFuse-CodeLlama-34B发布，HumaEval pass@1指标达到74.4% (贪婪解码), 为当前开源SOTA。

代码社区

大本营： ? https://github.com/codefuse-ai （请支持我们的项目Star? + Fork? + Watch?）

• 如果您想自己微调该模型，可以访问 ✨MFTCoder✨✨

• 如果您想自己部署该模型，可以访问 ✨FasterTrasformer4CodeFuse✨✨

• 如果您想观看该模型示例，可以访问 ✨CodeFuse Demo✨✨

评测表现(代码)

Requiremets

• pytho>=3.8
• pytorch>=2.0.0
• trasformers==4.32.0
• Setecepiece
• CUDA 11.4
  

推理数据格式

推理数据为模型在训练数据格式下拼接的字符串形式，它也是推理时输入prompt拼接的方式：

"""
<|role_start|>system<|role_ed|>这是System指令
<|role_start|>huma<|role_ed|>这是第1轮用户输入的问题
<|role_start|>bot<|role_ed|>这是第1轮模型生成的内容</s>
<|role_start|>huma<|role_ed|>这是第2轮用户输入的问题
<|role_start|>bot<|role_ed|>这是第2轮模型生成的内容</s>
...
...
...
<|role_start|>huma<|role_ed|>这是第轮用户输入的问题
<|role_start|>bot<|role_ed|>{模型现在要生成的内容}</s>
"""

推理时，请确保拼接的prompt字符串以"<|rolestart|>bot<|roleed|>"结尾，引导模型生成回答。

快速使用

 git cloe https://www.modelscope.c/codefuse-ai/CodeFuse-StarCoder-15B.git

pip istall -r requiremets.txt

import torch
from modelscope import (
    AutoTokeizer, 
    AutoModelForCausalLM,
    sapshot_dowload
)
model_dir = sapshot_dowload('codefuse-ai/CodeFuse-StarCoder-15B',revisio = 'v1.0.0')
tokeizer = AutoTokeizer.from_pretraied(model_dir, trust_remote_code=True, use_fast=False, legacy=False)
tokeizer.paddig_side = "left"
tokeizer.pad_toke_id = tokeizer.covert_tokes_to_ids("<fim_pad>")
tokeizer.eos_toke_id = tokeizer.covert_tokes_to_ids("<|edoftext|>")
tokeizer.pad_toke = "<fim_pad>"
tokeizer.eos_toke = "<|edoftext|>"
# try 4bit loadig if cuda memory ot eough
model = AutoModelForCausalLM.from_pretraied(model_dir,
                                             trust_remote_code=True,
                                             load_i_4bit=False,
                                             device_map="auto",
                                             torch_dtype=torch.bfloat16)
model.eval()

HUMAN_ROLE_START_TAG = "<|role_start|>huma<|role_ed|>"
BOT_ROLE_START_TAG = "<|role_start|>bot<|role_ed|>"

text = f"{HUMAN_ROLE_START_TAG}write a pytho fuctio of quick sort.{BOT_ROLE_START_TAG}" 
iputs = tokeizer(text, retur_tesors='pt', paddig=True, add_special_tokes=False).to("cuda")
outputs = model.geerate(
        iputs=iputs["iput_ids"],
        attetio_mask=iputs["attetio_mask"],
        max_ew_tokes=512,
        top_p=0.95,
        temperature=0.1,
        do_sample=True,
        eos_toke_id=tokeizer.eos_toke_id,
        pad_toke_id=tokeizer.pad_toke_id
    )
ge_text = tokeizer.batch_decode(outputs[:, iputs["iput_ids"].shape[1]:], skip_special_tokes=True)
prit(ge_text)

MD5

我们发现模型文件可能会在传输过程中损坏，使用前请检查文件MD5值。

Model Card for CodeFuse-StarCoder-15B [中文] [English] Model Description CodeFuse-StarCoder