Unichat-llama3-Chinese-70B

介绍

• 中国联通AI创新中心发布llama3-700亿参数中文指令微调模型，全参数微调非lora
• 本模型以Meta Llama 3为基础,增加中文数据进行训练,实现llama3模型高质量中文问答
• 模型上下文保持原生长度8K
• 模型使用llama3官方对话模板
• 基础模型 Meta-Llama-3-70B

? 数据

• 高质量指令数据，覆盖多个领域和行业，为模型训练提供充足的数据支持
• 微调指令数据经过严格的人工筛查，保证优质的指令数据用于模型微调

快速开始

import transformers
from transformers import AutoTokenizer, AutoModelForCausalLM
import torch

model_id = "UnicomAI/Unichat-llama3-Chinese-70B"

pipeline = transformers.pipeline(
    "text-generation",
    model=model_id,
    model_kwargs={"torch_dtype": torch.bfloat16},
    device="cuda",
)


messages = [
    {"role": "system", "content": "A chat between a curious user and an artificial intelligence assistant.The assistant gives helpful, detailed, and polite answers to the user's questions."},
    {"role": "user", "content": "你是谁"},
]


prompt = pipeline.tokenizer.apply_chat_template(
      messages,
      tokenize=False,
      add_generation_prompt=True
)

terminators = [
      pipeline.tokenizer.eos_token_id,
      pipeline.tokenizer.convert_tokens_to_ids("<|eot_id|>")
]


outputs = pipeline(
        prompt,
        max_new_tokens=2048,
        eos_token_id=terminators,
        do_sample=False,
        temperature=0.6,
        top_p=1,
        repetition_penalty=1.05
)
print(outputs[0]["generated_text"][len(prompt):])

资源

更多模型,数据集和训练相关细节请参考:

• Github：Unichat-llama3-Chinese