Itroductio

The MiiCPM-MoE-8x2B is a decoder-oly trasformer-based geerative laguage model.

The MiiCPM-MoE-8x2B adopt a Mixture-of-Experts(MoE) architecture, which has 8 experts per layer ad activates 2 of 8 experts for each toke.

Usage

This is a model versio after istructio tuig but without other rlhf methods. Chat template is automatically applied.

from modelscope import AutoModelForCausalLM, AutoTokeizer
import torch
torch.maual_seed(0)

path = 'opebmb/MiiCPM-MoE-8x2B'
tokeizer = AutoTokeizer.from_pretraied(path)
model = AutoModelForCausalLM.from_pretraied(path, torch_dtype=torch.bfloat16, device_map='cuda', trust_remote_code=True)

respods, history = model.chat(tokeizer, "山东省最高的山是哪座山, 它比黄山高还是矮？差距多少？", temperature=0.8, top_p=0.8)
prit(respods)

Note

1. You ca alse iferece with vLLM, which will be compatible with this repo ad has a much higher iferece throughput.
2. The precisio of model weights i this repo is bfloat16. Maual covertio is eeded for other kids of dtype.
3. For more details, please refer to our github repo.

Statemet

1. As a laguage model, MiiCPM-MoE-8x2B geerates cotet by learig from a vast amout of text.
2. However, it does ot possess the ability to comprehed or express persoal opiios or value judgmets.
3. Ay cotet geerated by MiiCPM-MoE-8x2B does ot represet the viewpoits or positios of the model developers.
4. Therefore, whe usig cotet geerated by MiiCPM-MoE-8x2B, users should take full resposibility for evaluatig ad verifyig it o their ow.

Introduction The MiniCPM-MoE-8x2B is a decoder-only transformer-based generative language model. Th