Fie-tuig o Itel Gaudi2

This model is a fie-tued model based o mistralai/Mistral-7B-v0.1 o the ope source dataset Ope-Orca/SlimOrca. The we alig it with DPO algorithm. For more details, you ca refer our blog: The Practice of Supervised Fie-tuig ad Direct Preferece Optimizatio o Itel Gaudi2.

Model date

Neural-chat-7b-v3-1 was traied betwee September ad October, 2023.

Evaluatio

We submit our model to opellmleaderboard, ad the model performace has bee improved sigificatly as we see from the average metric of 7 tasks from the leaderboard.

Traiig procedure

Traiig hyperparameters

The followig hyperparameters were used durig traiig:

• learig_rate: 1e-04
• traibatchsize: 1
• evalbatchsize: 2
• seed: 42
• distributed_type: multi-HPU
• um_devices: 8
• gradietaccumulatiosteps: 8
• totaltraibatch_size: 64
• totalevalbatch_size: 8
• optimizer: Adam with betas=(0.9,0.999) ad epsilo=1e-08
• lrschedulertype: cosie
• lrschedulerwarmup_ratio: 0.03
• um_epochs: 2.0

Traiig sample code

Here is the sample code to reproduce the model: Sample Code.

Prompt Template

### System:
{system}
### User:
{usr}
### Assistat:

Iferece with trasformers

import trasformers


model_ame = 'Itel/eural-chat-7b-v3-1'
model = trasformers.AutoModelForCausalLM.from_pretraied(model_ame)
tokeizer = trasformers.AutoTokeizer.from_pretraied(model_ame)

def geerate_respose(system_iput, user_iput):

    # Format the iput usig the provided template
    prompt = f"### System:\{system_iput}\### User:\{user_iput}\### Assistat:\"

    # Tokeize ad ecode the prompt
    iputs = tokeizer.ecode(prompt, retur_tesors="pt", add_special_tokes=False)

    # Geerate a respose
    outputs = model.geerate(iputs, max_legth=1000, um_retur_sequeces=1)
    respose = tokeizer.decode(outputs[0], skip_special_tokes=True)

    # Extract oly the assistat's respose
    retur respose.split("### Assistat:\")[-1]


# Example usage
system_iput = "You are a math expert assistat. Your missio is to help users uderstad ad solve various math problems. You should provide step-by-step solutios, explai reasoigs ad give the correct aswer."
user_iput = "calculate 100 + 520 + 60"
respose = geerate_respose(system_iput, user_iput)
prit(respose)

# expected respose
"""
To calculate the sum of 100, 520, ad 60, we will follow these steps:

1. Add the first two umbers: 100 + 520
2. Add the result from step 1 to the third umber: (100 + 520) + 60

Step 1: Add 100 ad 520
100 + 520 = 620

Step 2: Add the result from step 1 to the third umber (60)
(620) + 60 = 680

So, the sum of 100, 520, ad 60 is 680.
"""

Ethical Cosideratios ad Limitatios

eural-chat-7b-v3-1 ca produce factually icorrect output, ad should ot be relied o to produce factually accurate iformatio. eural-chat-7b-v3-1 was traied o Ope-Orca/SlimOrca based o mistralai/Mistral-7B-v0.1. Because of the limitatios of the pretraied model ad the fietuig datasets, it is possible that this model could geerate lewd, biased or otherwise offesive outputs.

Therefore, before deployig ay applicatios of eural-chat-7b-v3-1, developers should perform safety testig.

Disclaimer

The licese o this model does ot costitute legal advice. We are ot resposible for the actios of third parties who use this model. Please cosult a attorey before usig this model for commercial purposes.

Orgaizatios developig the model

The NeuralChat team with members from Itel/DCAI/AISE/AIPT. Core team members: Kaokao Lv, Liag Lv, Chag Wag, Wexi Zhag, Xuhui Re, ad Haihao She.

Useful liks

• Itel Neural Compressor lik
• Itel Extesio for Trasformers lik

Ope LLM Leaderboard Evaluatio Results

Detailed results ca be foud here

Fine-tuning on Intel Gaudi2 This model is a fine-tuned model based on mistralai/Mistral-7B-v0.1 on t