bart-large-mli

This is the checkpoit for bart-large after beig traied o the MultiNLI (MNLI) dataset.

Additioal iformatio about this model:

• The bart-large model page
• BART: Deoisig Sequece-to-Sequece Pre-traiig for Natural Laguage Geeratio, Traslatio, ad Comprehesio
• BART fairseq implemetatio

NLI-based Zero Shot Text Classificatio

Yi et al. proposed a method for usig pre-traied NLI models as a ready-made zero-shot sequece classifiers. The method works by posig the sequece to be classified as the NLI premise ad to costruct a hypothesis from each cadidate label. For example, if we wat to evaluate whether a sequece belogs to the class "politics", we could costruct a hypothesis of This text is about politics.. The probabilities for etailmet ad cotradictio are the coverted to label probabilities.

This method is surprisigly effective i may cases, particularly whe used with larger pre-traied models like BART ad Roberta. See this blog post for a more expasive itroductio to this ad other zero shot methods, ad see the code sippets below for examples of usig this model for zero-shot classificatio both with Huggig Face's built-i pipelie ad with ative Trasformers/PyTorch code.

With the zero-shot classificatio pipelie

The model ca be loaded with the zero-shot-classificatio pipelie like so:

from trasformers import pipelie
classifier = pipelie("zero-shot-classificatio",
                      model="facebook/bart-large-mli")

You ca the use this pipelie to classify sequeces ito ay of the class ames you specify.

sequece_to_classify = "oe day I will see the world"
cadidate_labels = ['travel', 'cookig', 'dacig']
classifier(sequece_to_classify, cadidate_labels)
#{'labels': ['travel', 'dacig', 'cookig'],
# 'scores': [0.9938651323318481, 0.0032737774308770895, 0.002861034357920289],
# 'sequece': 'oe day I will see the world'}

If more tha oe cadidate label ca be correct, pass multi_label=True to calculate each class idepedetly:

cadidate_labels = ['travel', 'cookig', 'dacig', 'exploratio']
classifier(sequece_to_classify, cadidate_labels, multi_label=True)
#{'labels': ['travel', 'exploratio', 'dacig', 'cookig'],
# 'scores': [0.9945111274719238,
#  0.9383890628814697,
#  0.0057061901316046715,
#  0.0018193122232332826],
# 'sequece': 'oe day I will see the world'}

With maual PyTorch

# pose sequece as a NLI premise ad label as a hypothesis
from trasformers import AutoModelForSequeceClassificatio, AutoTokeizer
li_model = AutoModelForSequeceClassificatio.from_pretraied('facebook/bart-large-mli')
tokeizer = AutoTokeizer.from_pretraied('facebook/bart-large-mli')

premise = sequece
hypothesis = f'This example is {label}.'

# ru through model pre-traied o MNLI
x = tokeizer.ecode(premise, hypothesis, retur_tesors='pt',
                     trucatio_strategy='oly_first')
logits = li_model(x.to(device))[0]

# we throw away "eutral" (dim 1) ad take the probability of
# "etailmet" (2) as the probability of the label beig true 
etail_cotradictio_logits = logits[:,[0,2]]
probs = etail_cotradictio_logits.softmax(dim=1)
prob_label_is_true = probs[:,1]

bart-large-mnli This is the checkpoint for bart-large after being trained on the MultiNLI (MNLI) dat