huggingface-transformers

How to get logits or raw model scores from Hugging Face text classification pipeline?


I'm using Hugging Face's pipeline for sentiment analysis with a model like distilbert-base-uncased-finetuned-sst-2-english.

The pipeline returns labels and scores like:

{'label': 'POSITIVE', 'score': 0.987}

However, I want the raw logits (before softmax) or full class probability distribution (not just the top one), so I can:

  1. Calibrate scores
  2. Apply temperature scaling
  3. Compute confidence intervals
from transformers import pipeline

clf = pipeline("sentiment-analysis")
result = clf("I love this movie!")
print(result)

I tried setting return_all_scores=True, but it still gives only post-softmax values.


Solution

  • This might help

    from transformers import AutoTokenizer, AutoModelForSequenceClassification
    import torch
    import torch.nn.functional as F
    
    # Load tokenizer and model
    model_name = "distilbert-base-uncased-finetuned-sst-2-english"
    tokenizer = AutoTokenizer.from_pretrained(model_name)
    model = AutoModelForSequenceClassification.from_pretrained(model_name)
    
    # Prepare inputs
    text = "I love this movie!"
    inputs = tokenizer(text, return_tensors="pt")
    
    # Get logits
    with torch.no_grad():
        outputs = model(**inputs)
        logits = outputs.logits  # Raw logits before softmax
    
    # Optionally convert to probabilities
    probs = F.softmax(logits, dim=1)
    
    # Print
    print("Logits:", logits)
    print("Probabilities:", probs)