Min-K% Probability Evaluation

The min_k_prob step evaluates whether specific content was likely part of a model’s pretraining data, providing insights into model memorization patterns and potential dataset contamination.

Overview

Min-K% Probability is a novel approach for detecting whether a given text was included in a model’s pretraining data. Unlike standard metrics like perplexity that consider all tokens equally, this method focuses specifically on the least likely tokens in a sequence, offering a more sensitive measure of model memorization. This evaluation technique was introduced by Shi et al. (2023) and has proven effective for detecting pretraining data without requiring access to reference models or knowledge of the original training distribution.

Technical Implementation

The Min-K% Probability method works on a simple yet powerful intuition: texts that a model has seen during training tend to have fewer “outlier” tokens with extremely low probabilities. When evaluating a text, the system first computes the conditional probability for each token given its preceding context. It then identifies the k% of tokens with the lowest probabilities (the outliers) and calculates their average log likelihood. This focused measurement provides a more discriminative signal than whole-sequence perplexity, as unseen texts are more likely to contain several extremely low-probability tokens, while seen texts generally have more uniform probability distributions even for their least likely tokens.

This selective approach to token probability analysis enables the detection of subtle memorization patterns that might be obscured when considering all tokens equally. The implementation leverages the model’s own probability distributions to reveal whether it has prior exposure to specific content, without requiring comparison models or complex calibration techniques.

Key Features

Min-K% Probability evaluations offer several valuable capabilities for understanding model behavior. The approach provides a direct measurement of model memorization, revealing which content a model has likely encountered during pretraining. This can uncover potential dataset contamination in benchmark evaluations, helping researchers ensure the validity of their performance measurements. The method also enables privacy auditing, making it possible to detect whether private information might have been included in training data. Additionally, it can identify potential copyright concerns by detecting memorized content from published works.

Because Min-K% Probability requires no reference models or knowledge of the training distribution, it can be applied to any model that provides token probabilities. This makes it particularly valuable for evaluating black-box commercial models where training details remain undisclosed.

Model Compatibility

This evaluation method requires access to token-level probabilities from the model. It works with local Hugging Face models where these probabilities are directly accessible but is not compatible with most API-based models that only provide text outputs. The approach is most effective with larger models, as their increased capacity tends to result in stronger memorization patterns that are more readily detectable. Like the cloze prompt evaluation, this method depends on accessing the model’s internal probability distributions rather than relying solely on generated text.

When to Use

Min-K% Probability evaluation is particularly valuable in several scenarios. When validating benchmark results, it helps determine whether test data might have been inadvertently included in the model’s pretraining, which could artificially inflate performance metrics. For copyright compliance assessment, it can detect whether published content has been memorized by the model, potentially raising intellectual property concerns. The method is also useful for privacy auditing, helping to identify whether sensitive information might have been included in training data. Additionally, it can evaluate the effectiveness of machine unlearning techniques designed to remove specific content from models.

This approach is most suitable when you need to make fine-grained determinations about specific content rather than evaluating general model capabilities. Its ability to work without reference models makes it especially valuable for analyzing proprietary systems where training details are unavailable.

Implementation Details

The core implementation of Min-K% Probability involves selecting the tokens with the lowest probabilities in a sequence and analyzing their likelihood patterns. After obtaining the token probabilities for a text, the method sorts them and selects the bottom k% (typically 20%) with the lowest probabilities. It then calculates the average log probability of these selected tokens, which serves as the memorization score. Higher average scores for the lowest-probability tokens suggest the content was likely seen during training.

The primary hyperparameter in this approach is the percentage of tokens (k) to consider. Research has shown that values around 20% typically provide good discrimination between seen and unseen content, but this can be tuned based on the specific use case and model characteristics. The method can be applied both to complete documents and to shorter text segments, with longer texts generally providing more reliable signals due to the increased sample size for token probability analysis.

Min-K% Probability Evaluation

Min-K% Probability Evaluation

Overview

Technical Implementation

Key Features

Model Compatibility

When to Use

Implementation Details

Further Reading