Detectors<a class="headerlink" href="#module-pytorch_ood.detector" title="Link to this heading">

See Implementation:

Parameters:

model – neural network to use. Can be None when using predict_logits(...) directly.
t – temperature value \(T\). Default is 1.

property device: The device of the detector’s owned torch state, if one can be inferred.

predict(x: Tensor) → Tensor

Apply the model and forward its logits to predict_logits(...).

Parameters:: x – input batch
Returns:: outlier scores

predict_logits(*args, **kwargs)

Parameters:: logits – logits given by the model

requires_fit = False: Whether fit(...) must be called before scoring.

static score(logits: Tensor, t: float | None = 1.0) → Tensor[source]

Parameters:

logits – logits for samples
t – temperature value

to(device) → Self

Move detector-owned modules and tensor state to device.

This is a detector-level analogue of nn.Module.to(...). It moves modules, tensors, and common container-valued state stored on the detector itself.

Parameters:: device – target torch device
Returns:: self

Monte Carlo Dropout (MCD)

class pytorch_ood.detector.MCD(model: Module, samples: int = 30, mode: str = 'var', batch_norm: bool = True)[source]

Bases: Detector

From the paper Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning. Forward-propagates the input through the model \(N\) times with activated dropout and averages the results.

In mean mode, the outlier score is calculated as

\[- \max_y \frac{1}{N} \sum_n^{N} \sigma_y(f_n(x))\]

where \(\sigma\) is the softmax function. In var mode, the scores are calculated as

\[\frac{1}{C} \sum_y^C \frac{1}{N} \sum_n^N ( \sigma_y(f_n(x)) - \mu_y )^2\]

where \(C\) is the number of classes and \(\mu_y\) is the class mean. This is the mean over the per class variance, which was used in Bayesian SegNet: Model Uncertainty in Deep Convolutional Encoder-Decoder Architectures for Scene Understanding.

See MCD Paper:: ICML
See Bayesian SegNet:: ArXiv

Warning

This implementations puts the model into evaluation mode (except for variants of the BatchNorm Layers). This could also affect other modules.

Parameters:

model – the module to use for the forward pass. Should output logits.
samples – number of iterations
mode – can be one of var or mean
batch_norm – keep batch norm layers in evaluation mode

property device: The device of the detector’s owned torch state, if one can be inferred.

n_samples: number \(N\) of samples

predict(x: Tensor) → Tensor[source]

Parameters:: x – input
Returns:: outlier score

requires_fit = False: Whether fit(...) must be called before scoring.

static run(model: Module, x: Tensor, samples: int, batch_norm=True) → Tuple[Tensor, Tensor][source]

Parameters:

model – neural network
x – input
samples – number of rounds
batch_norm – keep batch norm layers in evaluation mode

Returns:

mean and variance of softmax normalized model outputs

static run_mean(model: Module, x: Tensor, samples: int, batch_norm=True) → Tensor[source]

Assumes that the model outputs logits. More memory efficient implementation.

Parameters:

model – neural network
x – input
samples – number of rounds
batch_norm – keep batch norm layers in evaluation mode

Returns:

mean softmax output of the model

to(device) → Self

Move detector-owned modules and tensor state to device.

This is a detector-level analogue of nn.Module.to(...). It moves modules, tensors, and common container-valued state stored on the detector itself.

Parameters:: device – target torch device
Returns:: self

Temperature Scaling

class pytorch_ood.detector.TemperatureScaling(model: Module | None)[source]

Bases: MaxSoftmax

Implements temperature scaling from the paper On Calibration of Modern Neural Networks.

The method uses an additional set of validation samples to determine the optimal temperature value \(T\) to calibrate the softmax output.

The score is calculated as:

\[- \max_y \sigma_y(f(x) / T)\]

where \(\sigma\) is the softmax function, \(T\) is the optimal temperature and \(\sigma_y\) indicates the \(y^{th}\) value of the resulting probability vector.

See Paper:: ArXiv
Parameters:: model – neural network to use. Can be None when using fit_logits(...) and predict_logits(...) directly.

property device: The device of the detector’s owned torch state, if one can be inferred.

fit(data_loader: DataLoader) → Self

Extract logits from a loader and forward them to fit_logits(...).

Parameters:: data_loader – loader to extract logits from

fit_logits(logits: Tensor, labels: Tensor) → Self[source]

Optimize temperature using L-BFGS. Ignores OOD inputs.

Parameters:

logits – logits
labels – labels for logits

predict(x: Tensor) → Tensor[source]

Apply the model and forward its logits to predict_logits(...).

Parameters:: x – input batch
Returns:: outlier scores

predict_logits(*args, **kwargs)

Parameters:: logits – logits given by the model

requires_fit = True: Whether fit(...) must be called before scoring.

static score(logits: Tensor, t: float | None = 1.0) → Tensor

Parameters:

logits – logits for samples
t – temperature value

to(device) → Self

Move detector-owned modules and tensor state to device.

This is a detector-level analogue of nn.Module.to(...). It moves modules, tensors, and common container-valued state stored on the detector itself.

Parameters:: device – target torch device
Returns:: self

KL-Matching

class pytorch_ood.detector.KLMatching(model: Module | None)[source]

Bases: LogitsDetector

Implements KL-Matching from the paper Scaling Out-of-Distribution Detection for Real-World Settings.

For each class, an typical posterior distribution \(d_y = \mathbb{E}_{x \sim \mathcal{X}_{val}}[p(y \vert x)]\) is estimated, where \(y\) is the class with the maximum posterior \(y = \arg\max_y p(y \vert x)\), as predicted by the model. Note that the method does not require class labels for the validation set. During evaluation, the KL-Divergence between the observed and the typical posterior \(D_{KL}[p(y \vert x) \Vert d_y]\) is used as outlier score.

This method can also be applied to multi-class settings.

See Paper:: ArXiv
Parameters:: model – neural network, is assumed to output logits. Can be None when using fit_logits(...) and predict_logits(...) directly.

property device: The device of the detector’s owned torch state, if one can be inferred.

dists: ParameterDict: Typical posteriors per class

fit(data_loader: DataLoader) → Self

Extract logits from a loader and forward them to fit_logits(...).

Parameters:: data_loader – loader to extract logits from

fit_logits(logits: Tensor, labels: Tensor) → Self[source]

Estimates typical distributions for each class. Ignores OOD samples.

Parameters:

logits – logits
labels – class labels

predict(x: Tensor) → Tensor[source]

Calculates KL-Divergence between predicted posteriors and typical posteriors.

Parameters:: x – input tensor, will be passed through model
Returns:: Outlier scores

predict_logits(*args, **kwargs)

Parameters:: logits – logits predicted by the model

requires_fit = True: Whether fit(...) must be called before scoring.

to(device) → Self

Move detector-owned modules and tensor state to device.

This is a detector-level analogue of nn.Module.to(...). It moves modules, tensors, and common container-valued state stored on the detector itself.

Parameters:: device – target torch device
Returns:: self

Entropy

class pytorch_ood.detector.Entropy(model: Module | None)[source]

Bases: LogitsDetector

Implements Entropy-based OOD detection.

This methods calculates the entropy based on the logits of a classifier. Higher entropy means more uniformly distributed posteriors, indicating larger uncertainty. Entropy is calculated as

\[H(x) = - \sum_i^C \sigma_i(f(x)) \log( \sigma_i(f(x)) )\]

where \(\sigma_i\) indicates the \(i^{th}\) softmax value and \(C\) is the number of classes.

Parameters:: model – the model \(f\). Can be None when using predict_logits(...) directly.

property device: The device of the detector’s owned torch state, if one can be inferred.

predict(x: Tensor) → Tensor

Apply the model and forward its logits to predict_logits(...).

Parameters:: x – input batch
Returns:: outlier scores

predict_logits(*args, **kwargs)

Parameters:: logits – logits given by your model

requires_fit = False: Whether fit(...) must be called before scoring.

static score(logits: Tensor) → Tensor[source]

Parameters:: logits – logits of input

to(device) → Self

Move detector-owned modules and tensor state to device.

This is a detector-level analogue of nn.Module.to(...). It moves modules, tensors, and common container-valued state stored on the detector itself.

Parameters:: device – target torch device
Returns:: self

Generalized Entropy (GEN)

class pytorch_ood.detector.GEN(model: Module | None, gamma: float | None = 0.1)[source]

Bases: LogitsDetector

Implements GEN: Pushing the Limits of Softmax-Based Out-of-Distribution Detection.

GEN generalizes softmax-based OOD scoring by applying a power transform to the posterior probabilities. The score is defined as

\[G_\gamma(x) = \sum_{j=1}^{C} p_j(x)^\gamma \, (1 - p_j(x))^\gamma\]

where \(p_j(x) = \sigma_j(f(x))\) is the \(j^{th}\) softmax probability of the model output and \(\gamma \in (0, 1)\) controls the sensitivity.

A small \(\gamma\) (the paper recommends \(\gamma = 0.1\)) amplifies differences near \(p = 0\) and \(p = 1\), making the score highly sensitive to the shape of the full softmax distribution rather than only its maximum. In-distribution samples produce confident (peaky) posteriors with low scores, while OOD samples yield higher scores.

See Paper:

CVPR 2023

See Implementation:

Parameters:

model – the neural network \(f\). Can be None when using predict_logits(...) directly.
gamma – exponent \(\gamma\). Default is 0.1 as recommended by the paper.

property device: The device of the detector’s owned torch state, if one can be inferred.

gamma: float: Power-transform exponent

predict(x: Tensor) → Tensor

Apply the model and forward its logits to predict_logits(...).

Parameters:: x – input batch
Returns:: outlier scores

predict_logits(*args, **kwargs)

Parameters:: logits – logits given by the model

requires_fit = False: Whether fit(...) must be called before scoring.

static score(logits: Tensor, gamma: float = 0.1) → Tensor[source]

Parameters:

logits – logits of input
gamma – power-transform exponent \(\gamma\)

to(device) → Self

Move detector-owned modules and tensor state to device.

This is a detector-level analogue of nn.Module.to(...). It moves modules, tensors, and common container-valued state stored on the detector itself.

Parameters:: device – target torch device
Returns:: self

Logit-based

Logit-based methods are based on the observation that OOD inputs tend to yield different logits compared to ID data.

Maximum Logit

class pytorch_ood.detector.MaxLogit(model: Module | None)[source]

Bases: LogitsDetector

Implements the Max Logit Method for OOD Detection as proposed in Scaling Out-of-Distribution Detection for Real-World Settings.

\[- \max_y f_y(x)\]

where \(f_y(x)\) indicates the \(y^{th}\) logits value predicted by \(f\).

See Paper:: ArXiv
Parameters:: model – neural network to use. Can be None when using predict_logits(...) directly.

property device: The device of the detector’s owned torch state, if one can be inferred.

predict(x: Tensor) → Tensor

Apply the model and forward its logits to predict_logits(...).

Parameters:: x – input batch
Returns:: outlier scores

predict_logits(*args, **kwargs)

Parameters:: logits – logits as given by the model

requires_fit = False: Whether fit(...) must be called before scoring.

static score(logits: Tensor) → Tensor[source]

Parameters:: logits – logits for samples

to(device) → Self

Move detector-owned modules and tensor state to device.

This is a detector-level analogue of nn.Module.to(...). It moves modules, tensors, and common container-valued state stored on the detector itself.

Parameters:: device – target torch device
Returns:: self

OpenMax

class pytorch_ood.detector.OpenMax(model: Module | None, tailsize: int = 25, alpha: int = 10, euclid_weight: float = 1.0)[source]

Bases: LogitsDetector

Implementation of the OpenMax Layer as proposed in the paper Towards Open Set Deep Networks.

The method determines a center \(\mu_y\) for each class in the logits space of a model, and then creates a statistical model of the distances of correct classified inputs. It uses extreme value theory to detect outliers by fitting a weibull function to the tail of the distance distribution.

We use the pseudo-activation of the unknown class as outlier score.

See Paper:

See Implementation:

Parameters:

model – neural network, assumed to output logits. Can be None when using fit_logits(...) and predict_logits(...) directly.
tailsize – length of the tail to fit the distribution to
alpha – number of class activations to revise
euclid_weight – weight for the Euclidean distance.

property device: The device of the detector’s owned torch state, if one can be inferred.

fit(data_loader: DataLoader) → Self

Extract logits from a loader and forward them to fit_logits(...).

Parameters:: data_loader – loader to extract logits from

fit_logits(logits: Tensor, y: Tensor) → Self[source]

Determines parameters of the weibull functions for each class.

Parameters:

logits – logits given by the model
y – class labels

Returns:

predict(x: Tensor) → Tensor[source]

Parameters:: x – input, will be passed through the model to get logits

predict_logits(*args, **kwargs)

Parameters:: logits – logits given by model

requires_fit = True: Whether fit(...) must be called before scoring.

to(device) → Self

Move detector-owned modules and tensor state to device.

This is a detector-level analogue of nn.Module.to(...). It moves modules, tensors, and common container-valued state stored on the detector itself.

Parameters:: device – target torch device
Returns:: self

Energy Based (EBO)

class pytorch_ood.detector.EnergyBased(model: Module | None, t: float | None = 1.0)[source]

Bases: LogitsDetector

Implements the Energy Score of Energy-based Out-of-distribution Detection.

This methods calculates the negative energy for a vector of logits. This value can be used as outlier score.

\[E(x) = -T \log{\sum_i e^{f_i(x)/T}}\]

where \(f_i(x)\) indicates the \(i^{th}\) logit value predicted by \(f\).

See Paper:

NeurIPS

See Implementation:

Parameters:

model – neural network to use. Can be None when using predict_logits(...) directly.
t – Temperature value \(T\). Default is 1.

property device: The device of the detector’s owned torch state, if one can be inferred.

predict(x: Tensor) → Tensor

Apply the model and forward its logits to predict_logits(...).

Parameters:: x – input batch
Returns:: outlier scores

predict_logits(*args, **kwargs)

Parameters:: logits – logits given by the model

requires_fit = False: Whether fit(...) must be called before scoring.

static score(logits: Tensor, t: float | None = 1.0) → Tensor[source]

Parameters:

logits – logits of input
t – temperature value

t: float: Temperature

to(device) → Self

Move detector-owned modules and tensor state to device.

This is a detector-level analogue of nn.Module.to(...). It moves modules, tensors, and common container-valued state stored on the detector itself.

Parameters:: device – target torch device
Returns:: self

Weighted Energy Based (WEBO)

class pytorch_ood.detector.WeightedEBO(model: Module | None, weights: Tensor)[source]

Bases: LogitsDetector

Implements the Weighted Energy Based Score of VOS: Learning what you don’t know by virtual outlier synthesis.

This method calculates the energy from the weighted logits. The negative energy can be used as outlier score. The weights can be obtained, for example, by training with the pytorch_ood.loss.VOSRegLoss.

Overall, the score is defined as:

\[E(x) = - \log{\sum_i w_{i} e^{f_i(x)}}\]

where \(f_i(x)\) indicates the \(i^{th}\) logit value predicted by \(f\) and \(w\) indicates the weights.

Example Code:

weights = torch.nn.Linear(num_classes, 1))
detector = WeightedEBO(model, weights)
scores = detector(images)

See Paper:

See Implementation:

Parameters:

model – neural network \(f\) to use, is assumed to output logits. Can be None when using predict_logits(...) directly.
weights – weight vector of with shape \(C \times 1\) where \(C\) is the number of classes

property device: The device of the detector’s owned torch state, if one can be inferred.

predict(x: Tensor) → Tensor

Apply the model and forward its logits to predict_logits(...).

Parameters:: x – input batch
Returns:: outlier scores

predict_logits(*args, **kwargs)

Parameters:: logits – logits given by your model

requires_fit = False: Whether fit(...) must be called before scoring.

static score(logits: Tensor, weights: Tensor) → Tensor[source]

Parameters:

logits – logits of input
weights – weights as torch.nn.module

to(device) → Self

Move detector-owned modules and tensor state to device.

This is a detector-level analogue of nn.Module.to(...). It moves modules, tensors, and common container-valued state stored on the detector itself.

Parameters:: device – target torch device
Returns:: self

Feature-based

Mahalanobis Distance (MD)

class pytorch_ood.detector.Mahalanobis(model: Callable[[Tensor], Tensor] | None, eps: float = 0.002, norm_std: List | None = None)[source]

Implements the Mahalanobis Method from the paper A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attacks.

This method calculates a class center \(\mu_y\) for each class, and a shared covariance matrix \(\Sigma\) from the data. The outlier scores are then calculated as

\[- \max_k \lbrace (f(x) - \mu_k)^{\top} \Sigma^{-1} (f(x) - \mu_k) \rbrace\]

Also uses ODIN preprocessing if the given \(\epsilon > 0\)

See Implementation:

See Paper:

Parameters:

model – the Neural Network, should output features. Can be None when using fit_features(...) and predict_features(...) directly.
eps – magnitude for gradient based input preprocessing
norm_std – Standard deviations for input normalization

cov: Tensor: Covariance Matrix

property device: The device of the detector’s owned torch state, if one can be inferred.

eps: float: epsilon

fit(data_loader: DataLoader) → Self[source]

Fit parameters of the multi variate gaussian.

Parameters:: data_loader – dataset to fit on.

fit_features(z: Tensor, y: Tensor) → Self[source]

Fit parameters of the multi variate gaussian.

Parameters:

z – features
y – class labels

mu: Tensor: Centers

property n_classes: Number of classes the model is fitted for

precision: Tensor: Precision Matrix

predict(x: Tensor) → Tensor[source]

Parameters:: x – input tensor

predict_features(*args, **kwargs)

Calculates mahalanobis distance directly on features. ODIN preprocessing will not be applied.

Parameters:: z – features, as given by the model.

requires_fit = True: Whether fit(...) must be called before scoring.

to(device) → Self

Move detector-owned modules and tensor state to device.

This is a detector-level analogue of nn.Module.to(...). It moves modules, tensors, and common container-valued state stored on the detector itself.

Parameters:: device – target torch device
Returns:: self

Multi-Layer Mahalanobis Distance (MD)

class pytorch_ood.detector.MultiMahalanobis(model: List[Module], alpha: List[float] | None = None)[source]

Bases: StructuredDetector

Implements the Mahalanobis Method from the paper A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attacks which supports several layers.

For each of the given \(i\) layers, the method calculates a class center \(\mu_{iy}\) for each class, and a shared covariance matrix \(\Sigma_i\) from the data. The per-layer outlier scores are calculated as

\[M_i(x) = - \max_k \lbrace (f_i(x) - \mu_{ik})^{\top} \Sigma_i^{-1} (f_i(x) - \mu_{ik}) \rbrace\]

The final outlier score is the sum of all scores, weighted by \(\alpha\).

Example code is provided here

Note

This does not yet support ODIN preprocessing. Also, the \(\alpha\) values have to be determined manually.

See Implementation:

See Paper:

Parameters:

model – the neural network layers \(f_1(\cdot),...,f_n(\cdot)\), output of one will be used as input to the next.
alpha – weighting of the individual layers. Defaults to uniform weighting.

alpha: Per-layer weighting factors

cov: List[Tensor]: Covariance Matrices

property device: The device of the detector’s owned torch state, if one can be inferred.

fit(data_loader: DataLoader) → Self[source]

Fit one gaussian to the features of each layer. Will average over feature maps.

Parameters:: data_loader – dataset to fit on.
Returns:

fit_structured(zs: List[Tensor], y: Tensor) → Self[source]

Fit parameters of the multi variate gaussians.

Parameters:

zs – list of features for each layer
y – class labels

Returns:

mu: List[Tensor]: Centers

property n_classes: Number of classes the model is fitted for

precision: List[Tensor]: Precision Matrices

predict(x: Tensor) → Tensor[source]

Parameters:: x – input tensor

predict_structured(zs: List[Tensor], device=None) → Tensor[source]

Calculates mahalanobis distance directly on features. ODIN preprocessing will not be applied.

Parameters:

zs – list of per-layer features
device – device to use for computations

requires_fit = True: Whether fit(...) must be called before scoring.

to(device) → Self

Move detector-owned modules and tensor state to device.

This is a detector-level analogue of nn.Module.to(...). It moves modules, tensors, and common container-valued state stored on the detector itself.

Parameters:: device – target torch device
Returns:: self

Relative Mahalanobis Distance (RMD)

class pytorch_ood.detector.RMD(model: Callable[[Tensor], Tensor] | None)[source]

Bases: Mahalanobis

Implements the Relative Mahalanobis Distance (RMD) from the paper A Simple Fix to Mahalanobis Distance for Improving Near-OOD Detection.

This method calculates a class center \(\mu_y\) for each class, and a shared covariance matrix \(\Sigma\) from the data.

Additionally, it fits a background gaussian with mean \(\mu_0\) and covariance matrix \(\Sigma_0\) to all of the features and calculates outlier scores as

\[\min_k \lbrace d_k(f(x)) - d_0(f(x)) \rbrace\]

where \(d_k\) is the mahalanobis score for class \(k\) and \(d_0\) is the mahalanobis score under the background gaussian.

See Paper:: ArXiv
Parameters:: model – the Neural Network, should output features. Can be None when using fit_features(...) and predict_features(...) directly.

cov: Tensor: Covariance Matrix

property device: The device of the detector’s owned torch state, if one can be inferred.

eps: float: epsilon

fit(data_loader: DataLoader) → Self[source]

Fit parameters of the multi variate gaussian for the given loader. Ignores OOD Inputs.

Parameters:: data_loader – data loader with training data

fit_features(z: Tensor, y: Tensor) → Self[source]

Fit parameters of the multi variate gaussian. Ignores OOD inputs.

Parameters:

z – features
y – class labels

Returns:

mu: Tensor: Centers

property n_classes: Number of classes the model is fitted for

precision: Tensor: Precision Matrix

predict(x: Tensor) → Tensor[source]

Parameters:: x – input tensor

requires_fit = True: Whether fit(...) must be called before scoring.

to(device) → Self

Move detector-owned modules and tensor state to device.

This is a detector-level analogue of nn.Module.to(...). It moves modules, tensors, and common container-valued state stored on the detector itself.

Parameters:: device – target torch device
Returns:: self

Virtual Logit Matching (ViM)

class pytorch_ood.detector.ViM(model: Callable[[Tensor], Tensor] | None, d: int, w: Tensor, b: Tensor)[source]

Implements Virtual Logit Matching (ViM) from the paper ViM: Out-Of-Distribution with Virtual-logit Matching.

See Paper:: ArXiv
See Implementation:: GitHub

Note

Requires PyTorch ≥ 1.9 (torch.linalg).

Parameters:

model – neural network to use, is assumed to output features. Can be None when using fit_features(...) and predict_features(...) directly.
d – dimensionality of the principal subspace
w – weights \(W\) of the last layer of the network
b – biases \(b\) of the last layer of the network

alpha: float | None: the computed \(\alpha\) value

property device: The device of the detector’s owned torch state, if one can be inferred.

fit(data_loader: DataLoader) → Self[source]

Extracts features and logits, computes principle subspace and alpha. Ignores OOD samples.

Parameters:: data_loader – dataset to fit on

fit_features(features: Tensor, labels: Tensor) → Self[source]

Extracts features and logits, computes principle subspace and alpha. Ignores OOD samples.

Parameters:

features – features
labels – class labels

Returns:

predict(x: Tensor) → Tensor[source]

Parameters:: x – model input, will be passed through neural network

predict_features(*args, **kwargs)

Parameters:: x – features as given by the model

requires_fit = True: Whether fit(...) must be called before scoring.

to(device) → Self

Move detector-owned modules and tensor state to device.

This is a detector-level analogue of nn.Module.to(...). It moves modules, tensors, and common container-valued state stored on the detector itself.

Parameters:: device – target torch device
Returns:: self

Nearest Neighbor (kNN)

class pytorch_ood.detector.KNN(model: Callable[[Tensor], Tensor] | None, **knn_kwargs)[source]

Implements the detector from the paper Out-of-Distribution Detection with Deep Nearest Neighbors.

Fits a nearest neighbor model to the ID samples an uses the distance from the nearest neighbor as outlier score:

\[\min_{z \in \mathcal{D}} \lVert f(x) - f(z) \rVert_2\]

where \(\mathcal{D}\) is the dataset used to train the nearest neighbor model.

The original paper found that using contrastive pre-training could increase the performance.

See PMLR:

arXiv

Parameters:

model – neural network to use. Can be None when using fit_features(...) and predict_features(...) directly.
knn_kwargs – dict with keyword arguments that will be passed to the scikit learns k-NN

property device: The device of the detector’s owned torch state, if one can be inferred.

fit(data_loader: DataLoader) → Self[source]

Extracts features and fits the kNN-Model

Parameters:: data_loader – data loader

fit_features(z: Tensor, labels: Tensor) → Self[source]

Fits nearest neighbor model. Ignores OOD inputs.

Parameters:

z – features
labels – labels for features

predict(x: Tensor) → Tensor[source]

Parameters:: x – inputs, will be passed through model

predict_features(*args, **kwargs)

Parameters:

z – features
k – number of neighbors

requires_fit = True: Whether fit(...) must be called before scoring.

to(device) → Self

Move detector-owned modules and tensor state to device.

This is a detector-level analogue of nn.Module.to(...). It moves modules, tensors, and common container-valued state stored on the detector itself.

Parameters:: device – target torch device
Returns:: self

Nearest Neighbor Guidance (NNGuide)

class pytorch_ood.detector.NNGuide(model: Callable[[Tensor], Tensor], w: Tensor, b: Tensor, k: int = 10)[source]

Implements Nearest Neighbor Guidance for Out-of-Distribution Detection.

Guides classifier-based scores using k-NN similarity to an energy-weighted feature bank. The feature bank is constructed by scaling in-distribution training features with their corresponding energy scores. At inference, the outlier score is the negated product of the k-NN guidance (mean inner product with the energy-scaled feature bank) and the sample’s own energy:

\[s(x) = - \underbrace{\frac{1}{k} \sum_{z \in \mathcal{N}_k(x)} \langle f(x),\, E(z) \cdot f(z) \rangle}_{\text{guidance}} \cdot E(x)\]

where \(E(x) = \log \sum_i \exp(l_i(x))\) is the energy score, \(f(x)\) are the penultimate-layer features, and \(\mathcal{N}_k(x)\) are the \(k\) nearest neighbors in the energy-scaled feature bank measured by inner product.

The model passed to the constructor should extract penultimate-layer features. The classification head weights w and biases b are used internally to compute logits from features, similar to ViM.

See Paper:

arXiv

Parameters:

model – neural network that extracts penultimate-layer features
w – weight matrix of the classification head, shape (num_classes, feature_dim)
b – bias vector of the classification head, shape (num_classes,)
k – number of nearest neighbors for guidance (default: 10)

fit(data_loader: DataLoader, device=None) → Self[source]

Extract features from the data loader and build the energy-scaled feature bank.

Parameters:

data_loader – data loader with ID training data
device – device for feature extraction. If None, inferred from model.

fit_features(z: Tensor, labels: Tensor) → Self[source]

Build the energy-scaled feature bank from pre-extracted features.

Parameters:

z – features, shape (n, feature_dim)
labels – corresponding labels

predict(x: Tensor) → Tensor[source]

Parameters:: x – model inputs

predict_features(z: Tensor) → Tensor[source]

Compute the NNGuide outlier score from pre-extracted features.

Parameters:: z – features, shape (batch, feature_dim)

Simplified Hopfield Energy (SHE)

class pytorch_ood.detector.SHE(backbone: Callable[[Tensor], Tensor], head: Callable[[Tensor], Tensor])[source]

Implements Simplified Hopfield Energy from the paper Out-of-Distribution Detection based on In-Distribution Data Patterns Memorization with modern Hopfield Energy

For each class, SHE estimates the mean feature vector \(S_i\) of correctly classified instances. For some new instances with predicted class \(\hat{y}\), SHE then uses the inner product \(f(x)^{\top} S_{\hat{y}}\) as outlier score.

See Paper:

OpenReview

Parameters:

backbone – feature extractor
head – maps feature vectors to logits

property device: The device of the detector’s owned torch state, if one can be inferred.

fit(data_loader: DataLoader) → Self[source]

Extracts features and calculates mean patterns.

Parameters:: data_loader – data to fit

fit_features(z: Tensor, y: Tensor, batch_size: int = 1024) → Self[source]

Calculates mean patterns per class.

Parameters:

z – features to fit
y – labels
batch_size – how many samples we process at a time

predict(x: Tensor) → Tensor[source]

Parameters:: x – model inputs

predict_features(*args, **kwargs)

Parameters:: z – features as given by the model

requires_fit = True: Whether fit(...) must be called before scoring.

to(device) → Self

Move detector-owned modules and tensor state to device.

This is a detector-level analogue of nn.Module.to(...). It moves modules, tensors, and common container-valued state stored on the detector itself.

Parameters:: device – target torch device
Returns:: self

Gram Matrices Based (GM)

class pytorch_ood.detector.Gram(head: Module, feature_layers: List[Module], num_classes: int, num_poles_list: List[int] | None = None)[source]

Bases: StructuredDetector

Implements the on Gram matrices based Method from the paper Detecting Out-of-Distribution Examples with In-distribution Examples and Gram Matrices.

The Gram detector identifies OOD examples by analyzing feature correlations within the layers of a neural network using Gram matrices, which are computed as:

\[G^p_l = \left(F_l^p F_l^{p \top}\right)^{\frac{1}{p}}\]

Where \(F_l\) is the feature-map in layer \(l\). The Gram matrices capture the pairwise correlations between feature maps, which can be seen as capturing the image style. For each layer, matrices for several values of \(p\), called ‘’poles’’ are computed. During training, class-specific minimum and maximum bounds are calculated for each entry in the Gram matrices of the ID data in multiple layers of a neural network. For a test input \(x\), deviations are calculated layer-wise by comparing the Gram matrix values against the stored bounds. The total deviation across all layers \(l\) is normalized using the expected deviation for that layer:

\[\Delta(x) = \sum_{l} \frac{\delta_l(x)}{\mathbb{E}[\delta_l]}\]

See Implementation:

See Paper:

Parameters:

head – the head of the model
feature_layers – the layers of the model to be used for feature extraction
num_classes – the number of classes in the dataset
num_poles_list – the list of poles to be used for higher-order Gram matrices

property device: The device of the detector’s owned torch state, if one can be inferred.

fit(data_loader: DataLoader) → Self[source]

Calculate the minimum and maximum values for the Gram matrices of the training data.

Parameters:: data_loader – data loader for training data
Returns:: self

fit_structured(*args, **kwargs) → Self: Fit the detector directly on structured intermediate representations.

predict(x: Tensor) → Tensor[source]

Calculate deviation for inputs

Parameters:: x – input tensor, will be passed through model
Returns:: Gram based deviations

predict_structured(logits: Tensor, feature_list: List[Tensor]) → Tensor[source]

Parameters:

logits – logits given by your model
feature_list – list of features extracted from the model

Returns:

Gram based Deviations

requires_fit = True: Whether fit(...) must be called before scoring.

to(device) → Self

Move detector-owned modules and tensor state to device.

This is a detector-level analogue of nn.Module.to(...). It moves modules, tensors, and common container-valued state stored on the detector itself.

Parameters:: device – target torch device
Returns:: self

Neural Collapse Inspired (NCI)

class pytorch_ood.detector.NCI(encoder: Module, head: Linear, alpha: float = 0.0)[source]

Implements the Neural-Collapse Inspired OOD detector from the paper Detecting Out-of-distribution through the Lens of Neural Collapse.

Computes a global mean \(\mu_g\) of all features from the fitting set to center representations during inference. Let \(h\) be the representation of some input and \(z = h - \mu_g\) be the centered representation. The score is calculated as

\[- \frac{z \cdot w_c}{\lVert z \rVert_2} - \alpha \lVert h \rVert_1\]

where \(w_c\) is the weight vector for the class that the model predicted for the input, and \(\alpha\) is a hyper parameter that has to be determined manually.

The first term will penalize inputs whose representation does not align with the class vectors, while the second term penalizes inputs whose representation resides close to the origin.

See Paper:

CVPR

See Implementation:

Parameters:

encoder – model mapping inputs to features
head – the classification head of the model
alpha – weight for feature norm penalty. Will be ignored if \(\leq 0\)

property device: The device of the detector’s owned torch state, if one can be inferred.

fit(data_loader) → Self[source]

Parameters:: data_loader – data loader used to compute \(\mu_g\)

fit_features(z: Tensor, *args, **kwargs) → Self[source]

Parameters:: z – input features used to compute \(\mu_g\)

predict(x: Tensor) → Tensor[source]

Calculate outlier score for inputs, which will be passed through the encoder.

Parameters:: x – input tensor, will be passed through model
Returns:: outlier score

predict_features(*args, **kwargs)

Compute outlier scores based on features (without passing through encoder).

Parameters:: features – features given by the model

requires_fit = True: Whether fit(...) must be called before scoring.

to(device) → Self

Move detector-owned modules and tensor state to device.

This is a detector-level analogue of nn.Module.to(...). It moves modules, tensors, and common container-valued state stored on the detector itself.

Parameters:: device – target torch device
Returns:: self

Fast Decision Boundary Distance (fDBD)

class pytorch_ood.detector.fDBD(encoder: Module, head: Linear)[source]

Implements the Fast Decision Boundary Distance detector from the paper Fast Decision Boundary based Out-of-Distribution Detector.

Computes the closed-form distance from each sample’s penultimate-layer features to the decision boundaries of the linear classification head, averaged over all non-predicted classes and normalized by the distance to the training feature mean.

The score for a sample with features \(z\) and predicted class \(\hat{y}\) is:

\[- \frac{1}{|C|-1} \sum_{c \neq \hat{y}} \frac{| \text{logit}_{\hat{y}} - \text{logit}_c |} {\lVert w_{\hat{y}} - w_c \rVert_2 \cdot \lVert z - \mu \rVert_2}\]

where \(w_k\) are the weight vectors of the classification head and \(\mu\) is the mean of training features. This method is hyperparameter-free.

See Paper:

See Implementation:

Parameters:

encoder – model mapping inputs to penultimate-layer features
head – the linear classification head of the model

property device: The device of the detector’s owned torch state, if one can be inferred.

fit(data_loader: DataLoader) → Self[source]

Compute the training feature mean \(\mu\).

Parameters:: data_loader – data loader with training data

fit_features(z: Tensor, *args, **kwargs) → Self[source]

Compute the training feature mean directly from features.

Parameters:: z – training features

predict(x: Tensor) → Tensor[source]

Parameters:: x – input tensor, will be passed through the encoder

predict_features(*args, **kwargs)

Compute outlier scores from features.

Parameters:: z – penultimate-layer features
Returns:: outlier scores (higher = more OOD)

requires_fit = True: Whether fit(...) must be called before scoring.

to(device) → Self

Move detector-owned modules and tensor state to device.

This is a detector-level analogue of nn.Module.to(...). It moves modules, tensors, and common container-valued state stored on the detector itself.

Parameters:: device – target torch device
Returns:: self

Gaussian Mixture Model (GMM)

class pytorch_ood.detector.GMM(model: Callable[[Tensor], Tensor] | None, reg: float = 1e-06)[source]

Implements a class-conditional Gaussian Mixture Model (GMM) for Out-of-Distribution Detection.

Fits one Gaussian per class on penultimate-layer features, computing per-class means, covariance matrices, and mixing weights from the training data. The outlier score is the negative log-likelihood under the mixture:

\[-\log \sum_{k=1}^{K} \pi_k \, \mathcal{N}(z \mid \mu_k, \Sigma_k)\]

This extends Mahalanobis by allowing per-class covariance matrices and using the full mixture likelihood (logsumexp) instead of the max over classes.

Parameters:

model – neural network to use for feature extraction (can be None for feature-based interface)
reg – regularization added to the diagonal of each covariance matrix for numerical stability

property device: The device of the detector’s owned torch state, if one can be inferred.

fit(data_loader: DataLoader) → Self[source]

Extract features and fit the GMM.

Parameters:: data_loader – data loader with training data

fit_features(z: Tensor, labels: Tensor) → Self[source]

Fit one Gaussian per class directly on features. OOD-labeled samples are ignored.

Parameters:

z – features
labels – class labels

predict(x: Tensor) → Tensor[source]

Parameters:: x – input tensor, will be passed through the model

predict_features(*args, **kwargs)

Calculate outlier scores from features using the negative GMM log-likelihood.

Parameters:: z – features
Returns:: outlier scores (higher = more OOD)

requires_fit = True: Whether fit(...) must be called before scoring.

to(device) → Self

Move detector-owned modules and tensor state to device.

This is a detector-level analogue of nn.Module.to(...). It moves modules, tensors, and common container-valued state stored on the detector itself.

Parameters:: device – target torch device
Returns:: self

Gradient-based

Gradient-based detectors are based on the observation that the gradients (w.r.t. the model parameters or the inputs) for ID and OOD data behave differently.

GradNorm

class pytorch_ood.detector.GradNorm(model: Module, param_filter: Callable[[str], bool] | None = None)[source]

Bases: Detector

Detector from the paper Gradients as a Measure of Uncertainty in Neural Networks.

For each input sample, computes the binary cross-entropy loss between logits and a “confounding label”, which is a vector of all ones. Then, for each set of parameters in the model (as given by model.named_parameters()), computes up the squared \(\ell_2\)-norm of the gradients of the loss w.r.t. that parameter. The outlier score is the sum of these squared norms.

The idea is that higher gradient norms indicates that the model would require large parameter updates to accommodate the input, i.e., for such data, it is less familiar or more uncertain, and hence more likely to be OOD.

Note

OpenOOD uses only the gradients of the final classification head, which makes this computationally cheaper. You can achieve something similar by setting param_filter. Still, this method will compute gradients for all parameters unless you explicitly deactivate gradient calculation for parameters. For an example, see here

Note

On PyTorch ≥ 2.0, per-sample gradients are computed with torch.func.vmap + torch.func.grad in a single batched forward+backward pass. On PyTorch 1.x the original sequential loop over individual samples is used as a fallback.

Warning

The paper’s actual experiments (Section 4) concatenate the per-layer squared L2 norms into a feature vector and then train a 2-layer FC binary classifier on labeled ID and OOD gradient representations. The current implementation is a significant simplification: it sums all norms into a single scalar and uses it as a direct outlier score without any training. This simplification requires no OOD data but tends to perform poorly (AUROC ≈ 0.5) when ID and OOD datasets are of similar complexity, because the scalar sum loses the per-layer discriminative structure the classifier exploits. For an unsupervised gradient-based alternative see GradNormKL.

See Paper:

ICIP

Parameters:

model – A pre-trained classification model
param_filter – Function which indicates whether a named parameter should be included in the scoring. If none give, all parameters will be used.

property device: The device of the detector’s owned torch state, if one can be inferred.

predict(x: Tensor) → Tensor[source]

Compute outlier scores from input batch.

We will use the device of the model parameters for computations. On PyTorch ≥ 2.0, per-sample gradients are batched via torch.func; on older versions a sequential loop is used.

Parameters:: x – input, will be passed through network
Returns:: vector of outlier scores

requires_fit = False: Whether fit(...) must be called before scoring.

to(device) → Self

Move detector-owned modules and tensor state to device.

This is a detector-level analogue of nn.Module.to(...). It moves modules, tensors, and common container-valued state stored on the detector itself.

Parameters:: device – target torch device
Returns:: self

GradNormKL

class pytorch_ood.detector.GradNormKL(model: Module, param_filter: Callable[[str], bool] | None = None)[source]

Bases: Detector

Detector from the paper On the Importance of Gradients for Detecting Distributional Shifts in the Wild.

For each input sample, computes the KL divergence between the softmax output and a uniform distribution (implemented via binary cross-entropy with a uniform confounding label of \(1/C\) per class). The outlier score is the negated \(\ell_1\)-norm of the gradients of this loss w.r.t. the selected model parameters.

The key insight is that the gradient w.r.t. the logits simplifies to \(\text{softmax}(z) - 1/C\), which is zero when the model predicts a uniform distribution and grows as the prediction becomes more peaked. For in-distribution inputs the model is typically more confident (larger gradient norm) than for OOD inputs, so the negated norm gives higher scores to OOD samples, consistent with the convention that higher outlier scores indicate OOD data.

Note

The paper recommends using only the gradients of the final classification head (last FC layer) for computational efficiency. You can achieve this by setting param_filter and disabling gradient computation for the backbone via model.requires_grad_(False); model.fc.requires_grad_(True).

Note

On PyTorch ≥ 2.0, per-sample gradients are computed with torch.func.vmap + torch.func.grad in a single batched forward+backward pass. On PyTorch 1.x the original sequential loop over individual samples is used as a fallback.

See Paper:

NeurIPS

Parameters:

model – A pre-trained classification model.
param_filter – Function indicating whether a named parameter should be included in the scoring. If None, all parameters are used.

property device: The device of the detector’s owned torch state, if one can be inferred.

predict(x: Tensor) → Tensor[source]

Compute outlier scores for an input batch.

Uses the device of the model parameters for all computations. On PyTorch ≥ 2.0, per-sample gradients are batched via torch.func; on older versions a sequential loop is used.

Parameters:: x – input tensor, will be passed through the network
Returns:: vector of outlier scores (higher = more likely OOD)

requires_fit = False: Whether fit(...) must be called before scoring.

to(device) → Self

Move detector-owned modules and tensor state to device.

This is a detector-level analogue of nn.Module.to(...). It moves modules, tensors, and common container-valued state stored on the detector itself.

Parameters:: device – target torch device
Returns:: self

ODIN Preprocessing

class pytorch_ood.detector.ODIN(model: Module, criterion: Callable[[Tensor], Tensor] | None = None, eps: float = 0.05, temperature: float = 1000.0, norm_std: List[float] | None = None)[source]

Bases: Detector

Implements ODIN from the paper Enhancing The Reliability of Out-of-distribution Image Detection in Neural Networks.

ODIN is a preprocessing method for inputs that aims to increase the discriminability of the softmax outputs for ID and OOD data.

The operation requires two forward and one backward pass.

\[\hat{x} = x - \epsilon \ \text{sign}(\nabla_x \mathcal{L}(f(x) / T, \hat{y}))\]

where \(\hat{y}\) is the predicted class of the network.

See Paper:

See Implementation:

Parameters:

model – module to backpropagate through
criterion – loss function \(\mathcal{L}\) to use. If None is given, we will use negative log likelihood
eps – step size \(\epsilon\) of the gradient descent step
temperature – temperature \(T\) to use for scaling
norm_std – standard deviations used for normalization

criterion: criterion \(\mathcal{L}\)

property device: The device of the detector’s owned torch state, if one can be inferred.

eps: size \(\epsilon\) of the gradient step in the input space

predict(x: Tensor) → Tensor[source]

Calculates softmax outlier scores on ODIN pre-processed inputs.

Parameters:: x – input tensor
Returns:: outlier scores for each sample

requires_fit = False: Whether fit(...) must be called before scoring.

temperature: temperature value \(T\)

to(device) → Self

Move detector-owned modules and tensor state to device.

This is a detector-level analogue of nn.Module.to(...). It moves modules, tensors, and common container-valued state stored on the detector itself.

Parameters:: device – target torch device
Returns:: self

pytorch_ood.detector.odin_preprocessing(model: Module, x: Tensor, y: Tensor | None = None, criterion: Callable[[Tensor], Tensor] | None = None, eps: float = 0.05, temperature: float = 1000, norm_std: List[float] | None = None)[source]

Functional version of ODIN.

Parameters:

model – module to backpropagate through
x – sample to preprocess
y – the label \(\hat{y}\) which is used to evaluate the loss. If none is given, the models prediction will be used
criterion – loss function \(\mathcal{L}\) to use. If none is given, we will use negative log likelihood
eps – step size \(\epsilon\) of the gradient ascend step
temperature – temperature \(T\) to use for scaling
norm_std – standard deviations used during preprocessing

NAC-UE

Bases: Detector

Neuron Activation Coverage from the paper Neuron Activation Coverage: Rethinking Out-of-Distribution Detection and Generalization

See Paper:

ICLR

Parameters:

model – A classifier that returns logits of shape \((B, C)\), where \(B\) denotes the batch size and \(C\) the number of classes.
layers – Sequence of modules whose outputs \(z\) are used to compute NAC. For a ResNet-style architecture, e.g. [model.layer1, model.layer2, model.layer3, model.layer4].
m_bins – Number of histogram bins \(M\). Either a single value (shared across all layers) or one value per layer.
alpha – Sigmoid steepness parameter \(\alpha\). Either a single value (shared across all layers) or one value per layer.
o_star – Bin-filling parameter \(O^*\) (minimum count required for full coverage). Either a single value (shared across all layers) or one value per layer.
feature_reduce – Function mapping a layer output tensor to a 2D tensor of shape \((B, N)\), where \(B\) denotes the batch size and \(N\) the number of neurons. Defaults to: identity for tensors of shape \((B, N)\), spatial mean for tensors of shape \((B, C, H, W)\), otherwise flatten.
device – Optional device used during fitting and prediction.

property device: The device of the detector’s owned torch state, if one can be inferred.

fit(data_loader: DataLoader) → NACUE[source]

Fit the detector to a dataset. Some methods require this.

Parameters:: data_loader – dataset to fit on. This is usually the training dataset.
Raises:: ModelNotSetException – if model was not set

predict(x: Tensor) → Tensor[source]

Calculates outlier scores. Inputs will be passed through the model.

Parameters:

x – batch of data

Returns:

outlier scores for points

Raises:

RequiresFitException – if detector has to be fitted to some data
ModelNotSetException – if model was not set

requires_fit = True: Whether fit(...) must be called before scoring.

to(device) → Self

Move detector-owned modules and tensor state to device.

This is a detector-level analogue of nn.Module.to(...). It moves modules, tensors, and common container-valued state stored on the detector itself.

Parameters:: device – target torch device
Returns:: self

Activation Pruning

Activation pruning methods are based on the observation that OOD inputs cause unusual activations in the model, and that, by rectifying these unusual activations, we can often improve discriminability of ID and OOD samples.

Activation Shaping (ASH)

class pytorch_ood.detector.ASH(backbone: Callable[[Tensor], Tensor], head: Callable[[Tensor], Tensor], variant='ash-s', percentile: float = 0.65, detector: Callable[[Tensor], Tensor] | None = None)[source]

Implements ASH from the paper Extremely Simple Activation Shaping for Out-of-Distribution Detection.

ASH prunes the activations in some layer of the network (backbone) by removing a certain percentile of the highest activations. The remaining activations are modified, depending on the particular variant selected, and propagated through the remainder (head) of the network. Then uses the energy based outlier score. This approach has been shown to increase OOD detection rates while maintaining ID accuracy.

ASH-P: only prune, do not modify
ASH-B: binarize remaining activations
ASH-S: rescale remaining activations

The paper applies ASH after the last average pooling layer.

Example Code:

model = WideResNet()
detector = ASH(
    backbone = model.features_before_pool,
    head = model.forward_from_before_pool,
    detector=EnergyBased.score
)
scores = detector(images)

See Paper:

ICLR 2023

See Website:

github.io

Parameters:

variant – one of ash-p, ash-b, ash-s
backbone – first part of model to use, should output feature maps
head – second part of model used after applying ash, should output logits
percentile – amount of activations to modify
detector – detector that maps model outputs to outlier scores. Default is Energy based.

property device: The device of the detector’s owned torch state, if one can be inferred.

fit_feature_maps(feature_maps: Tensor, y: Tensor) → Self

Fit the detector directly on feature maps.

Parameters:

feature_maps – training feature maps to use for fitting.
y – corresponding class labels.

predict(x: Tensor) → Tensor[source]

Parameters:: x – input, will be passed through network

predict_feature_maps(*args, **kwargs)

Calculates outlier scores directly from feature maps.

Parameters:: feature_maps – batch of feature maps
Returns:: outlier scores for points

requires_fit = False: Whether fit(...) must be called before scoring.

to(device) → Self

Move detector-owned modules and tensor state to device.

This is a detector-level analogue of nn.Module.to(...). It moves modules, tensors, and common container-valued state stored on the detector itself.

Parameters:: device – target torch device
Returns:: self

ReAct

class pytorch_ood.detector.ReAct(backbone: Callable[[Tensor], Tensor], head: Callable[[Tensor], Tensor], threshold: float = 1.0, detector: Callable[[Tensor], Tensor] | None = None)[source]

Implements ReAct from the paper ReAct: Out-of-distribution Detection With Rectified Activations.

ReAct clips the activations in some layer of the network (backbone) and forward propagates the result through the remainder of the model (head). In the paper, ReAct is applied to the penultimate layer of the network.

The output of the network is then passed to an outlier detector that maps the output of the model to outlier scores.

Example Code:

model = WideResNet()
detector = ReAct(
    backbone = model.features,
    head = model.fc,
    detector = EnergyBased.score
)
scores = detector(images)

See Paper:

Parameters:

backbone – first part of model to use, should output feature maps
head – second part of model used after applying ash, should output logits
threshold – cutoff for activations
detector – detector that maps outputs to outlier scores. Default is energy based.

property device: The device of the detector’s owned torch state, if one can be inferred.

fit_feature_maps(feature_maps: Tensor, y: Tensor) → Self

Fit the detector directly on feature maps.

Parameters:

feature_maps – training feature maps to use for fitting.
y – corresponding class labels.

predict(x: Tensor) → Tensor[source]

Parameters:: x – input, will be passed through network

predict_feature_maps(*args, **kwargs)

Raises:: NotImplementedError

requires_fit = False: Whether fit(...) must be called before scoring.

to(device) → Self

Move detector-owned modules and tensor state to device.

This is a detector-level analogue of nn.Module.to(...). It moves modules, tensors, and common container-valued state stored on the detector itself.

Parameters:: device – target torch device
Returns:: self

DICE

class pytorch_ood.detector.DICE(model: Callable[[Tensor], Tensor] | None, w: Tensor, b: Tensor, p: float, detector: Callable[[Tensor], Tensor] | None = None)[source]

Implements DICE from the paper DICE: Leveraging Sparsification for Out-of-Distribution Detection.

See Paper:

Parameters:

model – feature extractor. Can be None when using fit_features(...) and predict_features(...) directly.
w – weights of last layer
b – bias of last layer
p – percentile of weights to drop

property device: The device of the detector’s owned torch state, if one can be inferred.

fit(data_loader: DataLoader) → Self[source]

Parameters:: data_loader – data loader to extract features from. OOD inputs will be ignored.

fit_features(z: Tensor, y: Tensor) → Self[source]

Calculates the masked weights. OOD Inputs will be ignored.

Parameters:

z – features
y – labels.

predict(x: Tensor) → Tensor[source]

Parameters:: x – input, will be passed through network

predict_features(*args, **kwargs)

Parameters:: x – features

requires_fit = True: Whether fit(...) must be called before scoring.

to(device) → Self

Move detector-owned modules and tensor state to device.

This is a detector-level analogue of nn.Module.to(...). It moves modules, tensors, and common container-valued state stored on the detector itself.

Parameters:: device – target torch device
Returns:: self

RankFeat

class pytorch_ood.detector.RankFeat(backbone: Callable[[Tensor], Tensor], head: Callable[[Tensor], Tensor], detector: Callable[[Tensor], Tensor] | None = None)[source]

Implements RankFeat from Rankfeat: Rank-1 Feature Removal for Out-of-Distribution Detection.

RankFeat removes the dominant rank-1 component from intermediate feature maps via SVD before forwarding through the remainder of the network. The intuition is that the leading singular vector captures generic, class-agnostic patterns shared between ID and OOD data. Removing it exposes subtler, class-specific structure that the energy score can exploit for better discrimination.

Concretely, given a feature map \(\mathbf{X} \in \mathbb{R}^{C \times HW}\), the method computes its (economy) SVD and subtracts the rank-1 approximation:

\[\mathbf{X}' = \mathbf{X} - \sigma_1 \, \mathbf{u}_1 \, \mathbf{v}_1^\top\]

The modified features \(\mathbf{X}'\) are then forwarded through the classification head, and the resulting logits are scored with the energy function.

Like ASH and ReAct, the model must be split into a backbone (up to and including the target convolutional block) and a head (the remaining layers including the classifier).

Example Code:

model = WideResNet()
detector = RankFeat(
    backbone=model.features_before_pool,
    head=model.forward_from_before_pool,
)
scores = detector(images)

See Paper:

NeurIPS 2022

See Implementation:

Parameters:

backbone – first part of the model, should output 4-D feature maps (B, C, H, W)
head – second part of the model applied after rank-1 removal, should output logits
detector – scoring function mapping logits to outlier scores. Default is score().

property device: The device of the detector’s owned torch state, if one can be inferred.

fit_feature_maps(feature_maps: Tensor, y: Tensor) → Self

Fit the detector directly on feature maps.

Parameters:

feature_maps – training feature maps to use for fitting.
y – corresponding class labels.

predict(x: Tensor) → Tensor[source]

Parameters:: x – input, will be passed through network
Returns:: outlier scores

predict_feature_maps(*args, **kwargs)

Calculates outlier scores directly from feature maps.

Parameters:: feature_maps – batch of feature maps
Returns:: outlier scores for points

requires_fit = False: Whether fit(...) must be called before scoring.

to(device) → Self

Move detector-owned modules and tensor state to device.

This is a detector-level analogue of nn.Module.to(...). It moves modules, tensors, and common container-valued state stored on the detector itself.

Parameters:: device – target torch device
Returns:: self

VRA

class pytorch_ood.detector.VRA(backbone: Callable[[Tensor], Tensor], head: Callable[[Tensor], Tensor], lower_percentile: float = 1.0, upper_percentile: float = 99.0, detector: Callable[[Tensor], Tensor] | None = None)[source]

Implements VRA from the paper Variational Rectified Activation for Out-of-Distribution Detection.

VRA is a two-sided version of ReAct that clips activations both above and below using percentile thresholds learned from In-Distribution data, then scores the result with an outlier detector (Energy-Based by default).

Unlike ReAct, which only clips activations from above at a fixed threshold, VRA learns per-dimension lower and upper clipping bounds from the training data using configurable percentiles.

Example Code:

model = WideResNet()
detector = VRA(
    backbone=model.features,
    head=model.fc,
)
detector.fit(train_loader)
scores = detector(images)

See Paper: