Models

Publications frequently use the same models, however, hyperparameters, pre-processing differ and are sometimes cumbersome to set up.

The purpose of this module is to minimize the effort required to reproduce the experiments of others by providing models, pre-processing and weights, as used in the original publications.

Vision

Wide ResNet

class pytorch_ood.model.WideResNet(num_classes, depth=40, widen_factor=2, drop_rate=0.3, in_channels=3, pretrained=None)[source]

Resnet Architecture with large number of channels and variable depth, which has been used in a number of publications.

Provides a number of pre-trained weights for models trained with OutlierExposureLoss, Energy Regularization or PixMix. Also includes models pre-trained on the variants on the ImageNet, which is known to increase the robustness.

See Paper:

BMVC

See Implementation:

GitHub

Parameters:

  • depth – depth of the network

  • num_classes – number of classes

  • widen_factor – factor used for channel increase per block

  • drop_rate – dropout probability

  • in_channels – number of input planes

  • pretrained – identifier of pretrained weights to load

Pretrained weights are taken from the corresponding publications.

Available Pre-Trained weights

Key

Paper

Description

imagenet32

Here

Pre-Trained on a downscaled version (\(32 \times 32\)) of the ImageNet.

imagenet32-nocifar

Here

Pre-Trained on a downscaled version (\(32 \times 32\)) of the ImageNet, excluding CIFAR-10 classes.

oe-cifar100-tune

Here

Model trained with Outlier Exposure using the 80 million TinyImages database on the CIFAR-100.

oe-cifar10-tune

Here

Model trained with Outlier Exposure using the 80 million TinyImages database on the CIFAR-10.

er-cifar10-tune

Here

Model trained with Energy Regularization using the 80 million TinyImages database on the CIFAR-10.

er-cifar100-tune

Here

Model trained with Energy Regularization using the 80 million TinyImages database on the CIFAR-100.

cifar100-pt

Here

Pre-Trained model for CIFAR-100.

cifar10-pt

Here

Pre-Trained model for CIFAR-10.

cifar10-pixmix

Here

Model trained with PixMix on CIFAR-10. widen_factor=4

cifar100-pixmix

Here

Model trained with PixMix on CIFAR-100. widen_factor=4
feature_list(x: Tensor) → List[Tensor][source]

Extracts features after encoder, pooling, and fully connected layer

features(x: Tensor) → Tensor[source]

Extracts (flattened) features before the last fully connected layer.

forward(x: Tensor) → Tensor[source]

Forward propagate

Parameters:

x – input images

Returns:

class logits

static norm_std_for(pretrained: str) → List[float][source]

Return normalization standard deviation values for pretrained model. This is sometimes required, for example for pytorch_ood.detector.ODIN.

static transform_for(pretrained: str) → Compose[source]

Return pre-processing used for the evaluation of a pretrained model

Language

Models used in pre-LLM papers for OOD detection.

GRU Classifier

class pytorch_ood.model.GRUClassifier(num_classes, n_vocab, embedding_dim=50)[source]

Classifier with token embedding and multi layer gated recurrent unit (GRU) for text classification, as used in the OOD/Error Detection baseline paper.

See Implementation:

GitHub

Parameters:

  • num_classes – number of classes in the dataset

  • n_vocab – size of the vocabulary, i.e. number of distinct tokens

  • embedding_dim – embedding size

features(x: Tensor) → Tensor[source]
Parameters:

x – batch of tokens

Returns:

features

forward(x: Tensor) → Tensor[source]
Parameters:

x – batch of tokens

Returns:

logits

Modules

Neural Network modules frequently used in OOD detection.

Class Centers

class pytorch_ood.model.ClassCenters(n_classes: int, n_features: int, fixed: bool = False)[source]

Several methods for OOD Detection propose to model a center \(\mu_y\) for each class. These centers are either static, or learned via gradient descent.

The centers are also known as class proxy, class prototype or class anchor.

Parameters:

  • n_classes – number of classes vectors

  • n_features – dimensionality of the space in which the centers live

  • fixed – False if the centers should be learnable parameters, True if they should be fixed at their initial position

forward(x: Tensor) → Tensor[source]
Parameters:

x – samples

Returns:

pairwise squared distance of samples to each center

property params: Parameter

Class centers \(\mu\)

predict(x: Tensor) → Tensor[source]

Make class membership predictions based on the softmin of the distances to each center.

Parameters:

x – embeddings of samples

Returns:

normalized pairwise distance of samples to each center

class pytorch_ood.model.RunningCenters(n_classes: int, n_embedding: int)[source]

Estimates class centers from batches of data using a running mean estimator.

Parameters:

  • n_classes – number of centers

  • n_embedding – dimensionality of embedding space

property centers: Tensor
Returns:

current class center estimates

forward(x: Tensor)[source]

Calculates distances to centers

Parameters:

x

Returns:

distance matrix

reset() → None[source]

Resets the running stats of online class center estimates.

update(x: Tensor, target: Tensor) → Tensor[source]

Update running centers

Parameters:

  • x – inputs

  • target – class labels

Returns:

per class mean of inputs