.. DO NOT EDIT.
.. THIS FILE WAS AUTOMATICALLY GENERATED BY SPHINX-GALLERY.
.. TO MAKE CHANGES, EDIT THE SOURCE PYTHON FILE:
.. "auto_examples/benchmarks/interface/imagenet_openood.py"
.. LINE NUMBERS ARE GIVEN BELOW.

.. only:: html

    .. note::
        :class: sphx-glr-download-link-note

        :ref:`Go to the end <sphx_glr_download_auto_examples_benchmarks_interface_imagenet_openood.py>`
        to download the full example code.

.. rst-class:: sphx-glr-example-title

.. _sphx_glr_auto_examples_benchmarks_interface_imagenet_openood.py:


OpenOOD v1.5 - ImageNet
========================

Reproduces the OpenOOD v1.5 benchmark for OOD detection on ImageNet, using a pre-trained ResNet 50.

.. GENERATED FROM PYTHON SOURCE LINES 8-23

.. code-block:: Python
   :lineno-start: 9


    import pandas as pd  # additional dependency, used here for convenience
    import torch
    from torchvision.models import resnet50
    from torchvision.models.resnet import ResNet50_Weights

    from pytorch_ood.benchmark import ImageNet_OpenOOD
    from pytorch_ood.detector import MaxSoftmax
    from pytorch_ood.utils import fix_random_seed

    fix_random_seed(123)

    device = "cuda:0"
    loader_kwargs = {"batch_size": 16, "num_workers": 12}


.. GENERATED FROM PYTHON SOURCE LINES 24-29

.. code-block:: Python
   :lineno-start: 24

    model = resnet50(ResNet50_Weights.IMAGENET1K_V1).eval().to(device)
    trans = ResNet50_Weights.IMAGENET1K_V1.transforms()

    print(trans)


.. GENERATED FROM PYTHON SOURCE LINES 30-31

If you want to test more detectors, you can just add them here

.. GENERATED FROM PYTHON SOURCE LINES 31-35

.. code-block:: Python
   :lineno-start: 31

    detectors = {
        "MSP": MaxSoftmax(model),
    }


.. GENERATED FROM PYTHON SOURCE LINES 36-38

The ImageNet root should contain at least the validation tar, the dev kit tar, and the meta.bin
that is generated by the torchvision ImageNet implementation.

.. GENERATED FROM PYTHON SOURCE LINES 38-53

.. code-block:: Python
   :lineno-start: 38

    results = []
    benchmark = ImageNet_OpenOOD(root="data", image_net_root="data/imagenet-2012/", transform=trans)


    with torch.no_grad():
        for detector_name, detector in detectors.items():
            print(f"> Evaluating {detector_name}")
            res = benchmark.evaluate(detector, loader_kwargs=loader_kwargs, device=device)
            for r in res:
                r.update({"Detector": detector_name})
            results += res

    df = pd.DataFrame(results)
    print((df.set_index(["Dataset", "Detector"]) * 100).to_csv(float_format="%.2f"))


.. GENERATED FROM PYTHON SOURCE LINES 54-62

This should produce a table with results for the following OOD datasets:

* SSBHard (near-OOD)
* NINCO (near-OOD)
* iNaturalist (far-OOD)
* Textures (far-OOD)
* OpenImagesO (far-OOD)



.. _sphx_glr_download_auto_examples_benchmarks_interface_imagenet_openood.py:

.. only:: html

  .. container:: sphx-glr-footer sphx-glr-footer-example

    .. container:: sphx-glr-download sphx-glr-download-jupyter

      :download:`Download Jupyter notebook: imagenet_openood.ipynb <imagenet_openood.ipynb>`

    .. container:: sphx-glr-download sphx-glr-download-python

      :download:`Download Python source code: imagenet_openood.py <imagenet_openood.py>`

    .. container:: sphx-glr-download sphx-glr-download-zip

      :download:`Download zipped: imagenet_openood.zip <imagenet_openood.zip>`


.. only:: html

 .. rst-class:: sphx-glr-signature

    `Gallery generated by Sphinx-Gallery <https://sphinx-gallery.github.io>`_