:orphan:


Image Classification
====================

The objective of this section is to outline a quick method for obtaining
baseline results for comparison purposes.

To run these examples, you have to install ``pandas`` as well as ``scikit-learn``
as additional dependencies:

.. code-block:: shell

    pip install pandas scikit-learn



.. raw:: html

    <div class="sphx-glr-thumbnails">

.. thumbnail-parent-div-open

.. thumbnail-parent-div-close

.. raw:: html

    </div>


Benchmark Interface
-------------------------

Ready-to-use benchmarks provide a simple interface to (approximately) replicate the experiments
of other publications. While they are convenient, this comes at the price of flexibility.



.. raw:: html

    <div class="sphx-glr-thumbnails">

.. thumbnail-parent-div-open

.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="Reproduces the OpenOOD v1.5 benchmark for OOD detection on CIFAR-10, using the WideResNet model from the Hendrycks baseline paper.">

.. only:: html

  .. image:: /auto_examples/benchmarks/interface/images/thumb/sphx_glr_cifar10_openood_thumb.png
    :alt:

  :ref:`sphx_glr_auto_examples_benchmarks_interface_cifar10_openood.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">OpenOOD v1.5 - CIFAR10</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="Reproduces the ODIN benchmark for OOD detection, from the paper Enhancing the reliability of out-of-distribution image detection in neural networks.">

.. only:: html

  .. image:: /auto_examples/benchmarks/interface/images/thumb/sphx_glr_cifar100_odin_thumb.png
    :alt:

  :ref:`sphx_glr_auto_examples_benchmarks_interface_cifar100_odin.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">ODIN - CIFAR100</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="Reproduces the OpenOOD v1.5 benchmark for OOD detection on ImageNet, using a pre-trained ResNet 50.">

.. only:: html

  .. image:: /auto_examples/benchmarks/interface/images/thumb/sphx_glr_imagenet_openood_thumb.png
    :alt:

  :ref:`sphx_glr_auto_examples_benchmarks_interface_imagenet_openood.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">OpenOOD v1.5 - ImageNet</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="Reproduces the ODIN benchmark for OOD detection, from the paper Enhancing the reliability of out-of-distribution image detection in neural networks.">

.. only:: html

  .. image:: /auto_examples/benchmarks/interface/images/thumb/sphx_glr_cifar10_odin_thumb.png
    :alt:

  :ref:`sphx_glr_auto_examples_benchmarks_interface_cifar10_odin.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">ODIN - CIFAR10</div>
    </div>


.. thumbnail-parent-div-close

.. raw:: html

    </div>


.. toctree::
   :hidden:

   /auto_examples/benchmarks/interface/cifar10_openood
   /auto_examples/benchmarks/interface/cifar100_odin
   /auto_examples/benchmarks/interface/imagenet_openood
   /auto_examples/benchmarks/interface/cifar10_odin


From Scratch
-------------------------

Code for manually running benchmarks. More boilerplate, but also more flexibility compared to the benchmark
interface.

We provide an example that replicates a commonly used benchmark that includes several Out-of-Distribution detectors,
each tested against several OOD datasets.
We subsequently calculate the average performance of each detector
across all datasets and sort the outcomes based on their
Area Under Receiver Operating Characteristic (AUROC) score in ascending order.



.. raw:: html

    <div class="sphx-glr-thumbnails">

.. thumbnail-parent-div-open

.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="The evaluation is the same as for CIFAR 10.">

.. only:: html

  .. image:: /auto_examples/benchmarks/manual/images/thumb/sphx_glr_cifar100_baseline_thumb.png
    :alt:

  :ref:`sphx_glr_auto_examples_benchmarks_manual_cifar100_baseline.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">CIFAR 100</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="Example benchmark code for CIFAR10">

.. only:: html

  .. image:: /auto_examples/benchmarks/manual/images/thumb/sphx_glr_cifar10_baseline_thumb.png
    :alt:

  :ref:`sphx_glr_auto_examples_benchmarks_manual_cifar10_baseline.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">CIFAR 10</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="Benchmark code for CIFAR-10 that reuses cached logits and pooled features for fitting detectors that expose the standard representation APIs. At evaluation time, cached logits are used for true logits detectors, while feature detectors that operate directly on pooled features can use the cached feature representation. Other detectors still use their regular predict() interface so their full model-time behavior is preserved.">

.. only:: html

  .. image:: /auto_examples/benchmarks/manual/images/thumb/sphx_glr_cifar10_cached_baseline_thumb.png
    :alt:

  :ref:`sphx_glr_auto_examples_benchmarks_manual_cifar10_cached_baseline.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">CIFAR 10 Cached Baseline</div>
    </div>


.. thumbnail-parent-div-close

.. raw:: html

    </div>


.. toctree::
   :hidden:

   /auto_examples/benchmarks/manual/cifar100_baseline
   /auto_examples/benchmarks/manual/cifar10_baseline
   /auto_examples/benchmarks/manual/cifar10_cached_baseline


.. only:: html

  .. container:: sphx-glr-footer sphx-glr-footer-gallery

    .. container:: sphx-glr-download sphx-glr-download-python

      :download:`Download all examples in Python source code: benchmarks_python.zip </auto_examples/benchmarks/benchmarks_python.zip>`

    .. container:: sphx-glr-download sphx-glr-download-jupyter

      :download:`Download all examples in Jupyter notebooks: benchmarks_jupyter.zip </auto_examples/benchmarks/benchmarks_jupyter.zip>`


.. only:: html

 .. rst-class:: sphx-glr-signature

    `Gallery generated by Sphinx-Gallery <https://sphinx-gallery.github.io>`_