:orphan:


Image Classification
====================

The objective of this section is to outline a quick method for obtaining
baseline results for comparison purposes.

To run these examples, you have to install ``pandas`` as well as ``scikit-learn``
as additional dependencies:

.. code-block:: shell

    pip install pandas scikit-learn



.. raw:: html

    <div class="sphx-glr-thumbnails">

.. thumbnail-parent-div-open

.. thumbnail-parent-div-close

.. raw:: html

    </div>


Benchmark Interface
-------------------------

Ready-to-use benchmarks provide a simple interface to (approximately) replicate the experiments
of other publications. While they are convenient, this comes at the price of flexibility.



.. raw:: html

    <div class="sphx-glr-thumbnails">

.. thumbnail-parent-div-open

.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="Reproduces the OpenOOD benchmark for OOD detection, using the WideResNet model from the Hendrycks baseline paper.">

.. only:: html

  .. image:: /auto_examples/benchmarks/interface/images/thumb/sphx_glr_cifar10_openood_thumb.png
    :alt:

  :ref:`sphx_glr_auto_examples_benchmarks_interface_cifar10_openood.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">OpenOOD - CIFAR10</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="Reproduces the ODIN benchmark for OOD detection, from the paper Enhancing the reliability of out-of-distribution image detection in neural networks.">

.. only:: html

  .. image:: /auto_examples/benchmarks/interface/images/thumb/sphx_glr_cifar100_odin_thumb.png
    :alt:

  :ref:`sphx_glr_auto_examples_benchmarks_interface_cifar100_odin.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">ODIN - CIFAR100</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="Reproduces the OpenOOD benchmark for OOD detection, using a pre-trained ResNet 50.">

.. only:: html

  .. image:: /auto_examples/benchmarks/interface/images/thumb/sphx_glr_imagenet_openood_thumb.png
    :alt:

  :ref:`sphx_glr_auto_examples_benchmarks_interface_imagenet_openood.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">OpenOOD - ImageNet</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="Reproduces the ODIN benchmark for OOD detection, from the paper Enhancing the reliability of out-of-distribution image detection in neural networks.">

.. only:: html

  .. image:: /auto_examples/benchmarks/interface/images/thumb/sphx_glr_cifar10_odin_thumb.png
    :alt:

  :ref:`sphx_glr_auto_examples_benchmarks_interface_cifar10_odin.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">ODIN - CIFAR10</div>
    </div>


.. thumbnail-parent-div-close

.. raw:: html

    </div>


.. toctree::
   :hidden:

   /auto_examples/benchmarks/interface/cifar10_openood
   /auto_examples/benchmarks/interface/cifar100_odin
   /auto_examples/benchmarks/interface/imagenet_openood
   /auto_examples/benchmarks/interface/cifar10_odin


From Scratch
-------------------------

Code for manually running benchmarks. More boilerplate, but also more flexibility compared to the benchmark
interface.

We provide an example that replicates a commonly used benchmark that includes several Out-of-Distribution detectors,
each tested against several OOD datasets.
We subsequently calculate the average performance of each detector
across all datasets and sort the outcomes based on their
Area Under Receiver Operating Characteristic (AUROC) score in ascending order.



.. raw:: html

    <div class="sphx-glr-thumbnails">

.. thumbnail-parent-div-open

.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="The evaluation is the same as for CIFAR 10.">

.. only:: html

  .. image:: /auto_examples/benchmarks/manual/images/thumb/sphx_glr_cifar100_baseline_thumb.png
    :alt:

  :ref:`sphx_glr_auto_examples_benchmarks_manual_cifar100_baseline.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">CIFAR 100</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="Example benchmark code for CIFAR10">

.. only:: html

  .. image:: /auto_examples/benchmarks/manual/images/thumb/sphx_glr_cifar10_baseline_thumb.png
    :alt:

  :ref:`sphx_glr_auto_examples_benchmarks_manual_cifar10_baseline.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">CIFAR 10</div>
    </div>


.. thumbnail-parent-div-close

.. raw:: html

    </div>


.. toctree::
   :hidden:

   /auto_examples/benchmarks/manual/cifar100_baseline
   /auto_examples/benchmarks/manual/cifar10_baseline


.. only:: html

  .. container:: sphx-glr-footer sphx-glr-footer-gallery

    .. container:: sphx-glr-download sphx-glr-download-python

      :download:`Download all examples in Python source code: benchmarks_python.zip </auto_examples/benchmarks/benchmarks_python.zip>`

    .. container:: sphx-glr-download sphx-glr-download-jupyter

      :download:`Download all examples in Jupyter notebooks: benchmarks_jupyter.zip </auto_examples/benchmarks/benchmarks_jupyter.zip>`


.. only:: html

 .. rst-class:: sphx-glr-signature

    `Gallery generated by Sphinx-Gallery <https://sphinx-gallery.github.io>`_