.. DO NOT EDIT. .. THIS FILE WAS AUTOMATICALLY GENERATED BY SPHINX-GALLERY. .. TO MAKE CHANGES, EDIT THE SOURCE PYTHON FILE: .. "auto_examples/segmentation/street_coco_transform.py" .. LINE NUMBERS ARE GIVEN BELOW. .. only:: html .. note:: :class: sphx-glr-download-link-note :ref:`Go to the end ` to download the full example code. .. rst-class:: sphx-glr-example-title .. _sphx_glr_auto_examples_segmentation_street_coco_transform.py: StreetHazards + COCO objects ----------------------------------------------- We train a Feature Pyramid Segmentation model with a ResNet-50 backbone pre-trained on the ImageNet on the :class:`StreetHazards`. During training, we insert random COCO objects as anomalies into the image to regularize the model. .. warning :: The results produced by this script vary. It is impossible to ensure the reproducibility of the exact numerical values at the moment, because the model includes operations for which no deterministic implementation exists at the time of writing. .. GENERATED FROM PYTHON SOURCE LINES 15-37 .. code-block:: Python :lineno-start: 15 import segmentation_models_pytorch as smp import torch from segmentation_models_pytorch.encoders import get_preprocessing_fn from segmentation_models_pytorch.metrics import iou_score from torch.utils.data import DataLoader from torchvision.transforms.functional import pad, to_tensor from pytorch_ood.dataset.img import StreetHazards from pytorch_ood.detector import Entropy from pytorch_ood.loss import EntropicOpenSetLoss from pytorch_ood.utils import OODMetrics, fix_random_seed from pytorch_ood.utils.transforms import InsertCOCO device = "cuda:0" batch_size = 4 num_epochs = 1 fix_random_seed(12345) g = torch.Generator() g.manual_seed(0) .. GENERATED FROM PYTHON SOURCE LINES 38-39 Setup preprocessing .. GENERATED FROM PYTHON SOURCE LINES 39-63 .. code-block:: Python :lineno-start: 39 preprocess_input = get_preprocessing_fn("resnet50", pretrained="imagenet") # for demonstration purposes, we set the probability of OOD to 1 coco_transform = InsertCOCO( coco_dir="data/coco", exclude_classes="Streethazards", p=1, ) def my_transform(img, target, use_coco_transform): if use_coco_transform: img, target = coco_transform(img, target) img = to_tensor(img)[:3, :, :] # drop 4th channel img = torch.moveaxis(img, 0, -1) img = preprocess_input(img) img = torch.moveaxis(img, -1, 0) # size must be divisible by 32, so we pad the image. img = pad(img, [0, 8]).float() target = pad(target, [0, 8]) return img, target .. GENERATED FROM PYTHON SOURCE LINES 64-65 Setup datasets .. GENERATED FROM PYTHON SOURCE LINES 65-79 .. code-block:: Python :lineno-start: 65 dataset = StreetHazards( root="data", subset="train", transform=lambda img, target: my_transform(img, target, True), download=True, ) dataset_test = StreetHazards( root="data", subset="test", transform=lambda img, target: my_transform(img, target, False), download=True, ) .. GENERATED FROM PYTHON SOURCE LINES 80-81 Setup model .. GENERATED FROM PYTHON SOURCE LINES 81-88 .. code-block:: Python :lineno-start: 81 model = smp.FPN( encoder_name="resnet50", encoder_weights="imagenet", in_channels=3, classes=13, ).to(device) .. GENERATED FROM PYTHON SOURCE LINES 89-90 Train model for some epochs .. GENERATED FROM PYTHON SOURCE LINES 90-131 .. code-block:: Python :lineno-start: 90 criterion = EntropicOpenSetLoss() optimizer = torch.optim.Adam(params=model.parameters(), lr=0.0001) loader = DataLoader( dataset, batch_size=batch_size, shuffle=True, num_workers=10, worker_init_fn=fix_random_seed, generator=g, ) ious = [] loss_ema = 0 ioe_ema = 0 for epoch in range(num_epochs): for n, (x, y) in enumerate(loader): optimizer.zero_grad() y, x = y.to(device), x.to(device) y_hat = model(x) loss = criterion(y_hat, y) loss.backward() optimizer.step() tp, fp, fn, tn = smp.metrics.get_stats( y_hat.softmax(dim=1).max(dim=1).indices.long(), y.long(), mode="multiclass", num_classes=13, ) iou = iou_score(tp, fp, fn, tn) loss_ema = 0.8 * loss_ema + 0.2 * loss.item() ioe_ema = 0.8 * ioe_ema + 0.2 * iou.mean().item() if n % 10 == 0: print( f"Epoch {epoch:03d} [{n:05d}/{len(loader):05d}] \t Loss: {loss_ema:02.2f} \t IoU: {ioe_ema:02.2f}" ) .. GENERATED FROM PYTHON SOURCE LINES 132-133 Evaluate .. GENERATED FROM PYTHON SOURCE LINES 133-152 .. code-block:: Python :lineno-start: 133 print("Evaluating") model.eval() loader = DataLoader(dataset_test, batch_size=4, worker_init_fn=fix_random_seed, generator=g) detector = Entropy(model) metrics = OODMetrics(mode="segmentation") with torch.no_grad(): for n, (x, y) in enumerate(loader): y, x = y.to(device), x.to(device) o = detector(x) # undo padding o = pad(o, [-8, -8]) y = pad(y, [-8, -8]) metrics.update(o, y) print(metrics.compute()) .. GENERATED FROM PYTHON SOURCE LINES 153-155 Output: {'AUROC': 0.9573410749435425, 'AUPR-IN': 0.5151191353797913, 'AUPR-OUT': 0.9991346001625061, 'FPR95TPR': 0.16139476001262665} .. _sphx_glr_download_auto_examples_segmentation_street_coco_transform.py: .. only:: html .. container:: sphx-glr-footer sphx-glr-footer-example .. container:: sphx-glr-download sphx-glr-download-jupyter :download:`Download Jupyter notebook: street_coco_transform.ipynb ` .. container:: sphx-glr-download sphx-glr-download-python :download:`Download Python source code: street_coco_transform.py ` .. container:: sphx-glr-download sphx-glr-download-zip :download:`Download zipped: street_coco_transform.zip ` .. only:: html .. rst-class:: sphx-glr-signature `Gallery generated by Sphinx-Gallery `_