Merge branch 'main' of https://github.com/fvmassoli/mocca-anomaly-det…

…ection into main

README.md

@@ -1,35 +1,42 @@
-# MOCCA: Multi-Layer One-Class Classification for Anomaly Detection
+# MOCCA: Multi-Layer One-Class ClassificAtion for Anomaly Detection
 
-This repository contains the code relative to the paper "[MOCCA: Multi-Layer One-Class Classification for Anomaly Detection](https://arxiv.org/abs/2012.12111)" by Fabio Valerio Massoli (ISTI - CNR), Fabrizio Falchi (ISTI - CNR), Alperen Kantarci (ITU), Şeymanur Akti (ITU), Hazim Kemal Ekenel (ITU), Giuseppe Amato (ISTI - CNR).
 
-It reports a new technique to detect anomalies based on a layer-wise paradigm to exploit the features maps generated at different depths of a Deep Learning model.
+<p align="center">
+<img src="https://github.com/fvmassoli/mocca-anomaly-detection/blob/main/images/mocca.png"  alt="MOCCA" width="700" height="450">
+</p>
 
-We are researchers, not a software company, and have no personnel devoted to documenting and maintaing this research code. Therefore this code is offered "AS IS". Exact reproduction of the numbers in the paper depends on exact reproduction of many factors, including the version of all software dependencies and the choice of underlying hardware (GPU model, etc). Therefore you should expect to need to re-tune your hyperparameters slight for your new setup.
 
 
-## Proposed Approach
+This repository contains the code relative to the paper "[MOCCA: Multi-Layer One-Class ClassificAtion for Anomaly Detection](https://arxiv.org/abs/2012.12111)" by Fabio Valerio Massoli (ISTI - CNR), Fabrizio Falchi (ISTI - CNR), Alperen Kantarci (ITU), Şeymanur Akti (ITU), Hazim Kemal Ekenel (ITU), Giuseppe Amato (ISTI - CNR).
 
+It reports a new technique to detect anomalies based on a layer-wise paradigm to exploit the features maps generated at different depths of a Deep Learning model.
 
-<p align="center">
-<img src="https://github.com/fvmassoli/mocca-anomaly-detection/blob/main/images/mocca.png"  alt="MOCCA" width="700" height="450">
-</p>
+**Please note:** 
+We are researchers, not a software company, and have no personnel devoted to documenting and maintaing this research code. Therefore this code is offered "AS IS". Exact reproduction of the numbers in the paper depends on exact reproduction of many factors, including the version of all software dependencies and the choice of underlying hardware (GPU model, etc). Therefore you should expect to need to re-tune your hyperparameters slightly for your new setup.
 
 
 ## How to run the code
 
+Before to run the code, make sure that your system has the proper packages installed. You can have a look at the [requirements.txt](https://github.com/fvmassoli/mocca-anomaly-detection/blob/main/requirements.txt) file.
+
 
 Minimal usage (CIFAR10):
 
 ```
-python3 main_cifar10.py -ptr -tr -tt -zl 128 -nc <normal class> -dp <path to CIFAR10 dataset>
+python main_cifar10.py -ptr -tr -tt -zl 128 -nc <normal class> -dp <path to CIFAR10 dataset>
 ```
 
 Minimal usage (MVTec):
 
 ```
-python3 main_mvtec.py -ptr -tr -tt -zl 128 -nc <normal class> -dp <path to CIFAR10 dataset> --use-selector
+python main_mvtec.py -ptr -tr -tt -zl 128 -nc <normal class> -dp <path to MVTec dataset> --use-selector
 ```
 
+Minimal usage (ShanghaiTech):
+
+```
+python main_shanghaitech.py -dp <path to ShanghaiTech dataset> -ee -tt -zl 1024 -ll -use 
+```
 
 ## Reference
 For all the details about the training procedure and the experimental results, please have a look at the [paper](https://arxiv.org/abs/2012.12111).

datasets/base.py

@@ -0,0 +1,183 @@
+from abc import ABCMeta
+from abc import abstractmethod
+
+import torch 
+import numpy as np 
+from torch.utils.data import Dataset
+
+
+class DatasetBase(Dataset):
+    """
+    Base class for all datasets.
+    """
+    __metaclass__ = ABCMeta
+
+    @abstractmethod
+    def test(self, *args):
+        """
+        Sets the dataset in test mode.
+        """
+        pass
+
+    @property
+    @abstractmethod
+    def shape(self):
+        """
+        Returns the shape of examples.
+        """
+        pass
+
+    @abstractmethod
+    def __len__(self):
+        """
+        Returns the number of examples.
+        """
+        pass
+
+    @abstractmethod
+    def __getitem__(self, i):
+        """
+        Provides the i-th example.
+        """
+        pass
+
+
+class OneClassDataset(DatasetBase):
+    """
+    Base class for all one-class classification datasets.
+    """
+    __metaclass__ = ABCMeta
+
+    @abstractmethod
+    def val(self, *args):
+        """
+        Sets the dataset in validation mode.
+        """
+        pass
+
+    @property
+    @abstractmethod
+    def test_classes(self):
+        """
+        Returns all test possible test classes.
+        """
+        pass
+
+
+class VideoAnomalyDetectionDataset(DatasetBase):
+    """
+    Base class for all video anomaly detection datasets.
+    """
+    __metaclass__ = ABCMeta
+
+    @property
+    @abstractmethod
+    def test_videos(self):
+        """
+        Returns all test video ids.
+        """
+        pass
+
+
+    @abstractmethod
+    def __len__(self):
+        """
+        Returns the number of examples.
+        """
+        pass
+
+    @property
+    def raw_shape(self):
+        """
+        Workaround!
+        """
+        return self.shape
+
+    @abstractmethod
+    def __getitem__(self, i):
+        """
+        Provides the i-th example.
+        """
+        pass
+
+    @abstractmethod
+    def load_test_sequence_gt(self, video_id):
+        # type: (str) -> np.ndarray
+        """
+        Loads the groundtruth of a test video in memory.
+        :param video_id: the id of the test video for which the groundtruth has to be loaded.
+        :return: the groundtruth of the video in a np.ndarray, with shape (n_frames,).
+        """
+        pass
+
+    @property
+    @abstractmethod
+    def collate_fn(self):
+        """
+        Returns a function that decides how to merge a list of examples in a batch.
+        """
+        pass
+
+
+class ToFloatTensor3D(object):
+    """ Convert videos to FloatTensors """
+    def __init__(self, normalize=True):
+        self._normalize = normalize
+
+    def __call__(self, sample):
+        if len(sample) == 3:
+            X, Y, _ = sample
+        else:
+            X = sample
+
+        # swap color axis because
+        # numpy image: T x H x W x C
+        X = X.transpose(3, 0, 1, 2)
+        #Y = Y.transpose(3, 0, 1, 2)
+
+        if self._normalize:
+            X = X / 255.
+
+        X = np.float32(X)
+        return torch.from_numpy(X)
+
+class ToFloatTensor3DMask(object):
+    """ Convert videos to FloatTensors """
+    def __init__(self, normalize=True, has_x_mask=True, has_y_mask=True):
+        self._normalize = normalize
+        self.has_x_mask = has_x_mask
+        self.has_y_mask = has_y_mask
+
+    def __call__(self, sample):
+        X = sample
+        # swap color axis because
+        # numpy image: T x H x W x C
+        X = X.transpose(3, 0, 1, 2)
+
+        X = np.float32(X)
+
+        if self._normalize:
+            if self.has_x_mask:
+                X[:-1] = X[:-1] / 255.
+            else:
+                X = X / 255.
+
+        return torch.from_numpy(X)
+
+
+class RemoveBackground:
+
+    def __init__(self, threshold: float):
+        self.threshold = threshold
+
+    def __call__(self, sample: tuple) -> tuple:
+        X, Y, background = sample
+
+        mask = np.uint8(np.sum(np.abs(np.int32(X) - background), axis=-1) > self.threshold)
+        mask = np.expand_dims(mask, axis=-1)
+
+        mask = np.stack([binary_dilation(mask_frame, iterations=5) for mask_frame in mask])
+
+        X *= mask
+
+        return X, Y, background

datasets/data_manager.py

@@ -4,9 +4,10 @@
 
 from .mvtec import MVTec_DataHolder
 from .cifar10 import CIFAR10_DataHolder
+from .shanghaitech import ShanghaiTech_DataHolder
 
 
-AVAILABLE_DATASETS = ('cifar10', 'shanghai', 'MVTec_Anomaly')
+AVAILABLE_DATASETS = ('cifar10', 'ShanghaiTech', 'MVTec_Anomaly')
 
 
 class DataManager(object):
@@ -58,8 +59,8 @@ def get_data_holder(self):
         if self.dataset_name == 'cifar10':
             return CIFAR10_DataHolder(root=self.data_path, normal_class=self.normal_class)
 
-        if self.dataset_name == 'shanghai':
-            raise NotImplementedError
+        if self.dataset_name == 'ShanghaiTech':
+            return ShanghaiTech_DataHolder(root=self.data_path,clip_length=self.clip_length)
 
         if self.dataset_name == 'MVTec_Anomaly':
             texture_classes = tuple(["carpet", "grid", "leather", "tile", "wood"])