Skip to content

Commit

Permalink
Fix codestyle
Browse files Browse the repository at this point in the history
  • Loading branch information
Valeriy Zolkin committed Sep 6, 2022
1 parent 4820b19 commit d8d0df6
Show file tree
Hide file tree
Showing 85 changed files with 4,105 additions and 4,047 deletions.
30 changes: 15 additions & 15 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# _Multi-Camera Person Tracking and Re-Identification_ (using video)
[![HitCount](http://hits.dwyl.com/samihormi/https://githubcom/samihormi/Multi-Camera-Person-Tracking-and-Re-Identification.svg)](http://hits.dwyl.com/samihormi/https://githubcom/samihormi/Multi-Camera-Person-Tracking-and-Re-Identification)
Simple model to _"Detect/Track"_ and _"Re-identify"_ individuals in different cameras/videos.
[![HitCount](http://hits.dwyl.com/samihormi/https://githubcom/samihormi/Multi-Camera-Person-Tracking-and-Re-Identification.svg)](http://hits.dwyl.com/samihormi/https://githubcom/samihormi/Multi-Camera-Person-Tracking-and-Re-Identification)
Simple model to _"Detect/Track"_ and _"Re-identify"_ individuals in different cameras/videos.

<img align="right" img src="assets/2.gif" width="380" /> <img align="left" img src="assets/1.gif" width="380" />
<p align="center">
Expand All @@ -11,14 +11,14 @@ Simple model to _"Detect/Track"_ and _"Re-identify"_ individuals in different ca
</p>


## Introduction
This project aims to track people in different videos accounting for different angles.
# # Introduction
This project aims to track people in different videos accounting for different angles.


The framework used to accomplish this task relies on MOT and ReID to track and re-identify ID's of humans, respectively.
The framework used to accomplish this task relies on MOT and ReID to track and re-identify ID's of humans, respectively.
The tracking can be completed using YOLO_v3 or YOLO_v4 and ReID relies on KaiyangZhou's Torchreid library.
## Installation

# # Installation
- Download [Anaconda](https://www.anaconda.com/products/individual) if it is not installed on your machine

- Clone the repository
Expand All @@ -39,7 +39,7 @@ pip install -r requirements.txt
```python
conda install pytorch torchvision cudatoolkit -c pytorch
```
## Convert models
# # Convert models
- Download the YOLO models for [YOLO_v3](https://drive.google.com/file/d/18fmQMegNsAzPte7tJeCxwf1iE8JUTQhQ/view?usp=sharing) and [YOLO_v4](https://drive.google.com/file/d/1w9furPagm3KytRW2uNooLcBoiYWDwbop/view?usp=sharing) and add them to /model_data/weights/
* YOLO_v3
```python
Expand All @@ -51,21 +51,21 @@ python convert_y4.py model_data\weights\yolov4.weights model_data\models\yolov4.
```

# Pre-trained models (.h5) (If you want to start right away)
- Download the Keras models for [YOLO_v3](https://drive.google.com/file/d/1a7JI-A920lrdt6OKya-qCXx-5ZUWvkMg/view?usp=sharing) and [YOLO_v4](https://drive.google.com/file/d/1pwFo4aHKPi0ztpL5tEYaXIr8RltYYQeY/view?usp=sharing) and add them to \model_data\models\
- Download the Keras models for [YOLO_v3](https://drive.google.com/file/d/1a7JI-A920lrdt6OKya-qCXx-5ZUWvkMg/view?usp=sharing) and [YOLO_v4](https://drive.google.com/file/d/1pwFo4aHKPi0ztpL5tEYaXIr8RltYYQeY/view?usp=sharing) and add them to \model_data\models\

- Download either one of the following Torchreid models [1](https://drive.google.com/file/d/1EtkBARD398UW93HwiVO9x3mByr0AeWMg/view?usp=sharing),[2](https://drive.google.com/open?id=15Ayri_sHtrctJ1Zb8qERjvdi66y6QaI4) and add them to \model_data\models\ (you might have to change the path in reid.py)

## Demo
You can try out your own videos by running demo.py.
Under the directory \videos\output, the program will generate a video of the tracking, as well as a video of the tracking and ReID. (as can be seen in the example above)
You should specify the path of the videos and the version of YOLO you would like to use (v3 or v4)
# # Demo

You can try out your own videos by running demo.py.
Under the directory \videos\output, the program will generate a video of the tracking, as well as a video of the tracking and ReID. (as can be seen in the example above)
You should specify the path of the videos and the version of YOLO you would like to use (v3 or v4)

```python
python demo.py --videos videos\init\Double1.mp4 videos\init\Single1.mp4 --version v3
```

## Acknowledgement
# # Acknowledgement
This model is build on top of the incredible work done in the following projects:
* https://github.com/nwojke/cosine_metric_learning
* https://github.com/KaiyangZhou/deep-person-reid
Expand Down
14 changes: 8 additions & 6 deletions convert_y3.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
#! /usr/bin/env python
# ! /usr/bin/env python
"""
Reads Darknet config and weights and creates Keras model with TF backend.
Expand Down Expand Up @@ -30,6 +30,7 @@
help='Plot generated Keras model and save as image.',
action='store_true')


def unique_config_sections(config_file):
"""Convert all config sections to have unique names.
Expand All @@ -48,6 +49,7 @@ def unique_config_sections(config_file):
output_stream.seek(0)
return output_stream


# %%
def _main(args):
config_path = 'model_data/cfg/yolov3.cfg'
Expand All @@ -67,7 +69,7 @@ def _main(args):
weights_file = open(weights_path, 'rb')
major, minor, revision = np.ndarray(
shape=(3, ), dtype='int32', buffer=weights_file.read(12))
if (major*10+minor)>=2 and major<1000 and minor<1000:
if (major * 10 + minor) >= 2 and major < 1000 and minor < 1000:
seen = np.ndarray(shape=(1,), dtype='int64', buffer=weights_file.read(8))
else:
seen = np.ndarray(shape=(1,), dtype='int32', buffer=weights_file.read(4))
Expand Down Expand Up @@ -148,7 +150,7 @@ def _main(args):

# Handle activation.
act_fn = None
print('yyy',activation)
print('yyy', activation)
if activation == 'leaky':
pass # Add advanced activation later.
elif activation != 'linear':
Expand All @@ -157,9 +159,9 @@ def _main(args):
activation, section))

# Create Conv2D layer
if stride>1:
if stride > 1:
# Darknet uses left and top padding instead of 'same' mode
prev_layer = ZeroPadding2D(((1,0),(1,0)))(prev_layer)
prev_layer = ZeroPadding2D(((1, 0), (1, 0)))(prev_layer)
conv_layer = (Conv2D(
filters, (size, size),
strides=(stride, stride),
Expand Down Expand Up @@ -200,7 +202,7 @@ def _main(args):
assert activation == 'linear', 'Only linear activation supported.'
all_layers.append(Add()([all_layers[index], prev_layer]))
prev_layer = all_layers[-1]

elif section.startswith('upsample'):
stride = int(cfg_parser[section]['stride'])
assert stride == 2, 'Only stride=2 supported.'
Expand Down
21 changes: 11 additions & 10 deletions convert_y4.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
#! /usr/bin/env python
# ! /usr/bin/env python
"""
Creates Keras model with TF backend.
Expand Down Expand Up @@ -29,6 +29,7 @@
parser.add_argument('weights_path', help='Path to Darknet weights file.')
parser.add_argument('output_path', help='Path to output Keras model file.')


class Yolo4(object):
def get_class(self):
classes_path = os.path.expanduser(self.classes_path)
Expand Down Expand Up @@ -72,7 +73,7 @@ def load_yolo(self):
weights_file = open(self.weights_path, 'rb')
major, minor, revision = np.ndarray(
shape=(3, ), dtype='int32', buffer=weights_file.read(12))
if (major*10+minor)>=2 and major<1000 and minor<1000:
if (major * 10 + minor) >= 2 and major < 1000 and minor < 1000:
seen = np.ndarray(shape=(1,), dtype='int64', buffer=weights_file.read(8))
else:
seen = np.ndarray(shape=(1,), dtype='int32', buffer=weights_file.read(4))
Expand All @@ -92,9 +93,9 @@ def load_yolo(self):

bn_index = 0
for i in range(len(convs_sorted)):
print('Converting {}/{}'.format(i+1,len(convs_sorted)))
print('Converting {}/{}'.format(i + 1, len(convs_sorted)))
if i == 93 or i == 101 or i == 109:
#no bn, with bias
# no bn, with bias
weights_shape = self.yolo4_model.layers[convs_sorted[i][1]].get_weights()[0].shape
bias_shape = self.yolo4_model.layers[convs_sorted[i][1]].get_weights()[0].shape[3]
filters = bias_shape
Expand All @@ -113,7 +114,7 @@ def load_yolo(self):
conv_weights = np.transpose(conv_weights, [2, 3, 1, 0])
self.yolo4_model.layers[convs_sorted[i][1]].set_weights([conv_weights, conv_bias])
else:
#with bn, no bias
# with bn, no bias
weights_shape = self.yolo4_model.layers[convs_sorted[i][1]].get_weights()[0].shape
size = weights_shape[0]
bn_shape = self.yolo4_model.layers[bns_sorted[bn_index][1]].get_weights()[0].shape
Expand Down Expand Up @@ -152,14 +153,13 @@ def load_yolo(self):
self.yolo4_model.save(self.model_path)
logging.info("model saved to: {}".format(self.model_path))


if self.gpu_num>=2:
if self.gpu_num >= 2:
self.yolo4_model = multi_gpu_model(self.yolo4_model, gpus=self.gpu_num)

self.input_image_shape = K.placeholder(shape=(2, ))
self.boxes, self.scores, self.classes = yolo_eval(self.yolo4_model.output, self.anchors,
len(self.class_names), self.input_image_shape,
score_threshold=self.score)
self.boxes, self.scores, self.classes = yolo_eval(
self.yolo4_model.output, self.anchors, len(self.class_names), self.input_image_shape,
score_threshold=self.score)

def __init__(self, score, iou, anchors_path, classes_path, model_path, weights_path, gpu_num=1):
self.score = score
Expand All @@ -174,6 +174,7 @@ def __init__(self, score, iou, anchors_path, classes_path, model_path, weights_p
def close_session(self):
self.sess.close()


if __name__ == '__main__':
anchors_path = 'model_data/anchors/yolov4_anchors.txt'
classes_path = 'model_data/classes/coco_classes.txt'
Expand Down
14 changes: 9 additions & 5 deletions core/backbone.py
Original file line number Diff line number Diff line change
@@ -1,9 +1,10 @@
#! /usr/bin/env python
# ! /usr/bin/env python
# coding=utf-8

import tensorflow as tf
import core.common as common


def darknet53(input_data):

input_data = common.convolutional(input_data, (3, 3, 3, 32))
Expand Down Expand Up @@ -36,6 +37,7 @@ def darknet53(input_data):

return route_1, route_2, input_data


def cspdarknet53(input_data):

input_data = common.convolutional(input_data, (3, 3, 3, 32), activate_type="mish")
Expand Down Expand Up @@ -96,14 +98,18 @@ def cspdarknet53(input_data):
input_data = common.convolutional(input_data, (3, 3, 512, 1024))
input_data = common.convolutional(input_data, (1, 1, 1024, 512))

input_data = tf.concat([tf.nn.max_pool(input_data, ksize=13, padding='SAME', strides=1), tf.nn.max_pool(input_data, ksize=9, padding='SAME', strides=1)
, tf.nn.max_pool(input_data, ksize=5, padding='SAME', strides=1), input_data], axis=-1)
input_data = tf.concat([
tf.nn.max_pool(input_data, ksize=13, padding='SAME', strides=1),
tf.nn.max_pool(input_data, ksize=9, padding='SAME', strides=1),
tf.nn.max_pool(input_data, ksize=5, padding='SAME', strides=1),
input_data], axis=-1)
input_data = common.convolutional(input_data, (1, 1, 2048, 512))
input_data = common.convolutional(input_data, (3, 3, 512, 1024))
input_data = common.convolutional(input_data, (1, 1, 1024, 512))

return route_1, route_2, input_data


def darknet53_tiny(input_data):
input_data = common.convolutional(input_data, (3, 3, 3, 16))
input_data = tf.keras.layers.MaxPool2D(2, 2, 'same')(input_data)
Expand All @@ -121,5 +127,3 @@ def darknet53_tiny(input_data):
input_data = common.convolutional(input_data, (3, 3, 512, 1024))

return route_1, input_data


34 changes: 24 additions & 10 deletions core/common.py
Original file line number Diff line number Diff line change
@@ -1,9 +1,10 @@
#! /usr/bin/env python
# ! /usr/bin/env python
# coding=utf-8

import tensorflow as tf

# import tensorflow_addons as tfa


class BatchNormalization(tf.keras.layers.BatchNormalization):
"""
"Frozen state" and "inference mode" are two separate concepts.
Expand All @@ -17,6 +18,7 @@ def call(self, x, training=False):
training = tf.logical_and(training, self.trainable)
return super().call(x, training)


def convolutional(input_layer, filters_shape, downsample=False, activate=True, bn=True, activate_type='leaky'):
if downsample:
input_layer = tf.keras.layers.ZeroPadding2D(((1, 0), (1, 0)))(input_layer)
Expand All @@ -26,13 +28,15 @@ def convolutional(input_layer, filters_shape, downsample=False, activate=True, b
strides = 1
padding = 'same'

conv = tf.keras.layers.Conv2D(filters=filters_shape[-1], kernel_size = filters_shape[0], strides=strides, padding=padding,
conv = tf.keras.layers.Conv2D(filters=filters_shape[-1], kernel_size=filters_shape[0],
strides=strides, padding=padding,
use_bias=not bn, kernel_regularizer=tf.keras.regularizers.l2(0.0005),
kernel_initializer=tf.random_normal_initializer(stddev=0.01),
bias_initializer=tf.constant_initializer(0.))(input_layer)

if bn: conv = BatchNormalization()(conv)
if activate == True:
if bn:
conv = BatchNormalization()(conv)
if activate is True:
if activate_type == "leaky":
conv = tf.nn.leaky_relu(conv, alpha=0.1)
elif activate_type == "mish":
Expand All @@ -47,31 +51,41 @@ def convolutional(input_layer, filters_shape, downsample=False, activate=True, b
# conv = tf.keras.activations.relu(tf.nn.softplus(conv), max_value=20)

return conv
def softplus(x, threshold = 20.):


def softplus(x, threshold=20.):
def f1():
return x

def f2():
return tf.exp(x)

def f3():
return tf.math.log(1 + tf.exp(x))
# mask = tf.greater(x, threshold)
# x = tf.exp(x[mask])
# return tf.exp(x)
return tf.case([(tf.greater(x, tf.constant(threshold)), lambda:f1()), (tf.less(x, tf.constant(-threshold)), lambda:f2())], default=lambda:f3())
return tf.case(
[(tf.greater(x, tf.constant(threshold)), lambda:f1()), (tf.less(x, tf.constant(-threshold)), lambda:f2())],
default=lambda: f3()
)
# return tf.case([(tf.greater(x, threshold), lambda:f1())])


def mish(x):
return tf.keras.layers.Lambda(lambda x: x*tf.tanh(tf.math.log(1+tf.exp(x))))(x)
return tf.keras.layers.Lambda(lambda x: x * tf.tanh(tf.math.log(1 + tf.exp(x))))(x)
# return tf.keras.layers.Lambda(lambda x: softplus(x))(x)
# return tf.keras.layers.Lambda(lambda x: x * tf.tanh(softplus(x)))(x)


def residual_block(input_layer, input_channel, filter_num1, filter_num2, activate_type='leaky'):
short_cut = input_layer
conv = convolutional(input_layer, filters_shape=(1, 1, input_channel, filter_num1), activate_type=activate_type)
conv = convolutional(conv , filters_shape=(3, 3, filter_num1, filter_num2), activate_type=activate_type)
conv = convolutional(conv, filters_shape=(3, 3, filter_num1, filter_num2), activate_type=activate_type)

residual_output = short_cut + conv
return residual_output


def upsample(input_layer):
return tf.image.resize(input_layer, (input_layer.shape[1] * 2, input_layer.shape[2] * 2), method='nearest')

Loading

0 comments on commit d8d0df6

Please sign in to comment.