📷 survAPI Surveillance Camera Analysis

Table of Contents

❓ Need surveillance, but low on server capacity?
💡 Train the camera to know when to record.
⚡ Introduction:
❓ Problem:
💡 Solution:
💪 Improvements over conventional Security Cameras:
🏆 Features:
ℹ️ Demo:
🔥 Deployment:
🤝 Contribution:
✎ Example Data from the Database:
🏛️ Architecture:
🔧 Compatibility:
☁️ Planned Features:
📖 Dictionary:
🌐 Useful Links
⚖️ License

❓ Need surveillance, but low on server capacity?

💡 Train the camera to know when to record.

CI Info:

⚡ Introduction:

This implementation demonstrates a Security-Camera Software which was written in javascript and uses cocossd offered by ml5 to detect objects.

The application is secured using express-session and bcrypt. This way only authenticated users can access your surveillance and you are able to allow registrations and create new roles to control this system.

Check out this link to see all screenshots

❓ Problem:

A business needs a security camera, but is low on server capacity. Thatswhy the business can not store videos 24/7.

💡 Solution:

Using Machine Learning, we can tell the camera when it has objects in view and use a lightweight backend to store the timestamps and also the type of objects which where detected.

Classification:

After detection, the software later classifies them as certain targets.

Persistence

If the classified object is one of the targets, it will be persisted in the database by giving information on its type and also adding a timestamp and date. Now we know when to start recording and stop after a predefined amount of time passed.

💪 Improvements over conventional Security Cameras:

installation is very easy and very customizable due to Open Source Technology
a clean webinterface makes your cameras accessible and configurable
detection of persons is a standard problem of Machine Learning and already has a very high precision
destroy video material in which nothing happens, you will only store critical/important moments

🏆 Features:

⭐ advance your security by getting additional information from your camera.	⭐ super simple setup.	⭐ modern dashboard and webpages to access the cameras.
⭐ combine any hardware with this software.	⭐ SurvAPI: The included surveillance API also offers all of the analysis tool to be called from other frameworks.	⭐ lightweight and fast.
⭐ optimized amount of video data.	⭐ Machine Learning used to detect certain targets.	⭐ predefine intervals to control data produced by the camera.
⭐ very simple database structure	⭐high security level by using bcrypt

ℹ️	Do you want to Request a feature? Click here to request

ℹ️ Demo:

⚠️ At the moment the demo may not work correctly in your browser. Thatswhy it is disabled.

INFORMATION: more to follow in the future.

🔥 Deployment:

1. Requirements:

NodeJS
Docker
Docker-compose
Text/Code-Editor of your choice (I recommend PHPStorm or VSCode (VScodium if you support open source!)

2. Setup:

To install NodeJS, you will have to do either:
- Windows: Visit this url
- Linux:
  sudo apt install nodejs npm -y
- MacOS:
  brew install node

3. Docker and Docker-compose:

The installation of Docker is different on each operating system.
- Windows: Visit [this.url]
- Linux:
  sudo apt install docker docker-compose
- MacOS:
  brew install docker docker-compose

4. Run database:

You have to navigate into src and then then run

docker-compose up

The command is the same on all operating systems

5. Install dependencies:

On all operating systems, run the following command in src:

npm install

6. Run the application:

Now after each change in your codebase you will have to run the following commands:

Tests:

Please run the tests as often as you can after you changed the code!

npm test

Compile ts to js:

npm run-script deploy

Finally run the application:

npm run-script devStart

or

nodemon

INFORMATION: more to follow in the future.

🤝 Contribution:

If you want to contribute, you need to fork this repository and write clean commits. What do I mean with clean commits?

Commits should not change different parts of the application, as long as they are not part of a specific problem which the commit tries to solve.
Commit messages should always be written using the english language.
Each commit message should be less than 50 letters and only describe one aspect.
Commits should not be too long. Please do not change 30 files in one commit.
There should be a minimum of understanding, what the business problem in this case is.
You should be able to keep the architecture and not contribute beginner’s code. (I am happy if a beginner contributes, because participating in a project can bring you further, but code should be readable, efficient and follow some principles)

What is the most important part right now?

Proof of Concept

Due by August 31th, 2021

The Proof of Concept will be a fully working Webapplication which is able to add a camera to its database and persist anything which is detected, if it is defined as a target.

Requirements:

❏ Add one or more cameras via the Webinterface

❏ Be able to switch between the cameras via Webinterface

❏ List all detections of a certain camera and make them clickable to gain more information

❏ Remove, or edit cameras from the Webinterface

❏ Be able to search and filter detections

Click here to get to the milestones

✎ Example Data from the Database:

1. Collection of `Detection’s

MockDatabase {db: Array(8), lastDetection: 1603531578818}
db: Array(9)
0: Detection {id: 1, objects: Array(1), date: 1603531508193, dateObject: Sat Oct 24 2020 11:25:08 GMT+0200 (Mitteleuropäische Sommerzeit)}
1: Detection {id: 2, objects: Array(1), date: 1603531518319, dateObject: Sat Oct 24 2020 11:25:18 GMT+0200 (Mitteleuropäische Sommerzeit)}
2: Detection
  date: 1603531528418
  dateObject: Sat Oct 24 2020 11:25:28 GMT+0200 (Mitteleuropäische Sommerzeit) {}
  id: 3
  objects: Array(1)
  0: {label: "person", confidence: 0.7510808706283569, x: 7.6421356201171875, y: 2.8800487518310547, width: 626.8524932861328, …}
  length: 1
  __proto__: Array(0)
  __proto__: Object
3: Detection {id: 4, objects: Array(0), date: 1603531538488, dateObject: Sat Oct 24 2020 11:25:38 GMT+0200 (Mitteleuropäische Sommerzeit)}
4: Detection {id: 5, objects: Array(1), date: 1603531548599, dateObject: Sat Oct 24 2020 11:25:48 GMT+0200 (Mitteleuropäische Sommerzeit)}
5: Detection {id: 6, objects: Array(1), date: 1603531558734, dateObject: Sat Oct 24 2020 11:25:58 GMT+0200 (Mitteleuropäische Sommerzeit)}
6: Detection
  date: 1603531568792
  dateObject: Sat Oct 24 2020 11:26:08 GMT+0200 (Mitteleuropäische Sommerzeit) {}
  id: 7
  objects: Array(0)
  length: 0
  __proto__: Array(0)
  __proto__: Object
7: Detection
  date: 1603531578818
  dateObject: Sat Oct 24 2020 11:26:18 GMT+0200 (Mitteleuropäische Sommerzeit) {}
  id: 8
  objects: Array(0)
  length: 0
  __proto__: Array(0)
  __proto__: Object
8: Detection
  date: 1603531588867
  dateObject: Sat Oct 24 2020 11:26:28 GMT+0200 (Mitteleuropäische Sommerzeit) {}
  id: 9
  objects: Array(1)
    0: {label: "person", confidence: 0.8875717520713806, x: 4.159679412841797, y: 1.1598587036132812, width: 632.5449562072754, …}
    length: 1
  __proto__: Array(0)
  __proto__: Object
  length: 9
  __proto__: Array(0)
  lastDetection: 1603531588867
  __proto__: Object

2. A single Detection

8: Detection
  date: 1603531588867
  dateObject: Sat Oct 24 2020 11:26:28 GMT+0200 (Mitteleuropäische Sommerzeit) {}
  id: 9
  objects: Array(1)
    0:
      confidence: 0.8875717520713806
      height: 477.72010803222656
      label: "person"
      normalized: {x: 0.006499499082565308, y: 0.002416372299194336, width: 0.9883514940738678, height: 0.9952502250671387}
      width: 632.5449562072754
      x: 4.159679412841797
      y: 1.1598587036132812
    __proto__: Object
    length: 1
  __proto__: Array(0)
  __proto__: Object
length: 9
__proto__: Array(0)
lastDetection: 1603531588867

3. A single Detection as JSON

{id: 1, objects: Array(1), date: 1615052099545, dateObject: Sat Mar 06 2021 18:34:59 GMT+0100 (Mitteleuropäische Normalzeit)}

The objects list:

{label: "person", confidence: 0.9367305040359497, x: 133.67712020874023, y: 16.696457862854004, width: 474.5007514...

Where it looks as follows:

confidence: 0.9367305040359497
height: 458.1805944442749
label: "person"
normalized: {x: 0.20887050032615662, y: 0.034784287214279175, width: 0.7414074242115021, height: 0.9545429050922394}
width: 474.5007514953613
x: 133.67712020874023
y: 16.696457862854004

🏛️ Architecture:

The project will be separated into the subsystems Camera, Surveillance, Database, Webviews where each subsystem exactly solves one of the given business problems.

To guarantee clean code, this project will profit from Domain Driven Design and from Self Contained System’s.

Camera:

The subsystem will fulfill the HARDWARE-part in this project. All types of cameras (which you find right below under Compatibility) will be connected via this subsystem. It defines a domain model which projects the real world properties of cameras into code and offers its configured cameras to all of the other subsystems. All of the input will run through this subsystem.

Surveillance:

The surveillance will process video material received from any video source, configured in the Camera subsystem. It makes use of ML5 and analyses the material based on the admins preferences. The analysis results will be persisted in the Database subsystem. It usually is not a separate subsystem, but for this project it will do best if the database can completely be substituted behind an interface which connects the Webviews and the Surveillance to it.

Database:

Use any type of database you want and store all of the analysis results. You can access the persisted data by using the Webviews or build your own UI to access it.

Webviews:

Basically a frontend which can be accessed from any webbrowser. You can configure the cameras, inspect detections in a nice way and get watch live material from your surveillance cameras.

🔧 Compatibility:

❏ IP Security Camera
❏ Analog MPX Security Camera
❏ Analog MPX Wireless Security Camera
❏ Wire-Free Security Camera Wiring
✓ Wi-Fi Security Camera Wiring

Problem:

This security software has to become a module in security networks which is located between the actual video input device and the DVR if the video input device is not wireless or NVR if the video input device is wireless.

Example Network (IP Security Camera):

In this system of an IP Security Camera you will find a cable which connects the NVR and the video input device by using power-over-ethernet (POE) technology to "provide both power and video transmission".

This setup leaves no space for an external software which directly controls the video input stream. This problem is architectural and has to be solved by this software to achieve the highest possible compatibility.

Solution:

1. Backwards Integration:

When using the Backwards Integration, the software should be run on a server which is directly connected to the DVR/NVR and will rewatch the video material recorded, filter unimportant video information and redirect it either to the DVR/NVR or to the server to reduce the amount of data saved.

2. Automated Video Analysis:

Using this method, the software is run on a server just as in the previous example. The software will then rewatch the video information received from the DVR/NVR and collect timestamps which flag video intervals in which target objects could be detected. You can then either keep your videos as they are or use the time stamps to remove unimportant sequences.

More solutions have to come, but at the moment there is no idea to manipulate the video input stream between the video input device and the DVR/NVR, which would be the ultimate use-case for this software.

☁️ Planned Features:

❏ SurvAPI, an API which will offer all of the information to the filesystem or to Web requests
❏ cloud support
❏ support of various sensors
❏ UI to manage connected devices / settings
❏ server implementation to offer a central responsibility for persistence (server 1:n cameras nets)

📖 Dictionary:

Word	Description
ML5	A Machine Learning Framework for javascript, based on TensorFlow and developed by the team of P5
CocoSSD	Object detection model that aims to localize and identify multiple objects in a single image. https://github.com/tensorflow/tfjs-models/tree/master/coco-ssd
Domain Driven Design (DDD)	https://en.wikipedia.org/wiki/Domain-driven_design
Self Contained System (SCS)	https://en.wikipedia.org/wiki/Self-contained_system_(software)

Files

README.adoc

Latest commit

History