Voisin-d-energie-ULB

"Voisin d'énergie"

Project made in collaboration with ULB (Université Libre de Bruxelles) - IRIDIA laboratory, Innoviris, IGEAT, BinHôme, Energy4Commons and BEAMS.
Wifi de Voisin d'énergie

Table of contents

1. About the Project
   • Built with
2. Getting Started
   • Prerequisites
3. Usage
4. Contributing
5. License
6. Contact

About The Project

The project consists in creating a backend architecture and a frontend system to visualize Fluksometer electrical data : consumption, production and total.

(back to top)

Built With

• python3
• Cassandra
• Javascript
• Bootstrap
• Nodejs
• socket.io
• express
• chartjs

(back to top)

Getting Started

This section describes how to setup the project locally.

Prerequisites

List of packages to install (Ubuntu) :

1) Frontend

• nodejs

  sudo apt install nodejs

• nodejs npm

  sudo apt install nodejs npm

• express

  npm install express

• socket.io

  npm install -g socket.io --save

• cassandra-driver

  npm install -g cassandra-driver

• body-parser

  npm install --save body-parser

• fs

  npm install fs

Backend

• python3 (> v3.8.10)

  sudo apt install python3

• numpy (> v1.17.4)

  sudo apt install python3-numpy

• pandas (> v1.1.0)

  sudo apt install python3-pandas

• tmpo (tmpo repository) (v0.2.10)

  sudo pip3 install tmpo

• cassandra-driver (> v3.25.0)

  sudo -H pip3 install cassandra-driver

• mail

   sudo apt install mailutils

Cassandra

echo "deb http://www.apache.org/dist/cassandra/debian 40x main" | sudo tee -a /etc/apt/sources.list.d/cassandra.sources.list
sudo apt install openjdk-8-jre-headless
cd /usr/lib/jvm/java-8-openjdk-amd64/
pwd
sudo nano ~/.bashrc   => set JAVA_HOME = <pwd>
sudo apt install curl
curl https://downloads.apache.org/cassandra/KEYS | sudo apt-key add -
sudo apt-get install cassandra
sudo pip install cqlsh 

(back to top)

Usage

In this section, we describe how to use the different scripts.

Prerequisite

1. First, start Cassandra service

   sudo service cassandra start

2. Start cqlsh and enter the 'flukso' keyspace

   cqlsh
   use flukso;

3. Execute any script using the instructions below :
4. Always make sure to stop cassandra service when we are done using it. It is very important.

   sudo service cassandra stop

Scripts

Here are the list of all the executable scripts aswell as their arguments :

• sync raw flukso data : The script automatically get the new Flukso data using the tmpo API and store it in Cassandra. No need to specify any arguments.

  sync_flukso.py

• preprocess Flukso sensors : The script contains a lot of different functions that are meant to be used before the raw data syncing. The script allows, among others, to create the neccessary Cassandra tables, as well as inserting the new data in them. However, those functions are automatically triggered using one command :

  preprocess_sensors_config.py [config]

  • The config argument allows to specify a config file path. The configuration file must be an Excel file containing those 3 tabs :
    1. Export_InstallationSensors : All flukso info :
       • Installation ID,
       • Start date,
       • Flukso ID,
       • Phase name,
       • Network factor,
       • Production factor,
       • Consumption factor,
       • Sensor ID,
       • Sensor Token
    2. Export_Access : All login IDS along with their corresponding installation IDs (group ids they belong to).
    3. InstallationCaptions : All installation IDs along with their captions (One sentence describing the nature of the group).

• Alerts : The script sends alert per email if it detects some unusual behaviour in the system

  alerts.py --mode MODE

  • The --mode argument is the behaviour we want to monitor for potential alerts. MODE :
    1. missing (default): Check if the number of missing data in the past is ok.
    2. sign : Check if the signs are correct in power data. It can be incorrect/incoherent if :
       • we see negative consumption values
       • we see positive production values
       • we see photovoltaic values during the night

• dump_csv : The purpose of this script is to dump csv from the database in many ways :

  • real time (get database history if no data saved yet, or get data between last saved date and now)
  • specific day of data
  • now also a specific home id and a specific date range

  dump_csv.py [--home HOME_Id] [--day DAY] [--start START_DAY] [--end END_DAY] output_filename 

  arguments :

  • output_filename : sensors config filename (mandatory)
  • home (optional): specific home id
  • day (optional): specific day (YYYY-MM-DD)
  • start (optional): start day (YYYY-MM-DD)
  • end (optional): end day (YYYY-MM-DD)

License

Distributed under the MIT License. See LICENSE.txt for more information.

Contact

• Alexandre Heneffe - [email protected]
• Guillaume Levasseur - [email protected]

(back to top)