Skip to content

Latest commit

 

History

History
445 lines (330 loc) · 22.6 KB

README.md

File metadata and controls

445 lines (330 loc) · 22.6 KB

Smart Sensor

Smart Sensor Animation

DISCLAIMER - This repository and its contents are not endorsed by, sponsored by, affiliated with, nor associated with the National Aeronautics and Space Administration (NASA). This repository is only a demonstration of some of the concepts that I explored while I was an intern at NASA's Langley Research Center.

Introduction

While I was an intern at NASA, I noticed a lot of interest in "smart" technologies, e.g., "Edge" computing and Internet of Things (IoT) devices; augmented and virtual reality; machine learning; etc.

Since there's no MicroCenter or Home Depot in space (and space delivery is not included in Amazon Prime), one of the things I thought about was a portable, customizable, plug-and-play data analyzer, aka a smart sensor suite (S3). For less than $100, the S3, with a set of basic sensors, could replace the following devices:

  • Gas Detector (O2, LEL, CO-H2S, NH3, NO2): $2571.91
  • Light Meter $94.95
  • Sound Level Meter: $76.68
  • Temperature and Humidity Meter: $19.99
  • Universal Remote Control: $9.99

In addition, users could add or create ad hoc sensors and actuators from simple components using wires, resistors, breadboards, etc. As a bonus, the S3 could communicate over Bluetooth, RF, and Wi-Fi wireless networks.

This device would also use machine learning, as well as rule-based, conditional logic, to analyze data on the "edge" and act on the results through actuators. This system would complement or replace rule-based, conditional logic systems, such as engine control units (i.e., ECU's, which use lookup tables), electronic flight control systems (i.e., EFCS's, which use flight control laws), heating, ventilation, and air conditioning (HVAC) control systems (which use target states), etc.

In this demo, we will create a device that collects and analyzes environmental data to determine if the thermal comfort level of a room complies with the American Society of Heating, Refrigerating and Air-Conditioning Engineers (ASHRAE) Standard 55-2017. This standard helps engineers design effective HVAC systems.

As a bonus, we will demonstrate how to collect and process data over-the-air (OTA) from satellite devices using the S3.

Parts

For this demo, we used:

  • One (1) Raspberry Pi 3 Model B+ with 5V/2.5A power source as the S3
  • One (1) GrovePi+ HAT (hardware attached on top) add-on board
  • One (1) temperature and humidity sensor with a Grove connector (we used a simple DHT11)
  • One (1) red and one (1) green LED's with Grove connectors
  • One (1) 16 x 2 LCD RGB Backlight with Grove connector
  • Four (4) 4-wire cables with four-pin female-to-female Grove connectors

This is not the only configuration the S3 can use; for our bonus demo, instead of connected sensors, the S3 uses a remote Raspberry Pi Zero W with a Waveshare Sense HAT (B).

Key Terms

  • Features (aka the x's, input variable names) - The names of each value in a vector of values (e.g., temperature, humidity, etc.). Together, they imply a relationship (e.g., hot or cold, etc.).
  • Labels (aka the y's, classes, targets, output variable names) - The name of the relationship between the values (e.g., hot or cold, etc.).
  • Weights (aka the thetas) - The actual values in the vector of features (e.g., 37 deg C, 100%, etc.).
  • Dataset - The collection of all the feature weights and label values.
  • Classifier - An algorithm that tries to predict the relationship between features from unlabeled data.
  • Model - An algorithm that accounts for the relationship between features and the correct label.

How it Works

  1. The application loads and parses the dataset and tests all classifiers against the dataset.
  2. The application creates a model using the highest scoring classifier.
  3. The application then requests data from the sensors.
  4. The application runs the data against the model using the selected classifier.
  5. The application returns the results:
    • If the result is "Neutral", it complies with ASHRAE Standard 55-2017. The application displays the data on its LCD screen and indicates compliance by lighting a green LED. The application can also forward this data to a supervisory control and data acquisition (SCADA) system via JSON.
    • If the result is anything but "Neutral"), it does NOT comply with ASHRAE Standard 55-2017. The application displays the data on its LCD screen and indicates non-compliance by lighting a red LED. The application can also forward this data to a supervisory control and data acquisition (SCADA) system via JSON and/or adjust the HVAC system autonomously.
  6. The application repeats steps 3 through 7 until it is shut off.

For this demo, we will only measure operative temperature and relative humidity. However, the S3 can also process atmospheric pressure, air speed, metabolic rate, and the insulating effects of clothing level, to predict if the occupants will feel cold, cool, slightly cool, neutral, slightly warm, warm, or hot.

atmo_pres air_speed rel_humid meta_rate cloth_lvl oper_temp sens_desc
1013.25 0.1 50.0 1.0 0.61 23.0 2
1013.25 0.1 60.0 1.0 0.61 26.0 3
1013.25 0.1 70.0 1.0 0.61 28.0 4

While this task would be easy to accomplish with simple if-else-then program, we also wanted to demonstrate the capabilities of machine learning.

Steps

  1. Install Raspbian for Robots on an SD card for the Raspberry Pi by following the instructions at https://www.dexterindustries.com/howto/install-raspbian-for-robots-image-on-an-sd-card/ to install Dexter Industries' version of Raspbian on your Raspberry Pi.

  2. Set up the Raspberry Pi with the GrovePi HAT:

  3. Increase the Raspberry Pi's swap file size to 1024 MB:

    • Input the following to stop the swapfile manager:

      sudo /etc/init.d/dphys-swapfile stop

    • Input the following to open the swap file configuration:

      sudo nano /etc/dphys-swapfile

    • Change CONF_SWAPSIZE=100 to CONF_SWAPSIZE=1024 and save (CTRLO, then CTRLX). Input the following to restart the swapfile manager:

      sudo /etc/init.d/dphys-swapfile start

    • Reboot your Raspberry Pi.

  4. Using the terminal, clone the Smart Sensor repository. By the way, since some of the scripts require elevated privileges (e.g., writing to a file, using the communications server, etc.), you will have to install all modules and packages using sudo or you will get a "Permission Denied" error:

    pi@dex:~ $ sudo git clone https://github.com/garciart/SmartSensor # Clone the repository
pi@dex:~ $ sudo chown -R pi:pi SmartSensor # If logged in as pi, assume ownership of all files in case you have to edit a script
pi@dex:~ $ cd SmartSensor
pi@dex:~ $ sudo chmod +x permissions.sh
pi@dex:~ $ sudo ./permissions.sh # Make the scripts executable

  5. Install the required dependencies:

    • Instead of a requirements.txt file, we created a shell script to install or update the dependencies:

      pi@dex:~/SmartSensor $ sudo chmod +x setup.sh
pi@dex:~/SmartSensor $ sudo ./setup.sh

    • Feel free to view the setup.sh script to get more information about the dependencies. To run the scripts, you will need to use Python 3. If you are using Python 3.5, that's ok. These scripts will run on Python 3.5 and higher. Also note that we use the newer Advanced Packaging Tool (APT) instead of apt-get.

    • While we tried to ensure the shell script is correct, if you run into an issue, install the problem package separately, and then re-run setup.sh. If you have more problems, feel free to contact us at [email protected].

  6. Our next step is to test the sensors:

    • Connect them as follows:

      • DHT (11 or 22) to digital port 7 on the Grove Pi HAT
      • RGB LCD to I2C port 2 on the Grove Pi HAT
      • Green LED to digital port 5 on the Grove Pi HAT
      • Red LED to digital port 6 on the Grove Pi HAT

    GrovePi Connections

    • Once they are connected, run the following command:
    pi@dex:~ /SmartSensor $ cd s3_scripts
pi@dex:~ /SmartSensor/s3_scripts $ ./sensors_test.py
    • We should see results similar to the following, but with different values for temperature and humidity:
    pi@dex:~ /SmartSensor/s3_scripts $ Temperature: 20.5C | Humidity: 30%

  7. Once the sensor test is complete, verify scikit-learn, TensorFlow, and the sensors work (disregard any TensorFlow warnings for now). We recommend examining the code as it runs:

    pi@dex:~ /SmartSensor/s3_scripts $ cd ..
pi@dex:~ /SmartSensor $ cd ml_scripts
pi@dex:~ /SmartSensor/ml_scripts $ ./tf_premade.py
pi@dex:~ /SmartSensor/ml_scripts $ ./sl_example.py
pi@dex:~ /SmartSensor/ml_scripts $ ./tf_example.py

  8. Notice that the scikit-learn example (sl_example.py) ran faster (4.6 sec vs 14.4 and 13.5 sec) and was much more accurate than the TensorFlow scripts (tf_premade.py and tf_example.py). We believe this is due to the dataset having six features, but with variations in only two of those features. If you run the scripts against the Iris dataset (remove the comments surrounding the Iris test and place them around the Thermal Comfort test), the accuracy of the TensorFlow scripts increases dramatically. However, for the S3, we will use the scikit classifiers.

    scikit-learn Classification Test.
...
Running samples using Gradient Boosting Classifier
(test score was 1.00%)...
Sample #1: Prediction: Slightly Cool (expected Slightly Cool)
Sample #2: Prediction: Neutral (expected Neutral)
Sample #3: Prediction: Slightly Warm (expected Slightly Warm)

Running samples using Random Forest Classifier
(test score was 1.00%)...
Sample #1: Prediction: Slightly Cool (expected Slightly Cool)
Sample #2: Prediction: Neutral (expected Neutral)
Sample #3: Prediction: Slightly Warm (expected Slightly Warm)
...
Elapsed time: 4.682486534118652 seconds.
Job complete. Have an excellent day.

    TensorFlow Classification Test using Premade Estimators.
...
Prediction is "Slightly Cool" (32.4%), expected "Slightly Cool"
Prediction is "Warm" (51.3%), expected "Neutral"
Prediction is "Warm" (85.8%), expected "Slightly Warm"
Elapsed time: 14.417850971221924 seconds.
Job complete. Have an excellent day.

    TensorFlow Classification Test using Keras.
...
X=[1013.25    0.1    50.      1.      0.61   23.  ], Predicted: Cool (1), Expected Slightly Cool
X=[1013.25    0.1    60.      1.      0.61   26.  ], Predicted: Cool (1), Expected Neutral
X=[1013.25    0.1    76.      1.      0.61   28.  ], Predicted: Cool (1), Expected Slightly Warm
Elapsed time: 13.566766738891602 seconds.
Job complete. Have an excellent day.

  9. Finally, we'll put everything together in smart_sensor.py:

    Smart Sensor Animation

    • Test the Sensors (Optional) - First, we will make sure all the sensors and actuators work. In our case, the sensor is the DHT-11, and the actuators are the two LEDs, and the RGB LCD (which started crapping out on us). For production, you may remove this code if you like.

    • Prepare the Model: Next, we will train all the classifiers as we did in sl_example.py. We will then pick the most accurate classifier for our model and retrain it against the whole data set. You will not get the same classifier all the time; during our test runs, we used the Extra Trees, the Gradient Boosting, and the Random Forest classifiers.

    • Check the Model (Optional): This is another optional step; We will check the accuracy of the selected model against some sample data. By the way, even though the data is unlabeled, we know what the resulting predictions should be. Once again, for production, you may remove this code if you like.

    • Collect Sensor Data: This is GrovePi specific code. We collect three samples of temperature and humidity from the DHT sensor, reformat it into a list of tuples, and send it for processing.

    • Process Sensor Data: Here, we run the collected data against the selected model. We collect the results, average them together, and make a determination of the conditions in the room based on the ASHRAE 7-point scale for thermal comfort.

    • Shutdown: Finally, we shut down the sensors and actuators to extend their working life.

    • Here are the results of a sample run:

    pi@dex:~ /SmartSensor/ml_scripts $ cd ..
pi@dex:~ /SmartSensor $ cd s3_scripts
pi@dex:~ /SmartSensor/s3_scripts $ ./smart_sensor.py
Smart Sensor Application.

Testing sensors...
Temperature: 22.0C | Humidity: 45.0%
Temperature: 22.0C | Humidity: 44.0%
Temperature: 22.0C | Humidity: 44.0%
Test complete.

Training and testing model...
Model selected: Extra Trees Classifier (1.00%).
Training and testing model complete.
Elapsed time: 15.283817291259766 seconds.

Checking model against unlabeled data...
Data to be evaluated:
Sample #1: [[1013.25, 0.1, 50.0, 1.0, 0.61, 23.0]] = Slightly Cool
Sample #2: [[1013.25, 0.1, 60.0, 1.0, 0.61, 26.0]] = Neutral
Sample #3: [[1013.25, 0.1, 76.0, 1.0, 0.61, 28.0]] = Slightly Warm
Prediction(s):
Sample #1: Prediction: Slightly Cool (expected Slightly Cool)
Sample #2: Prediction: Neutral (expected Neutral)
Sample #3: Prediction: Slightly Warm (expected Slightly Warm)

Collecting sensor data...
Sample #1 collected: [[1013.25, 0.1, 44.0, 1.0, 0.61, 22.0]]
Sample #2 collected: [[1013.25, 0.1, 44.0, 1.0, 0.61, 22.0]]
Sample #3 collected: [[1013.25, 0.1, 44.0, 1.0, 0.61, 22.0]]
Collection complete.

Processing sensor data...
Sensor data #1: Prediction: Slightly Cool
Sensor data #2: Prediction: Slightly Cool
Sensor data #3: Prediction: Slightly Cool
Overall sensation: 2 (Slightly Cool)

Shutting down board...
Job complete. Have an excellent day.
pi@dex:~ /SmartSensor/s3_scripts $

Additional Information

Data Source

Data verified using the Thermal Comfort Tool from the Center for the Built Environment (CBE) at the University of California, Berkeley.

Features

  • Atmospheric Pressure(atmo_pres)
  • Air Speed (air_speed)
  • Relative Humidity (rel_humid)
  • Metabolic Rate (meta_rate)
  • Clothing Level (cloth_lvl)
  • Operative Temperature (oper_temp)

Labels

  • Label: Sensation (sens_desc)
  • Label descriptions and value
    • Cold (0)
    • Cool (1)
    • Slightly Cool (2)
    • Neutral (3)
    • Slightly Warm (4)
    • Warm (5)
    • Hot (6)

Extra Credit

Like we stated earlier, for extra credit, we will demonstrate how to collect and process data over-the-air (OTA) from satellite devices using the S3 and a remote Raspberry Pi Zero W with a Waveshare Sense HAT (B). The Sense HAT is much more accurate than our DHT-11 and also provides us with barometric pressure readings.

You do not have to use our setup. You can use different combinations of devices, such as an ESP32, an ESP8226, a DHT-22, an M5StickC with an M5Stack sensor, etc. If you are going to use the Sense HAT (B), you need to install the BCM2835 library. For instructions, check out https://www.waveshare.com/wiki/Sense_HAT_(B)

  1. Turn on both the Raspberry Pi 3 Model B+ and the Pi Zero W.

    Note - By the way, for this demo, you do not need the GrovePi HAT or the GrovePi code. We just like the lights (and the board we printed out on our Ender 3. Thanks, Chris Cirone at https://www.thingiverse.com/thing:2161971!)

  2. Make and test the connection:

    • Once they are started, make sure sock_test_server.py is on the S3 and sock_test_client.py on the Pi Zero.

    • Get the IP address of each device (while there are many ways of doing this, we found the easiest way was to execute "hostname -I" at the command line)

    • Open both scripts and replace the HOST value with the IP address of the device (leave PORT 333 the same, unless you are already using it).

    • Test the connection by running sock_test_server.py on the S3 first, and then running sock_test_client.py on the Pi Zero (you may have to use sudo to run the scripts). You should get results similar to the following:

      • S3:
      pi@dex:~ /SmartSensor/s3_scripts $ sudo ./sock_test_server.py
Waiting for data...
Received 'Hello, friend.' from client!
Waiting for data...
Received 'Hello, friend.' from client!
Waiting for data...
Received 'Good-bye!' from client!
The client has signed off.
pi@dex:~ /SmartSensor/s3_scripts $
      • Pi Zero W:
      pi@raspberrypi:~ /SmartSensor $ sudo ./sock_test_client.py
Sending data...
Received 'Hello back!' from server!
Sending data...
Received 'Hello back!' from server!
Sending data...
Received 'Good-bye!' from server!
Signing off: Good-bye.
pi@raspberrypi:~ /SmartSensor $

  3. Get some data and process it using scikit-learn. Remember, it selects the best model after training, so your results may not be the same as ours:

    • S3:
    pi@dex:~ /SmartSensor/s3_scripts $ sudo ./smart_sensor_server.py
Starting Smart Sensor Server...

Training and testing model...
Model selected: Random Forest Classifier (1.00%).
Training and testing model complete.
Elapsed time: 16.272276639938354 seconds.

Checking model against unlabeled data...
Data to be evaluated:
Sample #1: [[1013.25, 0.1, 50.0, 1.0, 0.61, 23.0]] = Slightly Cool
Sample #2: [[1013.25, 0.1, 60.0, 1.0, 0.61, 26.0]] = Neutral
Sample #3: [[1013.25, 0.1, 76.0, 1.0, 0.61, 28.0]] = Slightly Warm
Prediction(s):
Sample #1: Prediction: Slightly Cool (expected Slightly Cool)
Sample #2: Prediction: Neutral (expected Neutral)
Sample #3: Prediction: Slightly Warm (expected Slightly Warm)

Waiting for sensor data...
Received [[[1031.76, 0.1, 36.83, 1.0, 0.61, 24.65]], [[1031.53, 0.1, 36.81, 1.0, 0.61, 24.65]], [[1031.57, 0.1, 36.83, 1.0, 0.61, 24.66]]] from client!
Collection complete.

Processing sensor data...
Sensor data #1: Prediction: Neutral
Sensor data #2: Prediction: Neutral
Sensor data #3: Prediction: Neutral
Overall sensation: 3 (Neutral)

Waiting for sensor data...
Received [[[1031.53, 0.1, 36.59, 1.0, 0.61, 24.68]], [[1031.55, 0.1, 36.59, 1.0, 0.61, 24.71]], [[1031.54, 0.1, 36.58, 1.0, 0.61, 24.68]]] from client!
Collection complete.

Processing sensor data...
Sensor data #1: Prediction: Neutral
Sensor data #2: Prediction: Neutral
Sensor data #3: Prediction: Neutral
Overall sensation: 3 (Neutral)

Waiting for sensor data...
Received [[[1031.57, 0.1, 36.73, 1.0, 0.61, 24.79]], [[1031.52, 0.1, 36.73, 1.0, 0.61, 24.76]], [[1031.51, 0.1, 36.72, 1.0, 0.61, 24.79]]] from client!
Collection complete.

Processing sensor data...
Sensor data #1: Prediction: Neutral
Sensor data #2: Prediction: Neutral
Sensor data #3: Prediction: Neutral
Overall sensation: 3 (Neutral)

Shutting down board...
Job complete. Have an excellent day.
pi@dex:~/SmartSensor/s3_scripts $
    • Pi Zero W:
    pi@raspberrypi:~ /SmartSensor $ sudo ./smart_sensor_client.py
Starting Smart Sensor Client...
Collecting atmospheric pressure, temperature, and relative humidity...
Sample #1 collected: [[1031.76, 0.1, 36.83, 1.0, 0.61, 24.65]]
Sample #2 collected: [[1031.53, 0.1, 36.81, 1.0, 0.61, 24.65]]
Sample #3 collected: [[1031.57, 0.1, 36.83, 1.0, 0.61, 24.66]]]
Sending data...
Received command for 30 second delay from server.
Collecting atmospheric pressure, temperature, and relative humidity...
Sample #1 collected: [[1031.53, 0.1, 36.59, 1.0, 0.61, 24.68]]
Sample #2 collected: [[1031.55, 0.1, 36.59, 1.0, 0.61, 24.71]]
Sample #3 collected: [[1031.54, 0.1, 36.58, 1.0, 0.61, 24.68]]
Sending data...
Received command for 30 second delay from server.
Collecting atmospheric pressure, temperature, and relative humidity...
Sample #1 collected: [[1031.57, 0.1, 36.73, 1.0, 0.61, 24.79]]
Sample #2 collected: [[1031.52, 0.1, 36.73, 1.0, 0.61, 24.76]]
Sample #3 collected: [[1031.51, 0.1, 36.72, 1.0, 0.61, 24.79]]
Sending data...
Shutting down client. Good-bye.
pi@raspberrypi:~ /SmartSensor $

And that's it! Remember, you do not have to use our setup to create a socket server and client. Good luck!

References