Skip to content

Time series storage and analysis

saroele edited this page May 20, 2014 · 8 revisions

Open Source

Dasling

https://github.com/dasling/dasling-FE

Short description

dasling interconnects sensors and actuators through calculation steps and stores all this in a DB.

dasling allows you to:

  • log sensor readings to a database
  • abstraction sensor readings to variables
  • perform calculations on these variables (custom plug-in algorithms)
  • store the results of these calculation, or calculate further on these results, and so on
  • do calculation based on historical sensor data (and calculation results)
  • send actions to actuators based on these calculations

dasling can be useful for a range of interesting stuff, e.g. interconnecting hardware around the globe, performing home automation, driving a led system, ...

Strong points

  • Distributed, hence some security by default
  • Talks MQTT

Weak points

  • Small community :)
  • Not tested, not bug free
  • Database performance compared to TS-focused databases like Kairos etc?
  • no built-in visualization, large development to do?

Kairosdb

https://code.google.com/p/kairosdb/

Short description

KairosDB is a fast distributed scalable time series database (Cassandra | HBase). SIngle node up to 40,000 data points data / sec.

Strong points

  • Fast !
  • REST support easy
  • Has UI
  • Community active

Weak points

  • Probably not easy to get up and running (Roel, you should know guys that have experimented with setting up Cassandra).
  • Not easy to experiment with new views on the data, since it's NOSQL.
  • JAVA :)
  • MQTT support? (if we bridge, what with QoS support?)
  • REST support overkill?

##ThingSpeak https://github.com/iobridge/ThingSpeak

Short description

ThingSpeak is an open source “Internet of Things” application and API to store and retrieve data from things using HTTP over the Internet or via a Local Area Network. With ThingSpeak, you can create sensor logging applications, location tracking applications, and a social network of things with status updates.

Strong points

  • Apparently more active nowadays

Weak points

##Elasticsearch/Kibana

Short description

Elasticsearch is build on top of Apache Lucene and permits efficient searches through semi-structured data using flexible queries. This is ideal to explore and understand data without going through the hassle of maintaining tables and performing complex queries in a relational DB. Kibana is an open source tool to visualize and interacts with logs and time-stamped data stored in Elasticsearch.

Strong points

  • No code required
  • Real-time analysis of streaming data
  • Highly scalable
  • Easy to setup nice looking UI.
  • Professional and extremely active community.

Weak points

  • TBD

homA

https://github.com/binarybucks/homA

Short description

HomA is a free and open source framework for building smart homes and connecting them to the internet of things. It is designed to provide an entry into the world of home automation while also remaining highly extensible and adaptable for advanced users.

To facilitate the development of own hardware and software extensions, HomA relies on completely open technologies and standards that are available on many platforms. Hence, it uses the lightweight and open MQTT protocol for both data storage and command transmission. The HomA framework consists of various independent software and hardware components that can be added the system simply by connecting them to the central broker. Once connected, they can publish and subscribe to various topics on the MQTT message bus in order to interact with other components. The architecture overview below should give you a good overview about the system.

HomA architecture

Strong points

Weak points

  • designed for single house, not for large community project?

FHEM

http://fhem.de/fhem.html

Short description

Fhem is a GPL'd perl server for house automation. It is used to automate some common tasks in the household like switching lamps / shutters / heating / etc. and to log events like temperature / humidity / power consumption.

The program runs as a server, you can control it via web or smartphone frontends, telnet or TCP/IP directly.

In order to use FHEM you'll need a 24/7 server (Fritz!Box, NAS, RPi, PC, MacMini, etc) with a perl interpreter and some attached hardware like the CUL, FHZ1300PC, etc. to access the actors and sensors. See the "Supported hardware" section below.

Strong points

  • Lots of interfaces to existing commercially available home automation HW

Weak points

  • Has little DB interfaces (could be checked better)
  • designed for single house, not for large community project?

emoncms

https://github.com/emoncms/emoncms

Short description

Strong points

Weak points

  • designed for single house, not for large community project?

Closed source solutions

Opensense

http://open.sen.se/

Short description

Strong points

  • Easy to set up
  • API documented
  • Control to with building block

Weak points

  • Data ownership
  • External database

Xively

https://xively.com/

Short description

Strong points

  • Easy to set up
  • API documented

Weak points

  • Data ownership
  • External database