Event Fabrication Simulator (ef-simØ)

Event Fabrication is the deliberate generation of synthetic events within an event-driven IoT architecture to compensate for missing or incomplete sensor data. In real-world IoT environments, missing events occur due to sensor malfunctions, network disruptions, or environmental conditions, which can negatively impact the accuracy and availability of complex event processing (CEP) systems. Event fabrication addresses this challenge by creating artificial events that mimic real-world data patterns, allowing CEP engines to maintain service reliability and completeness. This process can be achieved through various methods, such as utilizing past sensor data or applying time-series forecasting models like ARIMA to predict missing values. By integrating event fabrication mechanisms, IoT systems can enhance data quality, improve decision-making, and ensure uninterrupted operation even in non-ideal sensing conditions.

For more details on quality-aware event fabrication and hybrid simulation platforms, refer to the following paper:

A Hybrid Simulation Platform for Quality-Aware Evaluation of Complex Events in an IoT Environment
Dimitris Gkoulis, Cleopatra Bardaki, Mara Nikolaidou, George Kousiouris, Anargyros Tsadimas
Published in Simulation Modelling Practice and Theory, Volume 133, 2024

@article{GKOULIS2024102919,
    title = {A Hybrid Simulation Platform for quality-aware evaluation of complex events in an IoT environment},
    journal = {Simulation Modelling Practice and Theory},
    volume = {133},
    pages = {102919},
    year = {2024},
    issn = {1569-190X},
    doi = {https://doi.org/10.1016/j.simpat.2024.102919},
    url = {https://www.sciencedirect.com/science/article/pii/S1569190X24000339},
    author = {Dimitris Gkoulis and Cleopatra Bardaki and Mara Nikolaidou and George Kousiouris and Anargyros Tsadimas},
    keywords = {Hybrid simulation environments, Internet of Things (IoT), Quality-aware complex event processing (CEP), Quality evaluation},
    abstract = {Complex Event Processing (CEP) is a successful method to transform simple IoT events created by sensors into meaningful complex business events. To enhance availability, an event fabrication mechanism is integrated within the CEP model, generating synthetic events to offset missing data, resulting in a quality-aware CEP model. In this model, generated complex events are characterized by quality properties, namely completeness and timeliness. To empirically assess the quality of complex events through experimentation, we have developed a hybrid simulation platform. The platform’s dual nature stems from its distinctive approach of simulating sensor behaviors while concurrently running the quality-aware CEP IoT platform. Users can conduct experiments that closely mimic actual operational scenarios and have, in real-time, full visibility and control over all involved aspects, including composite transformations, quality assessment, event fabrication and its effectiveness, and aggregated reports. A representative experiment in an IoT-enabled greenhouse with missing events is presented to demonstrate the usefulness of the platform. The contribution of the hybrid simulation platform is twofold: provide (a) quality assessment of complex events, using two established quality properties for IoT environments with specific computation formulas and (b) a comprehensive testbed covering all aspects of a typical IoT setup for realistic experimentation. Together, these elements provide significant cost–benefit advantages by enabling researchers and practitioners to pre-optimize operational efficiency and decision-making in IoT systems.}
}

ef-simØ is a simulation platform designed to explore and evaluate various event fabrication methods in IoT environments. It enables researchers and practitioners to simulate missing data scenarios in streaming sensor data and test different approaches to compensate for data loss. The simulator models real-world sensor failures and employs techniques such as past-value reconstruction and time-series forecasting to generate synthetic events, ensuring the integrity and continuity of complex event processing (CEP) systems. By experimenting with different event fabrication methods, users can assess their effectiveness under varying conditions and determine optimal strategies for improving data availability and quality in IoT-based decision-making systems.

The simulation assumes an initially complete data stream with no missing events, which then experiences a total failure. However, there is a predefined time-to-fix period, during which the system is restored. The objective is to evaluate the reliability of simple event fabrication methods within this timeframe and determine the extent to which they can be trusted before more advanced techniques become necessary.

A common counterargument is that the failure might have started earlier than detected. The response to this concern is that the time-to-fix period inherently includes both the recovery of historical values—either by retrieving them through alternative means or by imputing large gaps in the data. Therefore, upon system restoration, the full sequence of historical values is available, even if some of these values have been reconstructed rather than directly sensed. This approach ensures a continuous and analyzable data stream despite temporary sensor failures.

To install and use the simulator, first create a virtual environment (Python 3.11 required):

python -m venv venv

Activate the virtual environment:

source venv/bin/activate

Install dependencies:

pip install -r requirements.txt

To run an exploratory analysis:

python -m ef_sim0.main exploratory_analysis

To run an experiment for temperature data:

python -m ef_sim0.main experiment TEMPERATURE