Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add release 0.95.0 #180

Merged
merged 4 commits into from
Jun 13, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/01_try-installation.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ best experience), Firefox or Edge.

## Install StreamPipes

<DownloadSection version={'0.93.0'}></DownloadSection>
<DownloadSection version={'0.95.0'} releaseDate={'2024-06-13'}></DownloadSection>

## Setup StreamPipes

Expand Down
215 changes: 215 additions & 0 deletions website-v2/blog/2024-06-13-release-095.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,215 @@
---
title: Apache StreamPipes release 0.95.0
author: Dominik Riemer
authorURL: "https://github.com/dominikriemer"
authorImageURL: "/img/riemer.png"
---

We're excited to share the release of StreamPipes 0.95.0, bringing with it a range of bug fixes and new features.
In total, the newest release closes over 200 issues.
This release doesn't include many large feature drops, we have worked a lot on making small things smoother and improving StreamPipes under the hood.
As we continue to work on the 1.0 release of Apache StreamPipes probably later this year, this release features many improvements to the end-user experience and bug fixes.

Let's delve into the key updates introduced in this release.

<!--truncate-->

The current release can be downloaded <a href="https://streampipes.apache.org/download.html">here</a>.

## Adapters & Pipeline Elements

### Improve deletion workflow of adapters

Within StreamPipes 0.93.0 we introduced the possibility to [edit existing adapters](https://streampipes.apache.org/blog/2023/11/28/release-093/#editing-existing-adapters).
This release improves adapter management by conveniently allowing to delete existing adapters even if they are used in a pipeline.
When deleting a new adapter, a dialog appears which allows users to delete all pipelines which are using the data stream produced by the adapter.

https://github.com/apache/streampipes/issues/1936

### Enhance data lake schema management

We've added a new option to the data lake sink which provides users more flexibility when writing data to the internal time-series storage.
Users can choose between the `Update Schema` and `Extend existing schema` options.
By choosing `Extend schema`, it is possible to also see "old" data in the data explorer after the schema has changed.

<img src="/img/blog/2024-06-13/data-lake-sink-configuration.png" style={{width: '70%', marginLeft: 'auto', marginRight: 'auto'}}/>

https://github.com/apache/streampipes/issues/2252

### New Adapter: Open Industry 4.0 devices

A new adapter in 0.95.0 is support for the [Open Industry 4.0](https://openindustry4.com/) specification over MQTT.
The adapter can be used to connect to a variety of devices which support the data format defined in the [Development Guidelines for Open Edge Computing](https://openindustry4.com/fileadmin/Dateien/Downloads/OEC_Development_Guideline_V1.1.1.pdf).

For instance, the adapter makes it easy to use IO-Link sensors together with an OI4-compatible IO-Link master.

https://github.com/apache/streampipes/pull/2511

### New Adapter(s): All PLC4X protocols supported as adapters

Since our "sister project" [Apache PLC4X](https://plc4x.apache.org) has introduced a new metadata API for gathering available drivers, StreamPipes 0.95.0 now supports a much larger collection of PLC connectors.
The list of newly supported protocols include:
* Allen Bradley ETH
* BACnet/IP
* Beckhoff TwinCat ADS
* EthernetIP
* Modbus ASCII/RTU/TCP

The already existing S7 adapter is now also available with additional configuration options.

<img src="/img/blog/2024-06-13/plc-adapter-overview.png"/>

Configurations on protocol and transport level are dynamically resolved from the PLC4X API. This ensures that users now have much greater flexibility when connecting to industrial protocols.

The screenshot below shows the advanced options panel for Modbus TCP:

<img src="/img/blog/2024-06-13/modbus-configuration.png"/>

The new adapters should be considered a beta feature and we still recommend to use the existing Modbus and S7 adapters until the new adapters have proven their stability.
More information on the usage of these new adapters and supported features can also be found on the PLC4X website.

https://github.com/apache/streampipes/issues/2632

### New Processor: Static Metadata enrichment

We added a new processor to add static metadata (such as sensor ID or location IDs) to any data stream within a pipeline.

https://github.com/apache/streampipes/issues/2350

### New Processor: Datetime from String

The new `Datetime from String` processor allows to convert a datetime string to a timestamp value and lets users conveniently choose the desired target time zone.

<img src="/img/blog/2024-06-13/datetime-from-string.png"/>

https://github.com/apache/streampipes/issues/1865

### New Sink: MS Teams

StreamPipes 0.95.0 comes with a new data sink for Microsoft Teams, which allows to publish notifications from a pipeline to a teams channel using Webhooks.

<img src="/img/blog/2024-06-13/ms-teams-sink.png"/>

https://github.com/apache/streampipes/pull/2248


### Remove deprecated adapters

We removed several outdated adapters that were deprecated in release 0.93.0:
* CoindeskBitcoinAdapter
* GdeltAdapter
* FlicMQTTAdapter
* TISensorTag
* IexCloudAdapter
* OpenSenseMapAdapter
* WikipediaAdapter

https://github.com/apache/streampipes/issues/2128

## UX improvements

### Display file types in different colors

Different file types in the `Files` overview are now better highlighted by a dedicated color.

https://github.com/apache/streampipes/issues/2107

### Download button for asset & files

In the asset and file overview, resources can be now directly downloaded by using a new download button.

https://github.com/apache/streampipes/issues/2192
https://github.com/apache/streampipes/issues/2074

## StreamPipes Python

As already mentioned in our [blog post](/blog/2024/03/27/anomaly-detection-with-python-functions/), we added an example to use an existing [ONNX](https://onnx.ai/) model as part of a StreamPipes function.

https://github.com/apache/streampipes/pull/2687

## Administration & Development

### Remove Consul entirely

Following the replacement of Consul with an internal service registration and discovery mechanism, StreamPipes 0.95.0 also removes Consul from all installation files.

https://github.com/apache/streampipes/issues/2133

## New Apache StreamPipes Go-Client

Apache StreamPipes now includes a new client library for Golang.
Our first version of the Go client interacts with the StreamPipes API and provides various ways to gather data from StreamPipes.

Here is an example on the usage of the Go client:

```go


package main

import (
"github.com/apache/streampipes/streampipes-client-go/streampipes"
"github.com/apache/streampipes/streampipes-client-go/streampipes/config"
"log"
)

/*
Here are some examples of using go client, including outputting the data returned by streams.
Only supports outputting model data
*/

func main() {

clientConfig := config.StreamPipesClientConfig{
Url: "http://localhost:8030",
Credential: config.StreamPipesApiKeyCredentials{
UserName: "<Your-User-Name>",
ApiKey: "<Your-API-Key>",
},
}

streamPipesClient, err := streampipes.NewStreamPipesClient(clientConfig)
if err != nil {
log.Fatal(err)
}

dataSeries, err := streamPipesClient.DataLakeMeasures().GetSingleDataSeries("measureName")
if err != nil {
log.Fatal(err)
}
dataSeries.Print()

/*
output format:

There are 2 pieces of DataSerie in the Dataseries
The 1 DataSeries
time msg test
2024-02-23T13:37:09.052Z go-client_test 2f4556
2024-02-23T13:37:26.044Z go-client_test 2f4556
2024-02-23T13:37:29.007Z go-client_test 2f4556
The 2 DataSeries
time msg test
2024-02-23T13:38:06.052Z go-client_test 2f4556
2024-02-23T13:38:35.044Z go-client_test 2f4556
2024-02-23T13:38:38.007Z go-client_test 2f4556

*/

}
```


https://github.com/apache/streampipes/pull/2437

First release, are working on more examples and better docs - stay tuned

## Final Remarks

We highly recommend updating to StreamPipes 0.95.0 to take advantage of these new features, bug fixes, and improvements.
For more detailed information, please refer to
the [release notes](https://github.com/apache/incubator-streampipes/blob/release/0.95.0/RELEASE_NOTES.md#0950).

We appreciate your continued support and valuable feedback.
StreamPipes is continually evolving, and we are dedicated to providing you with a powerful and reliable platform for
your IIoT streaming data needs.
2 changes: 1 addition & 1 deletion website-v2/docusaurus.config.js
Original file line number Diff line number Diff line change
Expand Up @@ -100,7 +100,7 @@ module.exports = {
textColor: 'white',
isCloseable: false,
content:
'Apache StreamPipes 0.93.0 is available! ⭐️',
'Apache StreamPipes 0.95.0 is available! ⭐️',
},
"image": "img/favicon.png",
"footer": {
Expand Down
16 changes: 1 addition & 15 deletions website-v2/sidebars.json
Original file line number Diff line number Diff line change
Expand Up @@ -53,7 +53,6 @@
"items": [
"pe/org.apache.streampipes.processors.transformation.jvm.booloperator.counter",
"pe/org.apache.streampipes.processors.filters.jvm.processor.booleanfilter",
"pe/org.apache.streampipes.processors.filters.jvm.processor.booleanfilter",
"pe/org.apache.streampipes.processors.transformation.jvm.booloperator.inverter",
"pe/org.apache.streampipes.processors.transformation.jvm.booloperator.logical",
"pe/org.apache.streampipes.processors.transformation.jvm.booloperator.timer",
Expand All @@ -62,7 +61,6 @@
"pe/org.apache.streampipes.processors.transformation.jvm.duration-value",
"pe/org.apache.streampipes.processors.textmining.jvm.chunker",
"pe/org.apache.streampipes.processors.filters.jvm.compose",
"pe/org.apache.streampipes.processors.filters.jvm.compose",
"pe/org.apache.streampipes.processors.transformation.jvm.count-array",
"pe/org.apache.streampipes.processors.siddhi.count",
"pe/org.apache.streampipes.processors.transformation.jvm.datetime",
Expand Down Expand Up @@ -93,25 +91,18 @@
"pe/org.apache.streampipes.processors.transformation.jvm.booloperator.timekeeping",
"pe/org.apache.streampipes.processors.transformation.jvm.measurementunitconverter",
"pe/org.apache.streampipes.processors.filters.jvm.enrich",
"pe/org.apache.streampipes.processors.filters.jvm.enrich",
"pe/org.apache.streampipes.processors.filters.jvm.schema",
"pe/org.apache.streampipes.processors.filters.jvm.schema",
"pe/org.apache.streampipes.processors.filters.jvm.movingaverage",
"pe/org.apache.streampipes.processors.filters.jvm.movingaverage",
"pe/org.apache.streampipes.processors.textmining.jvm.namefinder",
"pe/org.apache.streampipes.processors.transformation.jvm.processor.state.labeler.number",
"pe/org.apache.streampipes.processors.transformation.jvm.round",
"pe/org.apache.streampipes.processors.filters.jvm.numericalfilter",
"pe/org.apache.streampipes.processors.filters.jvm.numericalfilter",
"pe/org.apache.streampipes.processors.siddhi.numericalfilter",
"pe/org.apache.streampipes.processors.filters.jvm.numericaltextfilter",
"pe/org.apache.streampipes.processors.filters.jvm.numericaltextfilter",
"pe/org.apache.streampipes.processors.textmining.jvm.partofspeech",
"pe/org.apache.streampipes.processors.filters.jvm.project",
"pe/org.apache.streampipes.processors.filters.jvm.project",
"pe/org.apache.streampipes.processor.imageclassification.qrcode",
"pe/org.apache.streampipes.processors.filters.jvm.limit",
"pe/org.apache.streampipes.processors.filters.jvm.limit",
"pe/org.apache.streampipes.processors.textmining.jvm.sentencedetection",
"pe/org.apache.streampipes.processors.transformation.jvm.processor.booloperator.edge",
"pe/org.apache.streampipes.processors.transformation.jvm.split-array",
Expand All @@ -121,16 +112,11 @@
"pe/org.apache.streampipes.processors.transformation.jvm.stringoperator.timer",
"pe/org.apache.streampipes.processors.transformation.jvm.processor.stringoperator.state",
"pe/org.apache.streampipes.processors.filters.jvm.sdt",
"pe/org.apache.streampipes.processors.filters.jvm.sdt",
"pe/org.apache.streampipes.processors.filters.jvm.merge",
"pe/org.apache.streampipes.processors.filters.jvm.merge",
"pe/org.apache.streampipes.processors.transformation.jvm.taskduration",
"pe/org.apache.streampipes.processors.filters.jvm.textfilter",
"pe/org.apache.streampipes.processors.filters.jvm.textfilter",
"pe/org.apache.streampipes.processors.filters.jvm.threshold",
"pe/org.apache.streampipes.processors.filters.jvm.threshold",
"pe/org.apache.streampipes.processors.filters.jvm.throughputmon",
"pe/org.apache.streampipes.processors.filters.jvm.throughputmon",
"pe/org.apache.streampipes.processors.transformation.jvm.processor.timestampextractor",
"pe/org.apache.streampipes.processors.textmining.jvm.tokenizer",
"pe/org.apache.streampipes.processors.siddhi.topk",
Expand Down Expand Up @@ -214,4 +200,4 @@
"faq-common-problems"
]
}
}
}
2 changes: 1 addition & 1 deletion website-v2/src/pages/download.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ const Downloads: FC = () => (
<div className="installation-manual-part">
<h5 className="feature-item-margin"><span className="feature-highlights-bg">Installation</span></h5>
<ul>
<DownloadSection version={'0.93.0'} showMoreInfo={true} releaseDate={'2023-11-27'}></DownloadSection>
<DownloadSection version={'0.95.0'} showMoreInfo={true} releaseDate={'2024-06-13'}></DownloadSection>
<hr/>
</ul>
</div>
Expand Down
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
62 changes: 62 additions & 0 deletions website-v2/versioned_docs/version-0.95.0/01_try-installation.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,62 @@
---
id: try-installation
title: Installation
sidebar_label: Installation
---

import DownloadSection from '@site/src/components/download/DownloadSection.tsx';

The easiest way to install StreamPipes is our Docker-based installation. For production-grade deployments, we also
recommend looking at our Kubernetes support, which is also part of the installation kit.

## Prerequisites

The Docker-based installation requires **Docker** and **Docker Compose** to be installed on the target machine.
Installation instructions can be found below.

:::info Install Docker
Go to https://docs.docker.com/installation/ and follow the instructions to install Docker for your OS. Make sure
docker can be started as a non-root user (described in the installation manual, don’t forget to log out and in
again) and check that Docker is installed correctly by executing docker-run hello-world
:::

### Supported operating systems

The Docker-based installation supports the operating systems **Linux**, **Mac OS X** and **Windows 10 upwards**. Older windows
versions are not fully compatible with Docker. Linux VMs running under Windows might cause network problems with Docker,
therefore some manual work might be needed to make StreamPipes run properly.

### Web Browser

The StreamPipes application itself will be accessible through a web browser. We recommend a recent version of Chrome (
best experience), Firefox or Edge.

## Install StreamPipes

<DownloadSection version={'0.95.0'} releaseDate={'2024-06-13'}></DownloadSection>

## Setup StreamPipes

Once you've opened the browser at the URL given above, you should see the StreamPipes application as shown below. At
initial startup, StreamPipes automatically performs an installation process.
After the installation has finished, continue by clicking on "Go to login
page", once all components are successfully configured.

On the login page, enter your credentials, then you should be forwarded to the home page.

Congratulations! You've successfully managed to install StreamPipes. Now we're ready to build our first pipeline!

<div class="my-carousel docs-carousel">
<img src="/img/01_try-installation/04_home.png" alt="Home page"/>
</div>

:::danger Errors during the installation process
In most cases, errors during the installation are due to an under-powered system.
If there is a problem with any of the components, please restart the whole system (`docker-compose
down` and eventually also delete the volumes).
Please also make sure that you've assigned enough memory available to Docker.
:::

## Next Steps

That's it! Have a look at the usage guide to learn how to use Apache StreamPipes.
Loading
Loading