-
Notifications
You must be signed in to change notification settings - Fork 20
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
* draft release blog post * Update blog post, add 0.95.0 release * Remove duplicated pipeline elements from sidebar * Create release 0.95.0 --------- Co-authored-by: bossenti <[email protected]>
- Loading branch information
1 parent
52b7334
commit 33dbbff
Showing
184 changed files
with
13,849 additions
and
18 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,215 @@ | ||
| --- | ||
| title: Apache StreamPipes release 0.95.0 | ||
| author: Dominik Riemer | ||
| authorURL: "https://github.com/dominikriemer" | ||
| authorImageURL: "/img/riemer.png" | ||
| --- | ||
|
|
||
| We're excited to share the release of StreamPipes 0.95.0, bringing with it a range of bug fixes and new features. | ||
| In total, the newest release closes over 200 issues. | ||
| This release doesn't include many large feature drops, we have worked a lot on making small things smoother and improving StreamPipes under the hood. | ||
| As we continue to work on the 1.0 release of Apache StreamPipes probably later this year, this release features many improvements to the end-user experience and bug fixes. | ||
|
|
||
| Let's delve into the key updates introduced in this release. | ||
|
|
||
| <!--truncate--> | ||
|
|
||
| The current release can be downloaded <a href="https://streampipes.apache.org/download.html">here</a>. | ||
|
|
||
| ## Adapters & Pipeline Elements | ||
|
|
||
| ### Improve deletion workflow of adapters | ||
|
|
||
| Within StreamPipes 0.93.0 we introduced the possibility to [edit existing adapters](https://streampipes.apache.org/blog/2023/11/28/release-093/#editing-existing-adapters). | ||
| This release improves adapter management by conveniently allowing to delete existing adapters even if they are used in a pipeline. | ||
| When deleting a new adapter, a dialog appears which allows users to delete all pipelines which are using the data stream produced by the adapter. | ||
|
|
||
| https://github.com/apache/streampipes/issues/1936 | ||
|
|
||
| ### Enhance data lake schema management | ||
|
|
||
| We've added a new option to the data lake sink which provides users more flexibility when writing data to the internal time-series storage. | ||
| Users can choose between the `Update Schema` and `Extend existing schema` options. | ||
| By choosing `Extend schema`, it is possible to also see "old" data in the data explorer after the schema has changed. | ||
|
|
||
| <img src="/img/blog/2024-06-13/data-lake-sink-configuration.png" style={{width: '70%', marginLeft: 'auto', marginRight: 'auto'}}/> | ||
|
|
||
| https://github.com/apache/streampipes/issues/2252 | ||
|
|
||
| ### New Adapter: Open Industry 4.0 devices | ||
|
|
||
| A new adapter in 0.95.0 is support for the [Open Industry 4.0](https://openindustry4.com/) specification over MQTT. | ||
| The adapter can be used to connect to a variety of devices which support the data format defined in the [Development Guidelines for Open Edge Computing](https://openindustry4.com/fileadmin/Dateien/Downloads/OEC_Development_Guideline_V1.1.1.pdf). | ||
|
|
||
| For instance, the adapter makes it easy to use IO-Link sensors together with an OI4-compatible IO-Link master. | ||
|
|
||
| https://github.com/apache/streampipes/pull/2511 | ||
|
|
||
| ### New Adapter(s): All PLC4X protocols supported as adapters | ||
|
|
||
| Since our "sister project" [Apache PLC4X](https://plc4x.apache.org) has introduced a new metadata API for gathering available drivers, StreamPipes 0.95.0 now supports a much larger collection of PLC connectors. | ||
| The list of newly supported protocols include: | ||
| * Allen Bradley ETH | ||
| * BACnet/IP | ||
| * Beckhoff TwinCat ADS | ||
| * EthernetIP | ||
| * Modbus ASCII/RTU/TCP | ||
|
|
||
| The already existing S7 adapter is now also available with additional configuration options. | ||
|
|
||
| <img src="/img/blog/2024-06-13/plc-adapter-overview.png"/> | ||
|
|
||
| Configurations on protocol and transport level are dynamically resolved from the PLC4X API. This ensures that users now have much greater flexibility when connecting to industrial protocols. | ||
|
|
||
| The screenshot below shows the advanced options panel for Modbus TCP: | ||
|
|
||
| <img src="/img/blog/2024-06-13/modbus-configuration.png"/> | ||
|
|
||
| The new adapters should be considered a beta feature and we still recommend to use the existing Modbus and S7 adapters until the new adapters have proven their stability. | ||
| More information on the usage of these new adapters and supported features can also be found on the PLC4X website. | ||
|
|
||
| https://github.com/apache/streampipes/issues/2632 | ||
|
|
||
| ### New Processor: Static Metadata enrichment | ||
|
|
||
| We added a new processor to add static metadata (such as sensor ID or location IDs) to any data stream within a pipeline. | ||
|
|
||
| https://github.com/apache/streampipes/issues/2350 | ||
|
|
||
| ### New Processor: Datetime from String | ||
|
|
||
| The new `Datetime from String` processor allows to convert a datetime string to a timestamp value and lets users conveniently choose the desired target time zone. | ||
|
|
||
| <img src="/img/blog/2024-06-13/datetime-from-string.png"/> | ||
|
|
||
| https://github.com/apache/streampipes/issues/1865 | ||
|
|
||
| ### New Sink: MS Teams | ||
|
|
||
| StreamPipes 0.95.0 comes with a new data sink for Microsoft Teams, which allows to publish notifications from a pipeline to a teams channel using Webhooks. | ||
|
|
||
| <img src="/img/blog/2024-06-13/ms-teams-sink.png"/> | ||
|
|
||
| https://github.com/apache/streampipes/pull/2248 | ||
|
|
||
|
|
||
| ### Remove deprecated adapters | ||
|
|
||
| We removed several outdated adapters that were deprecated in release 0.93.0: | ||
| * CoindeskBitcoinAdapter | ||
| * GdeltAdapter | ||
| * FlicMQTTAdapter | ||
| * TISensorTag | ||
| * IexCloudAdapter | ||
| * OpenSenseMapAdapter | ||
| * WikipediaAdapter | ||
|
|
||
| https://github.com/apache/streampipes/issues/2128 | ||
|
|
||
| ## UX improvements | ||
|
|
||
| ### Display file types in different colors | ||
|
|
||
| Different file types in the `Files` overview are now better highlighted by a dedicated color. | ||
|
|
||
| https://github.com/apache/streampipes/issues/2107 | ||
|
|
||
| ### Download button for asset & files | ||
|
|
||
| In the asset and file overview, resources can be now directly downloaded by using a new download button. | ||
|
|
||
| https://github.com/apache/streampipes/issues/2192 | ||
| https://github.com/apache/streampipes/issues/2074 | ||
|
|
||
| ## StreamPipes Python | ||
|
|
||
| As already mentioned in our [blog post](/blog/2024/03/27/anomaly-detection-with-python-functions/), we added an example to use an existing [ONNX](https://onnx.ai/) model as part of a StreamPipes function. | ||
|
|
||
| https://github.com/apache/streampipes/pull/2687 | ||
|
|
||
| ## Administration & Development | ||
|
|
||
| ### Remove Consul entirely | ||
|
|
||
| Following the replacement of Consul with an internal service registration and discovery mechanism, StreamPipes 0.95.0 also removes Consul from all installation files. | ||
|
|
||
| https://github.com/apache/streampipes/issues/2133 | ||
|
|
||
| ## New Apache StreamPipes Go-Client | ||
|
|
||
| Apache StreamPipes now includes a new client library for Golang. | ||
| Our first version of the Go client interacts with the StreamPipes API and provides various ways to gather data from StreamPipes. | ||
|
|
||
| Here is an example on the usage of the Go client: | ||
|
|
||
| ```go | ||
|
|
||
|
|
||
| package main | ||
|
|
||
| import ( | ||
| "github.com/apache/streampipes/streampipes-client-go/streampipes" | ||
| "github.com/apache/streampipes/streampipes-client-go/streampipes/config" | ||
| "log" | ||
| ) | ||
|
|
||
| /* | ||
| Here are some examples of using go client, including outputting the data returned by streams. | ||
| Only supports outputting model data | ||
| */ | ||
|
|
||
| func main() { | ||
|
|
||
| clientConfig := config.StreamPipesClientConfig{ | ||
| Url: "http://localhost:8030", | ||
| Credential: config.StreamPipesApiKeyCredentials{ | ||
| UserName: "<Your-User-Name>", | ||
| ApiKey: "<Your-API-Key>", | ||
| }, | ||
| } | ||
|
|
||
| streamPipesClient, err := streampipes.NewStreamPipesClient(clientConfig) | ||
| if err != nil { | ||
| log.Fatal(err) | ||
| } | ||
|
|
||
| dataSeries, err := streamPipesClient.DataLakeMeasures().GetSingleDataSeries("measureName") | ||
| if err != nil { | ||
| log.Fatal(err) | ||
| } | ||
| dataSeries.Print() | ||
|
|
||
| /* | ||
| output format: | ||
| There are 2 pieces of DataSerie in the Dataseries | ||
| The 1 DataSeries | ||
| time msg test | ||
| 2024-02-23T13:37:09.052Z go-client_test 2f4556 | ||
| 2024-02-23T13:37:26.044Z go-client_test 2f4556 | ||
| 2024-02-23T13:37:29.007Z go-client_test 2f4556 | ||
| The 2 DataSeries | ||
| time msg test | ||
| 2024-02-23T13:38:06.052Z go-client_test 2f4556 | ||
| 2024-02-23T13:38:35.044Z go-client_test 2f4556 | ||
| 2024-02-23T13:38:38.007Z go-client_test 2f4556 | ||
| */ | ||
|
|
||
| } | ||
| ``` | ||
|
|
||
|
|
||
| https://github.com/apache/streampipes/pull/2437 | ||
|
|
||
| First release, are working on more examples and better docs - stay tuned | ||
|
|
||
| ## Final Remarks | ||
|
|
||
| We highly recommend updating to StreamPipes 0.95.0 to take advantage of these new features, bug fixes, and improvements. | ||
| For more detailed information, please refer to | ||
| the [release notes](https://github.com/apache/incubator-streampipes/blob/release/0.95.0/RELEASE_NOTES.md#0950). | ||
|
|
||
| We appreciate your continued support and valuable feedback. | ||
| StreamPipes is continually evolving, and we are dedicated to providing you with a powerful and reliable platform for | ||
| your IIoT streaming data needs. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Binary file added
BIN
+52.4 KB
website-v2/static/img/blog/2024-06-13/data-lake-sink-configuration.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
62 changes: 62 additions & 0 deletions
62
website-v2/versioned_docs/version-0.95.0/01_try-installation.md
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,62 @@ | ||
| --- | ||
| id: try-installation | ||
| title: Installation | ||
| sidebar_label: Installation | ||
| --- | ||
|
|
||
| import DownloadSection from '@site/src/components/download/DownloadSection.tsx'; | ||
|
|
||
| The easiest way to install StreamPipes is our Docker-based installation. For production-grade deployments, we also | ||
| recommend looking at our Kubernetes support, which is also part of the installation kit. | ||
|
|
||
| ## Prerequisites | ||
|
|
||
| The Docker-based installation requires **Docker** and **Docker Compose** to be installed on the target machine. | ||
| Installation instructions can be found below. | ||
|
|
||
| :::info Install Docker | ||
| Go to https://docs.docker.com/installation/ and follow the instructions to install Docker for your OS. Make sure | ||
| docker can be started as a non-root user (described in the installation manual, don’t forget to log out and in | ||
| again) and check that Docker is installed correctly by executing docker-run hello-world | ||
| ::: | ||
|
|
||
| ### Supported operating systems | ||
|
|
||
| The Docker-based installation supports the operating systems **Linux**, **Mac OS X** and **Windows 10 upwards**. Older windows | ||
| versions are not fully compatible with Docker. Linux VMs running under Windows might cause network problems with Docker, | ||
| therefore some manual work might be needed to make StreamPipes run properly. | ||
|
|
||
| ### Web Browser | ||
|
|
||
| The StreamPipes application itself will be accessible through a web browser. We recommend a recent version of Chrome ( | ||
| best experience), Firefox or Edge. | ||
|
|
||
| ## Install StreamPipes | ||
|
|
||
| <DownloadSection version={'0.95.0'} releaseDate={'2024-06-13'}></DownloadSection> | ||
|
|
||
| ## Setup StreamPipes | ||
|
|
||
| Once you've opened the browser at the URL given above, you should see the StreamPipes application as shown below. At | ||
| initial startup, StreamPipes automatically performs an installation process. | ||
| After the installation has finished, continue by clicking on "Go to login | ||
| page", once all components are successfully configured. | ||
|
|
||
| On the login page, enter your credentials, then you should be forwarded to the home page. | ||
|
|
||
| Congratulations! You've successfully managed to install StreamPipes. Now we're ready to build our first pipeline! | ||
|
|
||
| <div class="my-carousel docs-carousel"> | ||
| <img src="/img/01_try-installation/04_home.png" alt="Home page"/> | ||
| </div> | ||
|
|
||
| :::danger Errors during the installation process | ||
| In most cases, errors during the installation are due to an under-powered system. | ||
| If there is a problem with any of the components, please restart the whole system (`docker-compose | ||
| down` and eventually also delete the volumes). | ||
| Please also make sure that you've assigned enough memory available to Docker. | ||
| ::: | ||
|
|
||
| ## Next Steps | ||
|
|
||
| That's it! Have a look at the usage guide to learn how to use Apache StreamPipes. |
Oops, something went wrong.