Skip to content

Commit

Permalink
chore(extensions-docs): update extensions doc assets to recent develo…
Browse files Browse the repository at this point in the history
…pment state (#147)

Co-authored-by: dominikriemer <[email protected]>
  • Loading branch information
github-actions[bot] and dominikriemer authored Dec 9, 2023
1 parent 0e11353 commit f3910d6
Show file tree
Hide file tree
Showing 4 changed files with 98 additions and 3 deletions.
Original file line number Diff line number Diff line change
@@ -0,0 +1,78 @@
---
id: org.apache.streampipes.processors.transformation.jvm.datetime
title: Datetime From String
sidebar_label: Datetime From String
---

<!--
~ Licensed to the Apache Software Foundation (ASF) under one or more
~ contributor license agreements. See the NOTICE file distributed with
~ this work for additional information regarding copyright ownership.
~ The ASF licenses this file to You under the Apache License, Version 2.0
~ (the "License"); you may not use this file except in compliance with
~ the License. You may obtain a copy of the License at
~
~ http://www.apache.org/licenses/LICENSE-2.0
~
~ Unless required by applicable law or agreed to in writing, software
~ distributed under the License is distributed on an "AS IS" BASIS,
~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
~ See the License for the specific language governing permissions and
~ limitations under the License.
~
-->



<p align="center">
<img src="/img/pipeline-elements/org.apache.streampipes.processors.transformation.jvm.datetime/icon.png" width="150px;" class="pe-image-documentation"/>
</p>

***

## Overview

The "Datetime From String" processor is a handy tool that helps convert human-readable datetime information into a
format that machines can understand. This is particularly useful when dealing with data that includes dates and times.

### Why Use This Processor?

In the context of event streams, you may encounter dates and times formatted for human readability but not necessarily
optimized for computer processing. The "Datetime From String" processor addresses this by facilitating the conversion
of human-readable datetime information within your continuous stream of events.

***

## How It Works

When you input a data stream into this processor containing a datetime in a specific format (such as "2023-11-24 15:30:
00"), it
undergoes a transformation. The processor converts it into a computer-friendly format called a ZonedDateTime object.

### Example

Let's say you have an event stream with a property containing values like "2023-11-24 15:30:00" and you want to make
sure your computer understands it. You can use
this processor to convert it into a format that's machine-friendly.

***

## Getting Started

To use this processor, you need one thing in your data:

1. **Datetime String**: This is the name of the event property that contains the human-readable datetime string, like "2023-11-24 15:30:00".


### Configuration

The only thing you need to configure is the time zone.
1. **Time Zone**: Specify the time zone that applies to your datetime if it doesn't already have this information.This ensures that the processor understands the context of your
datetime.

## Output

After the conversion happens, the processor adds a new piece of information to your data stream:

* **timestringInMillis**: This is the transformed datetime in a format that computers can easily work with (UNIX timestamp in milliseconds).
* **timeZone**: The name of the timezone the `dateTime` value refers to. Can be used to reconstitute the actual time.
22 changes: 19 additions & 3 deletions docs/pe/org.apache.streampipes.sinks.internal.jvm.datalake.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,8 @@ sidebar_label: Data Lake
## Description

Stores events in the internal data lake so that data can be visualized in the live dashboard or in the data explorer.
Simply create a pipeline with a data lake sink, switch to one of the data exploration tool and start exploring your data!
Simply create a pipeline with a data lake sink, switch to one of the data exploration tool and start exploring your
data!

***

Expand All @@ -46,7 +47,22 @@ This sink requires an event that provides a timestamp value (a field that is mar

## Configuration

### Index
### Identifier

The name of the storage group of this event.
The name of the measurement (table) where the events are stored.

### Schema Update Options

The Schema Update Options dictate the behavior when encountering a measurement (table) with the same identifier.

#### Option 1: Update Schema

- **Description:** Overrides the existing schema.
- **Effect on Data:** The data remains in the data lake, but accessing old data is restricted to file export.
- **Impact on Features:** Other StreamPipes features, such as the Data Explorer, will only display the new event schema.

#### Option 2: Extend Existing Schema

- **Description:** Keeps old event fields in the event schema.
- **Strategy:** This follows an append-only strategy, allowing continued work with historic data.
- **Consideration:** Old properties may exist for which no new data is generated.
1 change: 1 addition & 0 deletions website-v2/sidebars.json
Original file line number Diff line number Diff line change
Expand Up @@ -70,6 +70,7 @@
"pe/org.apache.streampipes.processors.filters.jvm.compose",
"pe/org.apache.streampipes.processors.transformation.jvm.count-array",
"pe/org.apache.streampipes.processors.siddhi.count",
"pe/org.apache.streampipes.processors.transformation.jvm.datetime",
"pe/org.apache.streampipes.processors.transformation.jvm.fieldhasher",
"pe/org.apache.streampipes.processors.transformation.jvm.field-mapper",
"pe/org.apache.streampipes.processors.transformation.jvm.fieldrename",
Expand Down
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.

0 comments on commit f3910d6

Please sign in to comment.