Skip to content

[hotfix][docs] Clarify Kryo behavior #26883

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open

[hotfix][docs] Clarify Kryo behavior #26883

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 10 additions & 1 deletion ...ent.zh/docs/dev/datastream/fault-tolerance/serialization/types_serialization.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -393,7 +393,7 @@ If you observe unexpected behavior, manually specify the return type using the `
#### Serialization of POJO types

The `PojoTypeInfo` is creating serializers for all the fields inside the POJO. Standard types such as
int, long, String etc. are handled by serializers we ship with Flink.
int, long, String, etc. are handled by serializers we ship with Flink.
For all other types, we fall back to [Kryo](https://github.com/EsotericSoftware/kryo).

If Kryo is not able to handle the type, you can ask the `PojoTypeInfo` to serialize the POJO using [Avro](https://avro.apache.org).
Expand Down Expand Up @@ -429,6 +429,15 @@ or via user-defined custom serializers. To do that, set:
pipeline.generic-types: false
```

{{< hint warning >}}
Note that Kryo will deserialize any class on the classpath
* when `pipeline.generic-types: true`
* the Flink job has a type definition such as `DataStream<Object>`, or `DataStream<Tuple2<Integer,Object>>`.

A malicious actor who knows the Flink job and controls the data input to Kryo will be able to instantiate classes which are
not intended for instantiation.
{{< /hint >}}

An exception will be raised whenever a data type is encountered that would go through Kryo.

## Defining Type Information using a Factory
Expand Down
4 changes: 4 additions & 0 deletions docs/content.zh/docs/ops/production_ready.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -81,6 +81,10 @@ It is a single point of failure within the cluster, and if it crashes, no new jo

Configuring [High Availability]({{< ref "docs/deployment/ha/overview" >}}), in conjunction with Apache Zookeeper or Flinks Kubernetes based service, allows for a swift recovery and is highly recommended for production setups.

### Harden Kryo Serialization

[Disable support for generic Kryo types]({{< ref "docs/dev/datastream/fault-tolerance/serialization/types_serialization" >}}#disabling-kryo-fallback), as this is a security and performance concern.

### Secure Flink Cluster Access

Flink is intentionally designed to support remote code execution.
Expand Down
11 changes: 10 additions & 1 deletion ...ontent/docs/dev/datastream/fault-tolerance/serialization/types_serialization.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -394,7 +394,7 @@ If you observe unexpected behavior, manually specify the return type using the `
#### Serialization of POJO types

The `PojoTypeInfo` is creating serializers for all the fields inside the POJO. Standard types such as
int, long, String etc. are handled by serializers we ship with Flink.
int, long, String, etc. are handled by serializers we ship with Flink.
For all other types, we fall back to [Kryo](https://github.com/EsotericSoftware/kryo).

If Kryo is not able to handle the type, you can ask the `PojoTypeInfo` to serialize the POJO using [Avro](https://avro.apache.org).
Expand Down Expand Up @@ -430,6 +430,15 @@ or via user-defined custom serializers. To do that, set:
pipeline.generic-types: false
```

{{< hint warning >}}
Note that Kryo will deserialize any class on the classpath
* when `pipeline.generic-types: true`
* the Flink job has a type definition such as `DataStream<Object>`, or `DataStream<Tuple2<Integer,Object>>`.

A malicious actor who knows the Flink job and controls the data input to Kryo will be able to instantiate classes which are
not intended for instantiation.
{{< /hint >}}

An exception will be raised whenever a data type is encountered that would go through Kryo.

## Defining Type Information using a Factory
Expand Down
4 changes: 4 additions & 0 deletions docs/content/docs/ops/production_ready.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -81,6 +81,10 @@ It is a single point of failure within the cluster, and if it crashes, no new jo

Configuring [High Availability]({{< ref "docs/deployment/ha/overview" >}}), in conjunction with Apache Zookeeper or Flinks Kubernetes based service, allows for a swift recovery and is highly recommended for production setups.

### Harden Kryo Serialization

[Disable support for generic Kryo types]({{< ref "docs/dev/datastream/fault-tolerance/serialization/types_serialization" >}}#disabling-kryo-fallback), as this is a security and performance concern.

### Secure Flink Cluster Access

Flink is intentionally designed to support remote code execution.
Expand Down