Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add schema change #27

Merged
merged 10 commits into from
Aug 12, 2024
Merged

Conversation

QuakeWang
Copy link
Contributor

close: #25

///
/// Reference: <https://github.com/apache/paimon/blob/release-0.8.2/paimon-core/src/main/java/org/apache/paimon/schema/SchemaChange.java#L36>
#[derive(Debug, PartialEq, Eq, Deserialize, Serialize)]
pub enum SchemaChange {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we don't need to specify types for every enum value. It's unlikely that we'll use SetOption without SchemaChange.

How about embedding the struct directly in this enum?

pub enum SchemaChange {
    SetOption { 
        key: String,
        value: String,
    }
    ...
}

// specific language governing permissions and limitations
// under the License.

use crate::spec::DataTypeRoot;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

DataTypeRoot has been removed. Please merge with main, thanks!

/// Reference: <https://github.com/apache/paimon/blob/release-0.8.2/paimon-core/src/main/java/org/apache/paimon/schema/SchemaChange.java#L410>
#[derive(Debug, Clone, PartialEq, Eq, Deserialize, Serialize)]
#[serde(rename_all = "camelCase")]
pub struct Move {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Move is a keyword in rust, how about naming it ColumnMove and ColumnMoveType?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Move is a keyword in rust, how about naming it ColumnMove and ColumnMoveType?

OK, I will do it.

use crate::spec::{DataType, IntType};

#[test]
fn test_set_option() {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would you like to add a test against real string?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would you like to add a test against real string?

Are these tests necessary? If they are not, I will move out and add serialize/deserialize tests for the schema_change.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If they are not, I will move out and add serialize/deserialize tests for the schema_change.

That makes more sense.

Copy link
Contributor Author

@QuakeWang QuakeWang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Xuanwo PTAL 👀


let schema_changes: Vec<SchemaChange> = serde_json::from_str(json_data).unwrap();

for change in schema_changes {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi, SchemaChange has implemented Eq, so we don't need to check it this way. Instead, we can:

assert_eq!(
    schema_changes, 
    vec![
        SchemaChange::SetOption {
            key: "snapshot.time-retained".to_string, 
            value: "2h".to_string()
        },
        ...
    ]);

Copy link
Contributor Author

@QuakeWang QuakeWang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

cc @Xuanwo

Copy link
Member

@Xuanwo Xuanwo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

@Xuanwo
Copy link
Member

Xuanwo commented Aug 11, 2024

Hi, @SteNicholas @Aitozi, this PR is happy to go.

@JingsongLi JingsongLi merged commit ec38d14 into apache:main Aug 12, 2024
7 checks passed
@QuakeWang QuakeWang deleted the add-schema_change branch August 12, 2024 02:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

spec: Impl SchemaChange
4 participants