Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs: update docs home page and add pandas integration #1905

Merged
merged 4 commits into from
Nov 26, 2023

Conversation

MrPowers
Copy link
Contributor

Description

Updated the docs home page to include more information about the project.

Also added a pandas integration page. Lots more integration pages to come soon hopefully!

Copy link

ACTION NEEDED

delta-rs follows the Conventional Commits specification for release automation.

The PR title and description are used as the merge commit message. Please update your PR title and description to match the specification.

@ion-elgreco ion-elgreco changed the title Update docs home page and add pandas integration docs: Update docs home page and add pandas integration Nov 25, 2023

Time travel is a powerful feature that pandas users cannot access with CSV or Parquet.

## Schema enforcement
Copy link
Collaborator

@ion-elgreco ion-elgreco Nov 25, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we perhaps want to have a short section on how you can overwrite the schema and the table at the same time?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ion-elgreco - I actually just tried to do write_deltalake("tmp/some-table", df, mode="overwrite") thinking that would overwrite the scheme and table at the same time and it still gave a ValueError surprisingly. Is there another syntax? Should I also create an issue to add schema evolution?

Copy link
Collaborator

@ion-elgreco ion-elgreco Nov 26, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You need to also pass overwrite_schema = True

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ion-elgreco - thanks for clarifying, added a section. Let me know how it looks!!

docs/index.md Outdated
## Why implement the Delta Lake transaction log protocol in Rust and Scala?

Delta Spark depends on Java and Spark, which is fine for many use cases, but not all Delta Lake users want to depend on these libraries.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should we also mention that this allows using delta in Rust or other native projects? in many of these cases using a JVM is not an option.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added a sentence to clarify this. Good idea!

@roeap roeap changed the title docs: Update docs home page and add pandas integration docs: update docs home page and add pandas integration Nov 25, 2023
Copy link
Collaborator

@ion-elgreco ion-elgreco left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice!!

@ion-elgreco ion-elgreco merged commit 8ca8d65 into delta-io:main Nov 26, 2023
24 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants