Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs: Re-org project README #4260

Merged
merged 14 commits into from
Feb 23, 2024
Merged

docs: Re-org project README #4260

Changes from all commits
Commits
Show all changes
14 commits
Select commit Hold shift + click to select a range
8870262
docs: Re-org project README
Xuanwo Feb 23, 2024
012d608
Fix build
Xuanwo Feb 23, 2024
71b2d52
Merge branch 'main' into polish-docs
Xuanwo Feb 23, 2024
59f5fa1
Update table
Xuanwo Feb 23, 2024
13d5abf
try
Xuanwo Feb 23, 2024
1a65342
Try
Xuanwo Feb 23, 2024
b3898b3
Push inside div
Xuanwo Feb 23, 2024
60d3a59
Insert a br
Xuanwo Feb 23, 2024
7b44c3a
try adding p
Xuanwo Feb 23, 2024
5180d4f
try
Xuanwo Feb 23, 2024
41e0fe0
give up
Xuanwo Feb 23, 2024
21cff00
what if
Xuanwo Feb 23, 2024
e0fbade
Polish
Xuanwo Feb 23, 2024
982b88a
Polish
Xuanwo Feb 23, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
158 changes: 57 additions & 101 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,103 +1,59 @@
# Apache OpenDAL™

**OpenDAL** is a data access layer that allows users to easily and efficiently retrieve data from various storage services in a unified way.

![OpenDAL Architectural](https://github.com/apache/opendal/assets/5351546/c81013b2-5455-4950-9d31-dbf272b07998)

Major components of the project include:

**Libraries**

- [Rust Core](core/README.md)
- [C Binding](bindings/c/README.md) *not released*
- [Cpp Binding](bindings/cpp/README.md) *not released*
- [Haskell Binding](bindings/haskell/README.md) *not released*
- [Java Binding](bindings/java/README.md)
- [Lua Binding](bindings/lua/README.md) *not released*
- [Node.js Binding](bindings/nodejs/README.md)
- [Python Binding](bindings/python/README.md)
- [Ruby Binding](bindings/ruby/README.md) *not released*
- [Swift Binding](bindings/swift/README.md) *not released*
- [Zig Binding](bindings/zig/README.md) *not released*

**Applications**

- [oli](bin/oli): OpenDAL Command Line Interface
- [oay](bin/oay): OpenDAL Gateway

**Services**

<details>
<summary>Standard Storage Protocols (like ftp, webdav)</summary>

- ftp: FTP and FTPS
- http: HTTP read-only services
- sftp: [SFTP](https://datatracker.ietf.org/doc/html/draft-ietf-secsh-filexfer-02) services *being worked on*
- webdav: [WebDAV](https://datatracker.ietf.org/doc/html/rfc4918) Service

</details>

<details>
<summary>Object Storage Services (like s3, gcs, azblob)</summary>

- azblob: [Azure Storage Blob](https://azure.microsoft.com/en-us/services/storage/blobs/) services
- cos: [Tencent Cloud Object Storage](https://www.tencentcloud.com/products/cos) services
- gcs: [Google Cloud Storage](https://cloud.google.com/storage) Service
- obs: [Huawei Cloud Object Storage](https://www.huaweicloud.com/intl/en-us/product/obs.html) Service (OBS)
- oss: [Aliyun Object Storage Service](https://www.aliyun.com/product/oss) (OSS)
- s3: [AWS S3](https://aws.amazon.com/s3/) alike services
- supabase: [Supabase Storage](https://supabase.com/docs/guides/storage) Service *being worked on*
- wasabi: [Wasabi](https://wasabi.com/) Cloud Storage

</details>

<details>
<summary>File Storage Services (like fs, azdls, hdfs)</summary>

- fs: POSIX alike file system
- azdls: [Azure Data Lake Storage Gen2](https://azure.microsoft.com/en-us/products/storage/data-lake-storage/) services (As known as [ABFS](https://learn.microsoft.com/en-us/azure/storage/blobs/data-lake-storage-abfs-driver))
- hdfs: [Hadoop Distributed File System](https://hadoop.apache.org/docs/r3.3.4/hadoop-project-dist/hadoop-hdfs/HdfsDesign.html)(HDFS)
- ipfs: [InterPlanetary File System](https://ipfs.tech/) HTTP Gateway
- ipmfs: [InterPlanetary File System](https://ipfs.tech/) MFS API *being worked on*
- webhdfs: [WebHDFS](https://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-hdfs/WebHDFS.html) Service

</details>

<details>
<summary>Consumer Cloud Storage Service (like gdrive, onedrive)</summary>

- gdrive: [Google Drive](https://www.google.com/drive/) *being worked on*
- onedrive: [OneDrive](https://www.microsoft.com/en-us/microsoft-365/onedrive/online-cloud-storage) *being worked on*
- icloud: [Icloud Drive](https://www.icloud.com.cn/iclouddrive/) *being worked on*

</details>

<details>
<summary>Key-Value Storage Service (like rocksdb, sled)</summary>

- cacache: [cacache](https://crates.io/crates/cacache) backend
- dashmap: [dashmap](https://github.com/xacrimon/dashmap) backend
- memory: In memory backend
- persy: [persy](https://crates.io/crates/persy) backend
- redis: [Redis](https://redis.io/) services
- rocksdb: [RocksDB](http://rocksdb.org/) services
- sled: [sled](https://crates.io/crates/sled) backend
- redb: [redb](https://crates.io/crates/redb) backend
- tikv: [tikv](https://tikv.org/) backend
- atomicserver: [Atomicserver](https://github.com/atomicdata-dev/atomic-server) services

</details>

<details>
<summary>Cache Storage Service (like memcached, moka)</summary>

- ghac: [GitHub Action Cache](https://docs.github.com/en/actions/using-workflows/caching-dependencies-to-speed-up-workflows) Service
- memcached: [Memcached](https://memcached.org/) service
- mini_moka: [Mini Moka](https://github.com/moka-rs/mini-moka) backend
- moka: [Moka](https://github.com/moka-rs/moka) backend
- vercel_artifacts: [Vercel Remote Caching](https://vercel.com/docs/concepts/monorepos/remote-caching) Service *being worked on*

</details>
# Apache OpenDAL™: *Access Data Freely*

[![](https://img.shields.io/badge/maillist-dev%40opendal.apache.org-blue)](mailto:[email protected])
[![](https://img.shields.io/discord/1081052318650339399?logo=discord&label=discord)](https://discord.gg/XQy8yGR2dg)

OpenDAL offers a unified data access layer, empowering users to seamlessly and efficiently retrieve data from diverse storage services. Our goal is to deliver a comprehensive solution for any languages, methods, integrations, and services.

<img src="https://opendal.apache.org/img/architectural.png" alt="OpenDAL Architectural" width="61.8%" />

## For *ANY* languages

| Name | Release |
|-----------------------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------|
| [Rust Core](core/README.md) | [![](https://img.shields.io/crates/v/opendal.svg)](https://crates.io/crates/opendal) |
| [C Binding](bindings/c/README.md) | - |
| [Cpp Binding](bindings/cpp/README.md) | - |
| [Dotnet Binding](bindings/dotnet/README.md) | - |
| [Go Binding](bindings/go/README.md) | - |
| [Haskell Binding](bindings/haskell/README.md) | - |
| [Java Binding](bindings/java/README.md) | [![](https://img.shields.io/maven-central/v/org.apache.opendal/opendal-java)](https://central.sonatype.com/artifact/org.apache.opendal/opendal-java) |
| [Lua Binding](bindings/lua/README.md) | - |
| [Node.js Binding](bindings/nodejs/README.md) | [![](https://img.shields.io/npm/v/opendal)](https://www.npmjs.com/package/opendal) |
| [Ocaml Binding](bindings/ocaml/README.md) | - |
| [PHP Binding](bindings/php/README.md) | - |
| [Python Binding](bindings/python/README.md) | [![](https://img.shields.io/pypi/v/opendal)](https://pypi.org/project/opendal/) |
| [Ruby Binding](bindings/ruby/README.md) | - |
| [Swift Binding](bindings/swift/README.md) | - |
| [Zig Binding](bindings/zig/README.md) | - |

## For *ANY* methods

| Name | Description | Release |
|--------------------------|--------------------------------------------------------------------|------------------------------------------------------------------------------|
| [oay](bin/oay/README.md) | Access data via API Gateway | [![](https://img.shields.io/crates/v/oay.svg)](https://crates.io/crates/oay) |
| [oli](bin/oli/README.md) | Access data via Command Line (alternative to s3cmd, s3cli, azcopy) | [![](https://img.shields.io/crates/v/oli.svg)](https://crates.io/crates/oli) |
| [ofs](bin/ofs/README.md) | Access data via POSIX file system API (alternative to s3fs) | [![](https://img.shields.io/crates/v/ofs.svg)](https://crates.io/crates/ofs) |

## For *ANY* integrations

| Name | Description | Release |
|---------------------------------------------------------------------|--------------------------------------------------------------------------------------------|----------------------------------------------------------------------------------------------------------------|
| [dav-server-opendalfs](integrations/dav-server/README.md) | Access data via integrations to [dav-server-rs](https://github.com/messense/dav-server-rs) | [![](https://img.shields.io/crates/v/dav-server-opendalfs.svg)](https://crates.io/crates/dav-server-opendalfs) |
| [object_store_opendal](integrations/object_store_opendal/README.md) | Access data via integrations to [object_store](https://docs.rs/object_store) | [![](https://img.shields.io/crates/v/object_store_opendal.svg)](https://crates.io/crates/object_store_opendal) |

## For *ANY* services

| Type | Services |
|--------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| Standard Storage Protocols | ftp, http, [sftp](https://datatracker.ietf.org/doc/html/draft-ietf-secsh-filexfer-02), [webdav](https://datatracker.ietf.org/doc/html/rfc4918), .. |
| Object Storage Services | [azblob](https://azure.microsoft.com/en-us/services/storage/blobs/), [cos](https://www.tencentcloud.com/products/cos), [gcs](https://cloud.google.com/storage), [obs](https://www.huaweicloud.com/intl/en-us/product/obs.html), [oss](https://www.aliyun.com/product/oss), [s3](https://aws.amazon.com/s3/), [supabase](https://supabase.com/docs/guides/storage), [b2](https://www.backblaze.com/), [openstack swift](https://docs.openstack.org/swift/latest/), [upyun](https://www.upyun.com/), [vercel_blob](https://vercel.com/docs/storage/vercel-blob), .. |
| File Storage Services | fs, [alluxio](https://docs.alluxio.io/os/user/stable/en/api/REST-API.html), [azdls](https://azure.microsoft.com/en-us/products/storage/data-lake-storage/), [azfile](https://learn.microsoft.com/en-us/rest/api/storageservices/file-service-rest-api), [chainsafe](https://storage.chainsafe.io/), [dbfs](https://docs.databricks.com/en/dbfs/index.html), [gridfs](https://www.mongodb.com/docs/manual/core/gridfs/), [hdfs](https://hadoop.apache.org/docs/r3.3.4/hadoop-project-dist/hadoop-hdfs/HdfsDesign.html), [ipfs](https://ipfs.tech/), [webhdfs](https://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-hdfs/WebHDFS.html), .. |
| Consumer Cloud Storage Service | [gdrive](https://www.google.com/drive/), [onedrive](https://www.microsoft.com/en-us/microsoft-365/onedrive/online-cloud-storage), [dropbox](https://www.dropbox.com/), [icloud](https://www.icloud.com/iclouddrive), [koofr](https://koofr.eu/), [pcloud](https://www.pcloud.com/), [seafile](https://www.seafile.com/), [yandex_disk](https://360.yandex.com/disk/), .. |
| Key-Value Storage Services | [cacache](https://crates.io/crates/cacache), [cloudflare_kv](https://developers.cloudflare.com/kv/), [dashmap](https://github.com/xacrimon/dashmap), memory, [etcd](https://etcd.io/), [foundationdb](https://www.foundationdb.org/), [persy](https://crates.io/crates/persy), [redis](https://redis.io/), [rocksdb](http://rocksdb.org/), [sled](https://crates.io/crates/sled), [redb](https://crates.io/crates/redb), [tikv](https://tikv.org/), [atomicserver](https://github.com/atomicdata-dev/atomic-server), .. |
| Database Storage Services | [d1](https://developers.cloudflare.com/d1/), [libsql](https://github.com/tursodatabase/libsql), [mongodb](https://www.mongodb.com/), [mysql](https://www.mysql.com/), [postgresql](https://www.postgresql.org/), [sqlite](https://www.sqlite.org/), .. |
| Cache Storage Services | [ghac](https://docs.github.com/en/actions/using-workflows/caching-dependencies-to-speed-up-workflows), [memcached](https://memcached.org/), [mini_moka](https://github.com/moka-rs/mini-moka), [moka](https://github.com/moka-rs/moka), [vercel_artifacts](https://vercel.com/docs/concepts/monorepos/remote-caching), .. |
| Git Based Storage Services | [huggingface](https://huggingface.co/), .. |

> Welcome to add any services that are not currently supported [here](https://github.com/apache/opendal/issues/5).

Expand All @@ -115,7 +71,7 @@ OpenDAL is an active open-source project. We are always open to people who want

- Start with [Contributing Guide](CONTRIBUTING.md).
- Submit [Issues](https://github.com/apache/opendal/issues/new) for bug report or feature requests.
- Discuss at [dev mailing list](mailto:dev-subscribe@opendal.apache.org) ([subscribe](mailto:[email protected]?subject=(send%20this%20email%20to%20subscribe)) / [unsubscribe](mailto:[email protected]?subject=(send%20this%20email%20to%20unsubscribe)) / [archives](https://lists.apache.org/[email protected]))
- Discuss at [dev mailing list](mailto:[email protected]) ([subscribe](mailto:[email protected]?subject=(send%20this%20email%20to%20subscribe)) / [unsubscribe](mailto:[email protected]?subject=(send%20this%20email%20to%20unsubscribe)) / [archives](https://lists.apache.org/[email protected]))
- Asking questions in the [Discussions](https://github.com/apache/opendal/discussions/new?category=q-a).
- Talk to community directly at [Discord](https://discord.gg/XQy8yGR2dg).

Expand Down
Loading