diff --git a/README.md b/README.md index 9532333c3996..93813fac2f91 100644 --- a/README.md +++ b/README.md @@ -1,103 +1,59 @@ -# Apache OpenDAL™ - -**OpenDAL** is a data access layer that allows users to easily and efficiently retrieve data from various storage services in a unified way. - -![OpenDAL Architectural](https://github.com/apache/opendal/assets/5351546/c81013b2-5455-4950-9d31-dbf272b07998) - -Major components of the project include: - -**Libraries** - -- [Rust Core](core/README.md) -- [C Binding](bindings/c/README.md) *not released* -- [Cpp Binding](bindings/cpp/README.md) *not released* -- [Haskell Binding](bindings/haskell/README.md) *not released* -- [Java Binding](bindings/java/README.md) -- [Lua Binding](bindings/lua/README.md) *not released* -- [Node.js Binding](bindings/nodejs/README.md) -- [Python Binding](bindings/python/README.md) -- [Ruby Binding](bindings/ruby/README.md) *not released* -- [Swift Binding](bindings/swift/README.md) *not released* -- [Zig Binding](bindings/zig/README.md) *not released* - -**Applications** - -- [oli](bin/oli): OpenDAL Command Line Interface -- [oay](bin/oay): OpenDAL Gateway - -**Services** - -
-Standard Storage Protocols (like ftp, webdav) - -- ftp: FTP and FTPS -- http: HTTP read-only services -- sftp: [SFTP](https://datatracker.ietf.org/doc/html/draft-ietf-secsh-filexfer-02) services *being worked on* -- webdav: [WebDAV](https://datatracker.ietf.org/doc/html/rfc4918) Service - -
- -
-Object Storage Services (like s3, gcs, azblob) - -- azblob: [Azure Storage Blob](https://azure.microsoft.com/en-us/services/storage/blobs/) services -- cos: [Tencent Cloud Object Storage](https://www.tencentcloud.com/products/cos) services -- gcs: [Google Cloud Storage](https://cloud.google.com/storage) Service -- obs: [Huawei Cloud Object Storage](https://www.huaweicloud.com/intl/en-us/product/obs.html) Service (OBS) -- oss: [Aliyun Object Storage Service](https://www.aliyun.com/product/oss) (OSS) -- s3: [AWS S3](https://aws.amazon.com/s3/) alike services -- supabase: [Supabase Storage](https://supabase.com/docs/guides/storage) Service *being worked on* -- wasabi: [Wasabi](https://wasabi.com/) Cloud Storage - -
- -
-File Storage Services (like fs, azdls, hdfs) - -- fs: POSIX alike file system -- azdls: [Azure Data Lake Storage Gen2](https://azure.microsoft.com/en-us/products/storage/data-lake-storage/) services (As known as [ABFS](https://learn.microsoft.com/en-us/azure/storage/blobs/data-lake-storage-abfs-driver)) -- hdfs: [Hadoop Distributed File System](https://hadoop.apache.org/docs/r3.3.4/hadoop-project-dist/hadoop-hdfs/HdfsDesign.html)(HDFS) -- ipfs: [InterPlanetary File System](https://ipfs.tech/) HTTP Gateway -- ipmfs: [InterPlanetary File System](https://ipfs.tech/) MFS API *being worked on* -- webhdfs: [WebHDFS](https://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-hdfs/WebHDFS.html) Service - -
- -
-Consumer Cloud Storage Service (like gdrive, onedrive) - -- gdrive: [Google Drive](https://www.google.com/drive/) *being worked on* -- onedrive: [OneDrive](https://www.microsoft.com/en-us/microsoft-365/onedrive/online-cloud-storage) *being worked on* -- icloud: [Icloud Drive](https://www.icloud.com.cn/iclouddrive/) *being worked on* - -
- -
-Key-Value Storage Service (like rocksdb, sled) - -- cacache: [cacache](https://crates.io/crates/cacache) backend -- dashmap: [dashmap](https://github.com/xacrimon/dashmap) backend -- memory: In memory backend -- persy: [persy](https://crates.io/crates/persy) backend -- redis: [Redis](https://redis.io/) services -- rocksdb: [RocksDB](http://rocksdb.org/) services -- sled: [sled](https://crates.io/crates/sled) backend -- redb: [redb](https://crates.io/crates/redb) backend -- tikv: [tikv](https://tikv.org/) backend -- atomicserver: [Atomicserver](https://github.com/atomicdata-dev/atomic-server) services - -
- -
-Cache Storage Service (like memcached, moka) - -- ghac: [GitHub Action Cache](https://docs.github.com/en/actions/using-workflows/caching-dependencies-to-speed-up-workflows) Service -- memcached: [Memcached](https://memcached.org/) service -- mini_moka: [Mini Moka](https://github.com/moka-rs/mini-moka) backend -- moka: [Moka](https://github.com/moka-rs/moka) backend -- vercel_artifacts: [Vercel Remote Caching](https://vercel.com/docs/concepts/monorepos/remote-caching) Service *being worked on* - -
+# Apache OpenDAL™: *Access Data Freely* + +[![](https://img.shields.io/badge/maillist-dev%40opendal.apache.org-blue)](mailto:dev@opendal.apache.org) +[![](https://img.shields.io/discord/1081052318650339399?logo=discord&label=discord)](https://discord.gg/XQy8yGR2dg) + +OpenDAL offers a unified data access layer, empowering users to seamlessly and efficiently retrieve data from diverse storage services. Our goal is to deliver a comprehensive solution for any languages, methods, integrations, and services. + +OpenDAL Architectural + +## For *ANY* languages + +| Name | Release | +|-----------------------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------| +| [Rust Core](core/README.md) | [![](https://img.shields.io/crates/v/opendal.svg)](https://crates.io/crates/opendal) | +| [C Binding](bindings/c/README.md) | - | +| [Cpp Binding](bindings/cpp/README.md) | - | +| [Dotnet Binding](bindings/dotnet/README.md) | - | +| [Go Binding](bindings/go/README.md) | - | +| [Haskell Binding](bindings/haskell/README.md) | - | +| [Java Binding](bindings/java/README.md) | [![](https://img.shields.io/maven-central/v/org.apache.opendal/opendal-java)](https://central.sonatype.com/artifact/org.apache.opendal/opendal-java) | +| [Lua Binding](bindings/lua/README.md) | - | +| [Node.js Binding](bindings/nodejs/README.md) | [![](https://img.shields.io/npm/v/opendal)](https://www.npmjs.com/package/opendal) | +| [Ocaml Binding](bindings/ocaml/README.md) | - | +| [PHP Binding](bindings/php/README.md) | - | +| [Python Binding](bindings/python/README.md) | [![](https://img.shields.io/pypi/v/opendal)](https://pypi.org/project/opendal/) | +| [Ruby Binding](bindings/ruby/README.md) | - | +| [Swift Binding](bindings/swift/README.md) | - | +| [Zig Binding](bindings/zig/README.md) | - | + +## For *ANY* methods + +| Name | Description | Release | +|--------------------------|--------------------------------------------------------------------|------------------------------------------------------------------------------| +| [oay](bin/oay/README.md) | Access data via API Gateway | [![](https://img.shields.io/crates/v/oay.svg)](https://crates.io/crates/oay) | +| [oli](bin/oli/README.md) | Access data via Command Line (alternative to s3cmd, s3cli, azcopy) | [![](https://img.shields.io/crates/v/oli.svg)](https://crates.io/crates/oli) | +| [ofs](bin/ofs/README.md) | Access data via POSIX file system API (alternative to s3fs) | [![](https://img.shields.io/crates/v/ofs.svg)](https://crates.io/crates/ofs) | + +## For *ANY* integrations + +| Name | Description | Release | +|---------------------------------------------------------------------|--------------------------------------------------------------------------------------------|----------------------------------------------------------------------------------------------------------------| +| [dav-server-opendalfs](integrations/dav-server/README.md) | Access data via integrations to [dav-server-rs](https://github.com/messense/dav-server-rs) | [![](https://img.shields.io/crates/v/dav-server-opendalfs.svg)](https://crates.io/crates/dav-server-opendalfs) | +| [object_store_opendal](integrations/object_store_opendal/README.md) | Access data via integrations to [object_store](https://docs.rs/object_store) | [![](https://img.shields.io/crates/v/object_store_opendal.svg)](https://crates.io/crates/object_store_opendal) | + +## For *ANY* services + +| Type | Services | +|--------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| Standard Storage Protocols | ftp, http, [sftp](https://datatracker.ietf.org/doc/html/draft-ietf-secsh-filexfer-02), [webdav](https://datatracker.ietf.org/doc/html/rfc4918), .. | +| Object Storage Services | [azblob](https://azure.microsoft.com/en-us/services/storage/blobs/), [cos](https://www.tencentcloud.com/products/cos), [gcs](https://cloud.google.com/storage), [obs](https://www.huaweicloud.com/intl/en-us/product/obs.html), [oss](https://www.aliyun.com/product/oss), [s3](https://aws.amazon.com/s3/), [supabase](https://supabase.com/docs/guides/storage), [b2](https://www.backblaze.com/), [openstack swift](https://docs.openstack.org/swift/latest/), [upyun](https://www.upyun.com/), [vercel_blob](https://vercel.com/docs/storage/vercel-blob), .. | +| File Storage Services | fs, [alluxio](https://docs.alluxio.io/os/user/stable/en/api/REST-API.html), [azdls](https://azure.microsoft.com/en-us/products/storage/data-lake-storage/), [azfile](https://learn.microsoft.com/en-us/rest/api/storageservices/file-service-rest-api), [chainsafe](https://storage.chainsafe.io/), [dbfs](https://docs.databricks.com/en/dbfs/index.html), [gridfs](https://www.mongodb.com/docs/manual/core/gridfs/), [hdfs](https://hadoop.apache.org/docs/r3.3.4/hadoop-project-dist/hadoop-hdfs/HdfsDesign.html), [ipfs](https://ipfs.tech/), [webhdfs](https://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-hdfs/WebHDFS.html), .. | +| Consumer Cloud Storage Service | [gdrive](https://www.google.com/drive/), [onedrive](https://www.microsoft.com/en-us/microsoft-365/onedrive/online-cloud-storage), [dropbox](https://www.dropbox.com/), [icloud](https://www.icloud.com/iclouddrive), [koofr](https://koofr.eu/), [pcloud](https://www.pcloud.com/), [seafile](https://www.seafile.com/), [yandex_disk](https://360.yandex.com/disk/), .. | +| Key-Value Storage Services | [cacache](https://crates.io/crates/cacache), [cloudflare_kv](https://developers.cloudflare.com/kv/), [dashmap](https://github.com/xacrimon/dashmap), memory, [etcd](https://etcd.io/), [foundationdb](https://www.foundationdb.org/), [persy](https://crates.io/crates/persy), [redis](https://redis.io/), [rocksdb](http://rocksdb.org/), [sled](https://crates.io/crates/sled), [redb](https://crates.io/crates/redb), [tikv](https://tikv.org/), [atomicserver](https://github.com/atomicdata-dev/atomic-server), .. | +| Database Storage Services | [d1](https://developers.cloudflare.com/d1/), [libsql](https://github.com/tursodatabase/libsql), [mongodb](https://www.mongodb.com/), [mysql](https://www.mysql.com/), [postgresql](https://www.postgresql.org/), [sqlite](https://www.sqlite.org/), .. | +| Cache Storage Services | [ghac](https://docs.github.com/en/actions/using-workflows/caching-dependencies-to-speed-up-workflows), [memcached](https://memcached.org/), [mini_moka](https://github.com/moka-rs/mini-moka), [moka](https://github.com/moka-rs/moka), [vercel_artifacts](https://vercel.com/docs/concepts/monorepos/remote-caching), .. | +| Git Based Storage Services | [huggingface](https://huggingface.co/), .. | > Welcome to add any services that are not currently supported [here](https://github.com/apache/opendal/issues/5). @@ -115,7 +71,7 @@ OpenDAL is an active open-source project. We are always open to people who want - Start with [Contributing Guide](CONTRIBUTING.md). - Submit [Issues](https://github.com/apache/opendal/issues/new) for bug report or feature requests. -- Discuss at [dev mailing list](mailto:dev-subscribe@opendal.apache.org) ([subscribe](mailto:dev-subscribe@opendal.apache.org?subject=(send%20this%20email%20to%20subscribe)) / [unsubscribe](mailto:dev-unsubscribe@opendal.apache.org?subject=(send%20this%20email%20to%20unsubscribe)) / [archives](https://lists.apache.org/list.html?dev@opendal.apache.org)) +- Discuss at [dev mailing list](mailto:dev@opendal.apache.org) ([subscribe](mailto:dev-subscribe@opendal.apache.org?subject=(send%20this%20email%20to%20subscribe)) / [unsubscribe](mailto:dev-unsubscribe@opendal.apache.org?subject=(send%20this%20email%20to%20unsubscribe)) / [archives](https://lists.apache.org/list.html?dev@opendal.apache.org)) - Asking questions in the [Discussions](https://github.com/apache/opendal/discussions/new?category=q-a). - Talk to community directly at [Discord](https://discord.gg/XQy8yGR2dg).