Skip to content

Releases: apache/opendal

v0.47.2

03 Jul 05:08
@Xuanwo Xuanwo
v0.47.2
This tag was signed with the committer’s verified signature.
Xuanwo Xuanwo
GPG key ID: E95B7C7CB352F6C1
Learn about vigilant mode.
a58e802
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Learn about vigilant mode.
Compare
Choose a tag to compare
v0.47.2

Relelase List

Name Version Next
core 0.47.1 0.47.2
integrations/cloudfilter 0.0.0 0.0.0
integrations/dav-server 0.0.3 0.0.4
integrations/fuse3 0.0.0 0.0.1
integrations/object_store 0.44.1 0.44.2
integrations/unftp-sbe 0.0.0 0.0.1
integrations/virtiofs 0.0.0 0.0.0
bin/oay 0.41.4 0.41.5
bin/ofs 0.0.5 0.0.6
bin/oli 0.41.4 0.41.5
bindings/c 0.44.6 0.44.7
bindings/cpp 0.45.4 0.45.5
bindings/dotnet 0.1.2 0.1.3
bindings/go 0.0.0 0.0.0
bindings/haskell 0.44.4 0.44.5
bindings/java 0.46.1 0.46.2
bindings/lua 0.1.2 0.1.3
bindings/nodejs 0.46.2 0.47.0
bindings/ocaml 0.0.0 0.0.0
bindings/php 0.1.2 0.1.3
bindings/python 0.45.4 0.45.5
bindings/ruby 0.1.2 0.1.3
bindings/swift 0.0.0 0.0.0
bindings/zig 0.0.0 0.0.0
  • bindings/nodejs 0.47.0 release failed, we will get it fixed in next release.

Breaking changes

Binding Node.js Public API

Now, the append operation has been removed. You can use below code instead.

op.write("path/to/file", Buffer.from("hello world"), { append: true });

What's Changed

Added

Changed

  • refactor(nodejs)!: Remove append api by @bxb100 in #4796
  • refactor(core): Remove unused layer MadsimLayer by @zzzk1 in #4788

Fixed

  • fix(services/aliyun-drive): list dir without trailing slash by @yuchanns in #4766
  • fix(unftp-sbe): remove buffer for get by @George-Miao in #4775
  • fix(services/aliyun-drive): write op cannot overwrite existing files by @yuchanns in #4781
  • fix(core/services/onedrive): remove @odata.count for onedrive list op by @imWildCat in #4803
  • fix(core): Gcs's RangeWrite doesn't support concurrent write by @Xuanwo in #4806
  • fix(tests/behavior): skip test of write_with_overwrite for ghac by @yuchanns in #4823
  • fix(docs): some typos in website and nodejs binding docs by @suyanhanx in #4814
  • fix(core/aliyun_drive): Fix write_multi_max_size might overflow by @Xuanwo in #4830

Docs

  • doc(unftp-sbe): adds example and readme by @George-Miao in #4777
  • doc(nodejs): update upgrade.md by @bxb100 in #4799
  • docs: Add README and rustdoc for fuse3_opendal by @Xuanwo in #4813
  • docs: use version variable in gradle, same to maven by @shoothzj in #4824

CI

Chore

New Contributors

Full Changelog: v0.47.1...v0.47.2

Contributors

imWildCat, Xuanwo, and 10 other contributors
Assets 2
Loading
0 Join discussion

v0.47.1

21 Jun 11:00
@Xuanwo Xuanwo
v0.47.1
This tag was signed with the committer’s verified signature.
Xuanwo Xuanwo
GPG key ID: E95B7C7CB352F6C1
Learn about vigilant mode.
c98e81f
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Learn about vigilant mode.
Compare
Choose a tag to compare
v0.47.1

What's Changed

Added

  • feat(core): sets default chunk_size and sends buffer > chunk_size directly by @evenyag in #4710
  • feat(services): add optional access_token for AliyunDrive by @yuchanns in #4740
  • feat(unftp-sbe): Add integration for unftp-sbe by @George-Miao in #4753

Changed

  • refactor(ofs): Split fuse3 impl into fuse3_opendal by @Xuanwo in #4721
  • refactor(ovfs): Split ovfs impl into virtiofs_opendal by @zjregee in #4723
  • refactor(*): tiny refactor to the Error type by @waynexia in #4737
  • refactor(aliyun-drive): rewrite writer part by @yuchanns in #4744
  • refactor(object_store): Polish implementation details of object_store by @Xuanwo in #4749
  • refactor(dav-server): Polish dav-server integration details by @Xuanwo in #4751
  • refactor(core): Remove unused size for RangeWrite. by @reswqa in #4755

Fixed

  • fix(s3): parse MultipartUploadResponse to check error in body by @waynexia in #4735
  • fix(services/aliyun-drive): unable to list / by @yuchanns in #4754

Docs

CI

  • ci: verify build website on site content changes by @tisonkun in #4712
  • ci: Fix cert for redis and add docs for key maintenance by @Xuanwo in #4718
  • ci(nodejs): Disable services-all on windows by @Xuanwo in #4762

Chore

New Contributors

Full Changelog: v0.47.0...v0.47.1

Contributors

Xuanwo, evenyag, and 8 other contributors
Assets 2
Loading
0 Join discussion

v0.47.0

10 Jun 15:03
@tisonkun tisonkun
v0.47.0
This tag was signed with the committer’s verified signature.
tisonkun tison
GPG key ID: CD8564CB6A21E526
Learn about vigilant mode.
f2aa333
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Learn about vigilant mode.
Compare
Choose a tag to compare
v0.47.0

Core - Upgrade to v0.47

Public API

Reader into_xxx APIs

Since v0.47, Reader's into_xxx APIs requires async and returns Result instead.

- let r = op.reader("test.txt").await?.into_futures_async_read(1024..2048);
+ let r = op.reader("test.txt").await?.into_futures_async_read(1024..2048).await?;

Affected API includes:

  • Reader::into_futures_async_read
  • Reader::into_bytes_stream
  • BlockingReader::into_std_read
  • BlockingReader::into_bytes_iterator

Raw API

Bring Streaming Read Back

As explained in core: Bring Streaming Read Back, we do need read streaming back for better performance and low memory usage.

So our oio::Read changed back to streaming read instead:

trait Read {
-  async fn read(&self, offset: u64, size: usize) -> Result<Buffer>;
+  async fn read(&mut self) -> Result<Buffer>;
}

All services and layers should be updated to meet this change.

Java Binding - Upgrade to v0.46

Breaking change

PR-4641 renames async Operator to AsyncOperator and BlockingOperator to Operator.

New features

PR-4626 implements OperatorInputStream and OperatorOutputStream which implements Java's core IO abstractions InputStream and OutputStream. Users can now read/write bytes streamlined without loading/preparing the bytes fully in memory.

What's Changed

Added

  • feat(core/types): change oio::BlockingReader to Arc by @hoslo in #4577
  • fix: format_object_meta should not require metakeys that don't exist by @rebasedming in #4582
  • feat: add checksums to MultiPartComplete by @JWackerbauer in #4580
  • feat(doc): update object_store_opendal README by @hanxuanliang in #4606
  • feat(services/aliyun-drive): support AliyunDrive by @yuchanns in #4585
  • feat(bindings/python): Update type annotations by @3ok in #4630
  • feat: implement OperatorInputStream and OperatorOutputStream by @tisonkun in #4626
  • feat(bench): add buffer benchmark by @zjregee in #4603
  • feat: Add Executor struct and Execute trait by @Xuanwo in #4648
  • feat: Add executor in OpXxx and Operator by @Xuanwo in #4649
  • feat: Implement and refactor concurrent tasks for multipart write by @Xuanwo in #4653
  • feat(core/types): blocking remove_all for object storage based services by @TennyZhuang in #4665
  • feat(core): Streaming reading while chunk is not set by @Xuanwo in #4658
  • feat(core): Add more context in error context by @Xuanwo in #4673
  • feat: init ovfs by @zjregee in #4652
  • feat: Implement retry for streaming based read by @Xuanwo in #4683
  • feat(core): Implement TimeoutLayer for concurrent tasks by @Xuanwo in #4688
  • feat(core): Add reader size check in complete reader by @Xuanwo in #4690
  • feat(core): Azblob supports azure workload identity by @Xuanwo in #4705

Changed

  • refactor(core): Align naming for AccessorDyn by @morristai in #4574
  • refactor(core): core doesn't expose invalid input error anymore by @Xuanwo in #4632
  • refactor(core): Return unexpected error while content incomplete happen by @Xuanwo in #4633
  • refactor(core): Change Read's behavior to ensure it reads the exact size of data by @Xuanwo in #4634
  • refactor(bin/ofs): Fuse API by @ho-229 in #4637
  • refactor(binding/java)!: rename blocking and async operator by @tisonkun in #4641
  • refactor(core): Use concurrent tasks to refactor block write by @Xuanwo in #4692
  • refactor(core): Migrate RangeWrite to ConcurrentTasks by @Xuanwo in #4696

Fixed

  • fix(devcontainer/post_create.sh): change pnpm@stable to pnpm@latest by @GG2002 in #4584
  • fix(bin/ofs): privileged mount crashes when external umount by @ho-229 in #4586
  • fix(bin/ofs): ofs read only mount by @ho-229 in #4602
  • fix(raw): Allow retrying request while decoding response failed by @Xuanwo in #4612
  • fix(core): return None if metadata unavailable by @NKID00 in #4613
  • fix(bindings/python): Use abi3 and increase MSPV to 3.11 by @Xuanwo in #4623
  • fix: Fetch the content length while end_bound is unknown by @Xuanwo in #4631
  • fix: ofs write behavior by @ho-229 in #4617
  • fix(core/types): remove_all not work under object-store backend by @TennyZhuang in #4659
  • fix(ofs): Close file during flush by @Xuanwo in #4680
  • fix(core): RetryLayer could panic when other threads raises panic by @Xuanwo in #4685
  • fix(core/prometheus): Fix metrics from prometheus not correct for reader by @Xuanwo in #4691
  • fix(core/oio): Make ConcurrentTasks cancel safe by only pop after ready by @Xuanwo in #4707

Docs

CI

  • ci: upgrade typos to 1.21.0 and ignore changelog by @hezhizhen in #4601
  • ci: Disable jfrog webdav tests for it's keeping failed by @Xuanwo in #4607
  • ci: use official typos github action by @shoothzj in #4635
  • build(deps): upgrade crc32c to 0.6.6 for nightly toolchain by @tisonkun in #4650

Chore

New Contributors

Full Changelog: v0.46.0...v0.47.0

Contributors

jokester, Xuanwo, and 21 other contributors
Assets 2
Loading

v0.46.0

06 May 01:26
@tisonkun tisonkun
v0.46.0
This tag was signed with the committer’s verified signature.
tisonkun tison
GPG key ID: CD8564CB6A21E526
Learn about vigilant mode.
84586e5
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Learn about vigilant mode.
Compare
Choose a tag to compare
v0.46.0

Upgrade to v0.46

Public API

MSRV Changed to 1.75

Since 0.46, OpenDAL requires Rust 1.75.0 or later to use features like RPITIT and AFIT.

Services Feature Flag

Starting with version 0.46, OpenDAL only includes the memory service by default to prevent compiling unnecessary service code. To use other services, please activate their respective feature flags.

Additionally, we have removed all reqwest-related feature flags:

  • Users must now directly use reqwest's feature flags for options like rustls, native-tls, etc.
  • The rustls feature is no longer enabled by default; it must be activated manually.
  • OpenDAL no longer offers the trust-dns option; users should configure the client builder directly.

Range Based Read

Since v0.46, OpenDAL transformed it's Read IO trait to range based instead of stateful poll based IO. This change will make the IO more efficient, easier for concurrency and ready for completion based IO.

opendal::Reader now have APIs like:

let r = op.reader("test.txt").await?;
let buf = r.read(1024..2048).await?;

Buffer Based IO

Since version 0.46, OpenDAL features a native Buffer struct that supports both contiguous and non-contiguous buffers. This update enhances IO efficiency by minimizing unnecessary byte copying and enabling vectored IO.

OpenDAL's Reader will return Buffer and Writer will accept Buffer as input. Users who have implemented their own IO traits should update their code to use the new Buffer struct.

let r = op.reader("test.txt").await?;
// read returns `Buffer`
let buf: Buffer = r.read(1024..2048).await?;

let w = op.writer("test.txt").await?;

// Buffer can be created from continues bytes.
w.write("hello, world").await?;
// Buffer can also be created from non-continues bytes.
w.write(vec![Bytes::from("hello,"), Bytes::from("world!")]).await?;

// Make sure file has been written completely.
w.close().await?;

To enhance usability, we've integrated bridges into bytes::Buf and bytes::BufMut, allowing users to directly interact with the bytes API.

let r = op.reader("test.txt").await?;
let mut bs = vec![];
// read_into accepts bytes::BufMut
let buf: Buffer = r.read_into(&mut bs, 1024..2048).await?;

let w = op.writer("test.txt").await?;

// write_from accepts bytes::Buf
w.write_from("hello, world".as_bytes()).await?;

// Make sure file has been written completely.
w.close().await?;

Bridge API

OpenDAL's Reader and Writer previously implemented APIs such as AsyncRead and AsyncWrite directly. This design was not user-friendly, as it could lead to unexpected costs that users were unaware of in advance.

Since v0.46, OpenDAL provides bridge APIs for Reader and Writer instead.

let r = op.reader("test.txt").await?;

// Convert into futures AsyncRead + AsyncSeek.
let reader = r.into_futures_async_read(1024..2048);
// Convert into futures bytes stream.
let stream = r.into_bytes_stream(1024..2048);

let w = op.writer("test.txt").await?;

// Convert into futures AsyncWrite
let writer = w.into_futures_async_write();
// Convert into futures bytes sink;
let sink = w.into_bytes_sink();

Raw API

Async in IO trait

Since version 0.46, OpenDAL has adopted Rust's native async_in_trait for our core IO traits, including oio::Read, oio::Write, and oio::List.

This update eliminates the need for manually written, poll-based state machines and simplifies the codebase. Consequently, OpenDAL now requires Rust version 1.75.0 or later.

Users who have implemented their own IO traits should update their code to use the new async trait syntax.

What's Changed

Added

  • feat(services/github): add github contents support by @hoslo in #4281
  • feat: Allow selecting webpki for reqwest by @arlyon in #4303
  • feat(services/swift): add support for storage_url configuration in swift service by @zjregee in #4302
  • feat(services/swift): add ceph test setup for swift by @zjregee in #4307
  • docs(website): add local content search based on lunr plugin by @m1911star in #4348
  • feat(services/sled): add SledConfig by @yufan022 in #4351
  • feat : Implementing config for part of services by @AnuRage-git in #4344
  • feat(bindings/java): explicit async runtime by @tisonkun in #4376
  • feat(services/surrealdb): support surrealdb service by @yufan022 in #4375
  • feat(bindings/java): avoid double dispose NativeObject by @tisonkun in #4377
  • feat : Implement config for services/foundationdb by @AnuRage-git in #4355
  • feat: add ofs ctrl-c exit unmount hook by @oowl in #4393
  • feat: Implement RFC-4382 Range Based Read by @Xuanwo in #4381
  • feat(ofs): rename2 lseek copy_file_range getattr API support by @oowl in #4395
  • feat(services/github): make access_token optional by @hoslo in #4404
  • feat(core/oio): Add readable buf by @Xuanwo in #4407
  • feat(ofs): add freebsd OS support by @oowl in #4403
  • feat(core/raw/oio): Add Writable Buf by @Xuanwo in #4410
  • feat(bin/ofs): Add behavior test for ofs by @ho-229 in #4373
  • feat(core/raw/buf): Reduce one allocation by Arc::from_iter by @Xuanwo in #4440
  • feat: ?Send async trait for HttpBackend when the target is wasm32 by @waynexia in #4444
  • feat: add HttpClient::with() constructor by @waynexia in #4447
  • feat: Move Buffer as public API by @Xuanwo in #4450
  • feat: Optimize buffer implementation and add stream support by @Xuanwo in #4458
  • feat(core): Implement FromIterator for Buffer by @Xuanwo in #4459
  • feat(services/ftp): Support multiple write by @xxxuuu in #4425
  • feat(raw/oio): block write change to buffer by @hoslo in #4466
  • feat(core): Implement read and read_into for Reader by @Xuanwo in #4467
  • feat(core): Implement into_stream for Reader by @Xuanwo in #4473
  • feat(core): Tune buffer operations based on benchmark results by @Xuanwo in #4468
  • feat(raw/oio): Use Buffer as cache in RangeWrite by @reswqa in #4476
  • feat(raw/oio): Use Buffer as cache in OneshotWrite by @reswqa in #4477
  • feat: add list/copy/rename for dropbox by @zjregee in #4424
  • feat(core): Implement write and write_from for Writer by @zjregee in #4482
  • feat(core): Add auto ranged read and concurrent read support by @Xuanwo in #4486
  • feat(core): Implement fetch for Reader by @Xuanwo in #4488
  • feat(core): Allow concurrent reading on bytes stream by @Xuanwo in #4499
  • feat: provide send-wrapper to contidionally implement Send for operators by @waynexia in #4443
  • feat(bin/ofs): privileged mount by @ho-229 in #4507
  • feat(services/compfs): init compfs by @George-Miao in #4519
  • feat(raw/oio): Add PooledBuf to allow reuse buffer by @Xuanwo in #4522
  • feat(services/fs): Add PooledBuf in fs to allow reusing memory by @Xuanwo in #4525
  • feat(core): add access methods for Buffer by @George-Miao in #4530
  • feat(core): implement IoBuf for Buffer by @George-Miao in #4532
  • feat(services/compfs): compio runtime and compfs structure by @George-Miao in #4534
  • feat(core): change Result to default error by @George-Miao in #4535
  • feat(services/github): support list recursive by @hoslo in #4423
  • feat: Add crc32c checksums to S3 Service by @JWackerbauer in #4533
  • feat: Add into_bytes_sink for Writer by @Xuanwo in #4541

Changed

  • refactor(core/raw): Migrate oio::Read to async in trait by @Xuanwo in #4336
  • refactor(core/raw): Align oio::BlockingRead API with oio::Read by @Xuanwo in #4349
  • refactor(core/oio): Migrate oio::List to async fn in trait by @Xuanwo in #4352
  • refactor(core/raw): Migrate oio::Write from WriteBuf to Bytes by @Xuanwo in https://github...
Read more

Contributors

jbampton, mobiusklein, and 26 other contributors
Assets 2
Loading

v0.45.1

02 Mar 13:38
@Xuanwo Xuanwo
v0.45.1
This tag was signed with the committer’s verified signature.
Xuanwo Xuanwo
GPG key ID: A61687076CEAD424
Learn about vigilant mode.
5079125
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Learn about vigilant mode.
Compare
Choose a tag to compare
v0.45.1

Pacakages

Name Version
core 0.45.1
bin/oay 0.41.1
bin/oli 0.41.1
bin/ofs 0.0.2
bindings/python 0.45.1
bindings/nodejs 0.45.1
bindings/c 0.44.3
bindings/zig 0.0.0
bindings/dotnet 0.1.1
bindings/haskell 0.44.3
bindings/java 0.45.1
bindings/lua 0.1.1
bindings/ruby 0.1.1
bindings/swift 0.0.0
bindings/ocaml 0.0.0
bindings/php 0.1.1
bindings/cpp 0.44.3
bindings/go 0.0.0
integrations/object_store 0.43.0
integrations/dav-server 0.0.1

What's Changed

Added

  • feat(services/vercel_blob): support vercel blob by @hoslo in #4103
  • feat(bindings/python): add ruff as linter by @asukaminato0721 in #4135
  • feat(services/hdfs-native): Add capabilities for hdfs-native service by @jihuayu in #4174
  • feat(services/sqlite): Add list capability supported for sqlite by @jihuayu in #4180
  • feat(services/azblob): support multi write for azblob by @wcy-fdu in #4181
  • feat(release): Implement releasing OpenDAL components seperately by @Xuanwo in #4196
  • feat: object store adapter based on v0.9 by @waynexia in #4233
  • feat(bin/ofs): implement fuse for linux by @ho-229 in #4179
  • feat(services/memcached): change to binary protocal by @hoslo in #4252
  • feat(services/memcached): setup auth test for memcached by @hoslo in #4259
  • feat(services/yandex_disk): setup test for yandex disk by @hoslo in #4264
  • feat: add ci support for ceph_rados by @ZhengLin-Li in #4191
  • feat: Implement Config for part of services by @Xuanwo in #4277
  • feat: add jfrog test setup for webdav by @zjregee in #4265

Changed

  • refactor(bindings/python): simplify async writer aexit by @suyanhanx in #4128
  • refactor(service/d1): Add D1Config by @jihuayu in #4129
  • refactor: Rewrite webdav to improve code quality by @Xuanwo in #4280

Fixed

  • fix: Azdls returns 403 while continuation contains = by @Xuanwo in #4105
  • fix(bindings/python): missed to call close for the file internally by @zzl221000 in #4122
  • fix(bindings/python): sync writer exit close raise error by @suyanhanx in #4127
  • fix(services/chainsafe): fix 423 http status by @hoslo in #4148
  • fix(services/webdav): Add possibility to answer without response if file isn't exist by @AJIOB in #4170
  • fix(services/webdav): Recreate root directory if need by @AJIOB in #4173
  • fix(services/webdav): remove base_dir component by @hoslo in #4231
  • fix(core): Poll TimeoutLayer::sleep once to make sure timer registered by @Xuanwo in #4230
  • fix(services/webdav): Fix endpoint suffix not handled by @Xuanwo in #4257
  • fix(services/webdav): Fix possible error with value loosing from config by @AJIOB in #4262

Docs

CI

  • ci: Use old version of seafile mc instead by @Xuanwo in #4107
  • ci: Refactor workflows layout by @Xuanwo in #4139
  • ci: Migrate hdfs default setup by @Xuanwo in #4140
  • ci: Refactor check.sh into check.py to get ready for multi components release by @Xuanwo in #4159
  • ci: Add test case for hdfs over gcs bucket by @ArmandoZ in #4145
  • ci: Add hdfs test case for s3 by @ArmandoZ in #4184
  • ci: Add hdfs test case for azurite by @ArmandoZ in #4185
  • ci: Add support for releasing all rust packages by @Xuanwo in #4200
  • ci: Fix dependabot not update by @Xuanwo in #4202
  • ci: reduce the open pull request limits to 1 by @jbampton in #4225
  • ci: Remove version suffix from package versions by @Xuanwo in #4254
  • ci: Fix fuzz test for minio s3 name change by @Xuanwo in #4266
  • ci: Mark python 3.13 is not supported by @Xuanwo in #4269
  • ci: Disable yandex disk test for too slow by @Xuanwo in #4274
  • ci: Split python CI into release and checks by @Xuanwo in #4275
  • ci(release): Make sure LICENSE and NOTICE files are included by @Xuanwo in #4283
  • ci(release): Refactor and merge the check.py into verify.py by @Xuanwo in #4284

Chore

Read more

Contributors

jayvdb, jbampton, and 15 other contributors
Assets 2
Loading
0 Join discussion

v0.45.0

05 Feb 15:55
@morristai morristai
v0.45.0
This tag was signed with the committer’s verified signature.
morristai Morris Tai
GPG key ID: DDBFEDDFD8121197
Learn about vigilant mode.
7190c07
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Learn about vigilant mode.
Compare
Choose a tag to compare
v0.45.0

Upgrade to v0.45

Core

Public API

BlockingLayer is not enabled by default

To further enhance the optionality of tokio, we have introduced a new feature called layers-blocking. The default usage of the blocking layer has been disabled. To utilize the BlockingLayer, please enable the layers-blocking feature.

TimeoutLayer deprecated with_speed

The with_speed API has been deprecated. Please use with_io_timeout instead.

Raw API

No raw API changes.

What's Changed

Added

  • feat(ofs): introduce ofs execute bin by @oowl in #4033
  • feat: add a missing news by @WenyXu in #4056
  • feat(services/koofr): set test for koofr by @suyanhanx in #4050
  • feat(layers/dtrace): Support User Statically-Defined Tracing(aka USDT) on Linux by @Zheaoli in #4053
  • feat(website): add missing news and organize the onboarding guide by @morristai in #4072
  • feat(services/chainsafe): setup test for chainsafe by @hoslo in #4089
  • docs: Add apache iceberg-rust as user by @liurenjie1024 in #4100

Changed

  • refactor!: avoid hard dep to tokio rt by @tisonkun in #4061
  • refactor: Implement IntoFuture for operator futures to remove an alloc by @Xuanwo in #4098

Fixed

  • fix(examples/cpp): display the results to standard output. by @SYaoJun in #4040
  • fix(service/icloud):Missing 'X-APPLE-WEBAUTH-USER cookie' and URL initialized failed by @bokket in #4029
  • fix: Implement timeout layer correctly by using timeout by @Xuanwo in #4059
  • fix(koofr): create_dir when exist by @hoslo in #4062
  • fix(seafile): test_list_dir_with_metakey by @hoslo in #4063
  • fix: list path recursive should not return path itself by @youngsofun in #4067
  • fix: Upgrade suppaftp to address build failure by @Xuanwo in #4091

Docs

  • docs: Remove not needed actions in release guide by @Xuanwo in #4037
  • docs: fix spelling errors in README.md by @SYaoJun in #4039
  • docs: New PMC member Liuqing Yue by @Xuanwo in #4047
  • docs: New Committer Yang Shuai by @Xuanwo in #4054
  • docs(services/sftp): add more explanation for endpoint config by @silver-ymz in #4055
  • docs: Add upgrade docs for core by @Xuanwo in #4095
  • docs: fix reference in rustdocs of FuturePresignWrite by @qrilka in #4097
  • docs: Add docs of options for xxx_with APIs by @Xuanwo in #4099

CI

  • ci(services/s3): Use minio/minio image instead by @Xuanwo in #4070
  • ci: Fix CI after moving out of workspacs by @Xuanwo in #4081

Chore

  • chore: Delete bindings/ruby/cucumber.yml by @tisonkun in #4030
  • chore(website): Bump download link for 0.44.2 release by @Zheaoli in #4034
  • chore(website): Update the release tips by @Zheaoli in #4036
  • chore: add doap file by @tisonkun in #4038
  • chore(website): Add extra artifacts check process in release document by @Zheaoli in #4041
  • chore(bindings/dotnet): update cargo.lock and set up ci by @suyanhanx in #4084
  • chore(bindings/dotnet): build os detect by @suyanhanx in #4085
  • chore(bindings/ocaml): pinning OCaml binding opendal version for release by @Ranxy in #4086
  • chore: Specify opendal version for haskell binding by @Xuanwo in #4093
  • chore: Bump to version 0.45.0 to start release process by @morristai in #4088

New Contributors

Full Changelog: v0.44.2...v0.45.0

Contributors

qrilka, liurenjie1024, and 14 other contributors
Assets 2
Loading

v0.44.2

20 Jan 16:05
@Zheaoli Zheaoli
v0.44.2
This tag was signed with the committer’s verified signature.
Zheaoli Nadeshiko Manju
GPG key ID: 878F445D9C6CE65E
Learn about vigilant mode.
c5c6934
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Learn about vigilant mode.
Compare
Choose a tag to compare
v0.44.2

What's Changed

Added

  • feat: add behavior tests for blocking buffer reader by @WenyXu in #3872
  • feat(services): add pcloud support by @hoslo in #3892
  • feat(services/hdfs): Atomic write for hdfs by @shbhmrzd in #3875
  • feat(services/hdfs): add atomic_write_dir to hdfsconfig debug by @shbhmrzd in #3902
  • feat: add MongodbConfig by @zjregee in #3906
  • RFC-3898: Concurrent Writer by @WenyXu in #3898
  • feat(services): add yandex disk support by @hoslo in #3918
  • feat: implement concurrent MultipartUploadWriter by @WenyXu in #3915
  • feat: add concurrent writer behavior tests by @WenyXu in #3920
  • feat: implement concurrent RangeWriter by @WenyXu in #3923
  • feat: add concurrent and buffer parameters into FuzzInput by @WenyXu in #3921
  • feat(fuzz): add azblob as test service by @suyanhanx in #3931
  • feat(services/webhdfs): Implement write with append by @hoslo in #3937
  • feat(core/bench): Add benchmark for concurrent write by @Xuanwo in #3942
  • feat(oio): add block_write support by @hoslo in #3945
  • feat(services/webhdfs): Implement multi write via CONCAT by @hoslo in #3939
  • feat(core): Allow retry in concurrent write operations by @Xuanwo in #3958
  • feat(services/ghac): Add workaround for AWS S3 based GHES by @Xuanwo in #3985
  • feat: Implement path cache and refactor gdrive by @Xuanwo in #3975
  • feat(services): add hdfs native layout by @shbhmrzd in #3933
  • feat(services/s3): Return error if credential is empty after loaded by @Xuanwo in #4000
  • feat(services/gdrive): Use trash instead of permanently deletes by @Xuanwo in #4002
  • feat(services): add koofr support by @hoslo in #3981
  • feat(icloud): Add basic Apple iCloud Drive support by @bokket in #3980

Changed

  • refactor: Merge compose_{read,write} into enum_utils by @Xuanwo in #3871
  • refactor(services/ftp): Impl parse_error instead of From by @bokket in #3891
  • docs: very minor English wording fix in error message by @gabrielgrant in #3900
  • refactor(services/rocksdb): Impl parse_error instead of From by @suyanhanx in #3903
  • refactor: Re-organize the layout of tests by @Xuanwo in #3904
  • refactor(services/etcd): Impl parse_error instead of From by @suyanhanx in #3910
  • refactor(services/sftp): Impl parse_error instead of From by @G-XD in #3914
  • refactor!: Bump MSRV to 1.75 by @Xuanwo in #3851
  • refactor(services/redis): Impl parse_error instead of From by @suyanhanx in #3938
  • refactor!: Revert the bump of MSRV to 1.75 by @Xuanwo in #3952
  • refactor(services/onedrive): Add OnedriveConfig to implement ConfigDeserializer by @Borber in #3954
  • refactor(service/dropbox): Add DropboxConfig by @howiieyu in #3961
  • refactor: Polish internal types and remove not needed deps by @Xuanwo in #3964
  • refactor: Add concurrent error test for BlockWrite by @Xuanwo in #3968
  • refactor: Remove not needed types in icloud by @Xuanwo in #4021

Fixed

  • fix: Bump pyo3 to fix false positive of unnecessary_fallible_conversions by @Xuanwo in #3873
  • fix(core): Handling content encoding correctly by @Xuanwo in #3907
  • fix: fix RangeWriter incorrect next_offset by @WenyXu in #3927
  • fix(oio::BlockWrite): fix write_once case by @hoslo in #3953
  • fix: Don't retry close if concurrent > 1 to avoid content lost by @Xuanwo in #3957
  • fix(doc): fix rfc typos by @howiieyu in #3971
  • fix: Don't call wake_by_ref in OperatorFuture by @Xuanwo in #4003
  • fix: async fn resumed after initiate part failed by @Xuanwo in #4013
  • fix(pcloud,seafile): use get_basename and get_parent by @hoslo in #4020
  • fix(ci): remove pr author from review candidates by @dqhl76 in #4023

Docs

CI

  • build(deps): bump actions/download-artifact from 3 to 4 by @dependabot in #3885
  • build(deps): bump once_cell from 1.18.0 to 1.19.0 by @dependabot in #3880
  • build(deps): bump napi-derive from 2.14.2 to 2.14.6 by @dependabot in #3879
  • build(deps): bump url from 2.4.1 to 2.5.0 by @dependabot in #3876
  • build(deps): bump mlua from 0.8.10 to 0.9.2 by @oowl in #3890
  • ci: Disable supabase tests for our test org has been paused by @Xuanwo in #3908
  • ci: Downgrade artifact actions until regression addressed by @Xuanwo in #3935
  • ci: Refactor fuzz to integrate with test planner by @Xuanwo in #3936
  • ci: Pick random reviewers from committer list by @Xuanwo in #4001

Chore

Read more

Contributors

gabrielgrant, Xuanwo, and 17 other contributors
Assets 2
Loading

v0.44.1

06 Jan 08:50
@Xuanwo Xuanwo
v0.44.1
This tag was signed with the committer’s verified signature.
Xuanwo Xuanwo
GPG key ID: A61687076CEAD424
Learn about vigilant mode.
d4aab1e
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.
Compare
Choose a tag to compare
v0.44.1

What's Changed

Added

  • feat(service/memcached): Add MemCachedConfig by @ankit-pn in #3827
  • feat(service/rocksdb): Add RocksdbConfig by @ankit-pn in #3828
  • feat(services): add chainsafe support by @hoslo in #3834
  • feat(bindings/python): Build all available services for python by @Xuanwo in #3836
  • feat: Adding Atomicserver config by @k-aishwarya in #3845
  • feat(oio::read): implement the async buffer reader by @WenyXu in #3811
  • feat(oio::read): implement the blocking buffer reader by @WenyXu in #3860
  • feat: adapt the CompleteReader by @WenyXu in #3861
  • feat: add basic behavior tests for buffer reader by @WenyXu in #3862
  • feat: add fuzz reader with buffer tests by @WenyXu in #3866
  • feat(ofs): implement ofs based on fuse3 by @Inokinoki in #3857

Changed

  • refactor: simplify bindings_python.yml by @messense in #3837
  • refactor: Add edge test for aws assume role with web identity by @Xuanwo in #3839
  • refactor(services/webdav): Add WebdavConfig to implement ConfigDeserializer by @kwaa in #3846
  • refactor: use TwoWays instead of TwoWaysReader and TwoWaysWriter by @WenyXu in #3863

Fixed

  • fix: Add tests for listing recursively on not supported services by @Xuanwo in #3826
  • fix(services/upyun): fix list api by @hoslo in #3841
  • fix: fix a bypass seek relative bug in BufferReader by @WenyXu in #3864
  • fix: fix the bypass read does not sync the cur of BufferReader by @WenyXu in #3865

Docs

  • docs: Add Apache prefix for all bindings by @Xuanwo in #3829
  • docs: Add apache prefix for python docs by @Xuanwo in #3830
  • docs: Add branding in README by @Xuanwo in #3831
  • docs: Add trademark for Apache OpenDAL™ by @Xuanwo in #3832
  • docs: Add trademark sign for core by @Xuanwo in #3833
  • docs: Enable doc_auto_cfg when docs cfg has been enabled by @Xuanwo in #3835
  • docs: Address branding for haskell and C bindings by @Xuanwo in #3840
  • doc: add 0.44.0 release link to download.md by @dqhl76 in #3868

CI

  • ci: Remove workflows that not running or ready by @Xuanwo in #3842
  • ci: Migrate ftp to test planner by @Xuanwo in #3843

Chore

  • chore(bindings/java): Add name and description metadata by @tisonkun in #3838
  • chore(website): improve a bit trademark refs by @tisonkun in #3847
  • chore: Fix clippy warnings found in rust 1.75 by @Xuanwo in #3849
  • chore(bindings/python): improve ASF branding by @tisonkun in #3850
  • chore(bindings/haskell): improve ASF branding by @tisonkun in #3852
  • chore(bindings/c): make c binding separate workspace by @suyanhanx in #3856
  • chore(bindings/haskell): support co-log-0.6.0 && ghc-9.4 by @silver-ymz in #3858
  • chore: Bump to version 0.44.1 to start release process by @Xuanwo in #3869

New Contributors

Full Changelog: v0.44.0...v0.44.1

Contributors

messense, Xuanwo, and 10 other contributors
Assets 2
Loading

v0.44.0

31 Dec 03:00
@dqhl76 dqhl76
v0.44.0
This tag was signed with the committer’s verified signature.
dqhl76 Liuqing Yue
GPG key ID: 4F43CB2BC2E9EFE5
Learn about vigilant mode.
b0c6ea2
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.
Compare
Choose a tag to compare
v0.44.0

Upgrade Note

Rust core

Public API

Moka Service Configuration

  • The thread_pool_enabled option has been removed.

List Prefix Supported

After RFC: List Prefix landed, we have changed the behavior of list a path without /. OpenDAL used to return NotADirectory error, but now we will return the list of entries that start with given prefix instead.

Nodejs binding

Public API

Now, the list operation returns Array<Entry> instead of a lister.
Also, we removed scan, you can use list('some/path', {recursive: true})/listSync('some/path', {recursive: true}) instead of scan('some/path')/scanSync('some/path').

What's Changed

Added

Changed

  • refactor(service/etcd): use EtcdConfig in from_map by @G-XD in #3703
  • refactor(object_store): upgrade object_store to 0.7. by @youngsofun in #3713
  • refactor: List must support list without recursive by @Xuanwo in #3721
  • refactor: replace ftp tls impl as rustls by @oowl in #3760
  • refactor: Remove never used Stream poll_reset API by @Xuanwo in #3774
  • refactor: Polish operator read_with by @Xuanwo in #3775
  • refactor: Migrate gcs builder to config based by @Xuanwo in #3786
  • refactor(service/hdfs): Add HdfsConfig to implement ConfigDeserializer by @shbhmrzd in #3800
  • refactor(raw): add parse_header_to_str fn by @hoslo in #3804
  • refactor(raw): refactor APIs like parse_content_disposition by @hoslo in #3815
  • refactor: Polish http_util parse headers by @Xuanwo in #3817

Fixed

  • fix(oli): Fix cp -r command returns invalid path error by @kebe7jun in #3687
  • fix(website): folder name mismatch by @suyanhanx in #3707
  • fix(binding/java): fix SPECIAL_DIR_NAME by @G-XD in #3715
  • fix(services/dropbox): Workaround for dropbox limitations for create_folder by @Xuanwo in #3719
  • fix(ocaml_binding): sort actual & expected to pass ci by @Young-Flash in #3733
  • fix(ci): Make sure merge_local_staging handles all subdir by @Xuanwo in #3788
  • fix(services/gdrive): fix return value of get_file_id_by_path by @G-XD in #3801
  • fix(core): List root should not return itself by @Xuanwo in #3824

Docs

  • docs: add maturity model check by @suyanhanx in #3680
  • docs(website): show maturity model by @suyanhanx in #3709
  • docs(website): only VOTEs from PPMC members are binding by @G-XD in #3710
  • doc: add 0.43.0 release link to download.md by @G-XD in #3729
  • docs: Add process on nominating committers and ppmc members by @Xuanwo in #3740
  • docs: Deploy website to nightlies for every tags by @Xuanwo in #3739
  • docs: Remove not released bindings docs from top level header by @Xuanwo in #3741
  • docs: Add dependencies list for all packages by @Xuanwo in #3743
  • docs: Update maturity docs by @Xuanwo in #3750
  • docs: update the RFC doc by @suyanhanx in #3748
  • docs(website): polish deploy to nightlies by @suyanhanx in #3753
  • docs: add event calendar in community page by @dqhl76 in #3767
  • docs(community): polish events by @suyanhanx in #3768
  • docs(bindings/ruby): reflect test framework refactor by @tisonkun in #3798
  • docs(website): add service Huggingface to website by @morristai in #3812
  • docs: update release docs to add cargo-deny setup by @dqhl76 in #3821

CI

  • build(deps): bump cacache from 11.7.1 to 12.0.0 by @dependabot in #3690
  • build(deps): bump prometheus-client from 0.21.2 to 0.22.0 by @dependabot in #3694
  • build(deps): bump github/issue-labeler from 3.2 to 3.3 by @dependabot in #3698
  • ci: Add behavior test for b2 by @Xuanwo in #3714
  • ci(cargo): Add frame pointer support in build flag by @Zheaoli in #3772
  • ci: Workaround ring 0.17 build issue, bring aarch64 and armv7l back by @Xuanwo in #3781
  • ci: Support CI test for s3_read_on_wasm by @Zheaoli in #3813

Chore

  • chore: bump aws-sdk-s3 from 0.38.0 to 1.4.0 by @memoryFade in #3704
  • chore: Disable obs test for workaround by @Xuanwo in #3717
  • chore: Fix bindings CI by @Xuanwo in #3722
  • chore(binding/nodejs,website): Replace yarn with pnpm by @suyanhanx in #3730
  • chore: Bring persy CI back by @Xuanwo in #3751
  • chore(bindings/python): upgrade pyo3 to 0.20 by @messense in #3758
  • chore: remove unused binding feature file by @tisonkun in #3757
  • chore: Bump governor from 0.5.1 to 0.6.0 by @G-XD in #3761
  • chore: Split bindings/ocaml to separate workspace by @Xuanwo in #3792
  • chore: Split bindings/ruby to separate workspace by @ho-229 in #3794
  • chore(bindings/php): bump ext-php-rs to support latest php & separate workspace by @suyanhanx in #3799
  • chore: Address comments from hackernews by @Xuanwo in #3805
  • chore(bindings/ocaml): dep opendal point to core by @suyanhanx in #3814
  • chore: Bump to v0.44.0 to start release process by @dqhl76 in #3819

New Contributors

Full Changelog: http...

Read more

Contributors

messense, Xuanwo, and 16 other contributors
Assets 2
Loading

v0.43.0

08 Dec 10:43
@G-XD G-XD
v0.43.0
146e960
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.
Compare
Choose a tag to compare
v0.43.0

Upgrade Note

Rust Core

Public API

List Recursive

After RFC-3526: List Recursive landed, we have changed the list API to accept recursive instead of delimiter:

Users will need to change the following usage:

  • op.list_with(path).delimiter("") -> op.list_with(path).recursive(true)
  • op.list_with(path).delimiter("/") -> op.list_with(path).recursive(false)

delimiter other than "" and "/" is not supported anymore.

Stat a dir path

After RFC: List Prefix landed, we have changed the behavior of stat a dir path:

Here are the behavior list:

Case Path Result
stat existing dir abc/ Metadata with dir mode
stat existing file abc/def_file Metadata with file mode
stat dir without / abc/def_dir Error NotFound or metadata with dir mode
stat file with / abc/def_file/ Error NotFound
stat not existing path xyz Error NotFound

Services like s3, azblob can handle stat("abc/") correctly by check if there are objects with prefix abc/.

Raw API

Lister Align

We changed our internal lister implementation to align with the list public API for better performance and readability.

  • trait Page => List
  • struct Pager => Lister
  • trait BlockingPage => BlockingList
  • struct BlockingPager => BlockingLister

Every call to next will return an entry instead a page of entries. Also, we changed our async list api into poll based instead of async_trait.

Java binding

Breaking change

Because of a TLS lib issue, we temporarily disable the services-ftp feature.

Node.js binding

Breaking change

Because of a TLS lib issue, we temporarily disable the services-ftp feature.

Python binding

Breaking change

Because of a TLS lib issue, we temporarily disable the services-ftp feature.

C binding

There are no API changes.

What's Changed

Added

  • feat(bindings/C): Add opendal_operator_rename and opendal_operator_copy by @jiaoew1991 in #3517
  • feat(binding/python): Add new API to convert between AsyncOperator and Operator by @Zheaoli in #3514
  • feat: Implement RFC-3526: List Recursive by @Xuanwo in #3556
  • feat(service): add alluxio rest api support by @hoslo in #3564
  • feat(bindings/python): add OPENDAL_DISABLE_RANDOM_ROOT support by @Justin-Xiang in #3550
  • feat(core): add Alluxio e2e test by @hoslo in #3573
  • feat(service): alluxio support write by @hoslo in #3566
  • feat(bindings/nodejs): add retry layer by @suyanhanx in #3484
  • RFC: Concurrent Stat in List by @morristai in #3574
  • feat(service/hdfs): enable rename in hdfs service by @qingwen220 in #3592
  • feat: Improve the read_to_end perf and add benchmark vs_fs by @Xuanwo in #3617
  • feat: Add benchmark vs aws sdk s3 by @Xuanwo in #3620
  • feat: Improve the performance of s3 services by @Xuanwo in #3622
  • feat(service): support b2 by @hoslo in #3604
  • feat(core): Implement RFC-3574 Concurrent Stat In List by @morristai in #3599
  • feat: Implement stat dir correctly based on RFC-3243 List Prefix by @Xuanwo in #3651
  • feat(bindings/nodejs): Add capability support by @suyanhanx in #3654
  • feat: disable ftp for python and java binding by @ZutJoe in #3659
  • feat(bindings/nodejs): read/write stream by @suyanhanx in #3619

Changed

  • refactor(services/persy): migrate tot test planner by @G-XD in #3476
  • refactor(service/etcd): Add EtcdConfig to implement ConfigDeserializer by @Xuxiaotuan in #3543
  • chore(service/tikv): rename Backend to TikvBackend by @caicancai in #3545
  • refactor(services/azblob): add AzblobConfig by @acehinnnqru in #3553
  • refactor(services/cacache): migrate to test planner by @G-XD in #3568
  • refactor(services/sled): migrate to test planner by @G-XD in #3569
  • refactor(services/webhdfs): migrate to test planner by @G-XD in #3578
  • refactor(core): Rename all Page to List by @Xuanwo in #3589
  • refactor: Change List API into poll based and return one entry instead by @Xuanwo in #3593
  • refactor(services/tikv): migrate to test planner by @G-XD in #3587
  • refactor(service/redis): Migrate task to new task planner by @sunheyi6 in #3374
  • refactor(oio): Polish IncomingAsyncBody::bytes by @Xuanwo in #3621
  • refactor(services/rocksdb): migrate to test planner by @G-XD in #3636
  • refactor(services/azfile): Check if dir exists before create by @ZutJoe in #3652
  • refactor: Polish concurrent list by @Xuanwo in #3658

Fixed

Docs

CI

  • ci(services/libsql): add rust test threads limit by @G-XD in #3540
  • ci(services/redb): migrate to test planner by @suyanhanx in #3518
  • ci: Disable libsql behavior test until we or upstream address them by @Xuanwo in #3552
  • ci: Add new Python binding reviewer by @Zheaoli in #3560
  • ci(bindings/nodejs): add aarch64 build support by @suyanhanx in #3567
  • ci(planner): Polish the workflow planner code by @Zheaoli in #3570
  • ci(core): Add dry run for rc tags by @Xuanwo in #3624
  • ci: Disable persy until it has been fixed by @Xuanwo in #3631
  • ci: Calling cargo to make sure rust has been setup by @Xuanwo in #3633
  • ci: Fix etcd with tls and auth failed to start by @Xuanwo in #3637
  • ci(services/etcd): Use ALLOW_NONE_AUTHENTICATION as workaround by @Xuanwo in #3638
  • ci: dry run publish on rc tags for python binding by @everpcpc in #3645
  • ci: Add java linux arm64 build by @Xuanwo in #3660
  • ci(java/binding): Use zigbuild for glibc 2.17 support by @Xuanwo in #3664
  • ci(bindings/python): remove aarch support by @G-XD in #3674

Chore

  • chore(servies/sftp): Upgrade openssh-sftp-client to 0.14 by @sd44 in https://g...
Read more

Contributors

vincentdephily, jtescher, and 25 other contributors
Assets 2
Loading