Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Eio.Pool.use ~never_block #657

Merged
merged 1 commit into from
Mar 26, 2024

Conversation

SGrondin
Copy link
Collaborator

@SGrondin SGrondin commented Dec 16, 2023

This PR adds the optional parameter ?never_block:bool to Eio.Pool.use. The default is false.

Quoting pool.mli:

If true and the pool has reached maximum capacity, then a fresh resource is created to ensure that this use call does not wait for a resource to become available. This resource is immediately disposed after [f x] returns.

In my opinion, this feature makes Eio.Pool a valid choice for a wider range of use cases.

Method Availability Overhead
Creating a resource every time ✅ Never blocks 🔴 No reuse
Eio.Pool.use 🔴 Can block ✅ Maximal reuse
Eio.Pool.use ~never_block:true ✅ Never blocks 🟡 Some reuse

@SGrondin SGrondin force-pushed the pool-never-block branch 2 times, most recently from c1e0aa1 to c91cf5c Compare January 6, 2024 16:26
@talex5 talex5 force-pushed the pool-never-block branch 2 times, most recently from 5b4abc9 to dbb74d3 Compare March 18, 2024 13:18
Copy link
Collaborator

@talex5 talex5 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks useful - thanks!

(I rebased and also updated the test to check that dispose works for the temporary resource)

lib_eio/pool.ml Outdated
| false when never_block ->
(* We are at capacity, but cannot block.
Create a new resource to run f but don't add it to the pool. *)
Atomic.set cell Finished;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This isn't quite right. The cell might contain a Resource by now. If so, we should use that.

(probably needs an Atomic.exchange cell Finished here)

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, I'll fix it this weekend if you don't get to it first

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pushed. I have a remaining question for you that I'm keeping for the devs meeting, so I think merging this PR should wait until after the meeting.

Copy link
Collaborator

@talex5 talex5 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

@talex5 talex5 merged commit 3be614e into ocaml-multicore:main Mar 26, 2024
5 checks passed
talex5 added a commit to talex5/opam-repository that referenced this pull request May 28, 2024
CHANGES:

New features:

- Add `Eio.Path.symlink` (@patricoferris ocaml-multicore/eio#715, reviewed by @talex5).

- Add `Eio.Pool.use ~never_block` (@SGrondin ocaml-multicore/eio#657, reviewed by @talex5).

- Add `Eio_unix.Net.import_socket_listening` (@alyssais ocaml-multicore/eio#733).

- Add `Eio.Time.Timeout.sleep` (@talex5 ocaml-multicore/eio#726).

Documentation:

- Add `examples/fs` showing how to walk a directory tree (@talex5 ocaml-multicore/eio#730).

- README: explain that `read_all` reads until shutdown (@talex5 ocaml-multicore/eio#717, reported by @Wenke-D).

- Use long dash in README title (@lucperkins ocaml-multicore/eio#718).

Linux backend:

- Require Linux >= 5.15 (@talex5 ocaml-multicore/eio#720, reviewed by @SGrondin and @avsm).
  Removes a work-around that required checking whether every flow was a tty.

- Don't call submit immediately before wait (@talex5 ocaml-multicore/eio#728).
  This is slightly faster and makes the traces clearer.

- Don't record submit events when there's nothing to submit (@talex5 ocaml-multicore/eio#729).
  Makes the traces a bit clearer.

- Split flow into its own file (@talex5 ocaml-multicore/eio#727).

- Add work-around for signals race (@talex5 ocaml-multicore/eio#734).

POSIX backend:

- Add `_BSD_SOURCE` flag to fix build on OpenBSD (@prgbln ocaml-multicore/eio#722).

- Fix sandboxed path resolution on OpenBSD (@jebrosen ocaml-multicore/eio#723, reviewed by @talex5).
  OpenBSD uses `ELOOP` when opening a symlink with `O_NOFOLLOW`.

Build and test:

- Benchmarks: record uname, Eio backend, and number of cores (@talex5 ocaml-multicore/eio#719).

- Update to MDX 2.4.1 for OCaml 5.2 (@talex5 ocaml-multicore/eio#712).
talex5 added a commit to talex5/opam-repository that referenced this pull request May 28, 2024
CHANGES:

New features:

- Add `Eio.Path.symlink` (@patricoferris ocaml-multicore/eio#715, reviewed by @talex5).

- Add `Eio.Pool.use ~never_block` (@SGrondin ocaml-multicore/eio#657, reviewed by @talex5).

- Add `Eio_unix.Net.import_socket_listening` (@alyssais ocaml-multicore/eio#733, reviewed by @talex5).

- Add `Eio.Time.Timeout.sleep` (@talex5 ocaml-multicore/eio#726).

Documentation:

- Add `examples/fs` showing how to walk a directory tree (@talex5 ocaml-multicore/eio#730).

- README: explain that `read_all` reads until shutdown (@talex5 ocaml-multicore/eio#717, reported by @Wenke-D).

- Use long dash in README title (@lucperkins ocaml-multicore/eio#718).

Linux backend:

- Require Linux >= 5.15 (@talex5 ocaml-multicore/eio#720, reviewed by @SGrondin and @avsm).
  Removes a work-around that required checking whether every flow was a tty.

- Don't call submit immediately before wait (@talex5 ocaml-multicore/eio#728).
  This is slightly faster and makes the traces clearer.

- Don't record submit events when there's nothing to submit (@talex5 ocaml-multicore/eio#729).
  Makes the traces a bit clearer.

- Split flow into its own file (@talex5 ocaml-multicore/eio#727).

- Add work-around for signals race (@talex5 ocaml-multicore/eio#734).

POSIX backend:

- Add `_BSD_SOURCE` flag to fix build on OpenBSD (@prgbln ocaml-multicore/eio#722).

- Fix sandboxed path resolution on OpenBSD (@jebrosen ocaml-multicore/eio#723, reviewed by @talex5).
  OpenBSD uses `ELOOP` when opening a symlink with `O_NOFOLLOW`.

Build and test:

- Benchmarks: record uname, Eio backend, and number of cores (@talex5 ocaml-multicore/eio#719).

- Update to MDX 2.4.1 for OCaml 5.2 (@talex5 ocaml-multicore/eio#712).
avsm pushed a commit to avsm/opam-repository that referenced this pull request Sep 5, 2024
CHANGES:

New features:

- Add `Eio.Path.symlink` (@patricoferris ocaml-multicore/eio#715, reviewed by @talex5).

- Add `Eio.Pool.use ~never_block` (@SGrondin ocaml-multicore/eio#657, reviewed by @talex5).

- Add `Eio_unix.Net.import_socket_listening` (@alyssais ocaml-multicore/eio#733, reviewed by @talex5).

- Add `Eio.Time.Timeout.sleep` (@talex5 ocaml-multicore/eio#726).

Documentation:

- Add `examples/fs` showing how to walk a directory tree (@talex5 ocaml-multicore/eio#730).

- README: explain that `read_all` reads until shutdown (@talex5 ocaml-multicore/eio#717, reported by @Wenke-D).

- Use long dash in README title (@lucperkins ocaml-multicore/eio#718).

Linux backend:

- Require Linux >= 5.15 (@talex5 ocaml-multicore/eio#720, reviewed by @SGrondin and @avsm).
  Removes a work-around that required checking whether every flow was a tty.

- Don't call submit immediately before wait (@talex5 ocaml-multicore/eio#728).
  This is slightly faster and makes the traces clearer.

- Don't record submit events when there's nothing to submit (@talex5 ocaml-multicore/eio#729).
  Makes the traces a bit clearer.

- Split flow into its own file (@talex5 ocaml-multicore/eio#727).

- Add work-around for signals race (@talex5 ocaml-multicore/eio#734).

POSIX backend:

- Add `_BSD_SOURCE` flag to fix build on OpenBSD (@prgbln ocaml-multicore/eio#722).

- Fix sandboxed path resolution on OpenBSD (@jebrosen ocaml-multicore/eio#723, reviewed by @talex5).
  OpenBSD uses `ELOOP` when opening a symlink with `O_NOFOLLOW`.

Build and test:

- Benchmarks: record uname, Eio backend, and number of cores (@talex5 ocaml-multicore/eio#719).

- Update to MDX 2.4.1 for OCaml 5.2 (@talex5 ocaml-multicore/eio#712).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants