Merge commit '4c95db3223336cc85cb495c90e58d6be3826beed' into masync

src/deps/miniasync/ChangeLog

@@ -1,13 +1,31 @@
+Thu Aug 11 2022 Łukasz Plewa <[email protected]>
+
+	* Version 0.2.0-rc1
+
+	common: use correct atomics in ringbuf
+	common: fix use-after-free in membuf in mt scenarios
+	common: replace improper 'RPMA' in man pages with 'MINIASYNC'
+	doc: add async property to documentation
+	doc: update async property documentation
+	doc: update code styling to properly appear in manpage
+	masync: drop unused #ifdefs that block portability
+	masync: add hashmap example
+	masync: add flush operation for DML data mover
+	masync: add flag indicating that future is async
+	masync: add destination_readback flag to persistent memory writes
+	masync: add macro to check if chain entry was initialized
+
+
 Fri May 20 2022 Weronika Lewandowska <[email protected]>
 
 	* Version 0.1.0
 
 	This is the first official release of the miniasync library.
- 
+
 	The library provides the API for asynchronous memory operations
 	through the use of features and runtime mechanisms and independence
 	in terms of hardware by implementing a virtual data mover.
-	
+
 	This release also contains:
 	- documentation on the implemented functionality
 	- basic examples

src/deps/miniasync/doc/data_mover_dml_get_vdm.3.md

@@ -47,12 +47,13 @@ structure *struct vdm* is needed by every **miniasync**(7) data mover operation.
 * **vdm_memcpy**(3) - memory copy operation
 * **vdm_memmove**(3) - memory move operation
 * **vdm_memset**(3) - memory set operation
+* **vdm_flush**(3) - cache flush operation
 
 # RETURN VALUE #
 
 The **data_mover_dml_get_vdm**() function returns a pointer to *struct vdm* structure.
 
 # SEE ALSO #
 
-**vdm_memcpy**(3), **vdm_memmove**(3), **vdm_memset**(3),
- **miniasync**(7), **miniasync_vdm_dml**(7) and **<https://pmem.io>**
+**vdm_flush**(3), **vdm_memcpy**(3), **vdm_memmove**(3), **vdm_memset**(3),
+**miniasync**(7), **miniasync_vdm_dml**(7) and **<https://pmem.io>**

src/deps/miniasync/doc/data_mover_threads_new.3.md

@@ -58,7 +58,7 @@ The **data_mover_threads_default**() function allocates and initialzied a new th
 data mover structure with default parameters. It spanws *12* threads and creates a
 ringbuffer with size of *128* bytes.
 
-Currently, thread data mover supports following notifer types:
+Currently, thread data mover supports following notifier types:
 
 * **FUTURE_NOTIFIER_NONE**
 

src/deps/miniasync/doc/manuals.txt

@@ -20,3 +20,4 @@ runtime_wait.3
 vdm_memcpy.3
 vdm_memmove.3
 vdm_memset.3
+vdm_flush.3

src/deps/miniasync/doc/miniasync_future.7.md

@@ -38,6 +38,9 @@ typedef void (*future_map_fn)(struct future_context *lhs,
 typedef void (*future_init_fn)(void *future,
 			struct future_context *chain_fut, void *arg);
 
+typedef int (*future_has_property_fn)(void *future,
+			enum future_property property);
+
 enum future_state {
 	FUTURE_STATE_IDLE,
 	FUTURE_STATE_COMPLETE,
@@ -73,13 +76,18 @@ struct future_notifier {
 	uint32_t padding;
 };
 
+enum future_property {
+	FUTURE_PROPERTY_ASYNC,
+};
+
 FUTURE(_name, _data_type, _output_type)
 FUTURE_INIT(_futurep, _taskfn)
 FUTURE_INIT_COMPLETE(_futurep)
 FUTURE_CHAIN_ENTRY(_future_type, _name)
 FUTURE_CHAIN_ENTRY_LAST(_future_type, _name)
 FUTURE_CHAIN_ENTRY_INIT(_entry, _fut, _map, _map_arg)
 FUTURE_CHAIN_ENTRY_LAZY_INIT(_entry, _init, _init_arg, _map, _map_arg)
+FUTURE_CHAIN_ENTRY_IS_INITIALIZED(_entry)
 FUTURE_CHAIN_INIT(_futurep)
 FUTURE_AS_RUNNABLE(_futurep)
 FUTURE_OUTPUT(_futurep)
@@ -103,6 +111,7 @@ A future contains the following context:
 * structure for data which is the required state needed to perform the task
 * structure for output to store the result of the task
 * the size of the data and output structures (both can be 0)
+* a pointer for the property checking function which checks if the future has a custom property
 
 A future definition must begin with an instance of the *struct future* type, which
 contains all common metadata for all futures, followed by the structures for
@@ -132,14 +141,24 @@ A future implementation supporting **FUTURE_NOTIFIER_WAKER** type of notifier ca
 use a **FUTURE_WAKER_WAKE(_wakerp)** macro to signal the caller that some progress
 can be made and the future should be polled again.
 
-TODO: Mention **FUTURE_NOTIFIER_POLLER** when it becomes supported.
+<!-- TODO: Mention **FUTURE_NOTIFIER_POLLER** when it becomes supported. -->
+
+Futures can contain custom properties. Information, whether the future contains
+the property or not, is returned by the **future_has_property** function.
+Concrete implementation of the **future_has_property_fn** function should return the
+information, whether given future property applies to the future. Individual futures
+can be initialized using `FUTURE_INIT_EXT(_futurep, _taskfn, _propertyfn)` macro to
+set the property checking function. Futures initialized regularly have no properties applied.
+
+Supported properties:
+* **FUTURE_PROPERTY_ASYNC** property indicates that the future is asynchronous.
 
 For more information about the usage of future API, see *examples* directory
 in miniasync repository <https://github.com/pmem/miniasync>.
 
 # MACROS #
 
-**FUTURE(_name, _data_type, _output_type)** macro defines a future structure with *\_name*
+`FUTURE(_name, _data_type, _output_type)` macro defines a future structure with *\_name*
 as its name. Besides internal data needed by the future API, the defined structure contains
 data member of *\_data_type* type and output member of *\_output_type* type. User can
 provide the data that may later be retrieved in the future task implementation using
@@ -148,51 +167,59 @@ retrieved using **future_context_get_output**(3) function. Combined size of data
 structures can be retrieved using **future_context_get_size**(3) function. When the user has
 no need for input or output data, *\_data_type* and *\_output_type* can be defined as empty structures.
 
-**FUTURE_INIT(_futurep, _taskfn)** macro assigns task function *\_taskfn* to the future pointed
+`FUTURE_INIT(_futurep, _taskfn)` macro assigns task function *\_taskfn* to the future pointed
 by *\_futurep*. Task function must be of the *future_task_fn* type.
 
-**FUTURE_INIT_COMPLETE(_futurep)** macro instantiates a new already completed future with no assigned
+`FUTURE_INIT_EXT(_futurep, _taskfn, _propertyfn)` macro assigns task function *\_taskfn* and property
+checking function *\_propertyfn* to the future pointed by *\_futurep*. Task function must be of the
+*future_task_fn* type and the property checking function must be of the type *future_has_property_fn*.
+
+`FUTURE_INIT_COMPLETE(_futurep)` macro instantiates a new already completed future with no assigned
 task. This is helpful for handling initialization errors during future creation or simply for convenience
 in instantly ready futures.
 
-**FUTURE_CHAIN_ENTRY(_future_type, _name)** macro defines the future chain entry of the *\_future_type*
+`FUTURE_CHAIN_ENTRY(_future_type, _name)` macro defines the future chain entry of the *\_future_type*
 type named *\_name*. Future chain entries are defined as the members of chained future data structure
 using this macro. Chained future can be composed of multiple future chain entries that will be
 executed sequentially in the order they were defined.
 
-**FUTURE_CHAIN_ENTRY_LAST(_future_type, _name)** macro can be optionally used to indicate the last
+`FUTURE_CHAIN_ENTRY_LAST(_future_type, _name)` macro can be optionally used to indicate the last
 future in a chain. This lets software to include additional state inside of the *\_data_type* since
 otherwise the chain task implementation would not be able to differentiate between an entry and
 other data.
 
-**FUTURE_CHAIN_ENTRY_INIT(_entry, _fut, _map, _map_arg)** macro initializes the future chain
+`FUTURE_CHAIN_ENTRY_INIT(_entry, _fut, _map, _map_arg)` macro initializes the future chain
 entry pointed by *\_entry*. It requires pointer to the future instance *\_fut*, address of the mapping
 function *\_map* and the pointer to the argument for the mapping function *\_map_arg*. *\_fut* can either be
-the instance of the future defined using **FUTURE(_name, _data_type, _output_type)** macro or a virtual
+the instance of the future defined using `FUTURE(_name, _data_type, _output_type)` macro or a virtual
 data mover future. *\_map* function must be of the *future_map_fn* type and is an optional parameter.
 *map* function should define the mapping behavior of the data and output structures between chained
 future entry *\_entry* that has finished and the chained future entry that is about to start its execution.
 Chained future instance must initialize all of its future chain entries using this macro.
 
-**FUTURE_CHAIN_ENTRY_LAZY_INIT(_entry, _init, _init_arg, _map, _map_arg)** macro intializes the
+`FUTURE_CHAIN_ENTRY_LAZY_INIT(_entry, _init, _init_arg, _map, _map_arg)` macro initializes the
 future chain entry pointed by *\_entry* but it does not initialize its underlying future. Instead
-it uses function *\_init* and its argument *\init_arg* to instantiate the future right before
-its needed. This lets software instantiate futures with arguments derived from the results of
-previous entries in the chain. The *\map* and *\map_arg* variables behave the
+it uses function *\_init* and its argument *\_init_arg* to instantiate the future right before
+it's needed. This lets software instantiate futures with arguments derived from the results of
+previous entries in the chain. The *\_map* and *\_map_arg* variables behave the
 same as in **FUTURE_CHAIN_ENTRY_INIT**.
 
-**FUTURE_CHAIN_INIT(_futurep)** macro initializes the chained future at the address *\_futurep*.
+`FUTURE_CHAIN_ENTRY_IS_INITIALIZED(_entry)` macro checks if the future chain entry, pointed by *\_entry*,
+is already initialized (either lazily or regularly). If it is initialized, its structure can be safely
+accessed and used.
+
+`FUTURE_CHAIN_INIT(_futurep)` macro initializes the chained future at the address *\_futurep*.
 
-**FUTURE_AS_RUNNABLE(_futurep)** macro returns pointer to the runnable form of the future pointed by
+`FUTURE_AS_RUNNABLE(_futurep)` macro returns pointer to the runnable form of the future pointed by
 *\_futurep*. Runnable form of the future is required as an argument in **runtime_wait**(3) and
 **runtime_wait_multiple**(3) functions.
 
-**FUTURE_OUTPUT(_futurep)** macro returns the output of the future pointed by *\_futurep*.
+`FUTURE_OUTPUT(_futurep)` macro returns the output of the future pointed by *\_futurep*.
 
-**FUTURE_BUSY_POLL(_futurep)** repeatedly polls the future pointed by *\_futurep* until
+`FUTURE_BUSY_POLL(_futurep)` repeatedly polls the future pointed by *\_futurep* until
 it completes its execution. This macro does not use optimized polling.
 
-**FUTURE_WAKER_WAKE(_wakerp)** macro performs implementation-defined wake operation. It takes
+`FUTURE_WAKER_WAKE(_wakerp)` macro performs implementation-defined wake operation. It takes
 a pointer to the waker structure of *struct future_waker* type.
 
 # SEE ALSO #

src/deps/miniasync/doc/miniasync_vdm.7.md

@@ -42,12 +42,14 @@ struct vdm {
 	vdm_operation_delete op_delete;
 	vdm_operation_start op_start;
 	vdm_operation_check op_check;
+	unsigned capabilities;
 };
 
 enum vdm_operation_type {
 	VDM_OPERATION_MEMCPY,
 	VDM_OPERATION_MEMMOVE,
 	VDM_OPERATION_MEMSET,
+	VDM_OPERATION_FLUSH,
 };
 
 enum vdm_operation_result {
@@ -57,8 +59,9 @@ enum vdm_operation_result {
 };
 
 struct vdm_operation_data {
-	void *op;
+	void *data;
 	struct vdm *vdm;
+	struct vdm_operation operation;
 };
 
 struct vdm_operation_output {
@@ -67,6 +70,8 @@ struct vdm_operation_output {
 	union {
 		struct vdm_operation_output_memcpy memcpy;
 		struct vdm_operation_output_memmove memmove;
+		struct vdm_operation_output_memset memset;
+		struct vdm_operation_output_flush flush;
 	} output;
 };
 ```
@@ -95,6 +100,7 @@ Currently, virtual data mover API supports following operation types:
 * **VDM_OPERATION_MEMCPY** - a memory copy operation
 * **VDM_OPERATION_MEMMOVE** - a memory move operation
 * **VDM_OPERATION_MEMSET** - a memory set operation
+* **VDM_OPERATION_FLUSH** - a cache flush operation
 
 For more information about concrete data mover implementations, see **miniasync_vdm_threads**(7),
 **miniasync_vdm_synchronous**(7) and **miniasync_vdm_dml**(7).
@@ -122,7 +128,7 @@ The *result* field can be set to one of the following values:
 
 # SEE ALSO #
 
-**vdm_memcpy**(3), **vdm_memmove**(3), **vdm_memset**(3),
+**vdm_flush**(3), **vdm_memcpy**(3), **vdm_memmove**(3), **vdm_memset**(3),
 **miniasync**(7), **miniasync_future**(7),
 **miniasync_vdm_dml**(7), **miniasync_vdm_synchronous**(7),
 **miniasync_vdm_threads**(7) and **<https://pmem.io>**

src/deps/miniasync/doc/miniasync_vdm_dml.7.md

@@ -49,19 +49,20 @@ An example of **DML** data mover API usage with flags can be found in **EXAMPLE*
 
 When the future is polled for the first time the data mover operation will be executed
 asynchronously under the control of **DML** library. **DML** data mover does not
-block the calling thread
+block the calling thread.
 
 To create a new **DML** data mover instance, use **data_mover_dml_new**(3) function.
 
 **DML** data mover provides the following flags:
 
-* **MINIASYNC_DML_F_MEM_DURABLE** - write to destination is identified as write to durable memory
+* **VDM_F_MEM_DURABLE** - write to destination is identified as write to durable memory
 
 **DML** data mover supports following operations:
 
 * **vdm_memcpy**(3) - memory copy operation
 * **vdm_memmove**(3) - memory move operation
 * **vdm_memset**(3) - memory set operation
+* **vdm_flush**(3) - cache flush operation
 
 **DML** data mover does not support notifier feature. For more information about
 notifiers, see **miniasync_future**(7).
@@ -80,6 +81,6 @@ struct vdm_memcpy_future memcpy_fut = vdm_memcpy(dml_mover, dest, src,
 # SEE ALSO #
 
 **data_mover_dml_new**(3), **data_mover_dml_get_vdm**(3),
-**vdm_memcpy**(3), **vdm_memmove**(3), **vdm_memset**(3),
+**vdm_flush**(3), **vdm_memcpy**(3), **vdm_memmove**(3), **vdm_memset**(3),
 **miniasync**(7), **miniasync_future**(7), **miniasync_vdm**(7),
 **<https://github.com/intel/DML>** and **<https://pmem.io>**

src/deps/miniasync/doc/miniasync_vdm_threads.7.md

@@ -55,7 +55,7 @@ Thread data mover supports following operations:
 * **vdm_memmove**(3) - memory move operation
 * **vdm_memset**(3) - memory set operation
 
-Thread data mover supports following notifer types:
+Thread data mover supports following notifier types:
 
 * **FUTURE_NOTIFIER_NONE** - no notifier
 * **FUTURE_NOTIFIER_WAKER** - waker

src/deps/miniasync/doc/runtime_wait.3.md

@@ -51,7 +51,13 @@ future with **future_poll**(3) function until completion.
 The **runtime_wait_multiple**() function works similar to the **runtime_wait**() function,
 additionally it facilitates polling of multiple futures in an array.  **runtime_wait_multiple**()
 function uniformly polls the first *nfuts* futures in the array pointed by *futs* until all
-of them complete execution.
+of them complete execution. Runtime execution can be influenced by future properties.
+For more information about the future properties, see **miniasync_future**(7).
+
+Properties, which affect runtime:
+* **FUTURE_PROPERTY_ASYNC** property should be applied to asynchronous futures.
+During **runtime_wait_multiple**() function, asynchronous futures have a priority over the
+synchronous ones and, in general, are being polled first.
 
 **miniasync**(7) runtime implementation makes use of the waker notifier feature to optimize
 future polling. For more information about the waker feature, see **miniasync_future**(7).

src/deps/miniasync/doc/vdm_flush.3.md

@@ -0,0 +1,59 @@
+---
+layout: manual
+Content-Style: 'text/css'
+title: _MP(VDM_FLUSH, 3)
+collection: miniasync
+header: VDM_FLUSH
+secondary_title: miniasync
+...
+
+[comment]: <> (SPDX-License-Identifier: BSD-3-Clause)
+[comment]: <> (Copyright 2022, Intel Corporation)
+
+[comment]: <> (vdm_flush.3 -- man page for miniasync vdm_flush operation)
+
+[NAME](#name)<br />
+[SYNOPSIS](#synopsis)<br />
+[DESCRIPTION](#description)<br />
+[RETURN VALUE](#return-value)<br />
+[SEE ALSO](#see-also)<br />
+
+# NAME #
+
+**vdm_flush**() - create a new flush virtual data mover operation structure
+
+# SYNOPSIS #
+
+```c
+#include <libminiasync.h>
+
+struct vdm_operation_output_flush {
+	uint64_t unused;
+};
+
+FUTURE(vdm_operation_future,
+	struct vdm_operation_data, struct vdm_operation_output);
+
+struct vdm_operation_future vdm_flush(struct vdm *vdm, void *dest, size_t n, uint64_t flags);
+```
+
+For general description of virtual data mover API, see **miniasync_vdm**(7).
+
+# DESCRIPTION #
+
+**vdm_flush**() initializes and returns a new flush future based on the virtual data mover
+implementation instance *vdm*. The parameters: *dest*, *n* are standard flush parameters.
+The *flags* represents data mover specific flags. The **flush** operation is
+implemented only for the DML data mover and cannot be used with synchronous and thread data movers.
+
+Flush future obtained using **vdm_flush**() will attempt to flush the *n* bytes of the processor
+caches at the *dest* address when its polled.
+
+## RETURN VALUE ##
+
+The **vdm_flush**() function returns an initialized *struct vdm_operation_future* flush future.
+
+# SEE ALSO #
+
+**vdm_memcpy**(3), **vdm_memmove**(3), **vdm_memset**(3), **miniasync**(7), **miniasync_vdm**(7),
+**miniasync_vdm_dml**(7) and **<https://pmem.io>**

src/deps/miniasync/doc/vdm_is_supported.3.md

@@ -37,7 +37,7 @@ static inline int vdm_is_supported(struct vdm *vdm, unsigned capability);
 Currently vdm defines the following capabilities:
 - **VDM_F_NO_CACHE_HINT** - If supported, user can pass this flag to the **vdm_memcpy**(), **vdm_memset**(), **vdm_memmove**()
 functions, to hint vdm to bypass CPU cache, and write the data directly to the memory. If not supported vdm will ignore this flag.
-- **VDM_F_MEM_DURABLE** -- If supported, user can pass this flag **vdm_memcpy**(), **vdm_memset**(), **vdm_memmove**() functions
+- **VDM_F_MEM_DURABLE** -- If supported, user can pass this flag to the **vdm_memcpy**(), **vdm_memset**(), **vdm_memmove**() functions
 to ensure that the data written has become persistent, when a future completes.
 
 ## RETURN VALUE ##
@@ -46,5 +46,5 @@ The **vdm_is_supported**() function returns nonzero if the given capability is s
 
 # SEE ALSO #
 
-**vdm_memmove**(3), **vdm_memset**(3), **miniasync**(7), **miniasync_vdm**(7),
+**vdm_flush**(3), **vdm_memcpy**(3), **vdm_memmove**(3), **vdm_memset**(3), **miniasync**(7), **miniasync_vdm**(7),
 **miniasync_vdm_dml**(7) and **<https://pmem.io>**