Support unverified third party APIs in controllers #180

euclidgame · 2023-07-18T18:48:55Z

This PR allows developers to call any (unverified) third-party APIs besides the Kubernetes client APIs when implementing the controller.

It adds Lib as a generic type to Reconciler. Lib also takes two generic types LibInputType and LibOutputType. Lib wraps all interactions with the third-party APIs, and it implements the trait ExternalLibrary<LibInputType, LibOutputType>. ExternalLibrary has a method process which calls the third-party API in a synchronous way: it takes the input from reconcile_core (i.e., LibInputType) and returns an output (Option<LibOutputType>).

During every reconcile loop, the reconciler will return an Option<Request> (instead of the original Option<KubeAPIRequest>) to the shim_layer. Request is an enum with two variants for the two types of effects issued by the controller: invoking Kubernetes client APIs or third-party APIs. The shim layer will do pattern matching to the Request accordingly, which will either invoke the Kubernetes client API as before, or invoke the third-party APIs by calling the developer-provided process.

Correspondingly, the input to the reconciler, Response, is also an enum that encodes the two types of responses from the Kubernetes client APIs or third-party APIs.

Signed-off-by: Wenjie Ma <[email protected]>

marshtompsxd · 2023-07-18T20:51:17Z

src/reconciler/exec/io.rs

+
+verus! {
+
+pub enum Receiver<T: View> {


Could you add more comments on what should be T here? Same for Response

marshtompsxd · 2023-07-18T20:55:38Z

src/reconciler/exec/external.rs

+
+verus! {
+
+pub trait ExternalLibrary<InputType: View, OutputType: View> {


I think this is used when the controller does not need to communicate with any external systems? Could we have some doc (comments) here?

marshtompsxd · 2023-07-18T20:56:55Z

src/controller_examples/zookeeper_controller/common/mod.rs

@@ -19,4 +19,16 @@ pub enum ZookeeperReconcileStep {
    Error,
 }

+pub enum Error {


Do we use these errors anywhere?

Signed-off-by: Wenjie Ma <[email protected]>

marshtompsxd · 2023-07-19T02:21:13Z

src/controller_examples/zookeeper_controller/exec/zookeeper_lib/helper_funcs.rs

+    fn handle(&self, e: WatchedEvent) {}
+}
+
+pub fn empty_func(zk: String) -> Result<(), Error> {


I might miss sth, but what is this empty_func used for?

marshtompsxd · 2023-07-19T02:21:46Z

src/controller_examples/zookeeper_controller/exec/zookeeper_lib/lib.rs

+    }
+}
+
+// pub open spec fn


nit: let's remove this line

marshtompsxd · 2023-07-19T02:23:50Z

src/reconciler/exec/io.rs

+// The response type should be consistent with the Request type.
+// T: the output type of the third-party library of the reconciler which should be defined by the developer.
+// A safe and common way is to have an enum type which has the corresponding types of the input type in the Request.
+// Anway, the process method in the ExternalLibrary, the input type in Request, output type in Response and the handling


What should it be? I mean, although I introduce concretely what T is for, it is up to the developers to define it together with the Lib, reconcile_core process.

oh I was just saying there is a typo

marshtompsxd · 2023-07-19T02:31:32Z

src/reconciler/exec/external.rs

+// Similarly, the OutputType composes the Response<?> type of a reconciler.
+// Note that we can encapsulate all the required libraries here, so each reconciler only has one ExternalLibrary type.
+pub trait ExternalLibrary<InputType: View, OutputType: View> {
+    fn process(input: InputType) -> Option<OutputType>;


Sometimes the third party APIs are async functions. Since process is not async, we probably should check whether Verus supports async block (in external functions) so that we can call async functions in process

What does that mean? I think we have already used some asyn blocks.

You mean the one in run_controller

let reconcile = |cr: Arc<K>, ctx: Arc<Data>| async move { return reconcile_with::<K, ResourceWrapperType, ReconcilerType, ReconcileStateType>( &ReconcilerType::default(), cr, ctx ).await; };

?

It is an async closure but not an async block, tho they are very similar

marshtompsxd · 2023-07-19T02:32:59Z

src/kubernetes_cluster/mod.rs

@@ -1,4 +1,4 @@
 // Copyright 2022 VMware, Inc.
 // SPDX-License-Identifier: MIT
-pub mod proof;
+// pub mod proof;


Do you want to fix the proof lemmas in this PR or later?

Signed-off-by: Wenjie Ma <[email protected]>

marshtompsxd · 2023-07-19T14:54:27Z

src/reconciler/exec/io.rs

+}
+
+pub open spec fn opt_request_to_view<T: View>(receiver: &Option<Request<T>>) -> Option<RequestView<T::V>> {
+    match receiver {


Can you change receiver to request here?

marshtompsxd · 2023-07-19T15:05:47Z

@euclidgame I also revised the description a bit. Please take a look to ensure it is still correct.

Signed-off-by: Wenjie Ma <[email protected]>

euclidgame added 4 commits July 17, 2023 23:49

Add request and response concerning third-party libraries

957f2de

Signed-off-by: Wenjie Ma <[email protected]>

Refactor the reconciler module

230f51c

Signed-off-by: Wenjie Ma <[email protected]>

Add an EmptyLib

01acf2e

Signed-off-by: Wenjie Ma <[email protected]>

Add reconcile_zk_node helper_function

e47edb0

Signed-off-by: Wenjie Ma <[email protected]>

marshtompsxd self-requested a review July 18, 2023 20:45

marshtompsxd reviewed Jul 18, 2023

View reviewed changes

euclidgame added 5 commits July 18, 2023 16:06

Rename the Receiver and Response

a57c2f7

Signed-off-by: Wenjie Ma <[email protected]>

Add comments for Request, Response and ExternalLibrary

2386335

Signed-off-by: Wenjie Ma <[email protected]>

Add comments about empty library

26f62c6

Signed-off-by: Wenjie Ma <[email protected]>

Make the simple_controller module pass

9be557d

Signed-off-by: Wenjie Ma <[email protected]>

Make the rabbitmq_controller pass

16cb6f8

Signed-off-by: Wenjie Ma <[email protected]>

euclidgame marked this pull request as ready for review July 18, 2023 22:13

euclidgame requested a review from marshtompsxd July 18, 2023 22:13

marshtompsxd reviewed Jul 19, 2023

View reviewed changes

euclidgame added 2 commits July 18, 2023 23:52

Remove useless code

5a9fba5

Signed-off-by: Wenjie Ma <[email protected]>

Fix typos in the comments

c2a8d67

Signed-off-by: Wenjie Ma <[email protected]>

euclidgame requested a review from marshtompsxd July 19, 2023 05:21

marshtompsxd reviewed Jul 19, 2023

View reviewed changes

marshtompsxd changed the title ~~Support third party library~~ Support calling unverified third party APIs when building controllers Jul 19, 2023

marshtompsxd changed the title ~~Support calling unverified third party APIs when building controllers~~ Support unverified third party APIs in controllers Jul 19, 2023

Change receiver to request in several places

a358dda

Signed-off-by: Wenjie Ma <[email protected]>

euclidgame requested a review from marshtompsxd July 19, 2023 15:30

marshtompsxd merged commit c6a13d5 into main Jul 19, 2023
1 check passed

marshtompsxd deleted the support_third_party_lib branch August 1, 2023 21:53

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Support unverified third party APIs in controllers #180

Support unverified third party APIs in controllers #180

euclidgame commented Jul 18, 2023 •

edited

Loading

marshtompsxd Jul 18, 2023

marshtompsxd Jul 18, 2023

marshtompsxd Jul 18, 2023

marshtompsxd Jul 19, 2023

marshtompsxd Jul 19, 2023

marshtompsxd Jul 19, 2023

euclidgame Jul 19, 2023

marshtompsxd Jul 19, 2023

marshtompsxd Jul 19, 2023

euclidgame Jul 19, 2023

marshtompsxd Jul 19, 2023

marshtompsxd Jul 19, 2023

marshtompsxd Jul 19, 2023

euclidgame Jul 19, 2023

marshtompsxd Jul 19, 2023

marshtompsxd commented Jul 19, 2023


		verus! {

		pub trait ExternalLibrary<InputType: View, OutputType: View> {

Support unverified third party APIs in controllers #180

Support unverified third party APIs in controllers #180

Conversation

euclidgame commented Jul 18, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

marshtompsxd commented Jul 19, 2023

euclidgame commented Jul 18, 2023 •

edited

Loading