Using Philote for SSBJ Case Study

[pymusa-09]

I am learning Philote for the remote execution of MDO frameworks, where the work packages from different users can be written in different frameworks or different structures, or sometimes computational software.
So, starting from learning the basics of Philote for the SSBJ Case study, it has four modules (Aerodynamics, Structure, Propulsion and Performance). Let us assume I can access two modules in my server (Aerodynamics and Structure), and the client has Propulsion and Performance Modules. To run the MDA loop, the client needs access to my tools.
So, I created the server ports and made my work modules available to gRPC. I created server and client codes on the same local machine for a simple case study.

server.py

from concurrent import futures
import grpc
import philote_mdo.general as pmdo
from ssbj_disciplines.aerodynamics import Aerodynamics # Import aerodynamic module
from ssbj_disciplines.structure import Structure # Import structure module
import numpy as np

Define scalers for the disciplines

scalers = {
'z': np.array([0.05, 45000., 1.6, 5.5, 55.0, 1000.0]),
'WT': 49909.58578,
'ESF': 1.0,
'Theta': 0.950978,
'D': 12193.7018,
'fin': 4.093062,
'dpdx': 1.0,
}

Aerodynamics Server

def start_aero_server():
server = grpc.server(futures.ThreadPoolExecutor(max_workers=10))
discipline = pmdo.ExplicitServer(discipline=Aerodynamics(scalers)) # Pass scalers
discipline.attach_to_server(server)
server.add_insecure_port("0.0.0.0:60051") # Aerodynamics server on port 50051
server.start()
print("Aerodynamics server running on port 60051")
server.wait_for_termination()

Structure Server

def start_structure_server():
server = grpc.server(futures.ThreadPoolExecutor(max_workers=10))
discipline = pmdo.ExplicitServer(discipline=Structure(scalers)) # Pass scalers
discipline.attach_to_server(server)
server.add_insecure_port("0.0.0.0:60051:60052") # Structure server on port 50052
server.start()
print("Structure server running on port 60052")
server.wait_for_termination()

if name == "main":
import multiprocessing

# Run both servers in parallel
aero_process = multiprocessing.Process(target=start_aero_server)
structure_process = multiprocessing.Process(target=start_structure_server)

aero_process.start()
structure_process.start()

aero_process.join()
structure_process.join()

client.py
from future import print_function
from six import iterkeys
import numpy as np

from openmdao.api import ExecComp, IndepVarComp, Group, Problem, NonlinearBlockGS, ScipyKrylov
from ssbj_disciplines.performance import Performance
from ssbj_disciplines.propulsion import Propulsion
from philote_client_wrapper import PhiloteClientWrapper

class SSBJ_MDA(Group):
"""
SSBJ Analysis with aerodynamics, performance, propulsion, and structure disciplines.
"""
def init(self, scalers):
super().init()
self.scalers = scalers

def setup(self):
    # Design variables
    self.add_subsystem('z_ini',
                       IndepVarComp('z', np.array([1.0, 1.0, 1.0, 1.0, 1.0, 1.0])),
                       promotes=['*'])
    self.add_subsystem('x_aer_ini', IndepVarComp('x_aer', 1.0), promotes=['*'])
    self.add_subsystem('x_str_ini',
                       IndepVarComp('x_str', np.array([1.0, 1.0])),
                       promotes=['*'])
    self.add_subsystem('x_pro_ini', IndepVarComp('x_pro', 1.0), promotes=['*'])

    # Remote Aerodynamics and Structure (Philote)
    sap_group = Group()
    sap_group.add_subsystem(
        'Aerodynamics',
        PhiloteClientWrapper(
            host="0.0.0.0:60051",
            port=60051,
            variable_defs={
                "inputs": {"z": {"shape": (6,), "units": None}},
                "outputs": {"L": {"shape": (1,), "units": None}, "WT": {"shape": (1,), "units": None}},
            },
        ),
        promotes=["z"]
    )
    sap_group.add_subsystem(
        'Structure',
        PhiloteClientWrapper(
            host="0.0.0.0:60051",
            port=60052,
            variable_defs={
                "inputs": {"z": {"shape": (6,), "units": None}},
                "outputs": {"WF": {"shape": (1,), "units": None}},
            },
        ),
        promotes=["z"]
    )

    # Local Propulsion Module
    sap_group.add_subsystem('Propulsion', Propulsion(self.scalers), promotes=['*'])
    sap_group.nonlinear_solver = NonlinearBlockGS()
    sap_group.nonlinear_solver.options['atol'] = 1.0e-3
    sap_group.linear_solver = ScipyKrylov()
    self.add_subsystem('Mda', sap_group, promotes=['*'])

    # Map Remote Outputs to Top-Level Variables
    self.add_subsystem(
        'MapAerodynamics',
        ExecComp(
            'L = Aerodynamics_L; WT = Aerodynamics_WT',
            L=1.0,
            WT=1.0,
        ),
        promotes=["L", "WT"]
    )
    self.add_subsystem(
        'MapStructure',
        ExecComp(
            'WF = Structure_WF',
            WF=1.0,
        ),
        promotes=["WF"]
    )

    # Performance Module
    self.add_subsystem('Performance', Performance(self.scalers), promotes=['*'])

    # Constraints
    cstrs = [
        'con_theta_up = Theta*' + str(self.scalers['Theta']) + '-1.04',
        'con_theta_low = 0.96-Theta*' + str(self.scalers['Theta']),
        'con_dpdx = dpdx*' + str(self.scalers['dpdx']) + '-1.04',
        'con1_esf = ESF*' + str(self.scalers['ESF']) + '-1.5',
        'con2_esf = 0.5-ESF*' + str(self.scalers['ESF']),
        'con_temp = Temp*' + str(self.scalers['Temp']) + '-1.02',
        'con_dt=DT',
    ]
    for i in range(5):
        cstrs.append('con_sigma' + str(i + 1) + ' = sigma[' + str(i) + ']*' + str(self.scalers['sigma'][i]) + '-1.09')
    self.add_subsystem('Constraints', ExecComp(cstrs, sigma=np.zeros(5)), promotes=['*'])

def init_ssbj_mda():
"""
Initialize and run the SSBJ MDA analysis.
"""
prob = Problem()

# Define scalers for design variables and outputs
scalers = {
    'z': np.array([0.05, 45000., 1.6, 5.500, 55.00, 1000.0]),  # Initial design variables
    'x_aer': 1.0,
    'x_str': np.array([0.25, 1.0]),
    'x_pro': 0.5,
    'WT': 1.0,
    'Theta': 1.0,
    'L': 1.0,
    'WF': 1.0,
    'D': 1.0,
    'ESF': 1.0,
    'WE': 1.0,
    'fin': 1.0,
    'SFC': 1.0,
    'R': 1.0,
    'DT': 1.0,
    'Temp': 1.0,
    'dpdx': 1.0,
    'sigma': np.array([1.0, 1.0, 1.0, 1.0, 1.0])
}

prob.model = SSBJ_MDA(scalers)
prob.setup()

# Initialize and run the model
prob.run_driver()

# Update scalers with outputs
for key in iterkeys(scalers):
    if key not in ['z', 'x_str', 'x_aer', 'x_pro']:
        scalers[key] = prob[key]

return scalers

if name == "main":
scalers = init_ssbj_mda()

# Print results
print("\n--- Optimized Results ---")
print(f"Design Variables (z): {scalers['z']}")
print(f"Wing Loading (WT): {scalers['WT']}")
print(f"Range (R): {scalers['R']}")
print(f"Lift Coefficient (L): {scalers['L']}")
print(f"Weight Fraction (WF): {scalers['WF']}")




philote_client_wrapper
import numpy as np

import grpc
from openmdao.api import ExplicitComponent
from philote_mdo.general import ExplicitClient

class PhiloteClientWrapper(ExplicitComponent):
"""
OpenMDAO-compatible wrapper for Philote ExplicitClient.
"""
def init(self, host, port, variable_defs):
super().init()
self.host = host
self.port = port
self.client = ExplicitClient(channel=grpc.insecure_channel(f"{host}:{port}"))
self.variable_defs = variable_defs # Input/output definitions for the discipline

def setup(self):
    # Define inputs based on the variable definitions
    for var_name, var_props in self.variable_defs["inputs"].items():
        self.add_input(var_name, shape=var_props["shape"], units=var_props.get("units"))

    # Define outputs based on the variable definitions
    for var_name, var_props in self.variable_defs["outputs"].items():
        self.add_output(var_name, val=np.zeros(var_props["shape"]), shape=var_props["shape"], units=var_props.get("units"))

def compute(self, inputs, outputs):
    try:
        # Prepare inputs for the Philote client
        philote_inputs = {key: np.array(value) for key, value in inputs.items()}

        # Call the remote compute function
        philote_outputs = self.client.run_compute(philote_inputs)

        # Map Philote outputs to OpenMDAO outputs
        for key, value in philote_outputs.items():
            outputs[key] = value
    except Exception as e:
        raise RuntimeError(f"Failed to call Philote client compute on {self.host}:{self.port}: {e}")

so, when I am running as a Client, it gives me error StatusCode.UNIMPLEMENTED, indicating the server did not recognize the method invoked by the client.

[chrislupp]

Hi, thanks for sharing this. As a heads up, I'm currently at a conference (AIAA SciTech), so there's a lot going on. I will try to take a look at it as soon as I have time and try to help you debug the issue (or see if there is an issue with philote).

I have seen the unimplemented error before in a few different contexts. It can be a bit misleading in terms of what the root cause is.

In the mean time, could you provide the philote-python version that you are using? If you aren't using a release version, could you let me know the git hash of the commit that you are working off of? Thanks. It will help pinpoint things (and whether I may have fixed something already but not release the version yet).

[pymusa-09]

Thank you, Chris. Have a great time at SciTech. I installed the latest available version, 0.6.1, from Git. I will look into it more thoroughly from my side and will discuss it when you are available after the conference. Thank you.

[chrislupp]

Sorry for the delay. I'm having some difficulty discerning what your main goal is. My first question is, is there a reason you are implementing your own client wrapper for OpenMDAO? Philote-Python does provide an OpenMDAO implementation for explicit and implicit components. If you are trying to prototype for another framework, I suggest using those components as a starting point.

Unfortunately, it looks like the actual discipline implementations are not included in the code. Therefore, I can't actually test the code to see if there is something causing the issue within the defined Philote discipline implementation.

Depending on what you are trying to achieve, I would suggest initially creating a simpler problem and then adding complexity once parts work. If you are trying to test your custom client wrapper, I would suggest testing that against the examples provided in Philote-Python. If you are trying to debug the remote philote servers, I would recommend starting with a single discipline and debugging it until it works, before adding in additional servers.

I can try to run it if you could simplify it down to a single discipline and provide all the files needed to run. I understand that not all disciplines can be shared, but I'm ok working with toy problems if that helps. If you do that, could you do me a favor and upload the files instead of pasting them into the textbox (because it's markdown, it makes the files very hard to follow)?

[pymusa-09]

Hello Chris, Thank you for the update. The reason I tried to use a client wrapper is that I have several modules (python files) that I want to call. And I have one main run file for calling all the modules.
The example I am attaching is here;
aerodynamics.txt
common.txt
dpdxcalc.txt
performance.txt
propulsion.txt
structure.txt
ssbj_mda.txt

[pymusa-09]

Now, I assume I own two modules on my server (Aerodynamics and Structure), and the client has other modules, such as Propulsion, Performance Modules, etc. To run the MDA loop, the client needs access to my modules. The tool has one main run file, ssbj_mda, that can be used to run the whole framework.

I created a Philote Server file for my two modules, and I pushed these on gRPC and made them available for the client, and then I created a Philote Client file asking for these two modules, and I am testing it on the same machine.

client.txt
philote_client_wrapper.txt
server.txt

I tried all the examples (including the Parabloid one and followed a similar procedure, and it worked). I followed the same procedure for this SSBJ problem and my guess is when the client runs the file, it can not access all the variables and values because there are multiple files and I checked the port connections etc, and they are active and exist and therefore, I do not think any issue from that side.

Could you please look into all these files and guide me accordingly?

Thank you.

Regards,

[chrislupp]

Ok, I've simplified the problem somewhat, only using the performance discipline as a server and calling it from OpenMDAO (via the built-in client). Here is the revised (and simplified) performance discipline (performance.py):

from __future__ import print_function
import numpy as np
from openmdao.api import ExplicitComponent
import philote_mdo.general as pmdo

# pylint: disable=C0103


 def performance(Z, fin, SFC, WT, WF):
    if Z[1] <= 36089.0:
        theta = 1.0 - 6.875e-6 * Z[1]
    else:
        theta = 0.7519
    R = 661.0 * np.sqrt(theta) * Z[2] * fin / SFC * np.log(abs(WT / (WT - WF)))
    return R


 class Performance(pmdo.ExplicitDiscipline):

     def setup(self):
        # Global Design Variable z=(t/c,h,M,AR,Lambda,Sref)
         self.add_input("z", shape=(6,))
        # Local Design Variable x_per=null
        # Coupling parameters
         self.add_input("WT", shape=(1,))
         self.add_input("WF", shape=(1,))
         self.add_input("fin", shape=(1,))
         self.add_input("SFC", shape=(1,))
        # Coupling output
         self.add_output("R", shape=(1,))

    def setup_partials(self):
         pass

     def compute(self, inputs, outputs):
        # Variables scaling
         Z = inputs["z"]
         fin = inputs["fin"]
         SFC = inputs["SFC"]
         WT = inputs["WT"]
         WF = inputs["WF"]

         outputs["R"] = performance(Z, fin, SFC, WT, WF)

     def compute_partials(self, inputs, J):
         pass

The key line being:

class Performance(pmdo.ExplicitDiscipline):

as this is not an OpenMDAO discipline. I did remove the scalers to make this easier to understand. However, you can add the scalers in as options (if you need help with that, let's open another discussion/topic).

Next, run the server (server.py):

from concurrent import futures
import grpc
import philote_mdo.general as pmdo
from performance import Performance


def run():
    server = grpc.server(futures.ThreadPoolExecutor(max_workers=10))

    discipline = pmdo.ExplicitServer(discipline=Performance())
    discipline.attach_to_server(server)

    server.add_insecure_port("[::]:50051")
    server.start()
    print("Server started. Listening on port 50051.")
    server.wait_for_termination()


if __name__ == "__main__":
    run()

Finally, let's create an OpenMDAO problem and run the remote discipline from it (run.py):

import numpy as np
import grpc
import openmdao.api as om
import philote_mdo.openmdao as pmom


prob = om.Problem()
model = prob.model

model.add_subsystem(
    "Performance",
    pmom.RemoteExplicitComponent(channel=grpc.insecure_channel("localhost:50051")),
    promotes=["*"],
)

prob.setup()

prob["z"] = np.ones(6)
prob["WT"] = 2.0
prob["WF"] = 3.0
prob["fin"] = 4.0
prob["SFC"] = 1.0

prob.run_model()

print(prob["R"])

This works for me (using the Philote-Python OpenMDAO wrapper component). You can do this for multiple different servers as well (just use different ports).

I know that's a lot simpler than the example you uploaded, but I think it's easier to convey the workflow with it. Let me know if you have more questions regarding this. Also, if you need help setting up options for the discipline, feel free to start a new discussion/thread.

[chrislupp]

I left out the partials in the example to make it simpler. Obviously, if you are doing gradient-based optimization, you'd have to provide them.

[chrislupp]

@pymusa-09, does this address your issue?

[pymusa-09]

Hello Chris, Thank you for the update. I highly appreciate your time. I had this idea for a single component as it follows the principle of the Paraboloid case; my issue mainly addresses MDA/MDO Frameworks. I want to call multiple modules through mda.py such that variables are called and used through one server file, valid with the client file.

I hope I am presenting my issue, as I think the SSBJ case study is one of the good examples for the MDA/MDO study. Is it possible to create a server and client file for mda.py, rather than modules itself, or what is the way forward for such integration?

Thank you.

[chrislupp]

I'm not entirely sure I understand. Do you intend to wrap an OpenMDAO group into a single server? Either way, you shouldn't have to write a custom client unless you intend to wrap into a (currently) unsupported framework. Can you clarify what you mean by modules (Python or other)?

[pymusa-09]

Let me break it into simpler parts;
For example, I have an aircraft sizing tool in Python (not necessarily in OpenMDAO, maybe or not)...Now, this aircraft sizing tool contains many files, such as aerodynamics, propulsion, performance, and so on. I have one main run file for all the analyses. I do not want to change the code or files..rather, I want to control or change this run file, calling all other files....
So can we apply the same procedure to the run file and create a Philote server and client for it such that it can call other files from their respective ports?

Why I am asking this is because, in MDO, we usually have a large number of dependent files, and we can not create pmdo classes for each...we usually use one integration run file to call all other files...
and in remote execution, if we want to access some other tool, say thermal analysis, it may also have similarly several files and we want to access the main run file through Philote and carry out the analysis.

I hope I am clear this time. Thank you.

[chrislupp]

Why I am asking this is because, in MDO, we usually have a large number of dependent files, and we can not
create pmdo classes for each

I'm not entirely sure where this sweeping statement comes from. I will say from my experience (of practicing MDO), I view that as generally not true (maybe true for edge cases. if this has been your experience, I won't argue). OpenMDAO requires every discipline called to be a Component class. Other frameworks do the same using other mechanisms. In fact, Philote-Python's interface to implement disciplines is so similar to OpenMDAO, they could be easily confused for each other. These classes are only a software mechanism to enable integration.

Again, you should not have to create a client for the analysis code. One of the useful things (at least imho) of Philote is that the client really only needs to be implemented once per framework (not per analysis code).

Regarding the aircraft sizing tool. From your comment, to me it sounds like you want to call the entire sizing tool from Philote and run it remotely. To do this, you would wrap the run file into a Philote discipline class. I started a Philote discipline implementation that can host a full OpenMDAO group. I'm pretty sure the function eval works (haven't tested the gradients yet), but there are no unit or integration tests for that class as of today. It's also not released code; it will be in the next Philote-Python release. If you aren't using OpenMDAO, then you would have to wrap it some other way.

If you want to wrap submodules of a sizing tool into Philote servers, you are going to have to create the discipline classes somehow. The analysis servers require a discipline class (explicit or implicit) to be provided, there is no way around it.

Reading your response, I also wonder if you are referring to the deployment of the servers (and number of files needed to do that). If you see this as a deployment issue, let me know and I can weigh in on that.

It is entirely possible that I am still missing what you want to do. If so, sorry. I can take another look at code if you want to simplify it down a bit (e.g., Sellar Problem). But if you could avoid using custom clients, that would actually help me as I think it muddies the water quite a bit.

[pymusa-09]

Thank you, Chris, for the comprehensive explanation. Firstly, I mean to say, in MDAO Frameworks, if we have a large number of files, changing each class to pmdo may be tricky (and I know the Component class is always there for OpenMDAO and some other class types for codes not in OpenMDAO framework). The other reason is collaborative architectures; the codes can be written in different frameworks. My aircraft sizing tool is in the OpenMDAO framework, and my collaborator's code may be in another, so changing classes may be tricky. Again, it is doable, and I understand that.

"Regarding the aircraft sizing tool. From your comment, to me it sounds like you want to call the entire sizing tool from Philote and run it remotely."......exactly, this is what I intend to do. I am using OpenMDAO, and it would be great if I could do it changing the main file only.

I will create the discipline class for my main run file and then see how it works. I will keep you updated.

Thank you.