[RFC] About building bridges in OroPlatform

[gplanchat]

I would like to discuss about the subject of combining several OroPlatform applications into one application. After quickly discussing this in #775 with @mbessolov & @alexandr-parkhomenko and also with @24198 from Marello, I may have a different method about how bridges are built from what Oro and Marello do. I think it could be good to explain it and my experiences based on use cases and working projects. What I am proposing does not prevent from building things the way they were built until now, it brings a new way to declare interactions in the Oro Stack.

How things are working nowadays

In OroPlatform, several bricks can collaborate to build an application as a whole. To keep things simple, let's use a concrete example I voluntarily restrict : OroCommerce and Marello.

On both sides we have a bundle responsible for product data : OroProductBundle and MarelloProductBundle. None of both inherit of the other (as would do an EE bundle with its CE siamese twin), there is no dependency between the two bundles.

To make those bundles speak to each other, common usage from Oro is to build another bundle, a bridge bundle, knowing how to speak to both sides. It exists in OroCRM and OroCommerce (eg. OroCalendarCommerceBridgeBundle and OroCalendarCRMBridgeBundle to connect both sides to OroCalendarBundle).

Existing use cases (OroCRM and OroCommerce) does not make usage of soft dependencies, everything is bundled and mandatory.

Where I want to drive my argumentation is about what Marello (and other apps) is going to become: OroPlatform applications that can live by itself, which can also be integrated into a larger application (DiamanteDesk has not been integrated this way, Akeneo is not an OroPlatform application anymore and Timelap is an OroCRM plugin and can't live by itself).

Where soft dependencies and bridge bundles can help

Let's say we have OroCommerce and Marello installed in a common application (an use case we are having right now). We would have the composer.json requiring oro/crm, oro/commerce and marellocommerce/marello, nothing else should be required. All three applications would require oro/platform and related dependencies they require independently (at the same version).

If we reduce our concerns to customers, here are the bundles in action:

• OroAccountBundle (from oro/crm)
• MarelloSalesBundle (from marellocomerce/marello)

We would like to make available accounts from CRM in Marello's sales order workflow and products, this is the logical union between both tools. Logical move would be to build this bridge bundle:

• MarelloOroCustomerBridgeBundle

Where oro/platform's PR #775 takes action is in the declaration of the bridge bundle yaml file, in Resouces/config/oro/bundles.yml:

bundles:
  - name: Marello\Bridge\OroCustomerBridge\MarelloOroCustomerBridgeBundle
    dependencies:
     - Marello\Bundle\CustomerBundle\MarelloCustomerBundle
     - Oro\Bundle\CustomerBundle\OroCustomerBundle

If PR #775 is merged, the whole source code (including bridges) could be stored in the same repository the app is (eg. for Marello in src/Marello/Bridge/) and the bridge bundles would not be loaded in the kernel unless OroCRM is present in the application.

What can do a bridge, and actually not fully documented (where the things happens)

A bridge can do lots of things:

• add DB relations between tables of 2 autonomous applications
• add additional form fields (using form extensions)
• add datagrid columns
• add steps to the workflows
• loads of other things I can't think of for today...

We have made some concrete usages of this for internal projects, goals are different from e-commerce, but still matches with what a BAP is.

3 independent applications were linked together:

• a Docker UI
• a project management application
• OroCRM

Bridges are:

• Docker UI -> Project Management (present in Docker UI repository)
• Project Management -> OroCalendarBundle (present in Project Management repository)
• Project Management -> OroAccountBundle (present in Project Management repository)
• Project Management -> OroTaskBundle (present in Project Management repository)

Below is an example of what can be done:

When the Docker UI is used alone, the "Project" field is not present, the deployment runs and is logged in the Docker UI bundle, the Docker containers are built by docker-compose up.

When both the Docker UI and the Project Management app are present in the same OroPlatform app, the field appears and DB relations are created, both handled by the bridge bundle.

Same concepts applies to the relation between the Project Management app and OroCRM. This let us build the applications we need, just like a lego brick game.

I hope my explanation is clear, see it as an argument to promote PR #775 feature

[mbessolov]

@gplanchat
OroPlatform is not a code dependency management/deployment tool - that is the responsibility of composer or some other deployment tool. Why not create a composer plugin to manage soft-dependencies between bundles, components and any plain non-Symfony libraries instead of trying to convert OroPlatform into code dependency manager?

Same applies to #775

[gplanchat]

If my message can be interpreted like I would promote OroPlatform as a dependency manager such as Composer, it is a mistake. Composer does this very well and there is no intent to replace it, I have no intention to reinvent the wheel (or integrate Composer into OroPlatform).

Let's see 3 use cases:

Using several repositories/composer packages

The first use case is using 2 composer packages, here a kiboko/foo-bap and a kiboko/foo-bap-crm-bridge. It is what you chose for integrating CRM and Commerce:

vendor/kiboko/foo-bap
  src/
    Kiboko/
      Bundle/
        ProjectBundle/
          Resources/config/oro/bundles.yml
          KibokoProjectBundle.php
vendor/kiboko/foo-bap-crm-bridge
  src/
    Kiboko/
      Bridge/
        OroAccountBridge/
          Resources/config/oro/bundles.yml
          KibokoOroAccountBridgeBundle.php

This is how it is done in every Oro application I know. If PR #775 is merged, this use case will continue to work the same way it does today.

Pros

• Composer is used to require each part (either bundle and bridge), dependencies and version constraints are properly handled
• bridge versions can be decoupled from the application version

Cons

• 2 repositories have to be maintained independently, it can be cumbersome in some cases
• if a bundle excludes the KibokoProjectBundle in its Resources/config/oro/bundles.yml file, the bridge will not automatically be excluded and produce unpredictable results

Using one repository/composer package (without PR #775)

This second use case handled by OroPlatform may not be optimal (and couldn't justify itself). In my example, everything would be in the kiboko/foo-bap component (bundle and bridge):

vendor/kiboko/foo-bap
  src/
    Kiboko/
      Bridge/
        OroAccountBridge/
          Resources/config/oro/bundles.yml
          KibokoOroAccountBridgeBundle.php
      Bundle/
        ProjectBundle/
          Resources/config/oro/bundles.yml
          KibokoProjectBundle.php

Pros

• Composer is used to require everything, dependencies and version constraints are properly handled
• an unique repository to be maintain

Cons

• bridge versions can't be decoupled from the application version
• bad part: the bridge is loaded in the kernel, the oro/crm must be required in the composer.json file
• if a bundle excludes the KibokoProjectBundle in its Resources/config/oro/bundles.yml file, the bridge will not automatically be excluded and produce unpredictable results

Using one repository/composer package (with PR #775)

This third use case not handled by OroPlatform without PR #775 brings a new way to write BAP components. In my example, everything would be in the kiboko/foo-bap component (bundle and bridge), it is the same code as in the previous use case:

vendor/kiboko/foo-bap
  src/
    Kiboko/
      Bridge/
        OroAccountBridge/
          Resources/config/oro/bundles.yml
          KibokoOroAccountBridgeBundle.php
      Bundle/
        ProjectBundle/
          Resources/config/oro/bundles.yml
          KibokoProjectBundle.php

Pros

• Composer is used to require everything, dependencies and version constraints are properly handled
• an unique repository to be maintain
• good part: the bridge is not loaded in the kernel if the oro/crm is not present
• if a bundle excludes the KibokoProjectBundle in its Resources/config/oro/bundles.yml file, the bridge will be excluded and no unpredictable consequences can happen

Cons

• bridge versions can't be decoupled from the application version

[mbessolov]

OroPlatform automatically adds and initializes all bundles that have been added to the code base of the application - it is done purely for developer convenience (to avoid manually editing AppKernel). It also comes with an option (exclusions) to prevent this behavior, if for some reason a developer doesn't want it to happen.

In new Symfony versions manual editing of AppKernel will no longer be required. I see no reason to keep the bundle autoloading that we already have, and even less so to add more dependency management capabilities to what we will eventually remove.

[gplanchat]

Yes Symfony 4 does this, but when would OroPlatform be upgraded to this version?

I am not requiring a complex dependency management to be developed, an implementation is provided for developer convenience. Scope is reduced, and it prevents from broken use cases that can happen today when using exclusions option.

[mbessolov]

'exclusions' has never been meant to be used for dependency management, building bridges or any other complex use cases. It is merely a workaround to disable a few bundles if a developer of the final application believes it is safe to do so. Nobody expected, that this option will be used in bundles to disable some other bundles.

We are going to work on upgrading OroPlatform to Symfony 3.4 LTS, which already includes and recommends to use flex for installing bundles (and will automatically enable them) - we'll see how it works and if any duplicate functions of OroPlatform can be retired.

[gplanchat]

"You can't guess how your software will be (mis-)used", they say ;)

Migrating to Sf 3.4 or 4+ with Flex is the logical move and a lot of people are hoping it, I think. I do not mean to replace Flex. The migration will take some time and will not happen until a major version is released, which is less than 3 months from now for 3.0 or during 2019 for 4.0, unless BC breaks can be tolerated.

For what I can see for now, there is a lot of work to do from form types to deprecation messages to be able to migrate to Sf 3.4. To me it seems unlikely to be done with such a reduced amount of time (unless you've started working on another repo I didn't see).

I'm suggesting a change for the next minor (2.6). If you tell me we will use Sf 3.4 in less than 3 months, let's close this PR and this thread now and I'll prepare for Sf 3.4 and Flex use right away.

[gplanchat]

Hello @mbessolov,

Now it is official Oro 3.x will be on Symfony 3.4, is there some insights about the usage of Flex in future versions ?

[mbessolov]

We'll work on Symfony Flex for 3.1 release.

[gplanchat]

Great, that's good news

Thanks for the info