Run Mendix in Cloud Foundry

This document contains general information on the Mendix Cloud Foundry Buildpack.

The buildpack is heavily tied to the Mendix Public Cloud, but can be used independently. Release notes are available for the buildpack, Mendix itself and the Mendix Public Cloud.

Requirements
Supported Mendix Versions
Buildpack Releases and Version Pinning
Lifecycle
How to Deploy
Infrastructure Configuration
- Connect a Database
- Connect an External File Store
Mendix Runtime Configuration
Java Configuration
Built-In Proxy Configuration
Telemetry Configuration
- New Relic
- Splunk
- AppDynamics
- Datadog
- Dynatrace
- Logging
Using the Buildpack without an Internet Connection
Troubleshooting
- Enabling the Mendix Debugger
- Rescue Mode
Developing and Contributing
License

This ToC was generated by the VS Code Markdown All-In-One Extension.

Requirements

The buildpack requires a cluster which supports the cflinuxfs4 stack.

Additionally, we recommend a base level knowledge of Cloud Foundry. You should at least be familiar with the Cloud Foundry CLI.

Supported Mendix Versions

The latest buildpack release supports all officially-supported Mendix versions (7, 8 and 9).

The following table shows which specific buildpack release introduced or removed support for specific Mendix versions.

Mendix Major Version	Supported	End-of-Support
`10`	`v4.24.0`	-
`9`	`v4.24.0`	-
`8`	`v3.4.0`	-
`7`	`v3.1.0`	-
`6`	`v1.0`	`v4.16.0`

We recommend using a maintained (LTS / MTS) Mendix version. Additionally, we recommend to always use / "pin to" a specific buildpack release.

Supported CFLinuxFS Versions

Buildpack	Supported	End-of-Support
`4.x`	`cflinuxfs3`	2023 Q3
`5.x`	`cflinuxfs4`	-

We recommend switching to version 5.x with cflinuxfs4 since cflinuxfs3 is no longer maintained.

Buildpack Releases and Version Pinning

We recommend "pinning to" - using a specific release of - the buildpack. This will prevent you from being affected by bugs that are inadvertently introduced, but you will need to set up a procedure to regularly move to new versions of the buildpack.

To push with a specific release of the buildpack, replace <RELEASE> in the buildpack URL below in your cf push command with the release you want to pin to, e.g. v4.11.0 :

cf push <YOUR_APP> -b https://github.com/mendix/cf-mendix-buildpack/releases/download/<RELEASE>/cf-mendix-buildpack.zip -p <YOUR_MDA>.mda -t 180

You can find the list of available releases here.

⚠️ Do not directly pin to the buildpack source code, but always pin to a specific release. The release is a runnable version of the buildpack and always contains the (binary) dependencies needed to run the buildpack directly; we do not guarantee that the source code by itself runs.

Lifecycle

The buildpack lifecycle has two main phases:

stage : Fetch the JRE, Mendix Runtime, and nginx and bundle these together with the application model into a Cloud Foundry droplet. This is handled by stage.py .
run : Start the various processes and run the application. start.py is for orchestration, the JVM is for executing the Mendix Model, and nginx is used as reverse proxy including handling access restrictions.

The staging phase accepts archives in .mda format (Mendix Deployment Archive). There is experimental support for .mpk archives (Mendix Project Package). If an .mpk file is pushed, mxbuild is executed using Mono in the compile phase as well, the run phase stays the same.

Pushing .mpk files is only supported on cflinuxfs4 for Mendix version 9.24 or higher. To push MPKs for older Mendix versions, you should stay on cflinuxfs3.

How to Deploy

There are specific guides for deploying Mendix apps to the Pivotal and IBM Bluemix Cloud Foundry platforms on our documentation page. This buildpack readme documents the more low-level details and CLI instructions.

1. Install Cloud Foundry CLI

Install the Cloud Foundry command line executable. You can find this on the releases page. Set up the connection to your preferred Cloud Foundry environment with cf login and cf target .

2. Push your App

The following commands push an MDA (Mendix Deployment Archive) that was built by Mendix Studio Pro to Cloud Foundry.

⚠️ Please replace <LINK-TO-BUILDPACK> in the commands below with a link to the version of the buildpack you are trying to deploy. Check this section for details on how to pick a release.

cf push <YOUR_APP> -b <LINK-TO-BUILDPACK> -p <YOUR_MDA>.mda -t 180

Pushing a project directory is also possible. This will move the build process (using MxBuild, a component of Mendix Studio Pro) to Cloud Foundry:

cd <PROJECT DIR>; cf push -b <LINK-TO-BUILDPACK> -t 180

Note that you might need to increase the startup timeout to prevent the database from being partially synchronized. This can be done either by specifying the -t 180 parameter like above, or by using the CF_STARTUP_TIMEOUT environment variable (in minutes) from the command line.

Also note that building the project in Cloud Foundry takes more time and requires enough memory in the compile step.

3. Configure an Admin Password

The first push generates a new app. In order to login to your application as admin, you can set the password using the ADMIN_PASSWORD environment variable. Keep in mind that the admin password should comply with the policy you have set in Mendix Studio Pro. For security reasons, it is recommended to set this environment variable once to create the admin user, then remove the environment variable and restart the app. Finally, log in to the app and change the password via the web interface. Similarly, the setting can be used to reset the password of an administrator.

cf set-env <YOUR_APP> ADMIN_PASSWORD "<YOURSECRETPASSWORD>"

4. Connect a Database

After configuring an admin password, proceed with connecting a database.

Infrastructure Configuration

Mendix applications can use a variety of databases and external file stores. The buildpack can configure the Mendix Runtime to use these databases and file stores.

⚠️ Support for file stores and databases in the buildpack is ultimately dependent on Mendix Runtime support. The buildpack only plays a role in configuring them in the Mendix Runtime.

Connect a Database

To deploy an app, you need to connect a PostgreSQL, MySQL or any other Mendix Runtime supported database instance which allows at least 5 connections to the database. Find out which services are available in your Cloud Foundry foundation with the marketplace command.

cf marketplace

Example: a Cloud Foundry Marketplace offers the elephantsql service, which offers the free turtle plan. All you need to do is give it a name and bind it to your application.

cf create-service elephantsql turtle <SERVICE_NAME>
cf bind-service <YOUR_APP> <SERVICE_NAME>

Note that not all databases are automatically picked up by the buildpack. If cf push returns an error like Could not parse database credentials , you need to set the DATABASE_URL variable manually or set database Mendix custom runtime variables to configure a database. Note these variables need to be prefixed with MXRUNTIME_ , as per example:

cf set-env <YOUR_APP> MXRUNTIME_DatabaseType PostgreSQL
cf set-env <YOUR_APP> MXRUNTIME_DatabaseJdbcUrl jdbc:postgresql://host/databasename
cf set-env <YOUR_APP> MXRUNTIME_DatabaseName databasename
cf set-env <YOUR_APP> MXRUNTIME_DatabaseUserName user
cf set-env <YOUR_APP> MXRUNTIME_DatabasePassword password

Now, push the application once more.

cf push <YOUR_APP> -b <LINK-TO-BUILDPACK> -p <YOUR_MDA>.mda

You can now log in to your application with the configured admin password.

For PostgreSQL, the buildpack supports setting additional parameters in the connection URI retrieved from the service binding. To set additional JDBC parameters, set the DATABASE_CONNECTION_PARAMS environment variable as JSON key-value string.

cf set-env <YOUR_APP> DATABASE_CONNECTION_PARAMS '{"tcpKeepAlive": "true", "connectionTimeout": 30, "loginTimeout": 15}'

⚠️ If you set DATABASE_URL as JDBC connection string (prefixed with jdbc: and including parameters, DATABASE_CONNECTION_PARAMS is not required.

To allow connection to an AWS RDS database the buildpack selects the regional CA certificate stored in rds-certificates. If the region's certificate doesn't exist, the buildpack will fail with an error Could not find database CA certificate in map.

⚠️ After the root CA rotation of AWS RDS on 22nd August 2024, only buildpacks v5.0.5 or higher will continue to work, all older buildpacks only import no longer valid certificates and no longer can establish a connection to AWS RDS.

Supported VCAP Schemas

Cloud Foundry database services are detected from Cloud Foundry service bindings (VCAP) and translated into Mendix Runtime configuration. In case no database service is bound, the fallback is the environment variable DATABASE_URL.

All database configuration code can be found in database.py. Service bindings have preference over DATABASE_URL. Service bindings are recognized on the identifier and/or tags.

PostgreSQL using RDS Service Broker

Selection based on tags ["database", "RDS", "postgresql"].

  "rds": [
   {
    "binding_name": null,
    "credentials": {
     "db_name": "databasename",
     "host": "hostname.local",
     "password": "password",
     "uri": "postgres://username:[email protected]:5432/databasename",
     "username": "username"
    },
    "instance_name": "03E2080E-BA38-4AD4-B3AC-5F2D7ED2483B-database",
    "label": "rds",
    "name": "03E2080E-BA38-4AD4-B3AC-5F2D7ED2483B-database",
    "plan": "rds-plan",
    "provider": null,
    "syslog_drain_url": null,
    "tags": [
     "database",
     "RDS",
     "postgresql"
    ],
    "volume_mounts": []
   }
  ]

SAP Hana using Service Broker

Selection based on name hana and tags ["hana", "database", "relational"].

  "hana": [
   {
    "binding_name": null,
    "credentials": {
     "certificate": "-----BEGIN CERTIFICATE-----\n<<certficate>>\n-----END CERTIFICATE-----\n",
     "driver": "com.sap.db.jdbc.Driver",
     "host": "hostname",
     "password": "password",
     "port": "21863",
     "schema": "USR_username",
     "url": "jdbc:sap://hostname:21863?encrypt=true\u0026validateCertificate=true\u0026currentschema=USR_username",
     "user": "USR_username"
    },
    "instance_name": "Hana Schema",
    "label": "hana",
    "name": "Hana Schema",
    "plan": "schema",
    "provider": null,
    "syslog_drain_url": null,
    "tags": [
     "hana",
     "database",
     "relational"
    ],
    "volume_mounts": []
   }
  ],

Connect an External File Store

The Mendix Runtime supports multiple external file stores: AWS S3, Azure Storage and Swift (Bluemix Object Storage).

All of these can be configured manually via Custom Runtime Settings, but the buildpack provides ways to more easily configure AWS S3, Azure Storage and Swift (Bluemix Object Storage).

Swift (Bluemix Object Storage) Settings

When deploying to Bluemix, you can create an Object Storage service and attach it to your app. No further configuration in necessary, you just need to restart your app.

The buildpack will set up the custom runtime settings for the Object Storage service based on the service binding. By default, a storage container will be created for you called mendix . If you want to use a different container name (for example if you are sharing the Object Storage service between multiple apps), you can configure the container name with the environment variable SWIFT_CONTAINER_NAME .

Azure Storage Service Settings

When deploying Mendix to Cloud Foundry on Azure with the Azure Service Broker, you can create an Azure Storage Service instance and attach it to your app. No further configuration in necessary, you just need to restart your app.

The buildpack will set up the custom runtime settings for the Azure Storage Service based on the service binding. By default, a storage container will be created for you called mendix . If you want to use a different container name (for example if you are sharing the Azure Storage service between multiple apps), you can configure the container name with the environment variable AZURE_CONTAINER_NAME .

S3 Settings

The buildpack can configure AWS S3 file stores in the Mendix Runtime in two ways:

Using IAM Credentials.
Using a Token Vending Machine

The buildpack will set up the custom runtime settings for AWS S3 based on environment variables or a service binding.

⚠️ Support for other file stores that offer the S3 API (S3 compatible file stores) is dependent on the Mendix Runtime and the level of compatibility offered by these non-AWS file stores.

Use IAM Credentials

Create an IAM user and provide IAM user credential using following environment variables.

S3_ACCESS_KEY_ID : credentials access key
S3_SECRET_ACCESS_KEY : credentials secret
S3_BUCKET_NAME : bucket name

Implement TVM (Token Vending Machine)

Create a TVM (Token Vending Machine) and provide TVM credential using following environment variable.

S3_TVM_ENDPOINT : tvm_endpoint
S3_TVM_USERNAME : tvm_username
S3_TVM_PASSWORD : tvm_password

Please check s3-tvm-spec for API documentation help for TVM.

The following environment variables are optional:

S3_PERFORM_DELETES : set to false to never delete items from the file store. This is useful when you use a highly redundant service without a separate backup mechanism, such as AWS S3.
S3_KEY_SUFFIX : if your bucket is multi-tenant you can append a string after each object name, you can restrict IAM users to objects with this suffix.
S3_ENDPOINT : for S3 itself this is not needed, for S3 compatible object stores set the domain on which the object store is available.
S3_USE_V2_AUTH : use Signature Version 2 Signing Process, this is useful for connecting to S3 compatible object stores like Riak-CS, or Ceph.
S3_USE_SSE : if set to true this will enable Server Side Encryption in S3

Mendix Runtime Configuration

The buildpack allows setting various Mendix Runtime configuration options.

Horizontal Scaling

Mendix allows scaling out horizontally. The absence of the need for a state store results in the fact that nothing needs to be configured for running Mendix in clustering mode. Based on the CF_INSTANCE_INDEX variable, the runtime starts either in leader or worker mode. The leader mode will do the database synchronization activities (when necessary), while the workers will automatically wait until that is finished.

When you make changes to your domain model, the Mendix Runtime will need to synchronize data model changes with the database on startup. This will only happen on instance 0 . The other instances will wait until the database is fully synchronized. This is determined via the CF_INSTANCE_INDEX environment variable. This is a built-in variable in Cloud Foundry, you do not need to set it yourself. If the environment variable is not present (this is the case in e.g. older Cloud Foundry versions) every instance will attempt to synchronize the database. A warning containing the text CF_INSTANCE_INDEX environment variable not found will be printed in the log.

For Mendix < 9.12, scheduled events will also only be executed on instance 0. See the section Configuring Scheduled Events.

In all horizontal scaling scenarios, extra care needs to be taken when programming Java actions. Examples of things to be avoided are:

Relying on singleton variables to keep global application state
Relying on scheduled events to make changes in memory. Scheduled events will only run on the primary instance for Mendix < 9.12.

Constants

The default values for constants will be used as defined in your project. However, you can override them with environment variables. You need to replace the dot with an underscore and prefix it with MX_ . So a constant like Module.Constant with value ABC123 could be set like this:

cf set-env <YOUR_APP> MX_Module_Constant "ABC123"

After changing environment variables, you need to restart your app. A full push is not necessary.

cf restart <YOUR_APP>

Scheduled Events

The scheduled events can be configured using environment variable SCHEDULED_EVENTS .

Possible values are ALL , NONE, or a comma separated list of the scheduled events that you would like to enable. For example: ModuleA. ScheduledEvent, ModuleB. OtherScheduledEvent

For Mendix versions < 9.12, when scaling to multiple instances, the scheduled events that are enabled via the settings above will only be executed on instance 0 . The other instances will not execute scheduled events at all.

Custom Runtime Settings

To configure any of the advanced Custom Runtime Settings, you can use setting name prefixed with MXRUNTIME_ as an environment variable.

For example, to configure the ConnectionPoolingMinIdle setting to value 10 , you can set the following environment variable:

cf set-env <YOUR_APP> MXRUNTIME_ConnectionPoolingMinIdle 10

If the setting contains a dot . you can use an underscore _ in the environment variable. So to set com.mendix.storage.s3.EndPoint to foo you can use:

cf set-env <YOUR_APP> MXRUNTIME_com_mendix_storage_s3_EndPoint foo

Client Certificates

To add client certificates to the Mendix runtime configuration, use the CLIENT_CERTIFICATES environment variable.

Example:

[
    {"pfx": "AaBbCc==", "password": "password1", "pin_to": ["Module.WS1", "/bla/bla"]},
    {"pfx": "DdEeFf==", "password": ""}
]

The buildpack ensures that the environment variable is converted into custom runtime settings, and client certificate support is dependent on runtime support. Please consult the runtime documentation on client certificates for the runtime version of your application for further support.

Client Certificate Format

The environment variable is in JSON format and is a list ( [] ) of client certificate objects. Each object contains the following fields:

Field	Type	Required	Example	Description
`pfx`	`string(base64)`	Yes	`"AaBbCc=="`	Certificate in PKCS#12 format, encoded in `base64`
`password`	`string`	Yes	`"password1"`	Certificate password. Can be blank.
`pin_to`	`[string]`	No	`["Module.WS1", "/bla/bla"]`	JSON list of Mendix modules or relative paths to pin the client certificate to

Data Snapshots

If you want to enable initializing your database and files from an existing data snapshot included in the MDA, set the environment variable USE_DATA_SNAPSHOT to true .

⚠️ This only works when the database is completely empty. If there are any Mendix tables defined in the database, the Mendix Runtime will refuse to overwrite it. If you have ever started an app before setting this environment variable (thereby initializing the database), you will not be able to import a data snapshot.

License Activation

To activate a license on your application you need license credentials. These credentials can be obtained by contacting Mendix Support.

cf set-env <YOUR_APP> LICENSE_ID <UUID>
cf set-env <YOUR_APP> LICENSE_KEY <LicenseKey>

An example UUID is aab8a0a1-1370-467e-918d-3a243b0ae160 and LicenseKey is a very long base64 string. The app needs to be restarted for the license to be effective.

Java Configuration

This buildpack provides the Adoptium JDK and JRE.

Java Version

The buildpack will automatically determine the Java version from metadata(it will have JavaVersion as json attribute). The default Java version is 8. For Mendix 8 and above, the default Java version is 11. Metadata JavaVersion attribute will have these values "11", "17" or in the future "21".

We do not recommend overriding the Java version for the buildpack. However, for cases where this is required, the JAVA_VERSION variable can be used to override the version number.

Example:

cf set-env <YOUR_APP> JAVA_VERSION 11

This will resolve the Java dependency using major version number 11. The JDK or JRE with this version number must be specified in dependencies.yml and either be present on the Mendix CDN or in the vendor directory (see here).

Minor versions are also supported, but only if they are specified in the dependency configuration correctly. Example:

cf set-env <YOUR_APP> JAVA_VERSION 11.0.15

This example will only work correctly if the specific file for version 11.0.15 is present.

Java Heap Size

The Java heap size is configured automatically based on best practices. You can tweak this to your needs by using another environment variable, in which case it is used directly.

cf set-env <YOUR_APP> HEAP_SIZE 512M

Java Max Metaspace Size

The Java Max Metaspace Size is configured automatically based on best practices. You can tweak this to your needs by using another environment variable, in which case it is used directly.

cf set-env <YOUR_APP> MAX_METASPACE_SIZE 512M

Java Garbage Collector

The Java garbage collector is configured automatically based on best practices. You can tweak this to your needs by using another environment variable, the accepted values are Serial or G1.

cf set-env <YOUR_APP> JVM_GARBAGE_COLLECTOR Serial

Java Virtual Machine (JVM) Settings

You can configure the Java properties by providing the JAVA_OPTS enviroment variable to the application.

Configure the JAVA_OPTS environment variable by using the cf set-env command.

cf set-env <YOUR_APP> JAVA_OPTS '["-Djava.rmi.server.hostname=127.0.0.1", "-Dcom.sun.management.jmxremote.authenticate=false", "-Dcom.sun.management.jmxremote.ssl=false", "-Dcom.sun.management.jmxremote.port=5000", "-Dcom.sun.management.jmxremote.rmi.port=5000"]'

Enabling TLSv1.0 and TLSv1.1 for Outgoing Connections

OpenJDK disabled the insecure / End-of-Life TLSv1.0 and TLSv1.1 protocols for outgoing connections by default in April 2021. Since the buildpack uses an OpenJDK distribution, these protocols are also disabled by default for the buildpack.

To re-enable TLSv1.0 and TLSv1.1 for outgoing connections, set the ENABLE_OUTGOING_TLS_10_11 environment variable to true, and restage your application.

⚠️ Enabling TLSv1.0 and TLSv1.1 for outgoing connections is at your own risk, and this functionality may disappear at any time.

Certificate Authorities

To import Certificate Authorities (CAs) into the Java trust store, use the CERTIFICATE_AUTHORITIES environment variable.

The contents of this variable should be a concatenated string containing a the additional CAs in PEM format that are trusted.

Example:

-----BEGIN CERTIFICATE-----
AaBbCc==
-----END CERTIFICATE-----
-----BEGIN CERTIFICATE-----
DdEeFf==
-----END CERTIFICATE-----

Note that if a certificate is signed by an intermediary, the complete certificate chain has to be added.

Built-In Proxy Configuration

The buildpack includes an nginx proxy which can add custom HTTP headers. It also allows to set access restrictions.

HTTP Headers

HTTP headers allow the client and the server to pass additional information with the request or the response which defines the operating parameters of an HTTP transaction. Few of the response headers can be configured via HTTP_RESPONSE_HEADERS environment variable and setting a JSON string value to configure multiple supported headers. See Environment Details - Developer Portal Guide | Mendix Documentation Section 4.2 for all supported headers and options.

For example, to configure X-Frame-Options , you can set HTTP_RESPONSE_HEADERS environment variable like below:

cf set-env <YOUR_APP> HTTP_RESPONSE_HEADERS '{"X-Frame-Options": "allow-from https://mendix.com"}'

to configure multiple supported headers, you can set it like below:

cf set-env <YOUR_APP> HTTP_RESPONSE_HEADERS '{"Referrer-Policy": "no-referrer-when-downgrade", "X-Content-Type-Options": "nosniff"}'

Enabling SameSite / Secure Cookie Header Injection for Mendix Runtime < 8.12

Google Chrome will - at a certain moment - enforce cookie security by requiring the SameSite and Secure atrributes for all cookies. Mendix runtime versions < 8.12 do not include these properties in cookies.

The buildpack can inject these two properties into all cookies for affected runtime versions.

This workaround is disabled by default. If your application supports injecting these cookies, you can choose to enable cookie header injection by setting the SAMESITE_COOKIE_PRE_MX812 environment variable to true .

Access Restrictions

The buildpack proxy has the possibility to set access restrictions for certain application paths. To do so, use the ACCESS_RESTRICTIONS environment variable.

Example:

{
    "/": {"ipfilter": ["10.0.0.0/8"], "client_cert": true, "satisfy": "any"},
    "/ws/MyWebService/": {"ipfilter": ["10.0.0.0/8"], "client_cert": true, "satisfy": "all"},
    "/CustomRequestHandler/": {"ipfilter": ["10.0.0.0/8"]},
    "/CustomRequestHandler2/": {"basic_auth": {"user1": "password", "user2": "password2"}},
}

Access Restrictions Format

The environment variable is in JSON format and is a collection ( {} ) of path restriction objects. Each object is defined by a path (relative to the application root URL). A restriction object applies to that path and all sub-paths. Inheritance can be overridden by adding a restriction object for a sub-path. These settings will then override the parent object.

Note that:

Access restrictions are not supported for reserved system paths
A path restriction object must at least contain one restriction object

A path restriction object is composed of the following fields:

Field	Type	Example	Description
`ipfilter`	`[string]`	`["10.0.0.0/8"]`	List of IPs to allow in CIDR format. An empty list will allow access from all IPs.
`client_cert`	`boolean`	`false`	Enables checking client certificates. The certificate for the exact path the restriction applies to must be correctly provided with the `CLIENT_CERTIFICATES` environment variable.
`basic_auth`	`{string: string, string: string, ...}`	`{"user1": "password", "user2": "password2"}`	Adds a HTTP Basic Authentication restriction. Multiple user / password combinations can be supplied in one JSON object.
`satisfy`	`string(any\|all)`	`"any"`	Defines how restrictions are evaluated. `any` is equivalent to logical `OR` , `all` to `AND` .
`issuer_dn`	`string`	`"CN=example.com,O=example Inc."`	Adds certificate pinning through the `"Ssl-Client-Issuer-Dn"` header. This header must be supplied through an upstream proxy.

Custom Locations

The buildpack proxy features a more general mechanism than access restrictions to configure custom nginx locations. To use this feature, set the NGINX_CUSTOM_LOCATIONS environment variable.

Example:

{
    "/custom_location_1": {"body": "internal;\nset $some_value_1 $other_value_1;"},
    "/custom_location_1/custom_location_2": {"body": "internal;\nset $some_value_2 $other_value_2;"},
}

Custom Locations Format

The environment variable is in JSON format and is a collection ( {} ) of custom location objects. Each object is defined by a path (relative to the application root URL). A custom location object applies to that path and all sub-paths. Inheritance can be overridden by adding a custom location object for a sub-path. These settings will then override the parent object.

The only allowed field in a custom location object is body. This field contains the escaped nginx configuration for that location. If there are more fields present, the custom location will be rejected.

Note that:

Access restrictions take precedence over custom locations, i.e. an access restriction with the same path as a custom location overrides that custom location
Custom locations are not supported for reserved system paths

Telemetry Configuration

The buildpack includes a variety of telemetry agents that can be configured to collect and forward metrics/logs from the Mendix Runtime.

New Relic

Set up New Relic integration

Fluent Bit is used to collect Mendix Runtime logs to New Relic.

The metrics are collected by the New Relic Java Agent and an integration with the Telegraf agent.

To enable the integration you must provide the following variables:

Environment variable	Value example	Default	Description
`NEW_RELIC_LICENSE_KEY`	`api_key`	-	License Key or API Key (docs)
`NEW_RELIC_METRICS_URI`	`https://metric-api.eu.newrelic.com/metric/v1`	-	Metrics endpoint API (docs)
`NEW_RELIC_LOGS_URI`	`https://log-api.eu.newrelic.com/log/v1`	-	Logs endpoint API (docs)
`NEW_RELIC_APP_NAME`	`MyApp`	application domain name	Optional. Mendix App name shown on New Relic
`LOGS_REDACTION`	`true`	`true`	Optional. Enables email address redaction from logs

⚠️ For the first usage of the New Relic integration, the Mendix app should be redeployed after setting the variables up.

Tags/Metadata in metrics and logs

In addition to the runtime application logs, the following JSON-formatted metadata is automatically sent to New Relic, both for the metrics collected by the agent and the custom ones, pushed by Telegraf:

environment_id - unique identifier of the environment;
instance_index - number of the application instance;
hostname - name of the application host;
application_name - default application name, retrieved from domain name;
model_version - model version of the Mendix runtime;
runtime_version - version of the Mendix runtime.

ℹ️ model_version and runtime_version are only available to the custom metrics.

Custom tags

You can also set up custom tags in the following format key:value. Below, are listed some suggested tags that you might want to use:

app:{app_name} – this enables you to identify all logs sent from your app (for example, app:customermanagement)
env:{environment_name} – this enables you to identify logs sent from a particular environment so you can separate out production logs from test logs (for example, env:accp)

Service-binding integration (on-prem only) - DEPRECATED

To enable New Relic, simply bind a New Relic service to this app and settings will be picked up automatically. Afterwards you have to restage your application to enable the New Relic agent.

This integration does not support logs or custom metrics.

⚠️ The default NEW_RELIC_APP_NAME for this integration used to be the environment ID of the application. Now the value is the domain name set to the application. If you want to keep using the environment id, you will have to set this variable yourself to that value.

Splunk

Set up Splunk integration

To collect Mendix Runtime logs to Splunk Cloud Platform, Fluent Bit is used.

To enable Splunk integration for a Mendix application, following environment variables should be configured.

⚠️ For the first usage of Splunk integration the Mendix app should be redeployed after setting the variables up.

Environment variable	Value example	Default	Description
`SPLUNK_HOST`	`test.splunkcloud.com`	-	Host of Splunk Cloud without 'http://'
`SPLUNK_PORT`	`8088`	`8088`	Port of Splunk Cloud
`SPLUNK_TOKEN`¹	`uuid token`	-	Token from Splunk Cloud dashboard
`SPLUNK_LOGS_REDACTION`	`true`	`true`	DEPRECATED - If `true` emails in log message are redacted - Use LOGS_REDACTION instead
`LOGS_REDACTION`	`true`	`true`	Enables email address redaction from logs

To create new token on Splunk Cloud dashboard go to Settings -> Data Input -> HTTP Event Collector and push button New Token in the top-right corner of the page.

Once the Mendix application is redeployed/restarted, the runtime application logs should appear on the Splunk Cloud under Search & Reporting. In the search line specify: source="http:your_token_name", click search button.

Metadata

In addition to the runtime application logs, the following JSON-formatted metadata is automatically sent to the Splunk Cloud Platform:

environment_id - unique identifier of the environment;
instance_index - number of the application instance;
hostname - name of the application host;
application_name - default application name, retrieved from domain name;
model_version - model version of the Mendix runtime;
runtime_version - version of the Mendix runtime.

You can filter the data by these fields on Splunk Cloud Platform web interface.

Custom tags

You can also set up custom tags in the following format key:value. We recommend that you add the following custom tags:

app:{app_name} – this enables you to identify all logs sent from your app (for example, app:customermanagement)
env:{environment_name} – this enables you to identify logs sent from a particular environment so you can separate out production logs from test logs (for example, env:accp)

AppDynamics

All the information collected with AppDynamics Java Agent and Machine Agent is pushed to Controller and displayed on the Controller UI.

Basic functionality

To enable AppDynamics, configure the following environment variables. These settings enable basic (Java Agent related) AppDynamics features. More information here: Environment variables.

Please note that AppDynamics requires Mendix 7.15 or higher.

Environment Variable	Value example	Default	Description
`APPDYNAMICS_CONTROLLER_PORT`	`443`	`443`	Port of AppDynamics controller
`APPDYNAMICS_CONTROLLER_SSL_ENABLED`	`true`	`true`	Set if SSL is required
`APPDYNAMICS_CONTROLLER_HOST_NAME`	`<acc_name>.test.com`	-	Name of AppDynamics host without 'http://'
`APPDYNAMICS_AGENT_APPLICATION_NAME`	`<test-accp>`	App name as on Dev. Portal	Name of the app to be displayed on Controller UI
`APPDYNAMICS_AGENT_ACCOUNT_NAME`	`<acc_name>`	-	See 'License/Account' on the Controller UI
`APPDYNAMICS_AGENT_ACCOUNT_ACCESS_KEY`	`<secret>`	-	See 'License/Account' on the Controller UI
`APPDYNAMICS_AGENT_NODE_NAME`¹	`<node>` (finally `<node>_<num>`, `<num>` is being added automatically)	node	How a node is displayed on the Controller UI
`APPDYNAMICS_AGENT_TIER_NAME`	`<env_id>`	App Environment UUID	How a tier is displayed on the Controller UI

The APPDYNAMICS_AGENT_NODE_NAME environment variable will be appended with the value of the CF_INSTANCE_ID variable. If you use node for APPDYNAMICS_AGENT_NODE_NAME , the AppDynamics agent will be configured as node-0 for instance 0 and node-1 for instance 1 , etc.

For more details about nodes and tiers: Tiers and Nodes.

Required variables: APPDYNAMICS_AGENT_ACCOUNT_NAME, APPDYNAMICS_AGENT_ACCOUNT_ACCESS_KEY and APPDYNAMICS_CONTROLLER_HOST_NAME.

If you have all the required environment variables specified above, the AppDynamics Java Agent will be configured for your application.

The other environment variables are optional and will be set to the default values. After configuring these environment variables, restart your app for the agent to be enabled. If the agent has not been installed it is necessary to redeploy the app.

Extended functionality

⚠️ This functionality is fully supported in Mendix version 9.7 and later. For older versions, a limited amount of metrics are available:

postgres. metrics,
mx.client. metrics.

The Mendix App metrics collected by the Telegraf Agent also can be pushed to AppDynamics Controller. These metrics are pushed to Machine Agent HTTP Listener.

To activate the Machine Agent provide following environment variable:

APPDYNAMICS_MACHINE_AGENT_ENABLED: true

After the variable is set up the application should be restarted. If the Machine Agent has not been installed it is necessary to redeploy the app.

Please note, that this variable is custom feature flag for the buildpack. It cannot be found in AppDynamics documentation. From the perspective of AppDynamics the Mendix App metrics are considered as a 'Custom Metrics'. So to observe these metrics you should open Metric Browser and navigate to:

Application Infrastructure Performance -> <tier_name> -> Custom Metrics -> Mx Runtime Statistics.

Datadog

The Datadog integration features a limited Datadog Agent installation included in the official Datadog Cloud Foundry Buildpack. The following information is collected:

Application metrics are collected by the Datadog IoT agent.
JMX metrics and APM traces are retrieved using the Datadog Java trace agent and collected by the Datadog agent.
PostgreSQL metrics are collected by an included Telegraf agent and sent directly to Datadog via the Datadog API.

To enable Datadog, configure the following environment variables:

Environment Variable	Description
`DD_API_KEY`	The Datadog API key. Can can be configured in the `Integrations -> API` screen of the user interface for your Datadog organization.
`DD_LOG_LEVEL`	Ensures that log messages are sent to Datadog. A safe level would be `INFO` , but it can be later adjusted to different levels: `CRITICAL` , `ERROR` , `WARNING` , or `DEBUG` .

If you're using a Datadog EU organization, you should also set the DD_SITE environment variable accordingly.

Additionally, the following integration-specific variables are available:

Environment Variable	Default Value	Description
`DATADOG_DATABASE_DISKSTORAGE_METRIC`	`true`	Enables a metric denoting the disk storage size available to the database. This metric is set in the `DATABASE_DISKSTORAGE` environment variable.
`DATADOG_DATABASE_RATE_COUNT_METRICS`	`false`	Enables additional rate / count database metrics currently not compatible with the Datadog PostgreSQL integration
`DATADOG_LOGS_REDACTION`	`true`	DEPRECATED - Enables email address redaction from logs - Use LOGS_REDACTION instead
`LOGS_REDACTION`	`true`	Enables email address redaction from logs

To receive metrics from the runtime, the Mendix Java Agent is added to the runtime as Java agent. This agent can be configured by passing a JSON in the environment variable METRICS_AGENT_CONFIG as described in Datadog for v4 Mendix Cloud.

Please note that application metric collection requires Mendix 7.14 or higher.

Presets

For correlation purposes, we set the Datadog service for you to match your application name. This name is derived in the following order:

Your Mendix service: tag if you have set this in the runtime settings or TAGS environment variable. Format: ["service:myfirstapp", "tag2:value2", ...].
Your Mendix app: tag if you have set this in the runtime settings or TAGS environment variable. Example: for app:myfirstapp , service will be set to myfirstapp.
The first part of the Cloud Foundry route URI configured for your application, without numeric characters. Example: for a route URI myfirstapp1000-test.example.com , service will be set to myfirstapp .

Additionally, we configure the following Datadog environment variables for you:

Environment Variable	Value	Can Be Overridden?	Description
`DD_HOSTNAME`	`<app>-<env>.mendixcloud.com<-instance>`	No	Human-readable host name for your application
`DD_ENV`	`<env>`	Yes	Reserved tag. Set to the value of the `env` tag. Defaults to `none` .
`DD_SERVICE`	`<app>`	Yes	Reserved tag. Set as as described before. Is only set when `DD_TRACE_ENABLED` is set to `true` .
`DD_VERSION`	`<model_version>`	Yes	Reserved tag. Set to the value of the `version` tag. Defaults to the Mendix model version of the application.
`DD_TAGS`	`tag1:value1,...:...`	Yes	Global tags for Datadog Agent(s). Derived from the runtime settings in Mendix Public Cloud or the `TAGS` environment variable.
`DD_TRACE_ENABLED`	`false`	Yes	Enables Datadog APM and the Trace Agent(s). Enabling Datadog APM is experimental and enables tracing via the Datadog Java Trace Agent tracing functionality.
`DD_PROFILING_ENABLED`	`false`	Yes	Enables Datadog APM and the Trace Agent(s). Enabling Datadog Profiling is experimental and can only be enabled for Mendix 7.23.1 and up, and requires tracing to be enabled.
`DD_JMXFETCH_ENABLED`	`true`	No	Enables Datadog Java Trace Agent JMX metrics fetching
`DD_SERVICE_MAPPING`	`<database>:<app>.db`	No	Links your database to your app in Datadog APM
`DD_LOGS_ENABLED`	`true`	No	Enables sending your application logs directly to Datadog
`DD_ENABLE_CHECKS`	`false`	Yes	Enables system metrics. These are disabled by default, as the metrics might be host metrics instead of container metrics.

Other environment variables can be set as per the Datadog Agent documentation.

Known Limitations

Telegraf does not support (Datadog) metric types correctly yet (e.g. rate, counter, gauge). This means that all database metrics are currently pushed to Datadog as a gauge.

The most important metrics ( before_xid_wraparound , connections , database_size , db.count , locks , max_connections , percent_usage_connections , table.count , deadlocks ) are gauges and are compatible with the Datadog PostgreSQL integration and associated dashboards.

If you do require the additional rate and counter metrics, there is a workaround available. First, set the DATADOG_DATABASE_RATE_COUNT_METRICS environment variable to true . After that variable is enabled, the rate and counter metrics are suffixed by either _count or _rate to prevent collisions with the official Datadog metrics. In the Datadog UI, the metric type and unit can be changed to reflect this. We also set a helpful interval tag ( 10s ) which can be used here. Additionally, gauge metrics can be rolled up in Datadog dashboards. The correct type and unit for other submitted metrics can be found here.

Dynatrace

The Dynatrace integration features Dynatrace OneAgent installation and telegraf output configuration.

Dynatrace OneAgent collects various system metrics
Custom metrics are provided via telegraf (only applicable for Mendix version 9.7 and above)

To enable Dynatrace ingestion, configure the following environment variables:

Environment Variable	Description
`DT_PAAS_TOKEN`	The token for integrating your Dynatrace environment with your Mendix app
`DT_SAAS_URL`	Monitoring endpoint url of the Dynatrace service
`DT_TENANT`	Environment id of your Dynatrace environment. (see: https://www.dynatrace.com/support/help/get-started/monitoring-environment/environment-id)
`DT_IS_MANAGED`	[Optional] Should be set to `true` for Dynatrace Managed
`DT_CLUSTER_ID`	[Optional] Can be used to tag process groups
`DT_CUSTOM_PROP`	[Optional] Can be used to provide metadata for process groups

These scopes must be included while creating the access token (DT_PAAS_TOKEN):

PaaS integration - Installer download
Ingest metrics

For the custom metrics (via telegraf), buildpack attaches these default tags to metrics that are pushed to Dynatrace:

app - Environment Id of the Mendix application
instance_index - Instance index that metrics belong to

Extra tags can be attached via TAGS environment variable (example value: [env:accp])

Logging

The buildpack provides several options to configure logging.

Buildpack Log Levels

To set the log level for the buildpack itself to DEBUG, set the BUILDPACK_XTRACE environment variable to true .

Mendix Runtime Log Levels

It is possible to configure Mendix Runtime log levels using environment variables. This allows getting a better insight in the behavior of your Mendix app. This happens by adding one or more environment variables starting with the name LOGGING_CONFIG. The part of the name after that is not relevant and only used to distinguish between multiple entries if necessary. Its value should be valid JSON, in the format:

{
    "LOGNODE": "LEVEL"
}

You can see the available Log Nodes in your application in Mendix Studio Pro. The level should be one of:

CRITICAL
ERROR
WARNING
INFO
DEBUG
TRACE

Example:

cf set-env <YOUR_APP> LOGGING_CONFIG '{"<LOG NODE VALUE>": "DEBUG"}'

Rate-limiting of Log Output

The buildpack has the ability to rate-limit the amount of log lines from the Mendix Runtime. This can be useful for apps that misbehave and cause problems for other users in a multi-tenant environment. Rate-limiting is done in log lines per second. Extra lines are dropped and the number of dropped messages is printed on stderr .

Example (1000 loglines/second):

cf set-env <YOUR_APP> LOG_RATELIMIT '1000'

Custom Runtime Metrics filtering

For the third-party integrations explained above, in addition to the metrics collected by the agents, custom runtime metrics are provided via telegraf. This configuration also has a filtering mechanism that allows users to specify metrics they allow or deny for the vendor they are using. To filter the ingestion of custom runtime metrics to third party APMs, users should provide a list of prefixes of the metrics they want to allow/deny using the environment variables listed below.

Note: Custom database metrics cannot be filtered by name, to turn them off, the APPMETRICS_INCLUDE_DB environment variable should be set to false.

APM_METRICS_FILTER_ALLOW

Comma-separated list of prefixes for the metrics to be allowed. By default, all metrics are allowed, even if they are not specified via this env var.

For example, to allow only the session metrics, APM_METRICS_FILTER_ALLOW should be set to mx.runtime.stats.sessions:

cf set-env <YOUR_APP> APM_METRICS_FILTER_ALLOW 'mx.runtime.stats.sessions'

APM_METRICS_FILTER_DENY

Comma-separated list of prefixes for the metrics to be denied.

For example, to deny all metrics starting with jetty or mx.runtime, the environment variable should be set to jetty,mx.runtime:

cf set-env <YOUR_APP> APM_METRICS_FILTER_DENY 'jetty,mx.runtime'

APM_METRICS_FILTER_DENY_ALL

If this environment variable is set to true, all metrics will be denied regardless of values of APM_METRICS_FILTER_ALLOW, APM_METRICS_FILTER_DENY, and APPMETRICS_INCLUDE_DB.

cf set-env <YOUR_APP> APM_METRICS_FILTER_DENY_ALL true

Using the Buildpack without an Internet Connection

If you are running Cloud Foundry without a connection to the Internet, you should specify an on-premises web server that hosts Mendix Runtime files and other buildpack dependencies. You can set the endpoint with the following environment variable:

BLOBSTORE: https://my-intranet-webserver.my-company.com/mendix/

The preferred way to set up this on-premises web server is as a transparent proxy to https://cdn.mendix.com/ . This prevents manual work by system administrators every time a new Mendix version is released. Alternatively you can vendorize the required dependencies.

A list of buildpack dependencies can be downloaded from here.

For more information about external dependency management, please check here.

Troubleshooting

The buildpack provides several facilities for troubleshooting.

Enabling the Mendix Debugger

You can enable the Mendix Debugger by setting a DEBUGGER_PASSWORD environment variable. This will enable and open up the debugger for the lifetime of this process and is to be used with caution. The debugger is reachable on https://DOMAIN/debugger/. You can follow the second half of this How To to connect with Mendix Studio Pro. To stop the debugger, unset the environment variable and restart the application.

Rescue Mode

Sometimes the app won't run because it exits with status code 143. Or, for any reason, the app is unable to start, leaving you unable to debug the issue from within the container. For these cases we have introduced a DEBUG_CONTAINER mode. To enable it:

cf set-env <YOUR_APP> DEBUG_CONTAINER true
cf restart <YOUR_APP>

Now your app will start in CloudFoundry (i.e. the Mendix Runtime will not start yet) and you can troubleshoot the problem with:

cf ssh <YOUR_APP>
export HOME=$HOME/app # this should not be needed but for now it is
export DEBUG_CONTAINER=false # while we are in the container turn it off, we could try to make this optional by detecting other environment variables that are present over ssh but not regular start
export PORT=1234 # so that nginx can start correctly
cd app
PYTHONPATH=:buildpack:lib python3 buildpack/start.py

After you are done, you can disable debug mode with:

cf unset-env <YOUR_APP> DEBUG_CONTAINER
cf restart <YOUR_APP>

Similarly, if you need to use m2ee-tools inside the container for debugging purposes, you can do the following:

cf ssh <YOUR_APP>
export PYTHONPATH=/home/vcap/app/.local/lib/python3.10/site-packages/:/home/vcap/app/lib/
python3

and in the interactive Python console:

import os
from m2ee.client import M2EEClient
client = M2EEClient('http://localhost:8082', os.environ['M2EE_PASSWORD'])

Developing and Contributing

Please see DEVELOPING.md and CONTRIBUTING.md.

License

This project is licensed under the Apache License v2 (for details, see the LICENSE file).

Name		Name	Last commit message	Last commit date
Latest commit History 1,833 Commits
.github		.github
bin		bin
buildpack		buildpack
dev		dev
etc		etc
lib/m2ee		lib/m2ee
tests		tests
.gitignore		.gitignore
.mdlrc		.mdlrc
.pre-commit-config.yaml		.pre-commit-config.yaml
.pylintrc		.pylintrc
.yamllint.yml		.yamllint.yml
CONTRIBUTING.md		CONTRIBUTING.md
DEVELOPING.md		DEVELOPING.md
LICENSE		LICENSE
Makefile		Makefile
README.md		README.md
dependencies-stage.yml		dependencies-stage.yml
dependencies.yml		dependencies.yml
pyproject.toml		pyproject.toml
requirements-dev.in		requirements-dev.in
requirements.in		requirements.in
requirements.txt		requirements.txt

License

mendix/cf-mendix-buildpack

Folders and files

Latest commit

History

Repository files navigation

Run Mendix in Cloud Foundry

Requirements

Supported Mendix Versions

Supported CFLinuxFS Versions

Buildpack Releases and Version Pinning

Lifecycle

How to Deploy

1. Install Cloud Foundry CLI

2. Push your App

3. Configure an Admin Password

4. Connect a Database

Infrastructure Configuration

Connect a Database

Supported VCAP Schemas

PostgreSQL using RDS Service Broker

SAP Hana using Service Broker

Connect an External File Store

Swift (Bluemix Object Storage) Settings

Azure Storage Service Settings

S3 Settings

Use IAM Credentials

Implement TVM (Token Vending Machine)

Mendix Runtime Configuration

Horizontal Scaling

Constants

Scheduled Events

Custom Runtime Settings

Client Certificates

Client Certificate Format

Data Snapshots

License Activation

Java Configuration

Java Version

Java Heap Size

Java Max Metaspace Size

Java Garbage Collector

Java Virtual Machine (JVM) Settings

Enabling TLSv1.0 and TLSv1.1 for Outgoing Connections

Certificate Authorities

Built-In Proxy Configuration

HTTP Headers

Enabling SameSite / Secure Cookie Header Injection for Mendix Runtime < 8.12

Access Restrictions

Access Restrictions Format

Custom Locations

Custom Locations Format

Telemetry Configuration

New Relic

Set up New Relic integration

Tags/Metadata in metrics and logs

Custom tags

Service-binding integration (on-prem only) - DEPRECATED

Splunk

Set up Splunk integration

Metadata

Custom tags

AppDynamics

Basic functionality

Extended functionality

Datadog

Presets

Known Limitations

Dynatrace

Logging

Buildpack Log Levels

Mendix Runtime Log Levels

Rate-limiting of Log Output

Custom Runtime Metrics filtering

APM_METRICS_FILTER_ALLOW

APM_METRICS_FILTER_DENY

APM_METRICS_FILTER_DENY_ALL

Using the Buildpack without an Internet Connection

Troubleshooting

Enabling the Mendix Debugger