DataWeave is Mule’s most powerful and versatile tool for transforming data. The Transform Message component carries out a transformation of your Mule message that follows a transform script, this transform script can be explicitly written in DataWeave code or you can use the UI to build it implicitly through dragging and dropping elements. DataWeave fully supports DataSense, allowing you to leverage metadata from connectors, schemas and sample documents to more easily design your transformations. DataSense provides content assist while you are coding and scaffolds and auto generates lines of code form actions performed in the UI. The Transform Message component offers you a preview of your output that is built on sample data and is updated in real time as you make changes to your transform, so that you can be sure of what you’ll be getting out of the other end.
|
Tip
|
If you have projects that are built with the deprecated DataMapper, a migrator tool is now included in Studio, which assists in converting a DataMapper map into DataWeave: Right click on a DataMapper, select Migrate to DataWeave, and follow the instructions. |
In Anypoint Studio, place a Transform Message element in a flow to create transforms using the DataWeave language. The editor helps you do this by offering a drag and drop UI, intelligent autocomplete suggestions, and an output preview that is updated in real time as you make changes. This generates a .dwl transformation file (or several of them) that stores your code and is packaged with your Mule application.
|
Tip
|
Besides the Transform Message Component described in this document, you can also invoke DataWeave language on any Mule component that accepts Mule Expression Language (MEL) by calling ti through a function. See MEL DataWeave Functions for more information. |
When adding a Transform Message element to a Mule Flow, it takes the elements from the incoming Mule Message as its inputs. It then performs the necessary actions to produce a Mule message as output for the next element in the flow.
If you click on an instance of the Transform Message element in your flow, its properties editor will be displayed:
Note that by default the properties editor of the Transform Message element displays two main regions:
-
On the left: is the graphical UI, which exposes the known input and output structures. The mappings between input and output fields are represented through lines drawn from one side to the other. You can easily click and drag one field onto another to map these.
-
On the right: the DataWeave code editor. This allows you to leverage the full power of DataWeave syntax, which includes many tools that allow you to aggregate, normalize, group, join, partition, pivot and filter. See the DataWeave language introduction for a guide on the DataWeave language syntax.
Both regions show different representations of the same transform, and any changes done to one representation are matched in real time by the other one.
|
Tip
|
When creating a new transformation, it’s a lot easier if you first add and configure any other elements on your flow that expose metadata. In this way, this metadata that other message processors expose to Studio is displayed in the input and output structures, without you needing to define the metadata explicitly. |
Two tree views show the known metadata contents of the incoming and the outgoing Mule Messages, allowing you to explore them and know what data is available for using as an input and where it can fit into, and how to reference each of these parts.
You can simplify or hide the graphical UI if you wish by clicking the icons on the top right to select between the different views for this properties editor:
By simple interactions with the graphical UI, you can intuitively build out much of your DataWeave code, often this is all you really need to do. Here some of the most common things you can do through the UI:
-
Drag an element on the input structure over to another on the output structure. This will cast a line that joins them and also add the necessary lines to the DataWeave code that describes this mapping.
-
Drag a high-level object that contains inner fields inside it onto another in the output. You will be prompted about how to represent this in the DW code.
-
Double click on an output field to add it into the DataWeave code with a static value. This will add an
Fxicon next to it, as well as a line to the DataWeave code that assigns a defaultnullvalue to the field. You can then change this value in the code to whatever you want.
-
Select an element to have its corresponding line in the DataWeave code highlighted. This is very helpful when dealing with large transforms.
-
If an input field is mapped to two or more output fields, you can right-click it and then select which of the multiple outputs you want to highlight in the DataWeave code.
-
Filter the views displayed in the input and output structures by typing a name in the search boxes at the top of either, only those fields that match your search are then displayed. This is particularly useful when dealing with large data structures with many nested elements.
In this section, you write the actual DataWeave code that carries out the transform. Sometimes, all you need to do can be automatically built by dragging elements in the GUI, but other times you may want to carry out more complex operations that involve aggregation, filtering, calculations, defining custom functions, etc… and for that you must write DataWeave code.
The header of a dataweave script defines directives that indicate the input, the output and constant values or functions that can be referenced anywhere in the code. See The DataWeave Header. These directives can be manually set on the code, or automatically defined based on the metadata on your flow or what you map in the UI.
If your transform outputs XML data, a namespace directive will be automatically added to your DataWeave header section, defining a default name for it. This namespace is then referenced in the body too.
%dw 1.0 %output application/xml %namespace ns0 http://mulesoft.org/tshirt-service
|
Note
|
Although DataWeave as a language supports adding input directives and naming these by any name you like, when using DataWeave in Anypoint Studio, it’s not necessary to declare any input directives for any of the components of the Mule Message that arrives to the DataWeave transformer (Payload, flow variables and input/outbound properties) nor for any system variables. These are already implicitly recognized as inputs and can be referenced anywhere in the DataWeave body without a need to include them in the header, their type is known from Mule metadata. |
For further reference about writing DataWeave code, see DataWeave Language Introduction
Many elements you can use in Studio expose metadata about their input and output to the rest of the flow. For example, when a connector is configured to carry out a specific operation, its output structure is known, and this is expected to be the input received by following component in the flow. When this information is available to the Transform Message component, it is interpreted as the expected input or the required output, depending on where each are placed relative to each other on the flow. This metadata is then displayed in the input or output structures in the UI.
Ideally you should have both the known input and the expected output as already defined. If the metadata definition is missing for the input or output, a notification on the corresponding section of the Graphical UI will advise you to provide it. There are several ways you can deal with this:
-
Make sure that other elements in the flow are fully configured. For example, a connector can’t expose its output structure until a given operation is chosen, as its output might differ depending on this. A Web Service Consumer can include a WSDL file that describes the service, or an HTTP connector can include a RAML API definition file.
-
You can manually define the output of a Mule component via its Metadata Tab. See Defining Metadata.
-
You can manually define the input of the Transform Message component by clicking on the Define Metadata link that appears in the notification that warns you about missing input metadata. See Defining Metadata.
TipThis same menu can also be accessed by opening the preview section, which will display a shortcut to set this up in case it’s still not configured.
To edit the already defined metadata on the input of the Transform Message component, simply right-click on the root of the corresponding section of the Graphical Ui and select Set Metadata to access the same options.
|
Tip
|
If you plan to create your transform entirely via the text editor section, you can skip specifying the metadata definition and reference elements directly. You can set the output type in the output directive of your DataWeave code. To define your metadata via XML, see DataWeave XML Reference. |
As part of the metadata definition of your input structure, DataWeave allows you to set up certain properties of the reader object so that it parses the input differently. This is only available with certain inptut formats, and each one of these has its own specific properties. In Anypoint Studio, there are two ways to set this up:
-
Configure the component that actually brings this information into your flow, by accessing its Metadata tab.
-
On the Transform Message component itself, right clicking on the root of the input section and selecting Reader Configuration to access a menu
NoteThis option won’t be available if the type of the input doesn’t allow for this kind of configuration. If the payload is of type unknown, you must change its type first.
You can also add this information through properties in the XML source of your Mule project. For this, see DataWeave XML reference
For a detailed reference of what properties can be set in the Reader Configuration of each format, see the corresponding reader properties section:
As part of the metadata definition of your output structure, DataWeave allows you to set up certain properties of the writer object so that it constructs the output differently. This is only available with certain output formats, and each one of these has its own specific properties.
These properties are simply written on the %output directive of your DataWeave code.
For a detailed reference of what properties can be set in the Writer Configuration of each format, see the corresponding reader properties section:
The sample data tab allows you to test out examples of what your input might look like. This sample data is used together with your DataWeave code to produce a sample output in The Preview Section, which gets updated in real time as you make changes. Through this, you can make sure that your transformation deals well with different use cases, special characters, etc.
You can access this tab by right clicking on the root of the input section and selecting Edit Sample Data.
|
Tip
|
This same menu can also be accessed by opening the preview section, which will display a shortcut to set this up in case it’s still not provided. |
|
Note
|
When the input is of types JSON, XML, CSV or flat file, the sample input contains plain code in the corresponding format. When the input is of type POJO or DataWeave, the sample input is written in DataWeave for more simplicity. In these cases the sample DataWeave code is merely a way to display the sample data, not a transformation in itself. |
Selecting this option opens a new tab in the input section with an empty scaffolding of your input structure, in which values are populated with the string ????. You can replace these values with more useful sample values to see how they are mapped out to the preview section.
|
Tip
|
You can always click the rescafold button to restore the sample data to its default state. Note that with this you’ll loose any sample data you’ve provided.
|
You can enable the preview section by clicking on the Preview button on the top-right of the editor.
This section presents a sample output, built by taking the sample input you provide and transforming it through the DataWeave transform. As you make changes in the DataWeave code, notice how the output data structure changes. If your transformer has multiple outputs, the Preview section will display the one corresponding to the currently selected transform.
|
Note
|
Note that samples defined in this section work within the scope of the Transform Message component, they don’t alter the metadata that’s propagated to other elements in the flow, and their values aren’t propagated onwards. They aren’t used in run time in any way, not as default values nor anything else, they’re only used in design time. |
If no sample is provided yet, this section features a shortcut that you can click to open the Edit Sample window and provide an input sample to construct the preview.
If you still haven’t set up the metadata structure for your input, when clicking on this shortcut you will be first prompted to set up the structure via the Defining Input and Output Structure window.
For your DataWeave code’s syntax to be evaluated, you must have the Preview Section enabled. With this enabled, any syntax errors are marked. Above your DataWeave code, an additional error notification can be opened to display further detail.
+
If you click this notification, a window opens detailing each error in your code and its cause.
+
A single Transform Message element can give shape to several different components of the output Mule message. Each of these output components must be defined in a separate .dwl file, written out in a separate tab of the Transform section. For example in one tab you may be defining the payload contents, in another those of an outbound property, and these will both be parts of the same output Mule message.
To add a new output, simply click the Add new target button at the top of the DataWeave code section.
Then you must specify where in the output Mule message to place the output of this new DataWeave transform. In case you’re creating a new variable or property, you must also set a name for it.
You can also change the target of an existing transform by clicking the Edit Current Target button, and in that way point the output of your transform to a different element in the outgoing Mule Message.
|
Note
|
Execution order of multiple outputs is not guaranteed. Ensure each transformation is independant of the order of execution and the other outputs. |
By default, DataWeave code is expressed inline within your Mule XML file. If you wish to keep it in a separate file and have your XML reference this file, you can easily do this from the DataWeave UI. In order to export the DataWeave code to a .dwl file, you need to do the following:
-
click the Edit Current Target button
-
Select the 'File' radio button
-
Type a name for your
.dwlfile -
Click OK
A file will be created under the 'src/main/resources' folder in your project containing your DataWeave code.
The Transform Message component can be configured to handle the execution of a transformation of a large payload at a deferred time, and you can set the maximum size for which it will use memory rather than the hard disk. No configuration is necessary in the Transform Message component, but you may finetune certain parameters if you wish. See DataWeave Memory Management.
-
View complete example projects that use DataWeave in the Anypoint Exchange