Skip to content

Commit 25d7eb1

Browse files
authored
Merge pull request #121 from oracle/Issue#86-discover-with-variables
Issue#86 discover with variables
2 parents a482d0b + 3b5bdb0 commit 25d7eb1

31 files changed

+3183
-26
lines changed

README.md

Lines changed: 203 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -845,6 +845,209 @@ When creating the archive, the tool will try to gather all binaries, scripts, an
845845
1. Any binaries referenced from the `ORACLE_HOME` will not be gathered, as they are assumed to exist in any target domain to which model-driven operations will be applied. Doing this is key to allowing the model to be WebLogic Server version independent.
846846
2. In its current form, the discover domain Tool will only gather binaries and scripts that are accessible from the local machine. Warnings will be generated for any binaries or scripts that cannot be found but the configuration for those binaries will still be collected, where possible. It is the user's responsibility to add those missing files to the archive in the appropriate locations and edit the the model, as needed, to point to those files inside the archive using the relative path inside the archive (for example, `wlsdeploy/applications/myapp.ear`).
847847

848+
## The Variable Injector Tool
849+
850+
The variable injector tool is used to tokenize a model with variables. The values for these variables are assigned using an external property file. This facilitates using the same domain model to create new domains in different environments. The variable injector tool can be run as an option in the Discover Domain tool, or or from the stand-alone command line interface.
851+
852+
To enable the Variable Injector during the Discover Domain, create a variable injector configuration by placing a json file named model_variable_injector.json into the \<WLSDEPLOY\>/lib directory using one or more of the pre-defined keywords and/or a CUSTOM list of files. A keyword points to an injector directive file. The tool applies the directives to the attributes in a model, and if the directive matches an attribute, then a property token with a unique variable name is injected into the model and replaces the attribute value. The variable name and model attribute value are placed into the external variable properties file.
853+
854+
NOTE: Variable injection on an attribute is only performed once. The property token is not replaced by any subsequent matches.
855+
856+
If variable injection is enabled, the Discover Domain Tool calls the variable injector after the model has been discovered and after all filters run, but before model validation.
857+
858+
The supported keywords are as follows:
859+
860+
- CREDENTIALS - All MBean credentials attribute values (user and password) are injected with a variable.
861+
862+
- HOST - All MBean Host attribute values in the model are injected with a variable.
863+
864+
- PORT - All MBean Port attribute values in the model are injected with a variable.
865+
866+
- TARGET - All MBean Target attribute values in the model are injected with a variable.
867+
868+
- TOPOLOGY - Common environmental MBean attributes found in the topology section of the model are injected with a variable.
869+
This includes server, machine and node manager ports, credentials and listen addresses, and cluster messaging modes, addresses and ports.
870+
871+
- URL - All MBean URL attribute values in the model are injected with a variable.
872+
873+
NOTE: The directives used by each pre-defined keyword are defined in an injector json file that is located in the \<WLSDEPLOY\>/lib/injectors folder. These files should not be changed, but could be used as .
874+
875+
Here is an example of a model_variable_injector.json file using the PORT keyword.
876+
877+
```json
878+
{
879+
"PORT": {}
880+
}
881+
```
882+
883+
Below is a model snippet that shows injected variables in the port attributes.
884+
885+
```$yaml
886+
887+
topology:
888+
Name: soa_domain
889+
AdminServerName: AdminServer
890+
Cluster:
891+
soa_cluster:
892+
osb_cluster:
893+
Server:
894+
AdminServer:
895+
ListenAddress: myadmin.example.com
896+
ListenPort: @@PROP:Server.AdminServer.ListenPort@@
897+
Machine: machine1
898+
SSL:
899+
Enabled: true
900+
ListenPort: @@PROP:Server.SSL.AdminServer.ListenPort@@
901+
soa_server1:
902+
ListenAddress: managed1.example.com
903+
ListenPort: @@PROP:Server.soa_server1.ListenPort@@
904+
Cluster: soa_cluster
905+
Machine: machine2
906+
SSL:
907+
Enabled: true
908+
ListenPort: @@PROP:Server.SSL.soa_server1.ListenPort@@
909+
soa_server2:
910+
ListenAddress: managed2.example.com
911+
ListenPort: @@PROP:Server.soa_server2.ListenPort@@
912+
Cluster: soa_cluster
913+
Machine: machine3
914+
SSL:
915+
Enabled: true
916+
ListenPort: @@PROP:Server.SSL.soa_server2.ListenPort@@
917+
```
918+
919+
And the resulting variable property file:
920+
921+
```
922+
Server.AdminServer.ListenPort=7001
923+
Server.AdminServer.SSL.ListenPort=7002
924+
Server.soa_server1.ListenPort=8001
925+
Server.soa_server1.SSL.ListenPort=8002
926+
Server.soa_server2.ListenPort=8001
927+
Server.soa_server2.SSL.ListenPort=8002
928+
```
929+
930+
To specify the name and location of the variable properties file for the Discover Tool, use the argument `-variable_properties_file` on the command line. Usage of the variable_properties_file argument without the presence of the model variable injector file in the \<WLSDEPLOY\>/lib directory will cause an error condition and the tool will exit. If the model variable injector file exists in the directory, but the command line argument is not used, the variable properties file is created with the following defaults: If the model_file command line argument is used on the Discover Domain run, the properties file name and location will be the same as the model file, with the file extension `.properties`. If only the archive file argument is present, the archive file name and location will be used.
931+
932+
As with the archive and model file, each run of the discover domain tool will overwrite the contents of an existing variable property file with the values from the current run.
933+
934+
### Custom Variable Injector
935+
936+
To designate custom injector directives, use the CUSTOM keyword in the model_variable_injector.json. The CUSTOM keyword requires a a list of one or more custom injector directive json files.
937+
938+
An injector directive contains a key that identifies an attribute to be tokenized, and an optional set of directive properties. The key is a period separated MBean hierarchy and attribute name as they are defined in the model. Always exclude the name of the model section from the injector key.
939+
940+
For example, an injector key for the Server SSL Listen Port is as below. This directive contains no additional properties.
941+
942+
```json
943+
{
944+
"Server.SSL.ListenPort": {}
945+
}
946+
```
947+
948+
NOTE: The hierarchy of MBeans in the model for the ListenPort attribute and note that the MBean name of AdminServer is NOT included in the directive:
949+
950+
```yaml
951+
topology:
952+
Server:
953+
AdminServer:
954+
ListenAddress: myadmin.example.com
955+
ListenPort: 7001
956+
Machine: machine1
957+
SSL:
958+
Enabled: true
959+
ListenPort: 7002
960+
```
961+
962+
These custom injector json files will be processed by the Variable Injector tool before keywords, each file processed in list order. A property injected into an attribute will not be replaced by any subsequent matches.
963+
964+
#### Custom directive properties
965+
966+
Include the following properties to refine the directive as specified.
967+
968+
- force:<attribute>
969+
If the MBean hierarchy exists in the model, but the attribute does not, then the attribute will be added and persisted to the discovered model. The value stored in the
970+
model is the weblogic default value.
971+
972+
- variable_value:
973+
Replace the model value with the specified value in the variable properties. This may be used in conjunction with the force directive, replacing the default value with the indicated value.
974+
975+
- regexp:
976+
A list of regexp patterns that will be applied to either the string values or map values of an attribute in the model. If the pattern matches, then the matching part of the
977+
string or dictionary will be injected with a property token and a unique variable name .
978+
-- pattern:
979+
the regular expression pattern to apply to the string value or map values of an attribute
980+
-- suffix:
981+
The suffix name to append to each resulting variable name in order to create a unique variable name
982+
983+
The regexp list is useful when only a segment of a string value or map needs to be tokenized (giving you a clean list of property values in the variable properties file). You can inject more than one token into a string or map with multiple patterns. However, when you have more than one pattern, you must provide a suffix for each. This allows the tool to generate a unique variable name for each token in the string or map.
984+
985+
The following is an example of how to effectively use the regexp directive list to search for a segment in a string value. In this example, we want to search for the host and port in each Oracle JDBC URL that uses the special Oracle URL notation, and create an entry for the host and port in the variable properties file.
986+
987+
In the model, we expect to find a URL like the following:
988+
989+
```$yaml
990+
JDBCSystemResource:
991+
Database1:
992+
JdbcResource:
993+
JDBCDriverParams:
994+
URL: 'jdbc:oracle:thin:@(DESCRIPTION=(ADDRESS_LIST=(ADDRESS=(PROTOCOL=TCP)(HOST=slc05til.us.oracle.com)(PORT=1521)))(CONNECT_DATA=(SERVICE_NAME=orcl.us.oracle.com)))'
995+
```
996+
997+
We create a directive in our custom injector json file:
998+
999+
```json
1000+
"JDBCSystemResource.JdbcResource.JDBCDriverParams.URL":
1001+
{
1002+
"regexp": [
1003+
{
1004+
"pattern": "(?<=PORT=)[\\w.-]+(?=\\))",
1005+
"suffix": "Port"
1006+
},
1007+
{
1008+
"pattern": "(?<=HOST=)[\\w.-]+(?=\\))",
1009+
"suffix": "Host"
1010+
}
1011+
]
1012+
},
1013+
```
1014+
1015+
During the Discover Domain tool run, the pattern is applied to the URL string and tokens injected into the string:
1016+
1017+
```
1018+
URL: 'jdbc:oracle:thin:@(DESCRIPTION=(ADDRESS_LIST=(ADDRESS=(PROTOCOL=TCP)(HOST=@@PROP:JDBCSystemResource.Database1.JdbcResource.JDBCDriverParams.URL--Host@@:)(PORT=@@PROP:JDBCSystemResource.Database1.JdbcResource.JDBCDriverParams.URL--Port@@)))(CONNECT_DATA=(SERVICE_NAME=orcl.us.oracle.com)))'
1019+
```
1020+
1021+
And The variables put in the properties file:
1022+
1023+
```
1024+
JDBCSystemResource.Database1.JdbcResource.JDBCDriverParams.URL--Host=slc05til.us.oracle.com
1025+
JDBCSystemResource.Database1.JdbcResource.JDBCDriverParams.URL--Port=1521
1026+
```
1027+
1028+
#### Selecting specific MBean names for variable injection
1029+
1030+
This final custom directive allows you to explicitly define which named entries for an MBean in the model you wish to inject properties. For instance, you might wish to tokenize an attribute just for a specific server. To define a list of one or more names for a specific MBean in the injector directive hierarchy, format the list as follows:
1031+
```
1032+
MBean[comma separated list of names]
1033+
``
1034+
To select only the admin server named AdminServer for a Server directive, use the format Server[AdminServer]. To select servers soa_server1 and soa_server2, format the key as Server[soa_server1,soa_server2]
1035+
1036+
The injector tool recognizes two KEYWORDS for a user list, MANAGED_SERVERS (all the managed servers in the model) and ADMIN_SERVER (The admin server in the model).
1037+
1038+
A custom injector for the admin server SSL listen port is:
1039+
1040+
```json
1041+
{
1042+
"Server[ADMIN_SERVER].SSL.ListenPort": {},
1043+
}
1044+
```
1045+
1046+
## Samples
1047+
1048+
A sample of a model_variable_injector.json file and a custom injector json file are installed in the WLSDEPLOY/samples directory.
1049+
1050+
8481051
## Downloading and Installing the Software
8491052

8501053
The Oracle WebLogic Server Deploy Tooling project repository is located at [`https://github.com/oracle/weblogic-deploy-tooling`](https://github.com/oracle/weblogic-deploy-tooling). Binary distributions of the `weblogic-deploy.zip` installer can be downloaded from the [GitHub Releases page](https://github.com/oracle/weblogic-deploy-tooling/releases). To install the software, simply unzip the `weblogic-deploy.zip` installer on a machine that has the desired versions of WebLogic Server installed. After being unzipped, the software is ready to use, just set the `JAVA_HOME` environment variable to point to a Java 7 or higher JDK and the shell scripts are ready to run.

core/src/main/python/discover.py

Lines changed: 46 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -12,20 +12,20 @@
1212
from java.lang import IllegalArgumentException
1313
from java.lang import IllegalStateException
1414
from java.lang import String
15-
1615
from oracle.weblogic.deploy.aliases import AliasException
1716
from oracle.weblogic.deploy.discover import DiscoverException
1817
from oracle.weblogic.deploy.util import CLAException
1918
from oracle.weblogic.deploy.util import FileUtils
2019
from oracle.weblogic.deploy.util import PyWLSTException
2120
from oracle.weblogic.deploy.util import TranslateException
22-
from oracle.weblogic.deploy.util import WebLogicDeployToolingVersion
2321
from oracle.weblogic.deploy.util import WLSDeployArchive
2422
from oracle.weblogic.deploy.util import WLSDeployArchiveIOException
23+
from oracle.weblogic.deploy.util import WebLogicDeployToolingVersion
2524
from oracle.weblogic.deploy.validate import ValidateException
2625

2726
sys.path.append(os.path.dirname(os.path.realpath(sys.argv[0])))
2827

28+
import wlsdeploy.tool.util.variable_injector as variable_injector
2929
from wlsdeploy.aliases.wlst_modes import WlstModes
3030
from wlsdeploy.exception import exception_helper
3131
from wlsdeploy.logging.platform_logger import PlatformLogger
@@ -36,8 +36,9 @@
3636
from wlsdeploy.tool.discover.multi_tenant_discoverer import MultiTenantDiscoverer
3737
from wlsdeploy.tool.discover.resources_discoverer import ResourcesDiscoverer
3838
from wlsdeploy.tool.discover.topology_discoverer import TopologyDiscoverer
39-
from wlsdeploy.tool.validate.validator import Validator
4039
from wlsdeploy.tool.util import filter_helper
40+
from wlsdeploy.tool.util.variable_injector import VariableInjector
41+
from wlsdeploy.tool.validate.validator import Validator
4142
from wlsdeploy.util import wlst_helper
4243
from wlsdeploy.util import model_translator
4344
from wlsdeploy.util.cla_utils import CommandLineArgUtil
@@ -58,8 +59,9 @@
5859

5960
__optional_arguments = [
6061
# Used by shell script to locate WLST
61-
CommandLineArgUtil.DOMAIN_TYPE_SWITCH,
6262
CommandLineArgUtil.MODEL_FILE_SWITCH,
63+
CommandLineArgUtil.DOMAIN_TYPE_SWITCH,
64+
CommandLineArgUtil.VARIABLE_PROPERTIES_FILE_SWITCH,
6365
CommandLineArgUtil.ADMIN_URL_SWITCH,
6466
CommandLineArgUtil.ADMIN_USER_SWITCH,
6567
CommandLineArgUtil.ADMIN_PASS_SWITCH
@@ -80,6 +82,7 @@ def __process_args(args):
8082
__verify_required_args_present(required_arg_map)
8183
__wlst_mode = __process_online_args(optional_arg_map)
8284
__process_archive_filename_arg(required_arg_map)
85+
__process_variable_filename_arg(optional_arg_map)
8386

8487
combined_arg_map = optional_arg_map.copy()
8588
combined_arg_map.update(required_arg_map)
@@ -135,7 +138,7 @@ def __process_online_args(optional_arg_map):
135138
raise ex
136139
optional_arg_map[CommandLineArgUtil.ADMIN_PASS_SWITCH] = String(password)
137140

138-
__logger.info('WLSDPLY-06020')
141+
__logger.info('WLSDPLY-06020', class_name=_class_name, method_name=_method_name)
139142
return mode
140143

141144

@@ -159,6 +162,28 @@ def __process_archive_filename_arg(required_arg_map):
159162
return
160163

161164

165+
def __process_variable_filename_arg(optional_arg_map):
166+
"""
167+
If the variable filename argument is present, the required model variable injector json file must exist in
168+
the WLSDEPLOY lib directory.
169+
:param optional_arg_map: containing the variable file name
170+
:raises: CLAException: if this argument is present but the model variable injector json does not exist
171+
"""
172+
_method_name = '__process_variable_filename_arg'
173+
174+
if CommandLineArgUtil.VARIABLE_PROPERTIES_FILE_SWITCH in optional_arg_map:
175+
variable_injector_file_name = variable_injector.get_default_variable_injector_file_name()
176+
try:
177+
FileUtils.validateExistingFile(variable_injector_file_name)
178+
except IllegalArgumentException, ie:
179+
ex = exception_helper.create_cla_exception('WLSDPLY-06021', optional_arg_map[
180+
CommandLineArgUtil.VARIABLE_PROPERTIES_FILE_SWITCH], variable_injector_file_name,
181+
ie.getLocalizedMessage(), error=ie)
182+
__logger.throwing(ex, class_name=_class_name, method_name=_method_name)
183+
raise ex
184+
return
185+
186+
162187
def __discover(model_context):
163188
"""
164189
Populate the model from the domain.
@@ -344,7 +369,7 @@ def __persist_model(model, model_context):
344369
if not model_file.delete():
345370
model_file.deleteOnExit()
346371
except (WLSDeployArchiveIOException, IllegalArgumentException), arch_ex:
347-
ex = exception_helper.create_discover_exception('WLSDPLY-06009', model_file.getAbsolutePath(),
372+
ex = exception_helper.create_discover_exception('WLSDPLY-20023', model_file.getAbsolutePath(),
348373
model_file_name, arch_ex.getLocalizedMessage(),
349374
error=arch_ex)
350375
__logger.throwing(ex, class_name=_class_name, method_name=_method_name)
@@ -368,25 +393,31 @@ def __check_and_customize_model(model, model_context):
368393
if filter_helper.apply_filters(model.get_model(), "discover"):
369394
__logger.info('WLSDPLY-06014', _class_name=_class_name, method_name=_method_name)
370395

371-
396+
inserted, variable_model, variable_file_name = VariableInjector(_program_name, model.get_model(), model_context,
397+
WebLogicHelper(
398+
__logger).get_actual_weblogic_version()).\
399+
inject_variables_keyword_file()
400+
if inserted:
401+
model = Model(variable_model)
372402
try:
373403
validator = Validator(model_context, wlst_mode=__wlst_mode)
374404

375405
# no variables are generated by the discover tool
376-
validator.validate_in_tool_mode(model.get_model(), variables_file_name=None,
377-
archive_file_name=model_context.get_archive_file_name())
406+
validator.validate_in_tool_mode(model.get_model(), variables_file_name=variable_file_name,
407+
archive_file_name=model_context.get_archive_file_name())
378408
except ValidateException, ex:
379409
__logger.warning('WLSDPLY-06015', ex.getLocalizedMessage(), class_name=_class_name, method_name=_method_name)
410+
return model
380411

381412

382-
def __log_and_exit(exit_code, _class_name, _method_name):
413+
def __log_and_exit(exit_code, class_name, _method_name):
383414
"""
384415
Helper method to log the exiting message and call sys.exit()
385416
:param exit_code: the exit code to use
386-
:param _class_name: the class name to pass to the logger
417+
:param class_name: the class name to pass to the logger
387418
:param _method_name: the method name to pass to the logger
388419
"""
389-
__logger.exiting(result=exit_code, class_name=_class_name, method_name=_method_name)
420+
__logger.exiting(result=exit_code, class_name=class_name, method_name=_method_name)
390421
sys.exit(exit_code)
391422

392423

@@ -433,12 +464,12 @@ def main(args):
433464
error=ex, class_name=_class_name, method_name=_method_name)
434465
__log_and_exit(CommandLineArgUtil.PROG_ERROR_EXIT_CODE, _class_name, _method_name)
435466

436-
__check_and_customize_model(model, model_context)
437-
467+
model = __check_and_customize_model(model, model_context)
438468
try:
439469
__persist_model(model, model_context)
470+
440471
except TranslateException, ex:
441-
__logger.severe('WLSDPLY-06012', _program_name, model_context.get_archive_file_name(), ex.getLocalizedMessage(),
472+
__logger.severe('WLSDPLY-20024', _program_name, model_context.get_archive_file_name(), ex.getLocalizedMessage(),
442473
error=ex, class_name=_class_name, method_name=_method_name)
443474
__log_and_exit(CommandLineArgUtil.PROG_ERROR_EXIT_CODE, _class_name, _method_name)
444475

core/src/main/python/encrypt.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -190,7 +190,7 @@ def __encrypt_model_and_variables(model_context):
190190

191191
if variable_change_count > 0:
192192
try:
193-
variable_helper.write_variables(variables, variable_file)
193+
variable_helper.write_variables(_program_name, variables, variable_file)
194194
__logger.info('WLSDPLY-04209', _program_name, variable_change_count, variable_file,
195195
class_name=_class_name, method_name=_method_name)
196196
except VariableException, ve:

0 commit comments

Comments
 (0)