diff --git a/README.md b/README.md index 7170e35..82fd4c0 100644 --- a/README.md +++ b/README.md @@ -1,10 +1,67 @@ # oe-service-personalization -## dependency -* oe-cloud -* oe-logger -* oe-expression -* oe-personalization +This module will apply operations such as field masking, hiding +fields, sorting, etc on top of traditional remote endpoints, +thereby "personalizing" them. With these limited set of +operations it is also possible to personalize data to a group +of clients. In other words, the same data can appear (and/or +behave differently) on, say, an android app, an ios app, and, +a browser app. + +Such granular differenciations are possible by describing them +in an property called `scope` on the personalization rule. +(This is made possible by the `oe-personalization` module). +Further, for such segmented personalizations to take effect, we +need the necessary header in the http request (as how it is in +the `scope`). + +Such kind of personalizations allows for us to be able to +derive analytics on the groups or segmentations which would +in-turn allow us to focus more on servicing those groups better. +For e.g. assume there is an api. Further assume, it is +personalized for both a mobile client and a browser client. +Once deployed, we can derive the analytics by looking at +server logs for the same api and decide which platform that api +is frequently accessed. Such information can be used to improve +the user experience on that platform. However, this is not in +scope of this document. + +> This documentation is best viewed in a github based +markdown viewer. E.g. Visual Code editor's inbuilt markdown +reader + +## Table Of Contents + +- [oe-service-personalization](#oe-service-personalization) + * [Table Of Contents](#table-of-contents) + * [Install and test](#install-and-test) + * [Main features](#main-features) + * [`PersonalizationRule` model](#-personalizationrule--model) + + [Important properties](#important-properties) + + [Acceptable values for `methodName`](#acceptable-values-for--methodname-) + * [How to use](#how-to-use) + - [1. Install the module to your application](#1-install-the-module-to-your-application) + - [2. Add config to your project's `app-list.json`:](#2-add-config-to-your-project-s--app-listjson--) + - [3. Add `ServicePersonalizationMixin` mixin to the model declaration.](#3-add--servicepersonalizationmixin--mixin-to-the-model-declaration) + - [4. Insert rules into the `PersonalizationRule` model.](#4-insert-rules-into-the--personalizationrule--model) + - [5. _(Optional)_ Configure custom functions path](#5---optional---configure-custom-functions-path) + * [Working Principle](#working-principle) + * [Supported operations](#supported-operations) + * [**fieldMask** options](#--fieldmask---options) + + [fieldMask for strings](#fieldmask-for-strings) + + [fieldMask for numbers](#fieldmask-for-numbers) + + [fieldMask for date](#fieldmask-for-date) + * [Operations on objects](#operations-on-objects) + * [Programmatic API](#programmatic-api) + + [1. Using model name, and, model data](#1-using-model-name--and--model-data) + + [2. Using model name, data, and, personalization rules](#2-using-model-name--data--and--personalization-rules) + * [Significance of pre-fetch/post-fetch operations](#significance-of-pre-fetch-post-fetch-operations) + * [Points to consider](#points-to-consider) + * [Test Synopsis](#test-synopsis) + * [Note on loopback relations](#note-on-loopback-relations) + * [Additional Locale support for dates](#additional-locale-support-for-dates) + + [Dynamic locale support](#dynamic-locale-support) + + [Dynamic locale support (with limited locales)](#dynamic-locale-support--with-limited-locales-) ## Install and test @@ -17,3 +74,766 @@ $ npm run test $ # Run test cases along with code coverage - code coverage report will be available in coverage folder $ npm run grunt-cover ``` + +## Main features + +- Customizing remote responses, or data +(i.e. queried via loopback model api), + to appear in a certain manner + - Based on user role + - Custom scope - for e.g. for android, or, ios clients +- Limiting personalization to apply to a particular remote method + +## `PersonalizationRule` model + +This is the framework model is used to store personalization rules. + +```json +{ + "name": "PersonalizationRule", + "base": "BaseEntity", + "plural": "PersonalizationRules", + "description": "Service Personalization metadata", + "idInjection": false, + "strict": true, + "options": { + "validateUpsert": true, + "isFrameworkModel": true + }, + "properties": { + "ruleName": { + "type": "string" + }, + "disabled": { + "type": "boolean", + "default": false + }, + "modelName": { + "type": "string", + "required": true, + "unique": true, + "notin": [ + "PersonalizationRule" + ] + }, + "personalizationRule": { + "type": "object", + "required": true + }, + "methodName" : { + "type": "string", + "default": "**", + "description": "The model methodName this rule should apply to. Should be the methodName (static/instance) or wildcards you specify in a afterRemote()/beforeRemote(). Default '**'" + } + }, + "validations": [], + "relations": {}, + "acls": [], + "methods": {}, + "mixins": {} +} +``` + +### Important properties +1. `modelName` - the target model for which the personalization should apply +2. `personalizationRule` - the json object which stores operations to apply. More in the `How to Use` and `Supported operations` sections +3. `disabled` - boolean flag which instructs framework to apply personalization or not - default _false_ (i.e. apply the personalizations) +4. `methodName` - the method for which personalization has to apply - default `**` - i.e. apply to all static and instance methods. See below for acceptable values. +5. `ruleName` - (_optional_) name for the personalization rule. Used for debugging. +6. `scope` - (_optional_) used to control personalization based on roles +or through http headers (by the api consumers). For e.g. +it can have a value `{ "roles" : ['admin'] }`... +it means, personalization will apply for a +logged-in `admin` user only. + +### Acceptable values for `methodName` + +It can accept the following patterns (wildcards and names) +- `**` (_default_) - all static and instance methods +- `*` - only static methods +- `*.*` or `prototype.*` - only instance methods +- **Valid static method name**. It can be standard, +or, a custom static remote method. +- **Valid instance method name**. It can be standard, +or, a custom instance remote method. +E.g. `prototype.foo`, where `foo` is a method +defined on the prototype of the model's constructor. + +See section `Notes on loopback relations` for more information. + +## How to use + +This documents a general usage pattern - achieving +personalization via call to the http endpoint +of the model. + +The same can be done through code. Refer to `Programmatic Api` section for the same. + +#### 1. Install the module to your application + +``` +npm install oe-service-personalization +``` +#### 2. Add config to your project's `app-list.json`: + +``` + { + "path": "oe-service-personalization", + "enabled": true + } +``` + +If you require role-based service personalization, add `oe-personalization` depedency prior to `oe-service-personalization` +``` + { + "path": "oe-personalization", + "enabled": true, + "autoEnableMixins": true + }, + { + "path": "oe-service-personalization", + "enabled": true + } +``` + +#### 3. Add `ServicePersonalizationMixin` mixin to the model declaration. + +Example: +```json +{ + "name": "ProductOwner", + "base": "BaseEntity", + "idInjection": true, + "properties": { + "name": { + "type": "string" + }, + "city": { + "type": "string", + "require" : true + } + }, + "validations": [], + "relations": { + "ProductCatalog": { + "type": "hasMany", + "model": "ProductCatalog" + }, + "address": { + "type" : "hasOne", + "model" : "AddressBook" + } + }, + "acls": [], + "methods": {}, + "mixins": { + "ServicePersonalizationMixin" : true + } +} + +``` +#### 4. Insert rules into the `PersonalizationRule` model. + +Example: +```json +{ + "disabled" : false, + "modelName" : "ProductCatalog", + "personalizationRule" : { + "fieldValueReplace" : { + "keywords" : { + "Alpha" : "A", + "Bravo" : "B" + } + } + }, + "scope" : { + "device" : "mobile" + } +} +``` + +The above example adds a `fieldValueReplace` operation to the +`keywords` property of `ProductCatalog` model. Additionally +we have provided `scope` for this rule to take effect only +when it is specified in the http headers of the request; it +is always a simple key/value pair. See section `Supported operations` +for info about more operations. + +To have this personalization apply only to a user of a specified +role (say to either `tellers` or `agents`), it must be defined as in the below example: + +Example: +```json +{ + "disabled" : false, + "modelName" : "ProductCatalog", + "personalizationRule" : { + "fieldValueReplace" : { + "keywords" : { + "Alpha" : "A", + "Bravo" : "B" + } + } + }, + "scope" : { + "roles" : ["teller", "agent"] + } +} +``` + + +#### 5. _(Optional)_ Configure custom functions path + +If there are custom function operations, add the path in the application's `config.json` +file. Alternatively, set the environment variable: +`custom_function_path` + +Example: (`config.json` snippet): + +```json + "servicePersonalization" : { + "customFunctionPath": "D:\\Repos\\oecloud.io\\oe-service-personalization_master\\test\\customFunction" + } +``` +Example: (via environment variable) (bash prompt): +```bash +$ export custom_function_path="/project/customFuncDir" +``` +> Note: the full path to the directory is required. + +## Working Principle + +All models with the `ServicePersonalizationMixin` +enabled will have functions attached +to its `afterRemote()` and `beforeRemote()` which will +do the personalization. + +The personalization records, stored in `PersonalizationRule`, +are queried according to scope, and, model participating in +the remote call. The information required for this +is obtained from the `HttpContext` which is an argument in +the callback of the aforementioned methods. + +After this, these steps are done: + +1. personalization is applied at the root model, i.e. the +one that participates in the remote call. +2. personalization is then applied to any relations +3. personalization is applied to all properties +of the root model, which are model constructors + +Personalization can happen through code also via `performServicePersonalization()` +or, `applyServicePersonalization()` +api calls. They follow the same process mentioned above, +however, there are a few limitations. Notably, those +operations which are meant to apply `post-fetch` will +be honoured, and, thus personalized. + +More details on `pre-fetch` and `post-fetch` in sections below. (_Significance of pre-fetch & post-fetch_) + +## Supported operations + +To keep this document brief, a short description about each +operation is only given. Please visit the tests to see +example usages. It is recommended to review the tests synopsis +section if debugging the tests are necessary. + +The one and only test file can be found in this project's +folder here: +``` +./test/test.js +``` + +Below is the list of all supported operations and their +corresponding tests: + +| Operation | Description | Aspect | Tests | +|--------------------|---------------------------------------------------------------------------------------------------------------|----------------------|---------------------------------------| +| lbFilter | This applies a loopback filter to the request; it can contain an _include_ clause or a _where_ clause. | pre-fetch | t21 | +| filter | Same as above, but only adds the where clause to the request i.e. a query-based filter | pre-fetch | t9 | +| sort | Performs a sort at the datasource level | pre-fetch | t4, t5, t6, t7, t8, t10, t11 | +| fieldReplace | Replaces the property name in the data with another text. (Not its value) | pre-fetch/post-fetch | t1, t15, t17 | +| fieldValueReplace | Replaces the property value in the data | pre-fetch/post-fetch | t22, t20, t19, t18, t17, t16, t3, t23 | +| fieldMask | Masks value in the field according to a regex pattern. More details in the section of fieldMask | post-fetch | t24, t25, t26, t27, t28, t29 | +| mask | Hides a field in the response | pre-fetch | t13 | +| postCustomFunction | Adds a custom function which can add desired customization to response. Please see step #5 in how to use. | post-fetch | t35, t36 | +| preCustomFunction | Adds a custom function which can add desired customization to the request. Please see step #5 in how to use. | pre-fetch | t35, t36 | + + +## **fieldMask** options + +Prior to version 2.4.0, a field mask definition looks like this: + +```json +{ + "modelName": "ProductCatalog", + "personalizationRule": { + "fieldMask": { + "modelNo": { + "pattern": "([0-9]{3})([0-9]{3})([0-9]{4})", + "maskCharacter": "X", + "format": "($1) $2-$3", + "mask": [ + "$3" + ] + } + } + }, + "scope": { + "region": "us" + } +} +``` + +This is still supported. The framework assumes +`modelNo` in this example to be of type `String` +and performs validation before insert into +`PersonalizationRule` model. + +The **fieldMask** operations can be applied to the following data types: +- String +- Number +- Date + +Validation will happen for the same at the time of creating +the PersonalzationRule record, i.e., type validation on the +field will take place against the same field in the target +model. + +### fieldMask for strings + +Formal way to specify masking data of type `String` is as follows: + +```json +{ + "modelName": "ProductCatalog", + "personalizationRule": { + "fieldMask": { + "modelNo": { + "stringMask" : { + "pattern": "([0-9]{3})([0-9]{3})([0-9]{4})", + "maskCharacter": "X", + "format": "($1) $2-$3", + "mask": [ + "$3" + ] + } + + } + } + }, + "scope": { + "region": "us" + } +} +``` + +### fieldMask for numbers + +Formal way to specify masking of numbers is as follows: +```json +{ + "modelName": "ProductCatalog", + "personalizationRule": { + "fieldMask": { + "modelNo": { + "numberMask" : { + "pattern": "([0-9]{3})([0-9]{3})([0-9]{4})", + "maskCharacter": "X", + "format": "($1) $2-$3", + "mask": [ + "$3" + ] + } + + } + } + }, + "scope": { + "region": "us" + } +} +``` + +> Note: the options are similar to that of `stringMask`. +Validation is done to determine if modelNo is of type `Number` + +### fieldMask for date + +Formal way to specify masking of dates are as follows: +```json +{ + "modelName": "XCustomer", + "personalizationRule": { + "fieldMask": { + "dob": { + "dateMask": { + "format": "MMM/yyyy" + } + } + } + } +} +``` +The `format` in a `dateMask` field accepts any valid joda-time string. It is also +assumed to be of the `en_us` locale by default. Characters +intended for masking can be embedded in the format string itself, +however, they are a limited set, as, certain commonly used +characters like `x` or `X` have special meaning in the joda +standard. + +For more info about joda-time format visit: https://js-joda.github.io/js-joda/manual/formatting.html#format-patterns + +See note on supporting other locales. +( `Additional Locale support for dates` ) + +## Operations on objects + +Operations such as `fieldMask`, `fieldValueReplace`, etc can be +applied on properties of type `Object`. + +The path to the nested property can be specified by using +the unicode character `\uFF0E` as seperator. + +Example (test `t31`): + +```json +{ + "modelName": "Customer", + "personalizationRule": { + "fieldReplace": { + "billingAddress\uFF0Estreet": "lane" + } + }, + "scope": { + "device": "android" + } +} +``` + +## Programmatic API + +There are two flavours of api wrt personalization. Both are available in the following namespace: + +``` +oe-service-personalization/lib/api +``` + +### 1. Using model name, and, model data + +Use `performServicePersonalizations()` api. + +The signature is as follows: + +```js +/** + * Personalizes data by mutating it. + * + * The personalization rules are queried + * using the modelName. + * + * @param {string} modelName - name of model + * @param {*} data - Array or Object + * @param {ServicePersonalizationOptions} options - service personalization options + * Two properties: + * - isBeforeRemote - always false + * - context - the HttpContext object + * @param {function} cb - the function to signal completion + * Has only one arguments - error + * @returns {undefined} - nothing + */ +function performServicePersonalizations(modelName, data, options, cb) { + // ... +} +``` + +Example + +```JavaScript + +const { performServicePersonalizations } = require('./../../../lib/api'); // or require('oe-service-personalization/lib/api'); +const loopback = require('loopback'); + +module.exports = function(PseudoProductOwner) { + PseudoProductOwner.remoteMethod('demandchain', { + description: 'Gets the stores, store addresses, and, contacts of a product owner', + accepts: [ + { + arg: 'id', + type: 'number', + description: 'the unique id of the owner', + required: true + }, + { + arg: 'options', + type: 'object', + http:function(ctx) { + return ctx; + } + } + ], + returns: { + arg: 'chain', + root: true, + type: 'object' + }, + http: { path: '/:id/demandchain', verb: 'get' } + }); + + PseudoProductOwner.demandchain = function(ownerId, options, done) { + if(typeof done === 'undefined' && typeof options === 'function') { + done = options; + options = {}; + }; + let ProductOwner = loopback.findModel('ProductOwner'); + let filter = { + "include": [ + { + "ProductCatalog" : { + "store": { + "store" : { + "addresses" : "phones" + } + } + } + }, + "address" + ], + "where": { "id": ownerId } + }; + ProductOwner.findOne(filter, options, function(err, result) { + if(err) { + return done(err); + } + let persOptions = { + isBeforeRemote: false, + context: options + } + performServicePersonalizations(ProductOwner.definition.name, result, persOptions, function(err){ + done(err, result); + }) + }); + }; +} +``` + +> Note: the `options` in the remote method function +definition, in the example above is, `HttpContext` + +### 2. Using model name, data, and, personalization rules + +Use the `applyServicePersonalization()` api + +Signature: +```js +/** + * Api for personalization. Rules can + * be manually passed as arguments to + * this function. + * + * @param {string} modelName - the model name. + * @param {*} data - object or array + * @param {array} personalizationRecords - the personalization rule as an array. + * @param {object} options - personalization options + * @param {function} done - callback to signal completion. Takes only one argument - error. + * @returns {undefined} - nothing + */ + function applyServicePersonalization(modelName, data, personalizationRecords, options, done) { + // ... + } +``` + +## Significance of pre-fetch/post-fetch operations + +It impacts how personalizations is done for relations, and, nested +data. + +Operations are individual actions you can perform on data, such as +**fieldMask**, or, **fieldValueReplace**, etc. + +A `pre-fetch` operation is applied before data is fetched from +a loopback datasource. For e.g. **lbFilter**, **filter**, etc + +A `post-fetch` operation is carried out after data is fetched +from a loopback datasource. For e.g. **fieldMask** + +Due to the way loopback relations are +implemented, only operations that `post-fetch` are honoured. +This is also the case when using the programmatic api +for service personalization (regardless of whether relations +are accessed or not). + +## Points to consider + +1. Datasource support. Datasources can be service-oriented. +(Such as a web service). +Hence support for sorting, filtering, etc may be limited. +Therefore operations which pre-fetch may not give expected +results. + +2. Using custom functions (`postCustomFunction` or `preCustomFunction`). +Please honour the pipeline stage and use the correct +operation. No point in trying to modify `ctx.result` in a +`preCustomFunction`. Also ensure path to the directory where +the custom functions are stored is configured correctly. + +3. Understand how pre-fetch/post-fetch applies to relations and nested data. See section `Significance of pre-fetch/post-fetch operations` + +4. See section `Notes on loopback relations` + +## Test Synopsis + +The following entity structure and relationships assumed for most of the tests. + +``` + ++-------------------+ +------------------------+ +------------------------+ +| | | | | | +| AddressBook | | PhoneNumber | | ProductCatalog | +| | | | | | ++-------------------+ +------------------------+ +------------------------+ +| line1 : string | | number : string (PK)| | name : string | +| line2 : string | | firstName : string | | category : string | +| landmark : string | | lastName : string | | desc : string | +| pincode : string | | | | price : object | ++-------------------+ +------------------------+ | isAvailable : boolean | + | modelNo : string | + | keywords : [string] | + +------------------------+ + + ++-----------------+ +---------------+ +--------------------------------+ +| | | | | | +| ProductOwner | | Store | | StoreStock | +| | | | | | ++-----------------+ +---------------+ +--------------------------------+ +| name : string | | name : string | | storeId : string (FK) | +| city : string | | | | productCatalogId : string (FK) | ++-----------------+ +---------------+ +--------------------------------+ + + +========================================================================================================================== + + +--------------+ + +--------+ ProductOwner +----------+ + + +--------------+ + + (hasMany-ProductCatalog) (hasOne-address) + + + + v v + +-------+--------+ +----+----+ +-------------+ + +------------>+ ProductCatalog | | Address +-----+(hasMany-phones)+---> | PhoneNumber | + | +----------------+ +----+----+ +-------------+ + | ^ + | | +(belongsTo-product) + + | (hasMany-addresses) + | +-------+ + + | +-+(belongsTo-store)+->+ Store +---------------+ + | | +-------+ + | | + +-----+------+ | + | StoreStock +--+ + +------------+ + +``` + +Note: All the models have the `ServicePersonalizationMixin` enabled. + +The `test` folder is meant to emulate a small oe-cloud application. + +To run as an application server: + +``` +$ node test/server.js +``` + +It is also recommended to attach an explorer component (such as +loopback-component-explorer) when running as a standalone application. + +## Note on loopback relations + +The standard names for instance methods +commonly refer to a loopback relation. The names +are governed by the loopback framework. The pattern +goes something like this: + +E.g. consider a simple Customer/Order relationship. +Assume the following description of a `Customer` +model: + +* `Customer` + - relations + - `orders` + - type: `hasMany` + - model: `Order` + +Assumes a client invokes the following api (GET): +``` +http://localhost:3000/api/Customers/2/orders +``` +The loopback framework creates a `methodString` on the `HttpContext` +object as follows: +``` +Customer.prototype.__get__orders +``` + +If the requirement is such that, only _this_ api call +should be personalized, _create a personalization record_ for `Order` + +This should ensure the required result in the desired remote call. + +> Note: both models should have `ServicePersonalizationMixin` enabled + +A developer always has the freedom to define a non-static +instance method with the same name, and, still have the +relation defined. One must always refrain from doing this. + +Do not collude with loopback's +internal naming standards. + +## Additional Locale support for dates + +To enable additional locale support, we must modify this module dependencies. + +### Dynamic locale support + +For multi-locale support, we must do the following: + +1. Uninstall `@js-joda/locale_en-us` +2. Install the following: `@js-joda/locale`, `cldr-data`, `cldrjs` modules +3. In the `./lib/utils.js` file, edit the `Locale` import line; +import from the generic module like follows: + +``` +const { Locale } = require('@js-joda/locale'); +``` +4. In the `dateMask` configuration, specify `locale` property. + +Acceptable values (`String`) for the `locale` property are: +- ENGLISH +- US (_default_) +- UK +- CANADA +- FRENCH +- FRANCE +- GERMAN +- GERMANY + +> Warning: These steps could increase the size of `node_modules`. + +### Dynamic locale support (with limited locales) + +For supporting only a few locales in a `dateMask` operation, +we need to only install those locales and the generic `@js-joda/locale` module. The general pattern +of the locale module will be: + +``` +@js-joda/locale_ +``` + +Also perform steps `3` and `4` in the previous section. + +Refer to this documentation for acceptable `` values: + +https://github.com/js-joda/js-joda-locale#use-prebuilt-locale-packages diff --git a/common/mixins/service-personalization-mixin.js b/common/mixins/service-personalization-mixin.js new file mode 100644 index 0000000..c779030 --- /dev/null +++ b/common/mixins/service-personalization-mixin.js @@ -0,0 +1,55 @@ +/** + * + * ©2016-2020 EdgeVerve Systems Limited (a fully owned Infosys subsidiary), + * Bangalore, India. All Rights Reserved. + * + */ + +/** + * This mixin will attach beforeRemote and afterRemote + * hooks and decide if the data needs to be service + * personalized. + * + * Therefore, it is necessary to enable the mixin + * configuration on the corresponding model definition, + * even if it does not directly participate in the + * service personalization (viz is the case with any + * form of relations - or related models). + * + * This will only personalize data for the remote endpoints. + */ + + +const logger = require('oe-logger'); +const log = logger('service-personalization-mixin'); +const { runPersonalizations } = require('./../../lib/service-personalizer'); +const { slice } = require('./../../lib/utils'); + +module.exports = function ServicePersonalizationMixin(TargetModel) { + log.debug(log.defaultContext(), `Applying service personalization for ${TargetModel.definition.name}`); + TargetModel.afterRemote('**', function ServicePersonalizationAfterRemoteHook() { + let args = slice(arguments); + let ctx = args[0]; + let next = args[args.length - 1]; + // let callCtx = ctx.req.callContext; + log.debug(ctx, `afterRemote: (enter) MethodString: ${ctx.methodString}`); + runPersonalizations(ctx, false, function (err) { + log.debug(ctx, `afterRemote: (leave${err ? '- with error' : ''}) MethodString: ${ctx.methodString}`); + next(err); + }); + }); + + TargetModel.beforeRemote('**', function ServicePersonalizationBeforeRemoteHook() { + let args = slice(arguments); + let ctx = args[0]; + let next = args[args.length - 1]; + + log.debug(ctx, `beforeRemote: (enter) MethodString: ${ctx.methodString}`); + + // let ctxInfo = parseMethodString(ctx.methodString); + runPersonalizations(ctx, true, function (err) { + log.debug(ctx, `beforeRemote: (leave${err ? '- with error' : ''}) MethodString: ${ctx.methodString}`); + next(err); + }); + }); +}; diff --git a/common/models/personalization-rule.json b/common/models/personalization-rule.json index 10ed3d0..caa9507 100644 --- a/common/models/personalization-rule.json +++ b/common/models/personalization-rule.json @@ -28,10 +28,16 @@ "personalizationRule": { "type": "object", "required": true + }, + "methodName" : { + "type": "string", + "default": "**", + "description": "The model methodName this rule should apply to. Should be the methodName (static/instance) or wildcards you specify in a afterRemote()/beforeRemote(). Default '**'" } }, "validations": [], "relations": {}, "acls": [], - "methods": {} + "methods": {}, + "mixins": {} } \ No newline at end of file diff --git a/lib/api.js b/lib/api.js new file mode 100644 index 0000000..d463cb0 --- /dev/null +++ b/lib/api.js @@ -0,0 +1,25 @@ +const { performServicePersonalizations, applyServicePersonalization } = require('./service-personalizer'); +module.exports = { + /** + * Standard api for personalization + */ + performServicePersonalizations, + + /** + * Api for personalization. Rules can + * be manually passed as arguments to + * this function. + * + * @param {string} modelName - the model name. + * @param {*} data - object or array + * @param {array} personalizationRecords - the personalization rule as an array. + * @param {object} options - personalization options + * @param {function} done - callback to signal completion. Takes only one argument - error. + * @returns {undefined} - nothing + */ + applyServicePersonalization: function applyServicePersonalizationWrapper(modelName, data, personalizationRecords, options, done) { + let { context } = options; + context._personalizationCache = {}; + applyServicePersonalization(modelName, data, personalizationRecords, options, done); + } +}; diff --git a/lib/service-personalizer.js b/lib/service-personalizer.js index bf55e57..c6bcdfa 100755 --- a/lib/service-personalizer.js +++ b/lib/service-personalizer.js @@ -1,14 +1,14 @@ /** * - * ©2018-2019 EdgeVerve Systems Limited (a fully owned Infosys subsidiary), + * ©2018-2020 EdgeVerve Systems Limited (a fully owned Infosys subsidiary), * Bangalore, India. All Rights Reserved. * */ /** * Service personalization module. Optimizes and applies one personalization function. * - * @module EV Service Personalizer - * @author Atul Pandit, gourav_gupta, pradeep_tippa + * @module oe-service-personalization/lib/service-personalizer + * @author Atul Pandit, gourav_gupta, pradeep_tippa, arun_jayapal (aka deostroll) */ var loopback = require('loopback'); @@ -20,6 +20,20 @@ var logger = require('oe-logger'); var log = logger('service-personalizer'); var customFunction; +// begin - task-import - import from utils +const { + nextTick, + parseMethodString, + createError, + formatDateTimeJoda, + isDate, + isString, + isObject, + isNumber, + validateMethodName, + REMOTES +} = require('./utils'); +// end - task-import /** * This function returns the necessary sorting logic to @@ -30,829 +44,1248 @@ var customFunction; * @returns {function} - comparator function to be used for sorting */ var sortFactoryFn = (reverse = false) => - (first, second) => first.type === 'fieldReplace' ? (reverse ? -1 : 1) : 0; + (first, second) => first.type === 'fieldReplace' ? (reverse ? -1 : 1) : (reverse ? 1 : -1); + +// execute custom function +function executeCustomFunction(ctx, instruction, cb) { + let customFunctionName = instruction.functionName; + customFunction[customFunctionName](ctx); + cb(); +} /** - * - * This function returns personalization rule for modelName if exists. - * - * @param {String} modelName - Model Name - * @param {object} ctx - context - * @param {callback} callback - callback function - * @function - * @name getPersonalizationRuleForModel + * Custom function */ -var getPersonalizationRuleForModel = function getPersonalizationRuleForModelFn(modelName, ctx, callback) { - log.debug(ctx.req.callContext, 'getPersonalizationRuleForModel called for model - ', modelName); +// getCustom function +function loadCustomFunction(fnCache) { + customFunction = fnCache; +} + +function getCustomFunction() { + return customFunction; +} - var PersonalizationRule = loopback.findModel('PersonalizationRule'); - var findByModelNameQuery = { - where: { - modelName: modelName, - disabled: false - } - }; +// (Arun 2020-04-28 21:01:57) - retaining the below function for future use - PersonalizationRule.find(findByModelNameQuery, ctx.req.callContext, function getPersonalizationRuleForModelFindCb(err, result) { - log.debug(ctx.req.callContext, 'Query result = ', result); - if (err) { - // TODO: Error getting personalization rule.. what should be done? Continue or stop? - log.debug(ctx.req.callContext, 'Error getting personalization rule for model [', modelName, ']. skipping personalization'); - return callback(null); +/* To be used when database doesnt support sort OR sort needs to be done in memory*/ +// function sortInMemory(ctx, options) { +// var result = ctx.result; +// if (typeof result === 'undefined') { +// return; +// } +// if (!Array.isArray(options)) { +// options = [options]; +// } +// var keys = []; +// var values = []; +// for (var index in options) { +// if (options.hasOwnProperty(index)) { +// var key = Object.keys(options[index])[0]; +// values.push(options[index][key]); +// key = key.indexOf('|') > -1 ? key.replace(/\|/g, '.') : key; +// keys.push(key); +// } +// } +// var updatedResults; +// if (Array.isArray(result)) { +// // lodash version 3.10.1 uses sortByOrder;version 4.0.0 uses OrderBy +// updatedResults = _.orderBy(result, keys, values); +// } +// if (updatedResults) { +// ctx.result = updatedResults; +// } +// } + + +// begin - task-utils - utility functions +const utils = { + + /** + * field replacer function + * + * @param {instance} record - the model instance or plain data + * @param {object} replacement - personalization rule (for field name replace) + * @param {string} value - new field name + * @returns {void} nothing + */ + replaceField(record, replacement, value) { + var pos = replacement.indexOf('\uFF0E'); + var key; + var elsePart; + if (pos !== -1) { + key = replacement.substr(0, pos); + elsePart = replacement.substr(pos + 1); + } else { + key = replacement; } - if (result && result.length > 0) { - log.debug(ctx.req.callContext, 'Returning personzalition rule'); - return callback(result[0]); + if (typeof record[key] !== 'undefined' && typeof record[key] === 'object') { + utils.replaceField(record[key], elsePart, value); + } else if (typeof record[key] !== 'undefined' && typeof record[key] !== 'object') { + if (record[key]) { + if (typeof record.__data !== 'undefined') { + record.__data[value] = record[key]; + delete record.__data[key]; + } else { + record[value] = record[key]; + delete record[key]; + } + } } - log.debug(ctx.req.callContext, 'Personalization rules not defined for model [', modelName, ']. skipping personalization'); - return callback(null); - }); -}; + }, -/** - * - * This function add functions to an array postProcessingFunctions which will execute to - * apply personalization rules after getting result. - * - * @param {Object} ctx - loopback context - * @param {Object} p13nRule - Personalization Rule - * @param {callback} callback - callback function - * @function - * @name applyPersonalizationRule - */ + /** + * field value replace function + * + * @param {instance} record - the model instance or data + * @param {object} replacement - the personalization rule (for field value replace) + * @param {string} value - the value to replace + * @returns {void} nothing + */ + replaceValue(record, replacement, value) { + var pos = replacement.indexOf('\uFF0E'); + var key; + var elsePart; + if (pos !== null && typeof pos !== 'undefined' && pos !== -1) { + key = replacement.substr(0, pos); + elsePart = replacement.substr(pos + 1); + } else { + key = replacement; + } -var applyPersonalizationRule = function applyPersonalizationRuleFn(ctx, p13nRule) { - var arr = []; - log.debug(ctx.req.callContext, 'applying Personalizing ctx with function - ', p13nRule); + if (typeof record[key] !== 'undefined' && Array.isArray(record[key])) { + var newValue = record[key]; + record[key].forEach(function (element, index) { + if (value[element]) { + newValue[index] = value[element]; + } + }); + if (typeof record.__data !== 'undefined') { + record.__data[key] = newValue; + } else { + record[key] = newValue; + } + } else if (typeof record[key] !== 'undefined' && typeof record[key] === 'object') { + utils.replaceValue(record[key], elsePart, value); + } else if (typeof record[key] !== 'undefined' && typeof record[key] !== 'object') { + if (value.hasOwnProperty(record[key])) { + if (typeof record.__data !== 'undefined') { + record.__data[key] = value[record[key]]; + } else { + record[key] = value[record[key]]; + } + } + } + }, + + replaceRecordFactory(fn) { + return function replaceRecord(record, replacements) { + var keys = Object.keys(JSON.parse(JSON.stringify(replacements))); + for (var attr in keys) { + if (keys.hasOwnProperty(attr)) { + fn(record, keys[attr], replacements[keys[attr]]); + } + } + return record; + }; + }, - var instructions = Object.keys(p13nRule); + noop() { + // do nothing + }, - // TODO:Check if all instructions can be applied in parallel in asynch way. - // instructions.forEach(function (instruction) { + // === BEGIN: query based filter functions === - for (var i in instructions) { - if (instructions.hasOwnProperty(i)) { - var instruction = instructions[i]; - switch (instruction) { - case 'lbFilter': - arr.push({ - type: 'lbFilter', - fn: async.apply(addLbFilter, ctx, p13nRule[instruction]) - }); - // arr.push(async.apply(addLbFilter, ctx, p13nRule[instruction])); - break; - case 'filter': - arr.push({ - type: 'filter', - fn: async.apply(addFilter, ctx, p13nRule[instruction]) - }); - // arr.push(async.apply(addFilter, ctx, p13nRule[instruction])); - break; - case 'fieldReplace': - arr.push({ - type: 'fieldReplace', - fn: async.apply(addFieldReplace, ctx, p13nRule[instruction]) - }); - // arr.push(async.apply(addFieldReplace, ctx, p13nRule[instruction])); - break; - case 'fieldValueReplace': - arr.push({ - type: 'fieldValueReplace', - fn: async.apply(addFieldValueReplace, ctx, p13nRule[instruction]) - }); - // arr.push(async.apply(addFieldValueReplace, ctx, p13nRule[instruction])); - break; - case 'sort': - arr.push({ - type: 'sort', - fn: async.apply(addSort, ctx, p13nRule[instruction]) - }); - // arr.push(async.apply(addSort, ctx, p13nRule[instruction])); - break; - case 'postCustomFunction': - arr.push({ - type: 'postCustomFunction', - fn: async.apply(executeCustomFunction, ctx, p13nRule[instruction]) - }); - break; - case 'union': - // unionResults(ctx, p13nRule[instruction]); - break; - case 'mask': - arr.push({ - type: 'mask', - fn: async.apply(maskFields, ctx, p13nRule[instruction]) - }); - // arr.push(async.apply(maskFields, ctx, p13nRule[instruction])); - break; - case 'fieldMask': - arr.push({ - type: 'fieldMask', - fn: async.apply(maskCharacters, ctx, p13nRule[instruction]) - }); - // arr.push(async.apply(maskCharacters, ctx, p13nRule[instruction])); - break; - default: + addWhereClause(ctx, instruction, cb) { + if (typeof instruction === 'string') { + exprLang(instruction).then(function addWhereClauseInstrResultCb(result) { + utils.addWheretoCtx(ctx, result.value.where, cb); + }); + } else { + utils.addWheretoCtx(ctx, instruction, cb); + } + }, + + addWheretoCtx(ctx, where, cb) { + var filter = ctx.args.filter; + // Shall we directly use mergeQuery util.? + if (filter) { + if (typeof filter === 'string') { + var filterQuery = JSON.parse(filter); + if (filterQuery && filterQuery.where) { + // not sure and | or ?. or will give more results, and will give none. + var newQuery = { + or: [where, filterQuery.where] + }; + filter = filterQuery; + filter.where = newQuery; + } else { + filter.where = where; + } + } else { + filter.where = { or: [where, filter.where] }; } + } else { + filter = {}; + filter.where = where; } - } - return arr.sort(sortFactoryFn()); -}; + ctx.args.filter = filter; + cb(); + }, -function execute(arr, callback) { - async.parallel(arr, function applyPersonalizationRuleAsyncParallelFn(err, results) { - if (err) { - return callback(err); + // === END: query based filter functions === + + // === BEGIN: lbFilter functions === + + // Processes a filter instruction. filter instruction schema is same like loopback filter schema. + addLbFilter(ctx, instruction, cb) { + utils.addLbFilterClause(ctx, instruction, cb); + }, + + /** + * Function to add filter clause in query. + */ + + addLbFilterClause(ctx, instruction, cb) { + if (typeof instruction === 'string') { + exprLang(instruction).then(function addLbFilterInstrResultCb(result) { + utils.addLbFiltertoCtx(ctx, result.value, cb); + }); + } else { + utils.addLbFiltertoCtx(ctx, instruction, cb); } - callback(); - }); -} + }, -function addFieldValueReplace(ctx, instruction, cb) { - return fieldValueReplacementFn(ctx, instruction, cb); -} + addLbFiltertoCtx(ctx, filter, cb) { + ctx.args.filter = ctx.args.filter || {}; + mergeQuery(ctx.args.filter, filter); + cb(); + }, + // === END: lbFilter functions === -/* eslint-disable no-loop-func */ -var applyReversePersonalizationRule = function applyReversePersonalizationRuleFn(ctx, p13nRule, callback) { - log.debug(ctx.options, 'Reverse Personalizing ctx with function - ', p13nRule); - var instructions = Object.keys(p13nRule); + createOrderExp(instruction, tempKeys) { + if (!Array.isArray(instruction)) { + instruction = [instruction]; + } - var arr = []; - // TODO:Check if all instructions can be applied in parallel in asynch way. - // instructions.forEach(function (instruction) { + var orderExp = []; - for (var i in instructions) { - if (instructions.hasOwnProperty(i)) { - var instruction = instructions[i]; - switch (instruction) { - case 'fieldReplace': - arr.push({ - type: 'fieldReplace', - fn: async.apply(addReverseFieldReplace, ctx, p13nRule[instruction]) - }); - // arr.push(async.apply(addReverseFieldReplace, ctx, p13nRule[instruction])); - break; - case 'fieldValueReplace': - arr.push({ - type: 'fieldValueReplace', - fn: async.apply(addReverseFieldValueReplace, ctx, p13nRule[instruction]) - }); - // arr.push(async.apply(addReverseFieldValueReplace, ctx, p13nRule[instruction])); + for (var i = 0; i < instruction.length; i++) { + var obj = instruction[i]; + var key = Object.keys(obj)[0]; + var val = obj[key]; + key = key.indexOf('|') > -1 ? key.replace(/\|/g, '.') : key; + + var index = tempKeys.length >= 1 ? tempKeys.indexOf(key) : -1; + + switch (val.toUpperCase()) { + case 'ASC': + case 'ASCENDING': + case '': + val = 'ASC'; break; - case 'preCustomFunction': - arr.push({ - type: 'preCustomFunction', - fn: async.apply(executeCustomFunction, ctx, p13nRule[instruction]) - }); + case 'DESC': + case 'DESCENDING': + case 'DSC': + val = 'DESC'; break; default: + val = null; + } + if (val && index === -1) { + var value = key + ' ' + val; + orderExp.push(value); } } + + return orderExp; + }, + + getMaskedString(stringRule, value) { + let { pattern, flags, format, mask, maskCharacter } = stringRule; + let char = maskCharacter || 'X'; + let rgx = flags ? new RegExp(pattern, flags) : new RegExp(pattern); + let groups = value.match(rgx); + let masking = mask || []; + // eslint-disable-next-line no-inline-comments + let newVal = format || []; // format can be an array or string + if (Array.isArray(newVal)) { + for (let i = 1, len = groups.length; i < len; i++) { + newVal.push(`$${i}`); + } + newVal = newVal.join(''); + } + masking.forEach(function (elem) { + newVal = newVal.replace(elem, new Array(groups[elem.substr(1)].length + 1).join(char)); + }); + for (let i = 0; i < groups.length; i++) { + newVal = newVal.replace('$' + i, groups[i]); + } + return newVal; + }, + + getMaskedDate(rule, value) { + let { format, locale } = rule; + return formatDateTimeJoda(value, format, locale); } - return arr.sort(sortFactoryFn(true)).map(x => x.fn); }; -/* eslint-enable no-loop-func */ +// end - task-utils - utility functions -function addReverseFieldValueReplace(ctx, instruction, cb) { - reverseFieldValueReplacementFn(ctx, instruction, cb); -} +// begin task-p13nFunctions -function addReverseFieldReplace(ctx, instruction, cb) { - reverseFieldReplacementFn(ctx, instruction, cb); -} +const p13nFunctions = { + /** + * Does field replace. + * + * Pre-Fetch: Yes + * Post-Fetch: Yes + * + * For pre-appilication a reverse rule is applied. + * @param {object} replacements + * @param {boolean} isBeforeRemote + */ + // eslint-disable-next-line no-inline-comments + fieldReplace(replacements, isBeforeRemote = false) { // Tests: t1, t15, t17 + let replaceRecord = utils.replaceRecordFactory(utils.replaceField); + + let execute = function (replacements, data, cb) { + if (Array.isArray(data)) { + let updatedResult = data.map(record => { + return replaceRecord(record, replacements); + }); + data = updatedResult; + } else { + data = replaceRecord(data, replacements); + } -/** - * Function to add 'where' clause in the datasource filter query. - */ + nextTick(cb); + }; + + if (isBeforeRemote) { + return function (data, callback) { + var revInputJson = {}; + var rule = replacements; + for (var key in rule) { + if (rule.hasOwnProperty(key)) { + var pos = key.lastIndexOf('\uFF0E'); + if (pos !== -1) { + var replaceAttr = key.substr(pos + 1); + var elsePart = key.substr(0, pos + 1); + revInputJson[elsePart + rule[key]] = replaceAttr; + } else { + revInputJson[rule[key]] = key; + } + } + } + // fieldReplacementFn(ctx, revInputJson, cb); + execute(revInputJson, data, callback); + }; + } -function addWhereClause(ctx, instruction, cb) { - if (typeof instruction === 'string') { - exprLang(instruction).then(function addWhereClauseInstrResultCb(result) { - addWheretoCtx(ctx, result.value.where, cb); - }); - } else { - addWheretoCtx(ctx, instruction, cb); - } -} + // ! for afterRemote case + return function (data, callback) { + execute(replacements, data, callback); + }; + }, -function addWheretoCtx(ctx, where, cb) { - var filter = ctx.args.filter; - // Shall we directly use mergeQuery util.? - if (filter) { - if (typeof filter === 'string') { - var filterQuery = JSON.parse(filter); - if (filterQuery && filterQuery.where) { - // not sure and | or ?. or will give more results, and will give none. - var newQuery = { - or: [where, filterQuery.where] - }; - filter = filterQuery; - filter.where = newQuery; + /** + * does a field value replace. + * + * Pre-Fetch: no + * Post-Fetch: yes + * + * @param {object} replacements - replacement rule + */ + // eslint-disable-next-line no-inline-comments + fieldValueReplace(replacements) { // Tests: t22, t20, t19, t18, t17, t16, t3, t23 + return function (data, callback) { + let replaceRecord = utils.replaceRecordFactory(utils.replaceValue); + if (Array.isArray(data)) { + let updatedResult = data.map(record => { + return replaceRecord(record, replacements); + }); + data = updatedResult; } else { - filter.where = where; + data = replaceRecord(data, replacements); } - } else { - filter.where = { or: [where, filter.where] }; - } - } else { - filter = {}; - filter.where = where; - } - ctx.args.filter = filter; - cb(); -} -/** - * Function to add filter clause in query. - */ + nextTick(callback); + }; + }, -function addLbFilterClause(ctx, instruction, cb) { - if (typeof instruction === 'string') { - exprLang(instruction).then(function addLbFilterInstrResultCb(result) { - addLbFiltertoCtx(ctx, result.value, cb); - }); - } else { - addLbFiltertoCtx(ctx, instruction, cb); - } -} + noop: function (data, cb) { + utils.noop(data); + nextTick(cb); + }, -function addLbFiltertoCtx(ctx, filter, cb) { - ctx.args.filter = ctx.args.filter || {}; - mergeQuery(ctx.args.filter, filter); - cb(); -} + /** + * Apply a sort. Mostly passed on to the Model.find() + * where the actual sorting is applied. (Provided the + * underlying datasource supports it) + * + * Pre-Fetch: Yes + * Post-Fetch: No + * + * @param {HttpContext} ctx - the context object + * @param {object} instruction - the personalization sort rule + */ + // eslint-disable-next-line no-inline-comments + addSort(ctx, instruction) { // Tests: t4, t5, t6, t7, t8, t10, t11 + return function (data, callback) { + utils.noop(data); + var dsSupportSort = true; + if (dsSupportSort) { + var query = ctx.args.filter || {}; + // TODO: (Arun 2020-04-24 19:43:38) - what if no filter? + if (query) { + if (typeof query.order === 'string') { + query.order = [query.order]; + } -/* - * Object wrapper to add a processing function in the context. - * Wraps the instrunctions and reference to the actual function to be called. - * Processing functions are invoked before posting data or after data is retried by the API - * - */ + var tempKeys = []; -function ProcessingFunction(instruction, fn) { - this.instruction = instruction; - this.fn = fn; + if (query.order && query.order.length >= 1) { + query.order.forEach(function addSortQueryOrderForEachFn(item) { + tempKeys.push(item.split(' ')[0]); + }); + } - this.execute = function processingFunctionExecuteFn(ctx) { - // console.log('this.instruction = ' + JSON.stringify(this.instruction)); - // console.log('this.fn = ' + this.fn); + // create the order expression based on the instruction passed + var orderExp = utils.createOrderExp(instruction, tempKeys); - this.fn(ctx, instruction); - }; -} + if (typeof query.order === 'undefined') { + query.order = orderExp; + } else { + query.order = query.order.concat(orderExp); + } + query.order = _.uniq(query.order); + ctx.args.filter = ctx.args.filter || {}; + ctx.args.filter.order = query.order; + nextTick(callback); + } + } + /** + * TODO: (Arun 2020-04-24 19:37:19) we may need to enable + * the below lines if the datasource in consideration is + * service-oriented, like a web service (for e.g.), and, + * not a traditional db like mongo or postgres + */ -function fieldReplacementFn(ctx, replacements, cb) { - var input; - var result = ctx.result || ctx.accdata; + // else { + // addPostProcessingFunction(ctx, 'sortInMemory', instruction, sortInMemory); + // cb(); + // } + }; + }, - if (typeof result !== 'undefined' && !_.isEmpty(result)) { - input = ctx.result || ctx.accdata; - log.debug(ctx.options, 'fieldValueReplacementFn called. Resultset = ', - input + ' Replacements = ' + replacements); - } else if (typeof ctx.req.body !== 'undefined' && !_.isEmpty(ctx.req.body)) { - input = ctx.req.body; - log.debug(ctx.req.callContext, 'reverseFieldValueReplacementFn called. Input = ', - input + ' Replacements = ' + replacements); - } else { - return cb(); - } + /** + * Mask helper function for masking a field + * + * @param {CallContext} ctx - the request context + * @param {object} instructions - personalization rule object + * + * Pre-Fetch: No + * Post-Fetch: Yes + * + * Example Rule - Masks a "category field" + * + * var rule = { + * "modelName": "ProductCatalog", + * "personalizationRule" : { + * "mask" : { + * "category": true + * } + * } + * }; + */ + // eslint-disable-next-line no-inline-comments + mask(ctx, instructions) { // Tests: t13 + return function (data, callback) { + utils.noop(data); + let dsSupportMask = true; + if (dsSupportMask) { + ctx.args.filter = ctx.args.filter || {}; + let query = ctx.args.filter; + // TODO: (Arun - 2020-04-24 11:16:19) Don't we need to handle the alternate case? + // i.e. when there is no filter ? + if (!query) { + return nextTick(callback); + } + let keys = Object.keys(instructions); + let exp = {}; + if (typeof query.fields === 'undefined') { + for (var i = 0, length = keys.length; i < length; i++) { + var key = keys[i]; + key = key.indexOf('|') > -1 ? key.replace(/\|/g, '.') : key; + exp[key] = false; + } + query.fields = exp; + } - // replace field function to replace record wrt replacement object and value - function replaceField(record, replacement, value) { - var pos = replacement.indexOf('\uFF0E'); - var key; - var elsePart; - if (pos !== null && pos !== 'undefined' && pos !== -1) { - key = replacement.substr(0, pos); - elsePart = replacement.substr(pos + 1); - } else { - key = replacement; - } + // TODO: (Arun - 2020-04-24 11:17:11) shouldn't we uncomment the following? - if (record[key] !== 'undefined' && typeof record[key] === 'object') { - replaceField(record[key], elsePart, value); - } else if (record[key] !== 'undefined' && typeof record[key] !== 'object') { - if (record[key]) { - if (typeof record.__data !== 'undefined') { - record.__data[value] = record[key]; - delete record.__data[key]; - } else { - record[value] = record[key]; - delete record[key]; - } + // else { + // var fieldList = query.fields; + // fieldList = _.filter(fieldList, function (item) { + // return keys.indexOf(item) === -1 + // }); + // query.fields = fieldList; + // } } - } - } - function replaceRecord(record, replacements) { - var keys = Object.keys(JSON.parse(JSON.stringify(replacements))); - for (var attr in keys) { - if (keys.hasOwnProperty(attr)) { - replaceField(record, keys[attr], replacements[keys[attr]]); - } - } - return record; - } + nextTick(callback); + }; + }, /** - * if input or result is array then iterates the process - * otherwise once calls update record function. + * add a filter to Model.find() + * + * Pre-Fetch: yes + * Post-Fetch: no + * + * @param {HttpContext} ctx - http context + * @param {object} instruction - personalization filter rule */ - if (Array.isArray(input)) { - var updatedResult = []; - for (var i in input) { - if (input.hasOwnProperty(i)) { - var record = input[i]; - updatedResult.push(replaceRecord(record, replacements)); + // eslint-disable-next-line no-inline-comments + addFilter(ctx, instruction) {// Tests: t9 + return function (data, callback) { + utils.noop(data); + var dsSupportFilter = true; + if (dsSupportFilter) { + utils.addWhereClause(ctx, instruction, callback); } - } - input = updatedResult; - } else { - var updatedRecord = replaceRecord(input, replacements); - input = updatedRecord; - } + }; + }, - process.nextTick(function () { - return cb(); - }); -} - -function reverseFieldReplacementFn(ctx, rule, cb) { - // var input = ctx.args.data; + /** + * Adds a loopback filter clause + * + * @param {HttpContext} ctx - context + * @param {object} instruction - the rule + */ + // eslint-disable-next-line no-inline-comments + addLbFilter(ctx, instruction) { // Tests: t21 + return function (data, callback) { + utils.noop(data); + utils.addLbFilter(ctx, instruction, callback); + }; + }, - if (rule !== null && typeof rule !== 'undefined') { - var revInputJson = {}; + /** + * applies masking values in a field. E.g + * masking first few digits of the phone + * number + * + * @param {object} instruction - mask instructions + */ + // eslint-disable-next-line no-inline-comments + addFieldMask(charMaskRules) { // Test t24, t25, t26, t27, t28, t29 + return function (data, callback) { + var input = data; - for (var key in rule) { - if (rule.hasOwnProperty(key)) { - var pos = key.lastIndexOf('\uFF0E'); + function modifyField(record, property, rule) { + var pos = property.indexOf('.'); if (pos !== -1) { - var replaceAttr = key.substr(pos + 1); - var elsePart = key.substr(0, pos + 1); - revInputJson[elsePart + rule[key]] = replaceAttr; + var key = property.substr(0, pos); + var innerProp = property.substr(pos + 1); } else { - revInputJson[rule[key]] = key; + key = property; + } + + let oldValue = record[key]; + + if (isString(oldValue) && 'stringMask' in rule) { + record[key] = utils.getMaskedString(rule.stringMask, oldValue); + } else if (isDate(oldValue) && 'dateMask' in rule) { + record[`$${key}`] = utils.getMaskedDate(rule.dateMask, oldValue); + } else if (isObject(oldValue)) { + modifyField(oldValue, innerProp, rule); + } else if (isNumber(oldValue) && 'numberMask' in rule) { + record[`$${key}`] = utils.getMaskedString(rule.numberMask, String(oldValue)); + } else if (isString(oldValue)) { + record[key] = utils.getMaskedString(rule, oldValue); } } - } - fieldReplacementFn(ctx, revInputJson, cb); - } else { - return cb(); - } -} -/** - * Field value replacement function. To be used when datasource does not support field value replacements. - * It simply iterates over the resultset and carries our field value replacements. - * - * @param {Object} ctx - loopback context - * @param {Object} replacements - field value replacement rule - * @function - * @name fieldValueReplacementFn - */ + function applyRuleOnRecord(record, charMaskRules) { + Object.keys(charMaskRules).forEach(function (key) { + modifyField(record, key, charMaskRules[key]); + }); + return record; + } + if (Array.isArray(input)) { + var updatedResult = []; + input.forEach(function (record) { + updatedResult.push(applyRuleOnRecord(record, charMaskRules)); + }); + input = updatedResult; + } else { + input = applyRuleOnRecord(input, charMaskRules); + } -/** - * Reverse Field value replacement function. To be used for reverting field value replacements. - * It simply iterates over the posted data and reverts field value replacements. - * - * @param {Object} ctx - loopback context - * @param {Object} rule - field value replacement rule - * @function - * @name reverseFieldValueReplacementFn - */ + // return cb(); + nextTick(callback); + }; + }, -function reverseFieldValueReplacementFn(ctx, rule, cb) { - // var input = ctx.args.data; - - if (rule !== null && typeof rule !== 'undefined') { - var revInputJson = {}; - for (var field in rule) { - if (rule.hasOwnProperty(field)) { - var temp = {}; - var rf = rule[field]; - for (var key in rf) { - if (rf.hasOwnProperty(key)) { - temp[rf[key]] = key; - } - } - revInputJson[field] = temp; + /** + * adds the post custom function added via + * config.json to the after remote + * + * Pre-Fetch: no + * Post-Fetch: yes + * @param {context} ctx + * @param {object} instruction + * @returns {function} function that applies custom function (async iterator function) + */ + // eslint-disable-next-line no-inline-comments + addCustomFunction(ctx, instruction) { // Tests t35, t36 + return function (data, callback) { + utils.noop(data); + executeCustomFunction(ctx, instruction, () => nextTick(callback)); + }; + } +}; + +// end task-p13nFunctions + +/** + * apply the personalization on the given data + * @param {object} ctx - http context object + * @param {bool} isBeforeRemote - flag indicating if this is invoked in a beforeRemote stage + * @param {object} instructions - object containing personalization instructions + * @param {*} data - data from the context. Could either be a List or a single model instance + * @param {function} done - the callback which receives the new data. First argument of function is an error object + * @returns {void} nothing + */ +function personalize(ctx, isBeforeRemote, instructions, data, done) { + let tasks = null; + if (isBeforeRemote) { + tasks = Object.entries(instructions).map(([operation, instruction]) => { + switch (operation) { + case 'mask': + return { type: 'mask', fn: p13nFunctions.mask(ctx, instruction) }; + case 'sort': + return { type: 'sort', fn: p13nFunctions.addSort(ctx, instruction) }; + case 'filter': + return { type: 'filter', fn: p13nFunctions.addFilter(ctx, instruction) }; + case 'fieldReplace': + return { type: 'fieldReplace', fn: p13nFunctions.fieldReplace(instruction, true) }; + case 'lbFilter': + return { type: 'lbFilter', fn: p13nFunctions.addLbFilter(ctx, instruction) }; + case 'preCustomFunction': + return { type: 'preCustomFunction', fn: p13nFunctions.addCustomFunction(ctx, instruction) }; + default: + return { type: `noop:${operation}`, fn: p13nFunctions.noop }; } - } - fieldValueReplacementFn(ctx, revInputJson, cb); + }); } else { - return process.nextTick(function () { - return cb(); + tasks = Object.entries(instructions).map(([operation, instruction]) => { + switch (operation) { + case 'fieldReplace': + return { type: 'fieldReplace', fn: p13nFunctions.fieldReplace(instruction) }; + case 'fieldValueReplace': + return { type: 'fieldValueReplace', fn: p13nFunctions.fieldValueReplace(instruction) }; + case 'fieldMask': + return { type: 'fieldMask', fn: p13nFunctions.addFieldMask(instruction) }; + case 'postCustomFunction': + return { type: 'postCustomFunction', fn: p13nFunctions.addCustomFunction(ctx, instruction) }; + default: + return { type: `noop:${operation}`, fn: p13nFunctions.noop }; + } }); } -} -// old code -function executeCustomFunctionFn(ctx, customFunctionName) { - // TODO: Security check - // var custFn = new Function('ctx', customFunction); - var custFn = function customFnn(ctx, customFunction) { - customFunction(ctx); + let asyncIterator = function ({ type, fn }, done) { + log.debug(ctx, `${isBeforeRemote ? 'beforeRemote' : 'afterRemote'}: applying function - ${type}`); + fn(data, function (err) { + done(err); + }); }; - log.debug(ctx.options, 'function - ', customFunction); - custFn(ctx); -} - -// execute custom function -function executeCustomFunction(ctx, instruction, cb) { - let customFunctionName = instruction.functionName; - customFunction[customFunctionName](ctx); - cb(); + async.eachSeries(tasks.sort(sortFactoryFn(isBeforeRemote)), asyncIterator, function asyncEachCb(err) { + if (err) { + done(err); + } else { + done(); + } + }); } - /** - * Processes a 'filter' instruction. This method checks if underlying datasource to which the model is attached to - * supports query based filtering. If yes, it adds a 'where' clause in the query. Otherwise creates a post processing - * function which performs filtering on the resultset retrieved. - * @param {object} ctx - context. - * @param {object} instruction - instructions. - * @param {function} cb - callback function. + * Checks if the model has relations, and, + * also determines if data consists of + * relational data, then recursively personalizes + * @param {function} Model - loopback model constructor + * @param {*} data - data to personalize - object or array + * @param {object} personalizationOptions - standard options passed to applyServicePersonalization() + * @param {callback} done - to signal completion of task + * @returns {undefined} - nothing */ -function addFilter(ctx, instruction, cb) { - // TODO: Check the datasource to which this model is attached. - // If the datasource is capable of doing filter queries add a where clause. +function checkRelationAndRecurse(Model, data, personalizationOptions, done) { + let { settings: { relations }, definition: { name } } = Model; + let { isBeforeRemote, context } = personalizationOptions; + let prefix = isBeforeRemote ? 'beforeRemote' : 'afterRemote'; + let personalizationCache = context._personalizationCache; + let rootInfo = personalizationCache && personalizationCache.info; + + if (relations) { + // Object.entries(Model.setti) + let relationItems = Object.entries(relations); + let relationsIterator = function relationProcessor([relationName, relation], done) { + // check if the related model has personalization + let relData; + let relModel = relation.model; + let applyFlag = false; + let methodName = rootInfo && rootInfo.methodName; + + // begin - implicitRelationCheck - checking for implicit relation call + if (rootInfo && !rootInfo.isStatic && methodName.startsWith('__') && methodName.includes(relationName)) { + //! this is an implicit relation http call + // E.g GET /api/Customers/2/Orders + // Here data will be that of Order + // model. + relData = data; + applyFlag = true; + // end - implicitRelationCheck + } else if (Array.isArray(data)) { + relData = data.reduce((carrier, record) => { + if (record.__data && typeof record.__data[relationName] !== 'undefined') { + carrier.push(record.__data[relationName]); + } + return carrier; + }, []); + relData = _.flatten(relData); + if (relData.length) { + applyFlag = true; + } + } else if (data.__data) { + relData = data.__data[relationName]; + applyFlag = !!relData; + } else if ((relData = data[relationName])) { // eslint-disable-line no-cond-assign + applyFlag = true; + } - var dsSupportFilter = true; + let callback = function (err) { + log.debug(context, `${prefix}: (leave${err ? '- with error' : ''}) processing relation "${name}/${relationName}"`); + done(err); + }; + callback.__trace = `${name}_${relationName}`; + if (applyFlag) { + log.debug(context, `${prefix}: (enter) processing relation: ${name}/${relationName}`); + // let { context: { _personalizationCache : { records }} } = personalizationOptions; + // return applyServicePersonalization(relModel, relData, records, ) + // return applyServicePersonalization(relModel, relData, personalizationOptions, callback); + return fetchPersonalizationRecords(context, relModel, function (err, relModelP13nRecords) { + if (err) { + return callback(err); + } + applyServicePersonalization(relModel, relData, relModelP13nRecords, personalizationOptions, callback); + }); + } + log.debug(context, `${prefix}: (leave) processing relation "${relationName}"/"${name}" - skipped`); + nextTick(done); + }; - if (dsSupportFilter) { - addWhereClause(ctx, instruction, cb); + return async.eachSeries(relationItems, relationsIterator, done); } - // else {} + + //! no relations + nextTick(function () { + done(); + }); } -// Processes a filter instruction. filter instruction schema is same like loopback filter schema. -function addLbFilter(ctx, instruction, cb) { - addLbFilterClause(ctx, instruction, cb); +function fetchPersonalizationRecords(ctx, forModel, cb) { + let filter = { where: { modelName: forModel, disabled: false } }; + let { req: { callContext } } = ctx; + PersonalizationRule.find(filter, callContext, cb); } /** - * Processes a 'fieldValueReplace' instruction. - * This method checks if underlying datasource to which the model is attached - * to supports 'field value' replacements. - * If yes, it delegates it to datasource by modifying the query. - * Otherwise creates a post processing function which performs field - * value replacements by iterating the results retrieved. + * Iterates properties of the lb model, if it + * is a model constructor, applies personalization + * recursively + * + * @param {function} Model - the loopback model + * @param {*} data - the data to personalize - object/array + * @param {object} options - standard options passed to applyServicePersonalization() + * @param {callback} cb - signal completion of activity + * @returns {undefined} - nothing */ +function checkPropsAndRecurse(Model, data, options, cb) { + let ctx = options.context; + let { isBeforeRemote } = options; + let modelName = Model.definition.name; + let done = err => { + log.debug(ctx, `${isBeforeRemote ? 'beforeRemote' : 'afterRemote'}: (leave${err ? '- with error' : ''}) applying personalization on properties of ${modelName}`); + cb(err); + }; -function fieldValueReplacementFn(ctx, replacements, cb) { - var input; + let getModelCtorProps = lbModel => { + let propHash = lbModel.definition.properties; + let props = Object.entries(propHash); + let ctorTestPrimitive = ctor => ctor === String || ctor === Number || ctor === Date || ctor === Boolean; + let unAllowedCtors = ['ObjectID', 'Object']; + let ctorTestNames = ctor => unAllowedCtors.includes(ctor.name); + let ctorTest = ctor => ctorTestPrimitive(ctor) || ctorTestNames(ctor); + let modelProps = props.reduce((carrier, item) => { + let [fieldName, propDef] = item; + let { type } = propDef; + let addFlag = false; + if (fieldName === 'id') { + return carrier; + } + if (typeof type === 'function' && !ctorTest(type)) { + addFlag = true; + } else if (typeof type === 'object' && Array.isArray(type)) { + addFlag = type.some(ctorFn => !ctorTest(ctorFn)); + } + if (addFlag) { + carrier.hasModelCtorProps = true; + carrier.modelProps.push(item); + } + return carrier; + }, { modelProps: [], hasModelCtorProps: false }); + return modelProps; + }; - var result = ctx.result || ctx.accdata; + let extractData = (key, data) => { + if (typeof data === 'object' && Array.isArray(data)) { + return _.flatten(data.map(item => extractData(key, item))); + } + if (data.__data) { + return data.__data[key]; + } - if (typeof result !== 'undefined' && !_.isEmpty(result)) { - input = ctx.result || ctx.accdata; - log.debug(ctx.options, 'fieldValueReplacementFn called. Resultset = ', - input + ' Replacements = ' + replacements); - } else if (typeof ctx.instance !== 'undefined' && !_.isEmpty(ctx.instance)) { - input = ctx.instance; - log.debug(ctx.options, 'reverseFieldValueReplacementFn called. Input = ', - input + ' Replacements = ' + replacements); - } else { - return process.nextTick(function () { - return cb(); + return data[key]; + }; + + let mInfo = getModelCtorProps(Model); + if (mInfo.hasModelCtorProps) { + return async.eachSeries(mInfo.modelProps, function propPersonalizerFn([key, propDef], cb) { + let { type } = propDef; + + let relations = Model.settings.relations; + let done = err => { + log.debug(ctx, `${isBeforeRemote ? 'beforeRemote' : 'afterRemote'}: (leave${err ? '- with error' : ''}) personalizing field ${modelName}/${key}`); + cb(err); + }; + log.debug(ctx, `${isBeforeRemote ? 'beforeRemote' : 'afterRemote'}: (enter) personalizing field ${modelName}/${key}`); + // begin - task01 + + // if the property is an embedsOne relation + // property, skip the processing + + //! Because they might already be personalized + + let isAnEmbedsOneRelationProp = Object.entries(relations).some(([relationName, relationDef]) => { + utils.noop(relationName); + // TODO: Add more checks here...(?) + return relationDef.type === 'embedsOne' && relationDef.property === key; + }); + // end - task01 + + + if (!isAnEmbedsOneRelationProp) { + let unpersonalizedData = null; + let modelCtorName = null; + let modelCtor = null; + let applyFlag = true; + + // begin - task02 - extract data and set + // applyFlag if we encounter a model + // constructor + + if (typeof type === 'function') { + //! this could be a plain model constructor + modelCtorName = type.name; + modelCtor = type; + unpersonalizedData = extractData(key, data); + } else { + //! this could be an array of model constructors + + //! Only one model constructor available(?) + modelCtor = type[0]; + modelCtorName = modelCtor.name; + unpersonalizedData = extractData(key, data); + } + // end if-else block - if(typeof type === 'function') + + applyFlag = typeof modelCtor.modelName === 'string'; + + // end - task02 + + if (applyFlag) { + return fetchPersonalizationRecords(ctx, modelCtorName, function (err, records) { + if (err) { + return done(err); + } + applyServicePersonalization(modelCtorName, unpersonalizedData, records, options, done); + }); + } + } + // end if-block if(!isAnEmbedsOneRelationProp) + nextTick(done); + }, function (err) { + done(err); }); } + nextTick(done); +} - function replaceValue(record, replacement, value) { - var pos = replacement.indexOf('\uFF0E'); - var key; - var elsePart; - if (pos !== null && typeof pos !== 'undefined' && pos !== -1) { - key = replacement.substr(0, pos); - elsePart = replacement.substr(pos + 1); +/** + * Personalization helper function. + * + * @param {string} modelName - name of the model + * @param {*} data - the data to personalize - object/array + * @param {[object]} records - the personalization records + * @param {object} options - the personalization options + * @param {callback} cb - to signal completion of task + * @return {undefined} - nothing + */ +function applyServicePersonalization(modelName, data, records, options, cb) { + let Model = loopback.findModel(modelName); + let ctx = options.context; + let isBeforeRemote = options.isBeforeRemote; + log.debug(ctx, `${isBeforeRemote ? 'beforeRemote' : 'afterRemote'}: (enter) applying personalization for model: ${modelName}`); + let done = err => { + log.debug(ctx, `${isBeforeRemote ? 'beforeRemote' : 'afterRemote'}: (leave${err ? ' - with error' : ''}) applying personalization for model: ${modelName}`); + cb(err); + }; + + let execute = function ServicePersonalizationExecute(records) { + if (records.length === 0) { + checkRelationAndRecurse(Model, data, options, function (err) { + if (err) { + return done(err); + } + checkPropsAndRecurse(Model, data, options, done); + }); } else { - key = replacement; + let entry = records[0]; + let personalizationRule = entry.personalizationRule; + let ctx = options.context; + let isBeforeRemote = options.isBeforeRemote; + personalize(ctx, isBeforeRemote, personalizationRule, data, function (err) { + if (err) { + return done(err); + } + checkRelationAndRecurse(Model, data, options, function (err) { + if (err) { + return done(err); + } + checkPropsAndRecurse(Model, data, options, done); + }); + }); } + }; + if (isRemoteMethodAllowed(ctx, records)) { + return execute(records); + } + nextTick(done); +} - if (typeof record[key] !== 'undefined' && Array.isArray(record[key])) { - var newValue = record[key]; - record[key].forEach(function (element, index) { - if (value[element]) { - newValue[index] = value[element]; - } +let PersonalizationRule = null; +/** + * Initializes this module for service personalization + * during applcation boot. Initializes observers on + * PersonalizationRule model. + * + * @param {Application} app - The Loopback application object + */ +function init(app) { + PersonalizationRule = app.models.PersonalizationRule; + let servicePersoConfig = app.get('servicePersonalization'); + if (servicePersoConfig && servicePersoConfig.customFunctionPath) { + loadCustomFunction(require(servicePersoConfig.customFunctionPath)); + } + PersonalizationRule.observe('before save', function PersonalizationRuleBeforeSave(ctx, next) { + log.debug(ctx, 'PersonalizationRule: before save'); + let data = ctx.__data || ctx.instance || ctx.data; + + let model = loopback.findModel(data.modelName); + if (typeof model === 'undefined') { + log.error(ctx, `PersonalizationRule: before save - model "${data.modelName}" is not found`); + return nextTick(function () { + let error = new Error(`Model: ${data.modelName} is not found`); + next(error); }); - if (typeof record.__data !== 'undefined') { - record.__data[key] = newValue; - } else { - record[key] = newValue; - } - } else if (typeof record[key] !== 'undefined' && typeof record[key] === 'object') { - replaceValue(record[key], elsePart, value); - } else if (typeof record[key] !== 'undefined' && typeof record[key] !== 'object') { - if (value.hasOwnProperty(record[key])) { - if (typeof record.__data !== 'undefined') { - record.__data[key] = value[record[key]]; - } else { - record[key] = value[record[key]]; + } + let { modelName, personalizationRule: { postCustomFunction, fieldMask }, methodName } = data; + if (postCustomFunction) { + let { functionName } = postCustomFunction; + if (functionName) { + if (!Object.keys(getCustomFunction()).includes(functionName)) { + return nextTick(function () { + next(new Error(`The custom function with name "${functionName}" does not exist`)); + }); } + } else { + return nextTick(function () { + let error = new Error('postCustomFunction not defined with functionName'); + next(error); + }); } } - } - function replaceRecord(record, replacements) { - var keys = Object.keys(JSON.parse(JSON.stringify(replacements))); - for (var attr in keys) { - if (keys.hasOwnProperty(attr)) { - replaceValue(record, keys[attr], replacements[keys[attr]]); + if (fieldMask) { + // eslint-disable-next-line no-inline-comments + let fMaskPropNames = Object.keys(fieldMask); // all the fields/properties for the fieldMask + + let fmError = msg => createError(`Invalid fieldMask: ${msg}`); + + let fMaskDef = fMaskPropNames.reduce((carrier, fieldName) => { + if (!carrier.hasInvalidFieldMaskDefinition) { + let maskDefAtRoot = fieldMask[fieldName]; + let { modelProps } = carrier; + let field = modelProps[fieldName]; + let fieldType = field.type; + let isConstructor = typeof field.type === 'function'; + let isStringMask = 'stringMask' in maskDefAtRoot; + let isNumberMask = 'numberMask' in maskDefAtRoot; + let isDateMask = 'dateMask' in maskDefAtRoot; + let isAMask = isStringMask || isDateMask || isNumberMask; + if (isStringMask && isConstructor && fieldType !== String) { + carrier.hasInvalidFieldMaskDefinition = true; + carrier.error = fmError(`${fieldName} is of type ${fieldType.name}, expected ${String.name}`); + } else if (isNumberMask && isConstructor && fieldType !== Number) { + carrier.hasInvalidFieldMaskDefinition = true; + carrier.error = fmError(`${fieldName} is of type ${fieldType.name}, expected ${Number.name}`); + } else if (isDateMask && isConstructor && fieldType !== Date) { + carrier.hasInvalidFieldMaskDefinition = true; + carrier.error = fmError(`${fieldName} is of type ${fieldType.name}, expected ${Date.name}`); + } else if (!isAMask && isConstructor && fieldType !== String) { + carrier.hasInvalidFieldMaskDefinition = true; + carrier.error = fmError(`${fieldName} is of type ${fieldType.name}, expected ${String.name}`); + } + } + return carrier; + }, { hasInvalidFieldMaskDefinition: false, modelProps: loopback.findModel(data.modelName).definition.properties }); + + if (fMaskDef.hasInvalidFieldMaskDefinition) { + return nextTick(() => next(fMaskDef.error)); } } - return record; - } - if (Array.isArray(input)) { - var updatedResult = []; - for (var i in input) { - if (input.hasOwnProperty(i)) { - var record = input[i]; - updatedResult.push(replaceRecord(record, replacements)); + if (methodName) { + let Model = loopback.findModel(modelName); + if (!validateMethodName(Model, methodName)) { + let e = createError(`methodName: ${methodName} for model ${modelName} is invalid`); + return nextTick(() => next(e)); } } - input = updatedResult; - } else { - var updatedRecord = replaceRecord(input, replacements); - input = updatedRecord; - } - process.nextTick(function () { - return cb(); + + nextTick(next); }); } -function addFieldReplace(ctx, instruction, cb) { - fieldReplacementFn(ctx, instruction, cb); -} +function isRemoteMethodAllowed(ctx, currentPersonalizationRecords) { + let { _personalizationCache: { info }} = ctx; + let firstRecord = currentPersonalizationRecords[0]; + + if (info && firstRecord && info.modelName === firstRecord.modelName) { + // ! this is probably called via a true remote pipeline + let { methodName } = firstRecord; + let {STAR, STAR_DOT_STAR, DOT, DOUBLE_STAR, PROTOTYPE_DOT_STAR, PROTOTYPE} = REMOTES; + let isPattern = methodName.includes(STAR); + let isPrototype = methodName.includes(PROTOTYPE); + let hasDot = methodName.includes(DOT); + + let allowFlag = false; + if (isPattern && !isPrototype && methodName === STAR) { + //! when * + // allow only static calls + allowFlag = info.isStatic; + } else if (isPattern && !isPrototype && methodName === DOUBLE_STAR) { + //! ** - allow everthing + allowFlag = true; + } else if (isPattern && !isPrototype && hasDot && methodName === STAR_DOT_STAR) { + //! all prototypes (*.*) + allowFlag = !info.isStatic; + } else if (isPattern && isPrototype && methodName === PROTOTYPE_DOT_STAR) { + //! all prototypes (prototype.*) + allowFlag = !info.isStatic; + } else if (!isPattern && !isPrototype) { + //! single static method (E.g. find, findById) + allowFlag = info.isStatic && info.methodName === methodName; + } else if (isPrototype && hasDot && !isPattern) { + // single prototype method - (Eg. prototype.__get__orders) + let protoMethod = methodName.substr(methodName.indexOf(DOT) + 1); + allowFlag = !info.isStatic && info.methodName === protoMethod; + } -// Function to add Sort (Order By) to the query. -function addSort(ctx, instruction, cb) { - // { order: 'propertyName ' } -- sort by single field - // { order: ['propertyName ', 'propertyName ',...] } --sort by mulitple fields - - var dsSupportSort = true; - if (dsSupportSort) { - var query = ctx.args.filter || {}; - if (query) { - if (typeof query.order === 'string') { - query.order = [query.order]; - } - var tempKeys = []; + // TODO: consider adding an invert flag to invert the allowFlag, + // for e.g. disallow only findById, allow others, + // or disallow a custom remote method + return allowFlag; + } + return true; +} - if (query.order && query.order.length >= 1) { - query.order.forEach(function addSortQueryOrderForEachFn(item) { - tempKeys.push(item.split(' ')[0]); - }); - } +/** + * Grabs data either from ctx.req.body + * or ctx.result appropriately + * + * @param {HttpContext} ctx - the http context + * @param {object} info - parsed context method string + * @param {boolean} isBeforeRemote - flag denoting the stage - beforeRemote/afterRemote + * @returns {object} - the model name and data are contained here + */ +function getPersonalizationMeta(ctx, info, isBeforeRemote) { + let { modelName } = info; + let data = null; - // create the order expression based on the instruction passed - var orderExp = createOrderExp(instruction, tempKeys); + let theModel = modelName; + let { req } = ctx; + let httpMethod = req.method; - if (typeof query.order === 'undefined') { - query.order = orderExp; - } else { - query.order = query.order.concat(orderExp); - } - query.order = _.uniq(query.order); - ctx.args.filter = ctx.args.filter || {}; - ctx.args.filter.order = query.order; - cb(); - } else { - cb(); - } + if (httpMethod === 'PUT' || httpMethod === 'POST' || httpMethod === 'PATCH') { + data = isBeforeRemote ? ctx.req.body : ctx.result; } else { - addPostProcessingFunction(ctx, 'sortInMemory', instruction, sortInMemory); - cb(); + data = isBeforeRemote ? {} : ctx.result; } + return { model: theModel, data }; } /** - * Custom function + * Main entry point for personalizations + * via remote method call + * + * @param {httpContext} ctx - context + * @param {boolean} isBeforeRemote - flag denoting the stage - beforeRemote/afterRemote + * @param {callback} cb - to signal the personalizations are complete. */ -// getCustom function -function loadCustomFunction(fnCache) { - customFunction = fnCache; -} - -function getCustomFunction() { - return customFunction; -} - -/* eslint-disable */ -function addCustomFunction(ctx, instruction, cb) { - // instruction has the customFunction Name - // Datasource does not support field name replacement. Add it as a post processing function - addPostProcessingFunction(ctx, 'customFunction', instruction, executeCustomFunctionFn); - cb(); -} -/* eslint-enable */ +function performPersonalizations(ctx, isBeforeRemote, cb) { + let { _personalizationCache: { records, info } } = ctx; -function createOrderExp(instruction, tempKeys) { - if (!Array.isArray(instruction)) { - instruction = [instruction]; - } + let pMeta = getPersonalizationMeta(ctx, info, isBeforeRemote); - var orderExp = []; - - for (var i = 0; i < instruction.length; i++) { - var obj = instruction[i]; - var key = Object.keys(obj)[0]; - var val = obj[key]; - key = key.indexOf('|') > -1 ? key.replace(/\|/g, '.') : key; - - var index = tempKeys.length >= 1 ? tempKeys.indexOf(key) : -1; - - switch (val.toUpperCase()) { - case 'ASC': - case 'ASCENDING': - case '': - val = 'ASC'; - break; - case 'DESC': - case 'DESCENDING': - case 'DSC': - val = 'DESC'; - break; - default: - val = null; - } - if (val && index === -1) { - var value = key + ' ' + val; - orderExp.push(value); - } - } + let options = { isBeforeRemote, context: ctx }; - return orderExp; -} -/* To be used when database doesnt support sort OR sort needs to be done in memory*/ -function sortInMemory(ctx, options) { - var result = ctx.result; - if (typeof result === 'undefined') { - return; - } - if (!Array.isArray(options)) { - options = [options]; - } - var keys = []; - var values = []; - for (var index in options) { - if (options.hasOwnProperty(index)) { - var key = Object.keys(options[index])[0]; - values.push(options[index][key]); - key = key.indexOf('|') > -1 ? key.replace(/\|/g, '.') : key; - keys.push(key); - } - } - var updatedResults; - if (Array.isArray(result)) { - // lodash version 3.10.1 uses sortByOrder;version 4.0.0 uses OrderBy - updatedResults = _.orderBy(result, keys, values); - } - if (updatedResults) { - ctx.result = updatedResults; - } + applyServicePersonalization(pMeta.model, pMeta.data, records, options, cb); } +const ALLOWED_INSTANCE_METHOD_NAMES = ['get', 'create', 'findById', 'updateById']; + /** - * Instantiate a new post processing function and adds to the request context. + * This methods detects some of loopback's + * standard methods like exists, count and + * exempts such requests to be personalized + * + * It also exempts requests which are + * DELETE or HEAD + * + * @param {HttpContext} ctx - the http context object + * @param {object} info - the parsed represetation of the context's methodString + * @returns {boolean} - flag indicating if we can apply personalization */ +function getCanApplyFlag(ctx, info) { + let { methodName, isStatic } = info; + let { req } = ctx; + let httpMethod = req.method; + + let isAllowedPrototype = name => { + let startIdx = 2; + let endIdx = name.indexOf('_', startIdx); + let extractedMethod = name.substr(startIdx, endIdx - startIdx); + return ALLOWED_INSTANCE_METHOD_NAMES.some(value => value === extractedMethod); + }; -function addPostProcessingFunction(ctx, func, instruction, fn) { - var callContext = ctx.req.callContext = ctx.req.callContext || {}; - - callContext.postProcessingFns = callContext.postProcessingFns || {}; - callContext.postProcessingFns[callContext.modelName] = callContext.postProcessingFns[callContext.modelName] || []; - var prcFn = new ProcessingFunction(instruction, fn); - var addPrcFn = callContext.postProcessingFns[callContext.modelName].some(function (processingFn) { - return processingFn.fn.name === prcFn.fn.name; - }); - if (!addPrcFn) { - if (func === 'fieldReplace') { - callContext.postProcessingFns[callContext.modelName].push(prcFn); - } else { - callContext.postProcessingFns[callContext.modelName].unshift(prcFn); - } + if (httpMethod === 'DELETE' || httpMethod === 'HEAD') { + return false; } - // console.log('callContext so far - ' + JSON.stringify(callContext)); -} -/* - * Function to mask the certain fields from the output field List - * */ -function maskFields(ctx, instruction, cb) { - var dsSupportMask = true; - if (dsSupportMask) { - ctx.args.filter = ctx.args.filter || {}; - var query = ctx.args.filter; - if (!query) { - return cb(); - } - var keys = Object.keys(instruction); - var exp = {}; - if (typeof query.fields === 'undefined') { - for (var i = 0, length = keys.length; i < length; i++) { - var key = keys[i]; - key = key.indexOf('|') > -1 ? key.replace(/\|/g, '.') : key; - exp[key] = false; - } - query.fields = exp; - } - // else { - // var fieldList = query.fields; - // fieldList = _.filter(fieldList, function (item) { - // return keys.indexOf(item) === -1 - // }); - // query.fields = fieldList; - // } + if (isStatic && methodName === 'exists') { + return false; } - cb(); -} - -function maskCharacters(ctx, charMaskRules, cb) { - var input; - var result = ctx.result || ctx.accdata; - if (result && !_.isEmpty(result)) { - input = result; - } else if (ctx.req.body && !_.isEmpty(ctx.req.body)) { - input = ctx.req.body; - } else { - return cb(); + if (!isStatic && methodName.startsWith('__') && !isAllowedPrototype(methodName)) { + return false; } - function modifyField(record, property, rule) { - var pos = property.indexOf('.'); - if (pos !== -1) { - var key = property.substr(0, pos); - var innerProp = property.substr(pos + 1); - } else { - key = property; - } + return true; +} - if (record[key] && typeof record[key] === 'object') { - modifyField(record[key], innerProp, rule); - } else if (record[key] && typeof record[key] !== 'object') { - var char = rule.maskCharacter || 'X'; - var flags = rule.flags; - var regex = flags ? new RegExp(rule.pattern, flags) : new RegExp(rule.pattern); - var groups = record[key].match(regex) || []; - var masking = rule.mask || []; - var newVal = rule.format || []; - if (Array.isArray(newVal)) { - for (let i = 1; i < groups.length; i++) { - newVal.push('$' + i); +/** + * Initializes the pipeline for personalization + * + * @param {HttpContext} ctx - the http context object + * @param {boolean} isBeforeRemote - flag denoting phase of the request + * @param {function} cb - callback function to signal completion + * @returns {undefined} - nothing + */ +function runPersonalizations(ctx, isBeforeRemote, cb) { + let {methodString} = ctx; + let info = parseMethodString(methodString); + let canApply = null; + + if (isBeforeRemote) { + canApply = getCanApplyFlag(ctx, info); + ctx._personalizationCache = { canApply }; + if (canApply) { + return fetchPersonalizationRecords(ctx, info.modelName, function (err, records) { + if (err) { + return cb(err); } - newVal = newVal.join(''); - } - masking.forEach(function (elem) { - newVal = newVal.replace(elem, new Array(groups[elem.substr(1)].length + 1).join(char)); + ctx._personalizationCache = Object.assign(ctx._personalizationCache, { info, records }); + performPersonalizations(ctx, isBeforeRemote, cb); }); - for (let i = 0; i < groups.length; i++) { - newVal = newVal.replace('$' + i, groups[i]); - } - // normally we set __data but now, lb3!!! - record[key] = newVal; } + } else if (ctx._personalizationCache.canApply) { + return performPersonalizations(ctx, isBeforeRemote, cb); } - - function applyRuleOnRecord(record, charMaskRules) { - Object.keys(charMaskRules).forEach(function (key) { - modifyField(record, key, charMaskRules[key]); - }); - return record; - } - - if (Array.isArray(input)) { - var updatedResult = []; - input.forEach(function (record) { - updatedResult.push(applyRuleOnRecord(record, charMaskRules)); - }); - input = updatedResult; - } else { - input = applyRuleOnRecord(input, charMaskRules); - } - - return cb(); + let stage = isBeforeRemote ? 'beforeRemote' : 'afterRemote'; + log.debug(ctx, `${stage}: Avoided personalization -> ${methodString}`); + nextTick(cb); } +/** + * Personalizes data by mutating it. + * + * The personalization rules are queried + * using the modelName. + * + * @param {string} modelName - name of model + * @param {*} data - Array or Object + * @param {ServicePersonalizationOptions} options - service personalization options + * Two properties: + * - isBeforeRemote - always false + * - context - the HttpContext object + * @param {function} cb - the function to signal completion + * Has only one arguments - error + * @returns {undefined} - nothing + */ +function performServicePersonalizations(modelName, data, options, cb) { + let ctx = options.context; + log.debug(ctx, `performServicePersonalizations: (enter) model -> ${modelName}`); + let done = err => { + log.debug(ctx, `performServicePersonalizations: (leave${err ? '- with error' : ''}) model -> ${modelName}`); + cb(err); + }; + ctx._personalizationCache = {}; + fetchPersonalizationRecords(ctx, modelName, function (err, records) { + if (err) { + return done(err); + } + applyServicePersonalization(modelName, data, records, options, done); + }); +} module.exports = { - getPersonalizationRuleForModel: getPersonalizationRuleForModel, - applyPersonalizationRule: applyPersonalizationRule, - applyReversePersonalizationRule: applyReversePersonalizationRule, - execute: execute, loadCustomFunction, - getCustomFunction + getCustomFunction, + applyServicePersonalization, + init, + runPersonalizations, + performServicePersonalizations }; diff --git a/lib/utils.js b/lib/utils.js new file mode 100644 index 0000000..962cd79 --- /dev/null +++ b/lib/utils.js @@ -0,0 +1,115 @@ +const _slice = [].slice; +const { DateTimeFormatter, LocalDateTime, nativeJs } = require('@js-joda/core'); +const { Locale } = require('@js-joda/locale_en-us'); +const { prototype: {toString} } = Object; + +const STAR = '*'; +const PROTOTYPE = 'prototype'; +const DOT = '.'; +const DOUBLE_STAR = '**'; +const STAR_DOT_STAR = '*.*'; +const PROTOTYPE_DOT_STAR = 'prototype.*'; + +module.exports = { + /** + * queue the function to the runtime's next event loop + * + * @param {function} cb - the callback function + * @returns {void} + */ + nextTick: cb => process.nextTick(cb), + + /** + * parses a context's methodString + * @param {string} str - method string + * @returns {object} - object representing the method string + */ + parseMethodString: str => { + return str.split('.').reduce((obj, comp, idx, arr) => { + let ret = {}; + let length = arr.length; + if (idx === 0) { + ret.modelName = comp; + } else if (length === 3 && idx !== length - 1) { + ret.isStatic = false; + } else if (length === 3 && idx === length - 1) { + ret.methodName = comp; + } else { + ret.isStatic = true; + ret.methodName = comp; + } + return Object.assign({}, obj, ret); + }, {}); + }, + + slice: arg => _slice.call(arg), + + createError: msg => new Error(msg), + + /** + * joda time formatter helper + * @param {Date} date - the javascript date object to format + * @param {string} format - joda format string + * @param {string} locale - as per js-joda/locale api + * @return {string} - the formatted date + */ + formatDateTimeJoda(date, format, locale = 'US') { + let ldt = LocalDateTime.from(nativeJs(date)); + let pattern = DateTimeFormatter.ofPattern(format).withLocale(Locale[locale]); + return ldt.format(pattern); + }, + + + isDate(object) { + return toString.call(object) === '[object Date]'; + }, + + isString(object) { + return toString.call(object) === '[object String]'; + }, + + isObject(object) { + return toString.call(object) === '[object Object]'; + }, + + isNumber(object) { + return toString.call(object) === '[object Number]'; + }, + + /** + * + * @param {ModelConstructor} Model - loopback model + * @param {string} methodName - method name + * @returns {boolean} - true/false - true if the methodName string is valid + */ + validateMethodName(Model, methodName) { + let hasAsterisk = methodName.includes(STAR); + let hasPrototype = methodName.includes(PROTOTYPE); + + if (hasAsterisk && methodName === STAR) { + return true; + } else if (hasAsterisk && methodName === DOUBLE_STAR) { + return true; + } else if (hasAsterisk && hasPrototype && (methodName === PROTOTYPE_DOT_STAR || methodName === STAR_DOT_STAR)) { + return true; + } else if (!hasPrototype && !hasAsterisk) { + //! static method + return !!Model[methodName]; + } else if (hasPrototype && !hasAsterisk) { + //! prototype method + let protoMethod = methodName.substr(methodName.indexOf(DOT) + 1); + return !!Model.prototype[protoMethod]; + } + return false; + }, + + REMOTES: { + STAR, + PROTOTYPE, + DOT, + DOUBLE_STAR, + STAR_DOT_STAR, + PROTOTYPE_DOT_STAR + } + +}; diff --git a/package.json b/package.json index fb45ec7..cc7b472 100644 --- a/package.json +++ b/package.json @@ -1,6 +1,6 @@ { "name": "oe-service-personalization", - "version": "2.2.0", + "version": "2.3.0", "description": "oe-cloud modularization project", "engines": { "node": ">=6" @@ -14,9 +14,10 @@ "grunt-cover": "grunt test-with-coverage" }, "dependencies": { - "assertion-error": "1.1.0", + "@js-joda/core": "2.0.0", + "@js-joda/locale_en-us": "3.1.1", "async": "2.6.1", - "lodash": "4.17.14", + "lodash": "4.17.20", "oe-cloud": "^2.0.0", "oe-expression": "^2.0.0", "oe-personalization": "^2.0.0" @@ -32,7 +33,7 @@ "grunt-contrib-clean": "2.0.0", "grunt-mocha-istanbul": "5.0.2", "istanbul": "0.4.5", - "md5": "^2.2.1", + "md5": "2.3.0", "mocha": "5.2.0", "oe-connector-mongodb": "^2.0.0", "oe-connector-oracle": "^2.0.0", diff --git a/server/boot/service-personalization.js b/server/boot/service-personalization.js index ffd47c2..e74df87 100644 --- a/server/boot/service-personalization.js +++ b/server/boot/service-personalization.js @@ -1,6 +1,6 @@ /** * - * ©2018-2019 EdgeVerve Systems Limited (a fully owned Infosys subsidiary), + * ©2018-2020 EdgeVerve Systems Limited (a fully owned Infosys subsidiary), * Bangalore, India. All Rights Reserved. * */ @@ -8,397 +8,14 @@ * This boot script brings the ability to apply personalization rules to the model. * * @memberof Boot Scripts - * @author Pradeep Kumar Tippa + * @author deostroll * @name Service Personalization */ -// TODO: without clean db test cases are not passing, need to clean up test cases. - -var loopback = require('loopback'); -var log = require('oe-logger')('service-personalization'); // var messaging = require('../../lib/common/global-messaging'); var servicePersonalizer = require('../../lib/service-personalizer'); -var personalizationRuleModel; - -module.exports = function ServicePersonalization(app, cb) { - log.debug(log.defaultContext(), 'In service-personalization.js boot script.'); - personalizationRuleModel = app.models.PersonalizationRule; - // requiring customFunction - let servicePersoConfig = app.get('servicePersonalization'); - if (servicePersoConfig && 'customFunctionPath' in servicePersoConfig) { - try { - servicePersonalizer.loadCustomFunction(require(servicePersoConfig.customFunctionPath)); - } catch (e) { - log.error(log.defaultContext(), 'require customFunction', e); - } - } - // Creating 'before save' and 'after save' observer hooks for PersonlizationRule model - personalizationRuleModel.observe('before save', personalizationRuleBeforeSave); - personalizationRuleModel.observe('after save', personalizationRuleAfterSave); - // Creating filter finding only records where disabled is false. - var filter = { - where: { - disabled: false - } - }; - // Creating options to fetch all records irrespective of scope. - var options = { - ignoreAutoScope: true, - fetchAllScopes: true - }; - // Using fetchAllScopes and ignoreAutoScope to retrieve all the records from DB. i.e. from all tenants. - personalizationRuleModel.find(filter, options, function (err, results) { - log.debug(log.defaultContext(), 'personalizationRuleModel.find executed.'); - if (err) { - log.error(log.defaultContext(), 'personalizationRuleModel.find error. Error', err); - cb(err); - } else if (results && results.length > 0) { - // The below code for the if clause will not executed for test cases with clean/empty DB. - // In order to execute the below code and get code coverage for it we should have - // some rules defined for some models in the database before running tests for coverage. - log.debug(log.defaultContext(), 'Some PersonalizationRules are present, on loading of this PersonalizationRule model'); - for (var i = 0; i < results.length; i++) { - // No need to publish the message to other nodes, since other nodes will attach the hooks on their boot. - // Attaching all models(PersonalizationRule.modelName) before save hooks when PersonalizationRule loads. - // Passing directly modelName without checking existence since it is a mandatory field for PersonalizationRule. - attachRemoteHooksToModel(results[i].modelName, { ctx: results[i]._autoScope }); - } - cb(); - } else { - cb(); - } - }); +module.exports = function ServicePersonalization(app) { + servicePersonalizer.init(app); }; -// Subscribing for messages to attach 'before save' hook for modelName model when POST/PUT to PersonalizationRule. -// messaging.subscribe('personalizationRuleAttachHook', function (modelName, options) { -// // TODO: need to enhance test cases for running in cluster and send/recieve messages in cluster. -// log.debug(log.defaultContext(), 'Got message to '); -// attachRemoteHooksToModel(modelName, options); -// }); - -/** - * This function is before save hook for PersonlizationRule model. - * - * @param {object} ctx - Model context - * @param {function} next - callback function - */ -function personalizationRuleBeforeSave(ctx, next) { - var data = ctx.data || ctx.instance; - // It is good to have if we have a declarative way of validating model existence. - var modelName = data.modelName; - if (loopback.findModel(modelName, ctx.options)) { - var nextFlag = true; - if (data.personalizationRule.postCustomFunction) { - if (!(Object.keys(servicePersonalizer.getCustomFunction()).indexOf(data.personalizationRule.postCustomFunction.functionName) > -1) && nextFlag) { - next(new Error('Module \'' + data.personalizationRule.postCustomFunction.functionName + '\' doesn\'t exists.')); - nextFlag = false; - } - } - if (data.personalizationRule.preCustomFunction) { - if (!(Object.keys(servicePersonalizer.getCustomFunction()).indexOf(data.personalizationRule.preCustomFunction.functionName) > -1) && nextFlag) { - next(new Error('Module \'' + data.personalizationRule.precustomFunction.functionName + '\' doesn\'t exists.')); - } - } - if (nextFlag) { - next(); - } - } else { - // Not sure it is the right way to construct error object to sent in the response. - var err = new Error('Model \'' + modelName + '\' doesn\'t exists.'); - next(err); - } -} - -/** - * This function is after save hook for PersonlizationRule model. - * - * @param {object} ctx - Model context - * @param {function} next - callback function - */ -function personalizationRuleAfterSave(ctx, next) { - log.debug(log.defaultContext(), 'personalizationRuleAfterSave method.'); - var data = ctx.data || ctx.instance; - // Publishing message to other nodes in cluster to attach the 'before save' hook for model. - // messaging.publish('personalizationRuleAttachHook', data.modelName, ctx.options); - log.debug(log.defaultContext(), 'personalizationRuleAfterSave data is present. calling attachBeforeSaveHookToModel'); - attachRemoteHooksToModel(data.modelName, ctx.options); - next(); -} - -/** - * This function is to attach remote hooks for given modelName to apply PersonalizationRule. - * - * @param {string} modelName - Model name - * @param {object} options - options - */ -function attachRemoteHooksToModel(modelName, options) { - // Can we avoid this step and get the ModelConstructor from context. - var model = loopback.findModel(modelName, options); - // Setting the flag that Personalization Rule exists, need to check where it will be used. - if (!model.settings._personalizationRuleExists) { - model.settings._personalizationRuleExists = true; - // We can put hook methods in an array an have single function to attach them. - // After Remote hooks - - afterRemoteFindHook(model); - afterRemoteFindByIdHook(model); - afterRemoteFindOneHook(model); - afterRemoteCreateHook(model); - afterRemoteUpsertHook(model); - afterRemoteUpdateAttributesHook(model); - - // Before Remote Hooks - beforeRemoteCreateHook(model); - beforeRemoteUpsertHook(model); - beforeRemoteUpdateAttributesHook(model); - beforeRemoteFindHook(model); - beforeRemoteFindOneHook(model); - beforeRemoteFindByIdHook(model); - } -} - -/** - * This function is to attach after remote hook for find for given model. - * - * @param {object} model - Model constructor object. - */ -function afterRemoteFindHook(model) { - model.afterRemote('find', function (ctx, modelInstance, next) { - log.debug(ctx.req.callContext, 'afterRemoteFindHook for ', model.modelName, ' called'); - afterRemotePersonalizationExec(model, ctx, next); - }); -} - -/** - * This function is to attach after remote hook for findById for given model. - * - * @param {object} model - Model constructor object. - */ -function afterRemoteFindByIdHook(model) { - model.afterRemote('findById', function (ctx, modelInstance, next) { - log.debug(ctx.req.callContext, 'afterRemoteFindByIdHook for ', model.modelName, ' called'); - afterRemotePersonalizationExec(model, ctx, next); - }); -} - -/** - * This function is to attach after remote hook for findOne for given model. - * - * @param {object} model - Model constructor object. - */ -function afterRemoteFindOneHook(model) { - model.afterRemote('findOne', function (ctx, modelInstance, next) { - log.debug(ctx.req.callContext, 'afterRemoteFindOneHook for ', model.modelName, ' called'); - afterRemotePersonalizationExec(model, ctx, next); - }); -} - -/** - * This function is to attach after remote hook for create for given model. - * - * @param {object} model - Model constructor object. - */ -function afterRemoteCreateHook(model) { - model.afterRemote('create', function (ctx, modelInstance, next) { - log.debug(ctx.req.callContext, 'afterRemoteCreateHook for ', model.modelName, ' called'); - afterRemotePersonalizationExec(model, ctx, next); - }); -} - -/** - * This function is to attach after remote hook for upsert for given model. - * - * @param {object} model - Model constructor object. - */ -function afterRemoteUpsertHook(model) { - model.afterRemote('upsert', function (ctx, modelInstance, next) { - log.debug(ctx.req.callContext, 'afterRemoteUpsertHook for ', model.modelName, ' called'); - afterRemotePersonalizationExec(model, ctx, next); - }); -} - -/** - * This function is to attach after remote hook for updateAttributes for given model. - * - * @param {object} model - Model constructor object. - */ -function afterRemoteUpdateAttributesHook(model) { - model.afterRemote('prototype.updateAttributes', function (ctx, modelInstance, next) { - log.debug(ctx.req.callContext, 'afterRemoteUpdateAttributes for ', model.modelName, ' called'); - afterRemotePersonalizationExec(model, ctx, next); - }); -} - -/** - * This function is to attach before remote hook for create for given model. - * - * @param {object} model - Model constructor object. - */ -function beforeRemoteCreateHook(model) { - model.beforeRemote('create', function (ctx, modelInstance, next) { - log.debug(ctx.req.callContext, 'beforeRemoteCreateHook for ', model.modelName, ' called'); - beforeRemotePersonalizationExec(model, ctx, next); - }); -} - -/** - * This function is to attach before remote hook for upsert for given model. - * - * @param {object} model - Model constructor object. - */ -function beforeRemoteUpsertHook(model) { - model.beforeRemote('upsert', function (ctx, modelInstance, next) { - log.debug(ctx.req.callContext, 'beforeRemoteUpsertHook for ', model.modelName, ' called'); - beforeRemotePersonalizationExec(model, ctx, next); - }); -} - -/** - * This function is to attach before remote hook for updateAttributes for given model. - * - * @param {object} model - Model constructor object. - */ -function beforeRemoteUpdateAttributesHook(model) { - model.beforeRemote('prototype.updateAttributes', function (ctx, modelInstance, next) { - log.debug(ctx.req.callContext, 'beforeRemoteUpdateAttributesHook for ', model.modelName, ' called'); - beforeRemotePersonalizationExec(model, ctx, next); - }); -} - -/** - * This function is to attach before remote hook for find for given model - * and modify ctx.args.filter if any corresponding personalization rule is there. - * - * @param {object} model - Model constructor object. - */ -function beforeRemoteFindHook(model) { - model.beforeRemote('find', function (ctx, modelInstance, next) { - log.debug(ctx.req.callContext, 'beforeRemoteFindHook ', model.modelName, 'called'); - servicePersonalizer.getPersonalizationRuleForModel(model.modelName, ctx, function servicePersonalizationAccessHookGetRuleCb(rule) { - if (rule !== null && typeof rule !== 'undefined') { - log.debug(ctx.req.callContext, 'beforeRemoteFindHook personalization rule found , rule: ', rule); - var fns = servicePersonalizer.applyPersonalizationRule(ctx, rule.personalizationRule); - fns = fns.filter(x => x.type !== 'postCustomFunction'); - fns = fns.map(x => x.fn); - servicePersonalizer.execute(fns, function (err) { - if (err) { - return next(err); - } - log.debug(ctx.req.callContext, 'filter', ctx.args.filter); - next(); - }); - } else { - log.debug(ctx.req.callContext, 'beforeRemoteFindHook no rules were found'); - next(); - } - }); - }); -} - -function beforeRemoteFindOneHook(model) { - model.beforeRemote('findOne', function (ctx, modelInstance, next) { - log.debug(ctx.req.callContext, 'beforeRemoteFindOneHook ', model.modelName, 'called'); - servicePersonalizer.getPersonalizationRuleForModel(model.modelName, ctx, function servicePersonalizationAccessHookGetRuleCb(rule) { - if (rule !== null && typeof rule !== 'undefined') { - log.debug(ctx.req.callContext, 'beforeRemoteFindOneHook personalization rule found , rule: ', rule); - var fns = servicePersonalizer.applyPersonalizationRule(ctx, rule.personalizationRule); - fns = fns.filter(x => x.type !== 'postCustomFunction'); - fns = fns.map(x => x.fn); - servicePersonalizer.execute(fns, function (err) { - if (err) { - return next(err); - } - log.debug(ctx.req.callContext, 'filter', ctx.args.filter); - next(); - }); - } else { - log.debug(ctx.req.callContext, 'beforeRemoteFindOneHook no rules were found'); - next(); - } - }); - }); -} - -function beforeRemoteFindByIdHook(model) { - model.beforeRemote('findById', function (ctx, modelInstance, next) { - log.debug(ctx.req.callContext, 'beforeRemoteFindByIdHook ', model.modelName, 'called'); - servicePersonalizer.getPersonalizationRuleForModel(model.modelName, ctx, function servicePersonalizationAccessHookGetRuleCb(rule) { - if (rule !== null && typeof rule !== 'undefined') { - log.debug(ctx.req.callContext, 'beforeRemoteFindByIdHook personalization rule found , rule: ', rule); - var fns = servicePersonalizer.applyPersonalizationRule(ctx, rule.personalizationRule); - fns = fns.filter(x => x.type !== 'postCustomFunction'); - fns = fns.map(x => x.fn); - servicePersonalizer.execute(fns, function (err) { - if (err) { - return next(err); - } - log.debug(ctx.req.callContext, 'filter', ctx.args.filter); - next(); - }); - } else { - log.debug(ctx.req.callContext, 'beforeRemoteFindByIdHook no rules were found'); - next(); - } - }); - }); -} - -/** - * This function is to do the execution personalization rules of after remote hook for given model. - * - * @param {object} model - Model constructor object. - * @param {object} ctx - context object. - * @param {function} next - callback function. - */ -function afterRemotePersonalizationExec(model, ctx, next) { - log.debug(ctx.req.callContext, 'afterRemotePersonalizationExec for ', model.modelName, ' called.'); - servicePersonalizer.getPersonalizationRuleForModel(model.modelName, ctx, function servicePersonalizationMixinBeforeCreateGetReverse(rule) { - if (rule !== null && typeof rule !== 'undefined') { - log.debug(ctx.req.callContext, 'afterRemotePersonalizationExec personalization rule found , rule: ', rule); - log.debug(ctx.req.callContext, 'applying PersonalizationRule now'); - log.debug(ctx.req.callContext, 'beforeRemoteFindHook personalization rule found , rule: ', rule); - var fns = servicePersonalizer.applyPersonalizationRule(ctx, rule.personalizationRule); - fns = fns.map(x => x.fn); - servicePersonalizer.execute(fns, function (err) { - if (err) { - return next(err); - } - log.debug(ctx.req.callContext, 'filter', ctx.args.filter); - next(); - }); - } else { - log.debug(ctx.req.callContext, 'afterRemotePersonalizationExec no rules were found'); - next(); - } - }); -} - -/** - * This function is to do the execution personalization rules of before remote hook for given model. - * - * @param {object} model - Model constructor object. - * @param {object} ctx - context object. - * @param {function} next - callback function. - */ -function beforeRemotePersonalizationExec(model, ctx, next) { - log.debug(ctx.req.callContext, 'beforeRemotePersonalizationExec for ', model.modelName, ' called.'); - servicePersonalizer.getPersonalizationRuleForModel(model.modelName, ctx, function servicePersonalizationMixinBeforeCreateGetReverse(rule) { - if (rule !== null && typeof rule !== 'undefined') { - log.debug(ctx.req.callContext, 'beforeRemotePersonalizationExec personalization rule found , rule: ', rule); - log.debug(ctx.req.callContext, 'applying PersonalizationRule now'); - var fns = servicePersonalizer.applyReversePersonalizationRule(ctx, rule.personalizationRule); - servicePersonalizer.execute(fns, function (err) { - if (err) { - return next(err); - } - log.debug(ctx.req.callContext, 'filter', ctx.args.filter); - next(); - }); - } else { - log.debug(ctx.req.callContext, 'beforeRemotePersonalizationExec no rules were found'); - next(); - } - }); -} diff --git a/test/app-list.json b/test/app-list.json index 973d628..0b36f67 100644 --- a/test/app-list.json +++ b/test/app-list.json @@ -5,7 +5,8 @@ }, { "path": "oe-personalization", - "enabled": true + "enabled": true, + "autoEnableMixins" : true }, { "path": "./", @@ -14,6 +15,7 @@ { "path": "./", "enabled": true, - "serverDir" : "test" + "serverDir" : "test", + "autoEnableMixins": true } ] diff --git a/test/common/models/address-book.json b/test/common/models/address-book.json new file mode 100644 index 0000000..140877b --- /dev/null +++ b/test/common/models/address-book.json @@ -0,0 +1,19 @@ +{ + "name":"AddressBook", + "base":"BaseEntity", + "properties": { + "line1": "string", + "line2": "string", + "landmark": "string", + "pincode":"string" + }, + "relations": { + "phones" : { + "type": "hasMany", + "model":"PhoneNumber" + } + }, + "mixins": { + "ServicePersonalizationMixin": true + } +} \ No newline at end of file diff --git a/test/common/models/phone-number.json b/test/common/models/phone-number.json new file mode 100644 index 0000000..73334ec --- /dev/null +++ b/test/common/models/phone-number.json @@ -0,0 +1,14 @@ +{ + "name": "PhoneNumber", + "properties": { + "number": { + "type": "string", + "id": true + }, + "firstName": "string", + "lastName": "string" + }, + "mixins": { + "ServicePersonalizationMixin": true + } +} \ No newline at end of file diff --git a/test/common/models/product-catalog.json b/test/common/models/product-catalog.json index 12dd980..0ad0b89 100644 --- a/test/common/models/product-catalog.json +++ b/test/common/models/product-catalog.json @@ -25,8 +25,16 @@ ] }, "validations": [], - "relations": {}, + "relations": { + "store":{ + "model":"StoreStock", + "type": "hasOne" + } + }, "acls": [], "methods": {}, - "strict": true + "strict": true, + "mixins": { + "ServicePersonalizationMixin" : true + } } \ No newline at end of file diff --git a/test/common/models/product-owner.js b/test/common/models/product-owner.js new file mode 100644 index 0000000..203be43 --- /dev/null +++ b/test/common/models/product-owner.js @@ -0,0 +1,65 @@ +const { applyServicePersonalization } = require('./../../../lib/service-personalizer'); // or require('oe-service-personalization/lib/service-personalizer'); + +module.exports = function(ProductOwner) { + ProductOwner.remoteMethod('demandchain', { + description: 'Gets the stores, store addresses, and, contacts of a product owner', + accepts: [ + { + arg: 'id', + type: 'number', + description: 'the unique id of the owner', + required: true + }, + { + arg: 'options', + type: 'object', + http:function(ctx) { + return ctx; + } + } + ], + returns: { + arg: 'chain', + root: true, + type: 'object' + }, + http: { path: '/:id/demandchain', verb: 'get' } + }); + + ProductOwner.demandchain = function(ownerId, options, done) { + if(typeof done === 'undefined' && typeof options === 'function') { + done = options; + options = {}; + }; + + let filter = { + "include": [ + { + "ProductCatalog" : { + "store": { + "store" : { + "addresses" : "phones" + } + } + } + }, + "address" + ], + "where": { "id": ownerId } + }; + ProductOwner.findOne(filter, options, function(err, result) { + // if(err) { + // done(err) + // } + // else { + // let persOpts = { + // isBeforeRemote: false, context: options + // }; + // applyServicePersonalization('ProductOwner', result, persOpts, function(err){ + // done(err, result); + // }); + // } + done(err, result); + }) + }; +} \ No newline at end of file diff --git a/test/common/models/product-owner.json b/test/common/models/product-owner.json index 59658f6..3bfe704 100644 --- a/test/common/models/product-owner.json +++ b/test/common/models/product-owner.json @@ -16,8 +16,15 @@ "ProductCatalog": { "type": "hasMany", "model": "ProductCatalog" - } + }, + "address": { + "type" : "hasOne", + "model" : "AddressBook" + } }, "acls": [], - "methods": {} + "methods": {}, + "mixins": { + "ServicePersonalizationMixin" : true + } } diff --git a/test/common/models/pseudo-product-owner.js b/test/common/models/pseudo-product-owner.js new file mode 100644 index 0000000..7c7dd6d --- /dev/null +++ b/test/common/models/pseudo-product-owner.js @@ -0,0 +1,64 @@ +const { performServicePersonalizations } = require('./../../../lib/api'); // or require('oe-service-personalization/lib/api'); +const loopback = require('loopback'); + +module.exports = function(PseudoProductOwner) { + PseudoProductOwner.remoteMethod('demandchain', { + description: 'Gets the stores, store addresses, and, contacts of a product owner', + accepts: [ + { + arg: 'id', + type: 'number', + description: 'the unique id of the owner', + required: true + }, + { + arg: 'options', + type: 'object', + http:function(ctx) { + return ctx; + } + } + ], + returns: { + arg: 'chain', + root: true, + type: 'object' + }, + http: { path: '/:id/demandchain', verb: 'get' } + }); + + PseudoProductOwner.demandchain = function(ownerId, options, done) { + if(typeof done === 'undefined' && typeof options === 'function') { + done = options; + options = {}; + }; + let ProductOwner = loopback.findModel('ProductOwner'); + let filter = { + "include": [ + { + "ProductCatalog" : { + "store": { + "store" : { + "addresses" : "phones" + } + } + } + }, + "address" + ], + "where": { "id": ownerId } + }; + ProductOwner.findOne(filter, options, function(err, result) { + if(err) { + return done(err); + } + let persOptions = { + isBeforeRemote: false, + context: options + } + performServicePersonalizations(ProductOwner.definition.name, result, persOptions, function(err){ + done(err, result); + }) + }); + }; +} \ No newline at end of file diff --git a/test/common/models/pseudo-product-owner.json b/test/common/models/pseudo-product-owner.json new file mode 100644 index 0000000..18c9244 --- /dev/null +++ b/test/common/models/pseudo-product-owner.json @@ -0,0 +1,4 @@ +{ + "name": "PseudoProductOwner", + "base": "Model" +} diff --git a/test/common/models/store-stock.json b/test/common/models/store-stock.json new file mode 100644 index 0000000..fe1c083 --- /dev/null +++ b/test/common/models/store-stock.json @@ -0,0 +1,19 @@ +{ + "name": "StoreStock", + "description": "Inventory of what is sold in the store", + "properties": { + }, + "relations": { + "store":{ + "type":"belongsTo", + "model": "Store" + }, + "product": { + "type":"belongsTo", + "model":"ProductCatalog" + } + }, + "mixins": { + "ServicePersonalizationMixin": true + } +} \ No newline at end of file diff --git a/test/common/models/store.json b/test/common/models/store.json new file mode 100644 index 0000000..932e236 --- /dev/null +++ b/test/common/models/store.json @@ -0,0 +1,20 @@ +{ + "name": "Store", + "base":"BaseEntity", + "properties": { + "name": "string" + }, + "relations": { + "addresses": { + "model": "AddressBook", + "type": "hasMany" + }, + "products": { + "type":"hasMany", + "model": "ProductCatalog" + } + }, + "mixins": { + "ServicePersonalizationMixin": true + } +} \ No newline at end of file diff --git a/test/model-config.json b/test/model-config.json index 6c83a4a..b926055 100644 --- a/test/model-config.json +++ b/test/model-config.json @@ -39,5 +39,25 @@ "ProductOwner" :{ "dataSource": "db", "public": true + }, + "AddressBook" : { + "dataSource" : "db", + "public": true + }, + "PhoneNumber" : { + "dataSource" : "db", + "public": true + }, + "Store" : { + "dataSource" : "db", + "public": true + }, + "StoreStock" : { + "dataSource" : "db", + "public": true + }, + "PseudoProductOwner" : { + "dataSource": "db", + "public": "true" } } diff --git a/test/test.js b/test/test.js index b75fa92..50027d3 100755 --- a/test/test.js +++ b/test/test.js @@ -1,6 +1,6 @@ var oecloud = require('oe-cloud'); var loopback = require('loopback'); - +var async = require('async'); oecloud.boot(__dirname, function (err) { if (err) { console.log(err); @@ -14,6 +14,7 @@ var chalk = require('chalk'); var chai = require('chai'); chai.use(require('chai-things')); var expect = chai.expect; +var _ = require('lodash'); var ProductCatalog; var ProductOwner; @@ -27,10 +28,11 @@ var basePath = app.get('restApiRoot'); var productCatalogUrl = basePath + '/ProductCatalogs'; var productOwnerUrl = basePath + '/ProductOwners'; describe(chalk.blue('service personalization test started...'), function () { - this.timeout(10000); + // this.timeout(10000); var accessToken; before('wait for boot scripts to complete', function (done) { app.on('test-start', function () { + // console.log('booted'); ProductCatalog = loopback.findModel('ProductCatalog'); ProductCatalog.destroyAll(function (err, info) { return done(err); @@ -721,6 +723,7 @@ describe(chalk.blue('service personalization test started...'), function () { }); }); + // TODO: (Arun - 2020-04-24 22:34:58) Is it meant to demonstrate reverse field value replace? it('t16 should replace field value names while posting when fieldValueReplace personalization is configured', function (done) { // Setup personalization rule @@ -1057,6 +1060,7 @@ describe(chalk.blue('service personalization test started...'), function () { }); }); + //TODO: (Arun 2020-04-24 22:40:29) - lbFilter should it be there in the first place? it('t21 should give filterd result when lbFilter is applied', function (done) { // Setup personalization rule var ruleForAndroid = { @@ -1419,7 +1423,7 @@ describe(chalk.blue('service personalization test started...'), function () { }); }); - describe('Relation Tests - ', function () { + describe('DOT Tests - ', function () { var AddressModel, CustomerModel; var defContext = {}; before('setup test data', done => { @@ -1437,6 +1441,9 @@ describe(chalk.blue('service personalization test started...'), function () { state: { type: 'string' } + }, + mixins: { + ServicePersonalizationMixin: true } }; @@ -1452,6 +1459,9 @@ describe(chalk.blue('service personalization test started...'), function () { model: 'Address', property: 'billingAddress' } + }, + mixins: { + ServicePersonalizationMixin: true } }; @@ -1715,6 +1725,964 @@ describe(chalk.blue('service personalization test started...'), function () { }); }); }); + + describe('Updation scenarios', function () { + it('t37 should apply personalization during an upsert for the same scope', function (done) { + let putData = { + id: 'watch3', + name: 'MultiTrix - MODEL 1100', + modelNo: '7983211100' + }; + + api.put(productCatalogUrl + '?access_token=' + accessToken) + .set('Accept', 'application/json') + .set('REMOTE_USER', 'testUser') + .set('region', 'kl') + .send(putData) + .expect(200) + .end(function (err, resp) { + if (err) { + done(err) + } + else { + var result = resp.body; + expect(result.id).to.equal(putData.id); + expect(result.name).to.equal(putData.name); + expect(result.modelNo).to.equal('798321XXXX'); + done(); + } + }) + }); + + it('t38 should apply personalization to a single record that is being updated', function (done) { + let putData = { + modelNo: '1234560000' + }; + + let apiUrl = productCatalogUrl + '/watch3' + `?access_token=${accessToken}`; + api.put(apiUrl) + .set('Accept', 'application/json') + .set('REMOTE_USER', 'testUser') + .set('region', 'kl') + .send(putData) + .expect(200) + .end(function (err, resp) { + if (err) { + done(err) + } + else { + let result = resp.body; + expect(result.id).to.equal('watch3'); + expect(result.modelNo).to.equal('123456XXXX'); + done(); + } + }) + }); + }); + + var httpResult; + + describe('Relation tests', function () { + + before('creating the product owner', done => { + let data = { + "name": "Swamy Patanjali", + "city": "Lucknow", + "id": 12 + }; + ProductOwner = loopback.findModel('ProductOwner'); + ProductOwner.create(data, function (err) { + done(err); + }); + }); + + before('creating a catalog', done => { + let data = [ + { + "name": "Patanjali Paste", + "category": "FMCG", + "desc": "Herbal paste that is all vegan", + "price": { "currency": "INR", "amount": 45 }, + "isAvailable": false, + "keywords": ["toothpaste", "herbal"], + "productOwnerId": 12, + "id": "prod1" + }, + { + "name": "Patanjali Facial", + "category": "Cosmetics", + "desc": "Ayurvedic cream to get rid of dark spots, pimples, etc", + "price": { "currency": "INR", "amount": 70 }, + "isAvailable": true, + "keywords": ["face", "herbal", "cream"], + "productOwnerId": 12, + "id": "prod2" + } + ]; + + ProductCatalog.create(data, function (err) { + done(err); + }); + }); + + before('creating stores', done => { + let data = [ + { + "name": "Patanjali Store 1", + "id": "store2" + }, + { + "name": "Patanjali Store 2", + "id": "store1" + } + ]; + let Store = loopback.findModel('Store'); + Store.create(data, function (err) { + done(err); + }); + }); + + before('creating store stock', done => { + let data = [ + { "storeId": "store1", "productCatalogId": "prod1" }, + { "storeId": "store2", "productCatalogId": "prod2" } + ]; + let StoreStock = loopback.findModel('StoreStock'); + StoreStock.create(data, function (err) { + done(err); + }); + }); + + before('create addresses', done => { + let data = [ + { + "line1": "5th ave", + "line2": "Richmond", + "landmark": "Siegel Building", + "pincode": "434532", + "id": "addr1", + "storeId": "store1" + }, + { + "line1": "7th ave", + "line2": "Wellington Broadway", + "landmark": "Carl Sagan's Office", + "pincode": "434543", + "id": "addr2", + "storeId": "store1" + }, + { + "line1": "Patanjali Lane", + "line2": "Patanjali Rd", + "landmark": "Near locality water tank", + "pincode": "473032", + "id": "addr3", + "productOwnerId": 12 + }, + { + "line1": "Orchard St", + "line2": "Blumingdale's", + "landmark": "Post Office", + "pincode": "673627", + "id": "addr4", + "storeId": "store2" + } + ]; + let AddressBook = loopback.findModel('AddressBook'); + AddressBook.create(data, function (err) { + done(err); + }); + }); + + before('creating phone numbers', done => { + let data = [ + { + "number": "2342229898", + "firstName": "Ethan", + "lastName": "Hunt", + "addressBookId": "addr1" + }, + { + "number": "2342229899", + "firstName": "Davy", + "lastName": "Jones", + "addressBookId": "addr1" + }, + { + "number": "2342222399", + "firstName": "Jack", + "lastName": "Sparrow", + "addressBookId": "addr2" + }, + { + "number": "8037894565", + "firstName": "Martha", + "lastName": "James", + "addressBookId": "addr3" + }, + { + "number": "2340022399", + "firstName": "Antonio", + "lastName": "Bandaras", + "addressBookId": "addr4" + }, + ]; + + let PhoneNumber = loopback.findModel('PhoneNumber'); + PhoneNumber.create(data, function (err) { + done(err); + }); + }); + + + + it('t39 should apply child model personalization when included from parent with no personalization', done => { + let data = { + productOwnerId: 1 + }; + + let url = `${productCatalogUrl}/watch3?access_token=${accessToken}`; + api.put(url) + .set('Accept', 'application/json') + .set('REMOTE_USER', 'testUser') + .set('region', 'kl') + .send(data) + .expect(200) + .end((err, resp) => { + if (err) { + done(err) + } + else { + let filter = { include: ["ProductCatalog"] }; + let escapedFilter = encodeURIComponent(JSON.stringify(filter)); + let url2 = `${productOwnerUrl}/1?filter=${escapedFilter}&access_token=${accessToken}`; + let res = resp.body; + expect(res.modelNo).to.include("XXXX"); + api.get(url2) + .set('Accept', 'application/json') + .set('REMOTE_USER', 'testUser') + .set('region', 'kl') + .expect(200).end((err, resp) => { + if (err) { + done(err); + } + else { + let result = resp.body; + expect(result.ProductCatalog).to.be.array; + let watch3item = result.ProductCatalog.find(item => item.id === 'watch3'); + expect(watch3item.modelNo).to.equal('123456XXXX'); + done(); + } + }); + } + }); + }); + + it('t40(a) should demonstrate personalization is being applied recursively', done => { + let data = [ + { + "modelName": "AddressBook", + "personalizationRule": { + "mask": { + "landmark": true + } + } + }, + { + "modelName": "PhoneNumber", + "personalizationRule": { + "fieldMask": { + "number": { + "pattern": "([0-9]{3})([0-9]{3})([0-9]{4})", + "maskCharacter": "X", + "format": "($1) $2-$3", + "mask": ["$1", "$2"] + } + } + } + } + ]; + + PersonalizationRule.create(data, {}, function (err) { + if (err) { + return done(err) + } + let filter = { + "include": [ + { + "ProductCatalog": { + "store": { + "store": { + "addresses": "phones" + } + } + } + }, + "address" + ], + "where": { "id": 12 } + }; + let filterString = encodeURIComponent(JSON.stringify(filter)); + let url = `${productOwnerUrl}/findOne?access_token=${accessToken}&&filter=${filterString}`; + api.get(url) + .set('Accept', 'application/json') + .set('REMOTE_USER', 'testUser') + .expect(200) + .end((err, resp) => { + if (err) { + done(err) + } + else { + let result = resp.body; + httpResult = result; + expect(result.ProductCatalog).to.be.array; + expect(result.address).to.be.object; + // console.log(JSON.stringify(result,null, 2)); + _.flatten( + _.flatten(result.ProductCatalog.map(item => item.store.store.addresses)) + .map(x => x.phones) + ).forEach(ph => { + let substr = ph.number.substr(0, 10); + expect(substr).to.equal('(XXX) XXX-'); + }); + + + done(); + } + }); + }); + }); + + it('t40(b) should apply service personalization to a related model invoked via remote call', function (done) { + let url = `${productOwnerUrl}/1/ProductCatalog?access_token=${accessToken}`; + api.get(url) + .set('Accept', 'application/json') + .set('REMOTE_USER', 'testUser') + .set('region', 'kl') + .expect(200) + .end((err, resp) => { + if (err) { + return done(err); + } + let result = resp.body; + // console.log(resp.body); + let idx = result.findIndex(r => r.id === 'watch3'); + expect(result[idx].modelNo).to.equal('123456XXXX'); + done(); + }); + }); + }); + + describe('Remote method tests', () => { + beforeEach('re-inserting the personalization rules', done => { + let data = [ + { + "modelName": "AddressBook", + "personalizationRule": { + "mask": { + "landmark": true + } + } + }, + { + "modelName": "PhoneNumber", + "personalizationRule": { + "fieldMask": { + "number": { + "pattern": "([0-9]{3})([0-9]{3})([0-9]{4})", + "maskCharacter": "X", + "format": "($1) $2-$3", + "mask": ["$1", "$2"] + } + } + } + } + ] + + PersonalizationRule.create(data, {}, function (err) { + done(err); + }); + }); + + it('t41(a) should demonstrate data getting personalized via a custom remote method of model that has mixin enabled', done => { + + let ownerId = 12; + let url = `${productOwnerUrl}/${ownerId}/demandchain?access_token=${accessToken}`; + api.get(url) + .set('Accept', 'application/json') + .set('REMOTE_USER', 'testUser') + .expect(200) + .end((err, resp) => { + if (err) { + done(err); + } + else { + let result = resp.body; + expect(result).to.deep.equal(httpResult); + done(); + } + }) + }); + + it('t41(b) should personalize the response of a custom remote method of model that does not have mixin enabled (api usage)', done => { + let ownerId = 12; + let pseudoProductOwnerUrl = '/api/PseudoProductOwners'; + let url = `${pseudoProductOwnerUrl}/${ownerId}/demandchain?access_token=${accessToken}`; + api.get(url) + .set('Accept', 'application/json') + .set('REMOTE_USER', 'testUser') + .expect(200) + .end((err, resp) => { + if (err) { + done(err); + } + else { + let result = resp.body; + expect(result).to.deep.equal(httpResult); + done(); + } + }) + }); + }); + + /** + * These tests describe how the property + * level personalizations work + * + */ + let CustomerRecords; + describe('property level personalizations', () => { + let ModelDefinition = null; + + before('creating models dynamically', done => { + ModelDefinition = loopback.findModel('ModelDefinition'); + let AccountModel = { + name: 'Account', + properties: { + "accountType": "string", + "openedOn": "date" + }, + relations: { + "customer": { + type: "embedsOne", + model: "Customer", + property: "linkedCustomer" + } + } + }; + + let KycModel = { + name: 'Kyc', + plural: 'Kyc', + properties: { + "criteria": "string", + "code": "string", + "remark": "string", + "score": "number" + } + }; + + let CustomerModel = { + name: "XCustomer", + base: "BaseEntity", + properties: { + firstName: "string", + lastName: "string", + salutation: "string", + dob: "date", + kycInfo: ['Kyc'], + custRef: "string", + aadhar: "number" + }, + relations: { + all_accounts: { + type: 'hasMany', + model: 'Account' + } + } + }; + + + async.eachSeries([KycModel, CustomerModel, AccountModel], function (spec, cb) { + spec.mixins = { ServicePersonalizationMixin: true }; //enabling service personalization mixin + ModelDefinition.create(spec, {}, function (err) { + cb(err); + }); + }, function (err) { + done(err); + }); + }); + + let Customer = null; + before('creating a new customers', done => { + Customer = loopback.findModel('XCustomer'); + let data = [ + { + id: 1, + firstName: 'Cust', + lastName: 'One', + salutation: 'Mr', + kycInfo: [ + { + 'criteria': 'isEmployed', + 'code': 'BCODE-0056', + 'remark': 'SC Bank', + 'score': 56.23 + }, + { + 'criteria': 'allowedAgeLimit', + 'code': 'BCODE-0057', + 'remark': 'witin 25 to 35', + 'score': 76.24 + } + ], + dob: new Date(1987, 3, 12), + custRef: "HDFC-VCHRY-12354", + aadhar: 12345678 + }, + { + id: 2, + firstName: 'Cust', + lastName: 'Two', + salutation: 'Mrs', + kycInfo: [ + { + 'criteria': 'isEmployed', + 'code': 'BCODE-0056', + 'remark': 'Unemployed', + 'score': -23.23 + }, + { + 'criteria': 'allowedAgeLimit', + 'code': 'BCODE-0058', + 'remark': 'witin 25 to 35', + 'score': 76.24 + } + ], + dob: new Date(1989, 3, 12), + custRef: "ICICI-BLR-0056", + aadhar: 45248632 + } + ]; + CustomerRecords = data; + Customer.create(data, {}, function (err) { + done(err); + }); + }); + + before('creating personalization rules', done => { + let data = { + modelName: 'Kyc', + personalizationRule: { + fieldMask: { + code: { + 'pattern': '([A-Z]{5})\\-(\\d{4})', + 'maskCharacter': '*', + 'format': '$1-$2', + 'mask': ['$1'] + } + } + } + }; + PersonalizationRule.create(data, {}, function (err) { + done(err); + }); + }); + + it('t42 when fetching a customer record the kycInfo field should also be personalized', done => { + let custUrl = `/api/XCustomers/1`; + api.get(custUrl) + .set('Accept', 'application/json') + .set('REMOTE_USER', 'testUser') + .expect(200) + .end((err, resp) => { + done(err); + let result = resp.body; + expect(result).to.be.object; + expect(result).to.have.property('firstName'); + expect(result.kycInfo).to.be.array; + result.kycInfo.forEach(kycItem => { + let lastFour = kycItem.code.substr(-4); + let expectedString = `*****-${lastFour}`; + expect(kycItem.code).to.equal(expectedString); + }); + }); + }); + }); + + /** + * These set of test cases describe + * advanced configuration for fieldMask + * operation + */ + + describe('Advanced fieldMask configurations', () => { + it('t43 applying string mask on a dob field (date) should throw an error', done => { + let record = { + modelName: 'XCustomer', + personalizationRule: { + fieldMask: { + dob: { + 'pattern': '([0-9]{3})([0-9]{3})([0-9]{4})', + 'maskCharacter': 'X', + 'format': '($1) $2-$3', + 'mask': ['$3'] + } + } + } + }; + + PersonalizationRule.create(record, {}, function (err) { + if (err) { + return done(); + } + expect(false, 'Should not happen').to.be.ok; + }); + }); + + before('setup rules', done => { + let data = { + modelName: 'XCustomer', + personalizationRule: { + fieldMask: { + custRef: { + stringMask: { + 'pattern': '(\\w+)\\-(\\w+)\\-(\\d+)', + 'maskCharacter': 'X', + 'format': '$1-$2-$3', + 'mask': ['$3'] + } + }, + dob: { + dateMask: { + format: 'MMM/yyyy' + } + }, + aadhar: { + numberMask: { + pattern: '(\\d{2})(\\d{2})(\\d{2})(\\d{2})', + format: '$1 $2 $3 $4', + mask: ['$3', '$4'], + maskCharacter: '*' + } + } + } + } + }; + + PersonalizationRule.create(data, {}, function (err) { + done(err); + }); + }); + let apiResponse; + before('fetch api response', done => { + let url = '/api/XCustomers/2'; + api.get(url) + .set('Accept', 'application/json') + .expect(200) + .end((err, resp) => { + if (err) { + return done(err); + } + let result = resp.body; + // expect(result).to.be.object; + // expect(result.custRef).to.equal('ICICI-BLR-XXXX'); + apiResponse = result; + done(); + }); + }); + + it('t44 should apply a fieldMask on the custRef field which is of type string', () => { + expect(apiResponse).to.be.object; + expect(apiResponse.custRef).to.equal('ICICI-BLR-XXXX'); + }); + + it('t45 should apply a fieldMask to the dob field and display only month and year', () =>{ + expect(apiResponse.dob).to.equal("Apr/1989"); + }); + + it('t46 should apply fieldMask on the aadhar field (numberMask)', () => { + expect(apiResponse.aadhar).to.equal('45 24 ** **'); + }); + }); + + /** + * these tests describe the gating of personalization rules + * based on method of the model. + */ + + describe('Method based service personalization', () => { + before('creating personalization rules', done => { + let data = { + modelName: 'XCustomer', + methodName: 'find', + personalizationRule: { + fieldMask: { + custRef: { + stringMask: { + 'pattern': '(\\w+)\\-(\\w+)\\-(\\d+)', + 'maskCharacter': 'X', + 'format': '$1-$2-$3', + 'mask': ['$3'] + } + } + } + } + }; + + PersonalizationRule.create(data, {}, function(err){ + done(err); + }); + }); + + let apiCall1Response; + before('api call 1 - doing find() remote', done => { + let url = '/api/XCustomers'; + api.get(url) + .set('Accept', 'application/json') + .expect(200) + .end((err, resp) => { + if (err) { + return done(err); + } + let result = resp.body; + apiCall1Response = result; + done(); + }); + }); + + let apiCall2Response; + before('api call 2 - doing findById() remote', done => { + let url = '/api/XCustomers/2'; + api.get(url) + .set('Accept', 'application/json') + .expect(200) + .end((err, resp) => { + if (err) { + return done(err); + } + let result = resp.body; + apiCall2Response = result; + done(); + }); + }); + + it('t47 should assert that only find() call (i.e, api call 1) is personalized', () => { + expect(apiCall1Response).to.be.array; + apiCall1Response.forEach(rec => { + expect(rec.custRef).to.include('X'); + }); + expect(apiCall2Response).to.be.object; + expect(apiCall2Response.custRef).to.not.include('X'); + }); + }); + + + /** + * These tests describe the role based gating + * of service personalization + */ + + describe('Role-based service personalization', () => { + let allUsers; + before('setting up users and roles', done => { + let User = loopback.findModel('User'); + let Role = loopback.findModel('Role'); + let RoleMapping = loopback.findModel('RoleMapping'); + expect(typeof User !== 'undefined').to.be.ok; + expect(typeof Role !== 'undefined').to.be.ok; + expect(typeof RoleMapping !== 'undefined').to.be.ok; + + User.create([ + { username: 'John', email: 'John@ev.com', password: 'password1' }, + { username: 'Jane', email: 'Jane@ev.com', password: 'password1' }, + { username: 'Bob', email: 'Bob@ev.com', password: 'password1' }, + { username: 'Martha', email: 'Martha@ev.com', password: 'password1' } + ], function(err, users){ + if(err){ + return done(err); + } + allUsers = users; + + Role.create([ + { name: 'admin'}, + { name: 'manager'}, + { name: 'teller' }, + { name: 'agent' }, + ], function(err, roles){ + if(err){ + return done(err); + } + + let assignUserRole = (user, role) => cb => role.principals.create({ + principalType: RoleMapping.USER, + principalId: user.id + }, function(err){ + cb(err); + }); + + ['John', 'Jane', "Bob", 'Martha'].forEach((name, idx) => { + expect(users[idx].username).to.equal(name); + }); + + async.eachSeries([ + assignUserRole(users[0], roles[0]), + assignUserRole(users[1], roles[1]), + assignUserRole(users[2], roles[2]), + assignUserRole(users[3], roles[3]), + ], (fn, done) => fn(done), err => { + done(err); + }); + }); + }); + }); + + before('setup personalization rules', done => { + let rules = [ + { + ruleName: 'for tellers', + modelName: 'XCustomer', + personalizationRule: { + fieldMask : { + aadhar: { + numberMask: { + pattern: '(\\d{2})(\\d{2})(\\d{2})(\\d{2})', + format: '$1-$2-$3-$4', + mask: ['$1', '$2', '$3'] + } + } + } + }, + scope: { + roles: ['teller'] + } + }, + { + ruleName: 'for agents', + modelName: 'XCustomer', + personalizationRule: { + fieldMask : { + custRef: { + stringMask: { + pattern: '(\\w+)\\-(\\w+)\\-(\\d+)', + format: '$1-$2-$3', + mask: ['$1', '$3'] + } + } + } + }, + scope: { + roles: ['agent'] + } + } + ]; + + PersonalizationRule.create(rules, function(err){ + done(err); + }); + }); + + let accessTokens; + before('create access tokens', done => { + let url = '/api/Users/login'; + async.map(allUsers, function(user, cb){ + let { username } = user; + api.post(url) + .set('Accept', 'application/json') + .send({ username, password: 'password1'}) + .expect(200) + .end((err, resp) => { + if(err) { + return cb(err); + } + cb(null, { username, token: resp.body.id }); + }); + }, function(err, results) { + if(err) { + return done(err); + } + accessTokens = results.reduce((carrier, obj) => Object.assign(carrier, {[obj.username]: obj.token}), {}); + done(); + }); + }); + + let tellerResponse; + before('access teller data via remote', done => { + let accessToken = accessTokens['Bob']; + let url = `/api/XCustomers/2?access_token=${accessToken}`; + api.get(url) + .set("Accept", 'application/json') + .expect(200) + .end((err, resp) => { + if(err) { + return done(err); + } + tellerResponse = resp.body; + done(); + }); + }); + + let agentResponse; + before('access agent data via remote', done => { + let accessToken = accessTokens['Martha']; + let url = `/api/XCustomers/2?access_token=${accessToken}`; + api.get(url) + .set("Accept", 'application/json') + .expect(200) + .end((err, resp) => { + if(err) { + return done(err); + } + agentResponse = resp.body; + done(); + }); + }); + + let managerResponse; + before('access manager data via remote', done => { + let accessToken = accessTokens['Jane']; + let url = `/api/XCustomers/2?access_token=${accessToken}`; + api.get(url) + .set("Accept", 'application/json') + .expect(200) + .end((err, resp) => { + if(err) { + return done(err); + } + managerResponse = resp.body; + done(); + }); + }); + + let adminResponse; + before('access admin data via remote', done => { + let accessToken = accessTokens['John']; + let url = `/api/XCustomers/2?access_token=${accessToken}`; + api.get(url) + .set("Accept", 'application/json') + .expect(200) + .end((err, resp) => { + if(err) { + return done(err); + } + adminResponse = resp.body; + done(); + }); + }); + + it('t48 should assert that agent and teller results are not identital', () => { + expect(tellerResponse).to.not.deep.equal(agentResponse); + let originalRecord = CustomerRecords[1]; + expect(tellerResponse.aadhar).to.equal('XX-XX-XX-' + originalRecord.aadhar.toString().substr(-2)); + expect(agentResponse.custRef).to.equal('XXXXX-BLR-XXXX'); + }); + + it('t49 should assert that manager and admin results are identical since there is no personalization rules applied', () => { + expect(managerResponse).to.deep.equal(adminResponse); + }); + + }); + });