Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Column 'event' cannot be null for long names events #201

Open
wants to merge 38 commits into
base: MOODLE_311_STABLE
Choose a base branch
from
Open
Changes from 1 commit
Commits
Show all changes
38 commits
Select commit Hold shift + click to select a range
930377b
Fix for the cleanup_history task with a large number of parameters
nhoobin Oct 18, 2021
51899f3
Merge pull request #153 from catalyst/fix_cleanup_helper_task
Oct 22, 2021
0b13e57
Update HTTP post action step to correctly obtain body contents.
nhoobin Nov 5, 2021
f8edb18
Add option for stringcompare to error instead of fail.
nhoobin Nov 5, 2021
151ea4d
Merge pull request #157 from catalyst/http-and-stringcompare-updates
nhoobin Nov 5, 2021
c1f3f5a
Add missing foreign key for 'stepconfigid' upgrades
keevan Nov 23, 2021
9e18ab3
Merge pull request #160 from catalyst/fix-schema-for-upgrades
Peterburnett Nov 23, 2021
f6569bf
Add configurable task processing settings (#158)
keevan Nov 26, 2021
bd12195
Update check on error/failed steps to allow for zero values (#162)
keevan Nov 26, 2021
1d953cc
Merge pull request #163 from catalyst/add-configurable-task-limits-an…
Peterburnett Nov 26, 2021
0d432d8
Fix casing on queuelimit string for consistency
keevan Nov 26, 2021
2dd3792
[#164] Fix Unexpected Moodle Internal errors/warnings in CI
keevan Jan 19, 2022
39b2c46
Merge pull request #165 from keevan/164-issue-fix-for-master
danmarsden Jan 19, 2022
ef86c7e
Add webservice action step (#172)
keevan Mar 10, 2022
a4d739c
chore: add reusable workflow for 3.5-3.10
keevan Mar 11, 2022
cf22448
docs: update supported branches in docs
keevan Mar 11, 2022
6a07b83
chore: disable behat for tests since there are no .feature test files
keevan Mar 11, 2022
205db78
Merge pull request #174 from catalyst/168-add-ci-and-update-support-b…
dmitriim Mar 11, 2022
a2fa46c
Improved anonymous execution user handling
Peterburnett Apr 21, 2022
065f363
Fixed JS lint warnings and recompile
Peterburnett Apr 21, 2022
a11c081
Merge pull request #181 from catalyst/webservice-user-safety-35
Peterburnett Apr 26, 2022
c4e72ac
Ensure return value is always of type array
Peterburnett Apr 29, 2022
8dcb389
Merge pull request #184 from catalyst/webservice-fix
Peterburnett Apr 29, 2022
ce4bcf1
fix: issue where the user executing the web service was being updated
keevan May 4, 2022
d0bd7b5
Merge pull request #185 from catalyst/webservice-improvements-to-who-…
Peterburnett May 4, 2022
18e294e
Fixed improper handling of transactions in webservice steps and logging
Peterburnett May 30, 2022
e3c3e96
Merge pull request #187 from catalyst/webservice-transaction
Peterburnett May 30, 2022
96a0209
Updated history filter to allow 0th step to be handled
Peterburnett May 31, 2022
a7e0b81
Merge pull request #188 from catalyst/filter-step0
keevan May 31, 2022
ed9d105
perf(db): add index to improve query times
keevan Oct 18, 2022
b8a4a8e
ci: disable phpdoc checks
keevan Oct 19, 2022
f658b40
Merge pull request #197 from catalyst/add-indexes-MOODLE_35_STABLE
keevan Oct 19, 2022
565a947
Issue#203 Insert tool_trigger_events id into event other data
Jun 5, 2023
321535c
Merge pull request #205 from catalyst/issue#203
dmitriim Jun 7, 2023
4055bf0
issue #209: allow different HTTP methods for http_post_action_step
dmitriim Oct 16, 2023
dd9e5ed
Merge pull request #212 from catalyst/issue-209-MOODLE_35_STABLE
dmitriim Oct 17, 2023
c77d9e8
Fix #232 - ci badge linking MOODLE_35_STABLE
x-iy Jan 15, 2025
65305d0
Merge pull request #235 from catalyst/issue232-35
dmitriim Jan 15, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Add webservice action step (#172)
* Add support to run webservice functions internally

- Will run on behalf of the designated user / in their context
- Will currently accept a JSON blob in the parameters, similar to the HTTP POST action.

* Add several adjustments to username handling and reducing side-effects

- Add handling to set user back to previous after running the step which should avoid unintended side affects
- Update handling of username so that if it's not provided, it uses the main admin user by default
- Also refactored the code a bit to make it easier to see what's happening at a higher level (from the execute method)

* Add custom validation support to all steps

This should allow one to verify certain fields for steps to match what is expected, if the functionality cannot be done from the standard formlib checks alone

* Adjust lang string & form to indicate behaviour of not setting username

* Add custom validation in webservice action step

- Validates username must exist for user selected, if set
- Validates the function parameters that would be called. Note that due to templating, this will change the field templated to a ZERO, which might not be as expected. The only time this may not be what you want is probably if you are trying to add a field which is dynamic based on the previous input's value (possible but not common)

* Fix PHPCS errors

* Add tests for webservice action step

* Fix more PHPCS issues (line too long)

* Adjust for backwards compatibility with JSON_THROW_ON_ERROR

This will now default to an array format (which is a valid input for a webservice function call) if the user did not provide parameters. Otherwise, it will attempt to decode the json and if it's still NULL, something went wrong.

* Fix more unrelated PHPCS issues

* Ensure parent form validation carries through also

* Add handling to transform the data per step defined in each step class

- In particular adding a transform for the webservice action step to prettify the 'params' (JSON input) so it is more readable/managable the next time it is loaded.

* Fix tests using assertStringContainsString for backward compatibility

* Version bump
keevan authored Mar 10, 2022
commit ef86c7e1a104ecec58229291c431298596a25900
5 changes: 4 additions & 1 deletion classes/helper/datafield_manager.php
Original file line number Diff line number Diff line change
@@ -38,6 +38,9 @@ trait datafield_manager {

protected $datafields = [];

/** @var string regex to determine data fields - should ideally be readonly */
protected $datafieldregex = '/\{([-_A-Za-z0-9]+)\}/u';

/**
* Get the data fields.
*
@@ -160,7 +163,7 @@ public function render_datafields($templatestr, $event = null, $stepresults = nu
};

return preg_replace_callback(
'/\{([-_A-Za-z0-9]+)\}/u',
$this->datafieldregex,
$callback,
$templatestr
);
301 changes: 301 additions & 0 deletions classes/steps/actions/webservice_action_step.php
Original file line number Diff line number Diff line change
@@ -0,0 +1,301 @@
<?php
// This file is part of Moodle - http://moodle.org/
//
// Moodle is free software: you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
//
// Moodle is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details.
//
// You should have received a copy of the GNU General Public License
// along with Moodle. If not, see <http://www.gnu.org/licenses/>.

namespace tool_trigger\steps\actions;

defined('MOODLE_INTERNAL') || die();

require_once($CFG->libdir . '/externallib.php');

/**
* Webservice action step class.
*
* @package tool_trigger
* @author Kevin Pham <[email protected]>
* @copyright Catalyst IT, 2022
* @license http://www.gnu.org/copyleft/gpl.html GNU GPL v3 or later
*/
class webservice_action_step extends base_action_step {

use \tool_trigger\helper\datafield_manager;

/** @var string $functionname Name of the function to be called */
protected $functionname;

/** @var int $username The user in which this action will be actioned in the context of */
protected $username;

/** @var int $params parameters that will be used in the corresponding web service function call */
protected $params;

/**
* The fields supplied by this step.
*
* @var array
*/
private static $stepfields = [
'data',
'error',
'exception',
];

protected function init() {
$this->functionname = $this->data['functionname'];
$this->username = $this->data['username'];
$this->params = $this->data['params'];
}

/**
* Returns the step name.
*
* @return string human readable step name.
*/
public static function get_step_name() {
return get_string('webserviceactionstepname', 'tool_trigger');
}

/**
* Returns the step name.
*
* @return string human readable step name.
*/
public static function get_step_desc() {
return get_string('webserviceactionstepdesc', 'tool_trigger');
}

/**
* Returns the user that should execute the webservice function
*
* @uses webservice_action_step::$username
* @return \stdClass user object
*/
private function get_user() {
global $DB;
$username = $this->render_datafields($this->username);

if (empty($username)) {
// If {username} is not set, then default it to the main admin user.
$user = get_admin();
} else {
// Assume the role of the provided user given their {username}.
$user = $DB->get_record('user', ['username' => $username], '*', MUST_EXIST);
}

// This bypasses the sesskey check for the external api call.
$user->ignoresesskey = true;

return $user;
}

/**
* Prepare and run the function set in the config and return the results.
*
* @uses webservice_action_step::$functionname
* @uses webservice_action_step::$params
* @return array results of the function run
*/
private function run_function() {
// Passing any data from previous steps through by applying template magic.
$functionname = $this->render_datafields($this->functionname);
$params = $this->render_datafields($this->params);

// Execute the provided function name passing with the given parameters.
$response = \external_api::call_external_function($functionname, json_decode($params, true));
return $response;
}

/**
* Runs the configured step.
*
* @param $trigger
* @param $event
* @param $stepresults - result of previousstep to include in processing this step.
* @return array if execution was succesful and the response from the execution.
*/
public function execute($step, $trigger, $event, $stepresults) {
global $USER;

$this->update_datafields($event, $stepresults);

// Store the previous user, setting it back once the step is finished.
$previoususer = $USER;

// Set the configured user as the one who will run the function.
$user = $this->get_user();
\core\session\manager::set_user($user);
set_login_session_preferences();

// Run the function and parse the response to a step result.
$response = $this->run_function();
if ($response['error']) {
return [false, $response['exception']];
}

// Restore the previous user to avoid any side-effects occuring in later steps / code.
\core\session\manager::set_user($previoususer);
set_login_session_preferences();

// Return the function call response as is. The shape is already normalised.
return [true, $response];
}

/**
* {@inheritDoc}
* @see \tool_trigger\steps\base\base_step::add_extra_form_fields()
*/
public function form_definition_extra($form, $mform, $customdata) {
// URL.
$attributes = ['size' => '50', 'placeholder' => 'my_function_name'];
$mform->addElement('text', 'functionname', get_string('webserviceactionfunctionname', 'tool_trigger'), $attributes);
$mform->setType('functionname', PARAM_RAW_TRIMMED);
$mform->addRule('functionname', get_string('required'), 'required');
$mform->addHelpButton('functionname', 'webserviceactionfunctionname', 'tool_trigger');

// Who.
$attributes = ['placeholder' => 'username'];
$mform->addElement('text', 'username', get_string('webserviceactionusername', 'tool_trigger'), $attributes);
$mform->setType('username', PARAM_ALPHANUMEXT);
$mform->addHelpButton('username', 'webserviceactionusername', 'tool_trigger');

// Params.
$attributes = ['cols' => '50', 'rows' => '5'];
$mform->addElement('textarea', 'params', get_string('webserviceactionparams', 'tool_trigger'), $attributes);
$mform->setType('params', PARAM_RAW_TRIMMED);
$mform->addHelpButton('params', 'webserviceactionparams', 'tool_trigger');
}

/**
* {@inheritDoc}
* @see \tool_trigger\steps\base\base_step::add_privacy_metadata()
*/
public static function add_privacy_metadata($collection, $privacyfields) {
return $collection->add_external_location_link(
'webservice_action_step',
$privacyfields,
'step_action_webservice:privacy:desc'
);
}

/**
* Get a list of fields this step provides.
*
* @return array $stepfields The fields this step provides.
*/
public static function get_fields() {
return self::$stepfields;
}

/**
* {@inheritDoc}
* @see \tool_trigger\steps\base\base_step::form_validation()
*/
public function form_validation($data, $files) {
global $DB;

$errors = [];

// Check if the username links to a valid user, if set.
if (!empty($data['username'])) {
try {
$DB->get_record('user', ['username' => $data['username']], '*', MUST_EXIST);
} catch (\Throwable $e) {
$errors['username'] = $e->getMessage();
}
}

// Check if the provided function (name) is a valid callable function.
if (!empty($data['functionname'])) {
try {
$errorfield = 'functionname';
$function = \external_api::external_function_info($data['functionname']);

$errorfield = 'params';

// Fill template fields with a number.
$transformcallback = function() {
return 0;
};

// Cannot use redner_datafields since we need to know of the
// datafields in advance. Will need to apply the change
// manually.
$params = preg_replace_callback(
$this->datafieldregex,
$transformcallback,
$data['params']
);

// Check if this is valid JSON before doing the function's validate_parameters check.
$preparedparams = [];
if (!empty($params)) {
$preparedparams = json_decode($params, true);
if (is_null($preparedparams)) {
throw new \Exception('Invalid Syntax');
}
}

// Execute the provided function name passing with the given parameters.
// $response = \external_api::call_external_function($functionname, json_decode($params, true));
// Check if the provided function parameters are valid.
call_user_func(
[$function->classname, 'validate_parameters'],
$function->parameters_desc,
$preparedparams
);
} catch (\Throwable $e) {
// Most usually a response saying the function name provided doesn't exist.
$errors[$errorfield] = $e->getMessage();
}
}

return $errors;
}


/**
* {@inheritDoc}
* @see \tool_trigger\steps\base\base_step::transform_form_data()
*/
public function transform_form_data($data) {
// Prettify the JSON data in params, if there is content there.
if (!empty($data['params'])) {
// Fill template fields with a number.
$replacemap = [];
$start = PHP_INT_MIN; // Unlikely numerical conflict.
$transformcallback = function($matches) use(&$replacemap, &$start) {
$replacemap[$start] = $matches[0];
return $start++;
};

// Replace all matches with markable values, so they can be swapped back later on to their template forms.
$params = preg_replace_callback(
$this->datafieldregex,
$transformcallback,
$data['params']
);

// Pretty print the JSON value so it's formatted.
$params = json_encode(json_decode($params), JSON_PRETTY_PRINT);

// THEN, replace the temporary values with the original template variables.
$params = str_replace(array_keys($replacemap), array_values($replacemap), $params);

// Update the params key in $data to apply the changes as part of the render.
$data['params'] = $params;
}
return $data;
}
}
15 changes: 14 additions & 1 deletion classes/steps/base/base_form.php
Original file line number Diff line number Diff line change
@@ -124,7 +124,6 @@ public function get_trigger_fields($eventname, $stepclass, $existingsteps, $step

if (!$isfirst) {
foreach ($existingsteps as $step) {

// Don't show fields for steps that may exist after this one.
if ($step['steporder'] >= $steporder && $steporder != -1) {
break;
@@ -235,4 +234,18 @@ public function definition() {
}
}

/**
* Custom validation that will be run if it exists in each step.
*
* @author Kevin Pham <[email protected]>
* @copyright Catalyst IT, 2022
*/
public function validation($data, $files) {
$errors = parent::validation($data, $files);
return array_merge(
$errors,
$this->step->form_validation($data, $files)
);
}

}
26 changes: 26 additions & 0 deletions classes/steps/base/base_step.php
Original file line number Diff line number Diff line change
@@ -178,4 +178,30 @@ public static function get_fields() {
throw new \Exception('Not implemented');

}

/**
* Custom validation of the form that could be configured per step as required.
*
* @param array $data — array of ("fieldname"=>value) of submitted data
* @param array $files — array of uploaded files "element_name"=>tmp_file_path
* @return array of "element_name"=>"error_description" if there are errors,
* or an empty array if everything is OK (true allowed for backwards compatibility too).
*/
public function form_validation($data, $files) {
return [];
}

/**
* Transform / process form data, if required.
*
* An example of when you might want this is when prettifying JSON inputs.
* Any step using this should override this to apply the transformations as
* needed.
*
* @param array $data — array of ("fieldname"=>value) of submitted data
* @return array $data — array of ("fieldname"=>value) of transformed - if required - data
*/
public function transform_form_data($data) {
return $data;
}
}
2 changes: 1 addition & 1 deletion classes/steps/lookups/roles_lookup_step.php
Original file line number Diff line number Diff line change
@@ -74,7 +74,7 @@ public function execute($step, $trigger, $event, $stepresults) {
foreach ($userroles as $role) {
foreach ($role as $key => $value) {
if (is_scalar($role->roleid)) {
$stepresults[$this->outputprefix . 'roles'][ $role->id][$key] = $value;
$stepresults[$this->outputprefix . 'roles'][$role->id][$key] = $value;
}
}
}
Loading