diff --git a/README.md b/README.md index 26871f06..198a0e74 100644 --- a/README.md +++ b/README.md @@ -95,6 +95,9 @@ customize for your environment, see the [project's documentation](https://git-pr Your contributions are at the core of making this a true open source project. Any contributions you make are **greatly appreciated**. See [`CONTRIBUTING.md`](CONTRIBUTING.md) for more information. +## Extensibility +Git Proxy exposes the ability to add custom functionality in the form of plugins which run during a git push. Plugins are loaded via configuration and can be added to a git-proxy deployment via a npm package or loaded from JavaScript files on disk. See [plugin documentation for details](./plugins/README.md). + ## Security If you identify a security vulnerability in the codebase, please follow the steps in [`SECURITY.md`](https://github.com/finos/git-proxy/security/policy). This includes logic-based vulnerabilities and sensitive information or secrets found in code. diff --git a/config.schema.json b/config.schema.json index 7f876bd8..55bb8ca3 100644 --- a/config.schema.json +++ b/config.schema.json @@ -36,6 +36,13 @@ "description": "Flag to enable CSRF protections for UI", "type": "boolean" }, + "pushPlugins": { + "type": "array", + "description": "List of plugins to run on push. Each value is either a file path or a module name.", + "items": { + "type": "string" + } + }, "authorisedList": { "description": "List of repositories that are authorised to be pushed to through the proxy.", "type": "array", diff --git a/packages/git-proxy-notify-hello/index.js b/packages/git-proxy-notify-hello/index.js deleted file mode 100644 index 93f0a14f..00000000 --- a/packages/git-proxy-notify-hello/index.js +++ /dev/null @@ -1,11 +0,0 @@ -const Step = require('@finos/git-proxy/src/proxy/actions').Step; -const plugin = require('@finos/git-proxy/src/plugin'); - -const helloPlugin = new plugin.ActionPlugin(async (req, action) => { - const step = new Step('HelloPlugin'); - console.log('This is a message from the HelloPlugin!'); - action.addStep(step); - return action; -}); - -module.exports.helloPlugin = helloPlugin; diff --git a/plugins/README.md b/plugins/README.md new file mode 100644 index 00000000..0220defd --- /dev/null +++ b/plugins/README.md @@ -0,0 +1,98 @@ +# Plugins +Git Proxy has a simple mechanism for exposing customization within the application for organizations who wish to augment Git Proxy's built-in features, add custom functionality or integrate Git Proxy & other systems within their environment. Plugins are authored via custom objects & functions in JavaScript and can be distributed as NPM packages or source files. Plugins are loaded at deployment via configuration of the git-proxy application server. + +## Loading plugins +In order to load a plugin, you must either install the plugin as a standalone npm package by adding the plugin as a dependency or specify a file path on disk to load a `.js` file as a module. + +### NPM +1. Add the plugin package as a dependency. +```json +{ + "name": "@finos/git-proxy", + ... + "dependencies: { + "foo-my-gitproxy-plugin": "^0.0.1", + "@bar/another-gitproxy-plugin": "^0.0.1", + } +} +``` + +2. Set the "pushPlugins" property in proxy.config.json to a list of modules to load into Git Proxy. These packages must exist in `node_modules/`. + +```json +{ + "pushPlugins": [ + "foo-my-gitproxyplugin", + "@bar/another-gitproxy-plugin" + ] +} +``` + +### Local files +1. Download the plugin's source files & run `npm install` to download any dependencies of the plugin itself. +2. Set the "pushPlugins" property in proxy.config.json to a list of files to load into Git Proxy. +```json +{ + "pushPlugins": [ + "./plugins/foo/index.js", + "/home/alice/gitproxy-push-plugin/index.js" + ] +} +``` + +### Environment variables (deprecated) +The previous implementation of plugins were loaded via the following two environment variables: + +- `GITPROXY_PLUGIN_FILES`: a list of comma-separated JavaScript files which point to Node modules that contain plugin objects +- `GITPROXY_PLUGIN_PACKAGES`: a list of comma-separated NPM packages which contain modules & plugin objects + +Any files or packages specified by these variables will continue to be loaded via the plugin loader if set. However, it is recommended to simply list either files or NPM packages to load as plugins via configuration as documented above. These environment variables will be removed in a future release. + +```bash +# Setting a list of plugin packages to load via env var when running git-proxy +$ export GITPROXY_PLUGIN_PACKAGES="foo-my-gitproxyplugin,@bar/another-gitproxy-plugin/src/plugins/baz" +$ npx -- @finos/git-proxy +``` + +## Writing plugins +Plugins are written as Node modules which export objects containing the custom behaviour. These objects must extend the classes exported by Git Proxy's `plugin/` module. The only class which is exported today for developers to extend Git Proxy is called the `PushActionPlugin` class. This class executes the custom behaviour on any `git push` going through Git Proxy. + +The `PushActionPlugin` class takes a single function into its constructor which is executed on a `git push`. This is then loaded into the push proxy's "chain" of actions. Custom plugins are executed after parsing of a push but before any builtin actions. It is important to be aware of the load order when writing plugins to ensure that one plugin does not conflict with another. The order specified in `pushPlugins` configuration setting is preserved by the loader. + +To write a custom plugin, import the `PushActionPlugin` class from `@finos/git-proxy` and create a new type with your custom function: + +```javascript +// plugin-foo/index.js +const PushActionPlugin = require('@finos/git-proxy/src/plugin').PushActionPlugin; + +class MyPlugin extends PushActionPlugin { + constructor() { + super((req, action) => { + console.log(req); // Log the express.Request object + // insert custom behaviour here using the Action object... + return action; + }) + } +} + +module.exports = new MyPlugin(); +``` + +> Note: use `peerDependencies` to depend on `@finos/git-proxy` in your plugin's package to avoid circular dependencies! + +## Sample plugin +Git Proxy includes a sample plugin that can be loaded with any deployment for demonstration purposes. This plugin is not published as a standalone NPM package and must be used as a local file during deployment. To use the sample plugin: + +1. Run `npm install` in [./plugins/git-proxy-hello-world](./git-proxy-hello-world/). +2. Set "pushPlugins" in `proxy.config.json`: +```json +{ + "pushPlugins": [ + "./plugins/git-proxy-hello-world/index.js" + ] +} +``` +3. Run Git Proxy from source: +``` +npm run start +``` diff --git a/plugins/git-proxy-hello-world/index.js b/plugins/git-proxy-hello-world/index.js new file mode 100644 index 00000000..ceceadab --- /dev/null +++ b/plugins/git-proxy-hello-world/index.js @@ -0,0 +1,43 @@ +const Step = require('@finos/git-proxy/src/proxy/actions').Step; +// eslint-disable-next-line no-unused-vars +const Action = require('@finos/git-proxy/src/proxy/actions').Action; +const ActionPlugin = require('@finos/git-proxy/src/plugin').ActionPlugin; +'use strict'; + +class HelloPlugin extends ActionPlugin { + constructor() { + super(function logMessage(req, action) { + const step = new Step('HelloPlugin'); + action.addStep(step); + console.log('This is a message from the HelloPlugin!'); + return action; + }) + } +} + +/** + * + * @param {Request} req + * @param {Action} action + * @return {Promise} Promise that resolves to an Action + */ +async function logMessage(req, action) { + const step = new Step('LogRequestPlugin'); + action.addStep(step); + console.log(`LogRequestPlugin: req url ${req.url}`); + console.log('LogRequestPlugin: action', JSON.stringify(action)); + return action; +} + +class LogRequestPlugin extends ActionPlugin { + constructor() { + super(logMessage) + } + +} + + +module.exports = { + hello: new HelloPlugin(), + logRequest: new LogRequestPlugin() +}; \ No newline at end of file diff --git a/packages/git-proxy-notify-hello/package.json b/plugins/git-proxy-hello-world/package.json similarity index 76% rename from packages/git-proxy-notify-hello/package.json rename to plugins/git-proxy-hello-world/package.json index 64a8d3d5..ffd7389a 100644 --- a/packages/git-proxy-notify-hello/package.json +++ b/plugins/git-proxy-hello-world/package.json @@ -8,6 +8,9 @@ "author": "Thomas Cooper", "license": "Apache-2.0", "dependencies": { - "@finos/git-proxy": "file:../.." + "express": "^4.18.2" + }, + "peerDependencies": { + "@finos/git-proxy": "^1.3.2" } } diff --git a/proxy.config.json b/proxy.config.json index 02f390a8..280113e9 100644 --- a/proxy.config.json +++ b/proxy.config.json @@ -95,5 +95,6 @@ "privateOrganizations": [], "urlShortener": "", "contactEmail": "", - "csrfProtection": true + "csrfProtection": true, + "pushPlugins": [] } diff --git a/src/config/index.js b/src/config/index.js index b2a68aec..527ea39d 100644 --- a/src/config/index.js +++ b/src/config/index.js @@ -23,6 +23,7 @@ const _privateOrganizations = defaultSettings.privateOrganizations; const _urlShortener = defaultSettings.urlShortener; const _contactEmail = defaultSettings.contactEmail; const _csrfProtection = defaultSettings.csrfProtection; +const _pushPlugins = defaultSettings.pushPlugins; // Get configured proxy URL const getProxyUrl = () => { @@ -142,6 +143,11 @@ const getCSRFProtection = () => { return _csrfProtection; }; +// Get loadable push plugins +const getPushPlugins = () => { + return _pushPlugins; +} + const getSSLKeyPath = () => { if (_userSettings && _userSettings.sslKeyPemPath) { _sslKeyPath = _userSettings.sslKeyPemPath; @@ -177,5 +183,6 @@ exports.getPrivateOrganizations = getPrivateOrganizations; exports.getURLShortener = getURLShortener; exports.getContactEmail = getContactEmail; exports.getCSRFProtection = getCSRFProtection; +exports.getPushPlugins = getPushPlugins; exports.getSSLKeyPath = getSSLKeyPath; exports.getSSLCertPath = getSSLCertPath; \ No newline at end of file diff --git a/src/plugin.js b/src/plugin.js index d2d3228c..717ecfb4 100644 --- a/src/plugin.js +++ b/src/plugin.js @@ -1,42 +1,34 @@ -const path = require('path'); +const config = require('./config'); const lpModule = import('load-plugin'); ('use strict'); /** - * Finds, registers and loads plugins used by git-proxy + * Registers and loads plugins used by git-proxy */ class PluginLoader { /** * Initialize PluginLoader with candidates modules (node_modules or relative * file paths). - * @param {Array.} names List of Node module/package names to load. - * @param {Array.} paths List of file paths to load modules from. + * @param {Array.} targets List of Node module package names or files to load. */ - constructor(names, paths) { - this.names = names; - this.paths = paths; + constructor(targets) { + this.targets = targets; /** - * @type {Array.} List of ProxyPlugin objects loaded. + * @type {ProxyPlugin[]} List of loaded ProxyPlugins * @public */ this.plugins = []; - } - - /** - * Load configured plugins as modules and set each concrete ProxyPlugin - * to this.plugins for use in proxying. - */ - load() { - const modulePromises = []; - for (const path of this.paths) { - modulePromises.push(this._loadFilePlugin(path)); + if (this.targets.length === 0) { + console.log('No plugins configured'); // TODO: log.debug() + return; } - for (const name of this.names) { - modulePromises.push(this._loadPackagePlugin(name)); + const modulePromises = []; + for (const target of this.targets) { + modulePromises.push(this._loadPlugin(target)); } Promise.all(modulePromises).then((vals) => { const modules = vals; - console.log(`Found ${modules.length} plugin modules`); + console.log(`Found ${modules.length} plugin modules`); // TODO: log.debug() const pluginObjPromises = []; for (const mod of modules) { pluginObjPromises.push(this._castToPluginObjects(mod)); @@ -45,45 +37,37 @@ class PluginLoader { for (const pluginObjs of vals) { this.plugins = this.plugins.concat(pluginObjs); } - console.log(`Loaded ${this.plugins.length} plugins`); + console.log(`Loaded ${this.plugins.length} plugins`); // TODO: log.debug() }); }); } /** - * Load a plugin module from a relative file path to the - * current working directory. - * @param {string} filepath - * @return {Module} - */ - async _loadFilePlugin(filepath) { - const lp = await lpModule; - const resolvedModuleFile = await lp.resolvePlugin(path.join(process.cwd(), filepath)); - return await lp.loadPlugin(resolvedModuleFile); - } - - /** - * Load a plugin module from the specified Node module. Only - * modules with the prefix "@finos" are supported. - * @param {string} packageName + * Load a plugin module from either a file path or a Node module. + * @param {string} target * @return {Module} */ - async _loadPackagePlugin(packageName) { + async _loadPlugin(target) { const lp = await lpModule; - const resolvedPackageFile = await lp.resolvePlugin(packageName, { - prefix: '@finos', - }); - return await lp.loadPlugin(resolvedPackageFile); + try { + const resolvedModuleFile = await lp.resolvePlugin(target); + return await lp.loadPlugin(resolvedModuleFile); + } catch (err) { + return Promise.reject(err); + } } /** * Set a list of ProxyPlugin objects to this.plugins * from the keys exported by the passed in module. - * @param {Module} pluginModule + * @param {object} pluginModule * @return {ProxyPlugin} */ async _castToPluginObjects(pluginModule) { const plugins = []; + if (pluginModule instanceof ProxyPlugin) { + return [pluginModule]; + } // iterate over the module.exports keys for (const key of Object.keys(pluginModule)) { if ( @@ -106,39 +90,70 @@ class ProxyPlugin {} /** * A plugin which executes a function when receiving a proxy request. */ +class PushActionPlugin extends ProxyPlugin { +/** + * Custom function executed as part of the action chain. The function + * must take in two parameters: an Express Request and the current Action + * executed in the chain. This function should return a Promise that resolves + * to an Action. + * + * @param {function} exec - A function that: + * - Takes in an Express Request object as the first parameter (`req`). + * - Takes in an Action object as the second parameter (`action`). + * - Returns a Promise that resolves to an Action. + */ + constructor(exec) { + super(); + this.exec = exec; + } +} + class ActionPlugin extends ProxyPlugin { - /** - * Custom function executed as part of the action chain. The function - * must take in two parameters, an {@link https://expressjs.com/en/4x/api.html#req Express Request} - * and the current Action executed in the chain. - * @param {Promise} exec A Promise that returns an Action & - * executes when a push is proxied. - */ constructor(exec) { super(); this.exec = exec; } } +/** + * + * @param {Array} targets A list of loadable targets for plugin modules. + * @return {PluginLoader} + */ const createLoader = async () => { - // Auto-register plugins that are part of git-proxy core - let names = []; - let files = []; - if (process.env.GITPROXY_PLUGIN_PACKAGES !== undefined) { - names = process.env.GITPROXY_PLUGIN_PACKAGES.split(','); - } - if (process.env.GITPROXY_PLUGIN_FILES !== undefined) { - files = process.env.GITPROXY_PLUGIN_FILES.split(','); + const loadTargets = [...config.getPushPlugins()] + if (process.env.GITPROXY_PLUGIN_FILES) { + console.log('Note: GITPROXY_PLUGIN_FILES is deprecated. Please configure plugins to load via configuration (proxy.config.json).') + const pluginFiles = process.env.GITPROXY_PLUGIN_FILES.split(','); + loadTargets.push(...pluginFiles); } - const loader = new PluginLoader(names, files); - if (names.length + files.length > 0) { - loader.load(); + if (process.env.GITPROXY_PLUGIN_PACKAGES) { + console.log('Note: GITPROXY_PLUGIN_PACKAGES is deprecated. Please configure plugins to load via configuration (proxy.config.json).') + const pluginPackages = process.env.GITPROXY_PLUGIN_PACKAGES.split(','); + loadTargets.push(...pluginPackages); } + const loader = new PluginLoader(loadTargets); return loader; }; -module.exports.defaultLoader = createLoader(); -module.exports.ProxyPlugin = ProxyPlugin; -module.exports.ActionPlugin = ActionPlugin; -// exported for testing only -module.exports.createLoader = createLoader; +/** + * Default PluginLoader used by the proxy chain. This is a singleton. + * @type {PluginLoader} + */ +let _defaultLoader; + +module.exports = { + get defaultLoader() { + if (!_defaultLoader) { + _defaultLoader = createLoader(); + } + return _defaultLoader; + }, + set defaultLoader(loader) { + _defaultLoader = loader; + }, + ProxyPlugin, + ActionPlugin, // deprecated + PushActionPlugin, + createLoader +} \ No newline at end of file diff --git a/src/proxy/chain.js b/src/proxy/chain.js index c389a989..12aeeafa 100644 --- a/src/proxy/chain.js +++ b/src/proxy/chain.js @@ -53,7 +53,7 @@ const getChain = async (action) => { if (!pluginsLoaded && pluginActions.length > 0) { console.log(`Found ${pluginActions.length}, inserting into proxy chain`); for (const pluginAction of pluginActions) { - if (pluginAction instanceof plugin.ActionPlugin) { + if (pluginAction instanceof plugin.PushActionPlugin || pluginAction instanceof plugin.ActionPlugin) { console.log(`Inserting plugin ${pluginAction} into chain`); // insert custom functions after parsePush but before other actions pushActionChain.splice(1, 0, pluginAction.exec); diff --git a/src/proxy/index.js b/src/proxy/index.js index 7d6d7874..8b24e2d3 100644 --- a/src/proxy/index.js +++ b/src/proxy/index.js @@ -7,6 +7,7 @@ const path = require("path"); const router = require('./routes').router; const config = require('../config'); const db = require('../db'); +const plugin = require('../plugin'); const { GIT_PROXY_SERVER_PORT: proxyHttpPort } = require('../config/env').Vars; const { GIT_PROXY_HTTPS_SERVER_PORT: proxyHttpsPort } = require('../config/env').Vars; @@ -26,7 +27,7 @@ const start = async () => { // Check to see if the default repos are in the repo list const defaultAuthorisedRepoList = config.getAuthorisedList(); const allowedList = await db.getRepos(); - + plugin.defaultLoader = plugin.createLoader(); defaultAuthorisedRepoList.forEach(async (x) => { const found = allowedList.find((y) => y.project === x.project && x.name === y.name); if (!found) {