diff --git a/docs/future/README.md b/docs/future/README.md deleted file mode 100644 index 09a65648f..000000000 --- a/docs/future/README.md +++ /dev/null @@ -1,11 +0,0 @@ -Future -====== - -A collection of docs describing and outlining the history of OpenFn internals -and ideas for the future. - -## Table of Contents - -- [History of Core](history.md) -- [Kit Components](kit-components.md) - diff --git a/docs/future/diagrams/README.md b/docs/future/diagrams/README.md deleted file mode 100644 index 2c2125cf4..000000000 --- a/docs/future/diagrams/README.md +++ /dev/null @@ -1,17 +0,0 @@ -# Diagrams - -Diagrams are created using [draw.io](https://app.diagrams.net/). - -Editing diagrams can be done in the browser via the Draw.io web app, however -in order to programmically export the files to SVG for use in markdown -it is suggested to install the desktop version. - -See [here](https://github.com/jgraph/drawio-desktop/releases) to get the -appropriate version for your OS. - -## Exporting to SVG - -1. Install draw.io -2. Export the diagram to SVG: - `drawio --export --format svg` - diff --git a/docs/future/diagrams/core-cli-requirements.drawio b/docs/future/diagrams/core-cli-requirements.drawio deleted file mode 100644 index b065be4d7..000000000 --- a/docs/future/diagrams/core-cli-requirements.drawio +++ /dev/null @@ -1 +0,0 @@ -7VnbctowEP0aHsP4BoTHBEI7nbaTSSbT5KkjbMVWIluuLAfTr+8KS75gg8kEhtDkCe9qZUt7zlmvTM+ehNkXjuLgB/Mw7VmGl/Xsac+yTNOw4Ed6lspjWIPc43PiKV/puCV/sQ5U3pR4OKkFCsaoIHHd6bIowq6o+RDnbFEPe2S0/tQY+bjhuHURbXp/EU8EuffcGpX+r5j4gX6yORznIyHSwWonSYA8tqi47KuePeGMifwqzCaYyuzpvOTzZhtGi4VxHIldJvjf7qz7+Ma4mV5cTSN2d3n3h5+ZQ7U4sdQ7xh4kQJmMi4D5LEL0qvRecpZGHpa3NcEqY74zFivnExZiqdBEqWDgCkRI1SismC/vwTD6QAdlP0hbG9OsZi2V9cgiMUMhodIxYSknmMPqf+LFKtS7kIjD0Jwy9zl3zQjVj00EZ88FjADAZb55ueONSVWuBB7m4i2ZVEQXiPtYbMt4iT2oBrMQwwZhIscUCfJSXwhS7PWLuBJguFAYvwJvtcoXRFP1pJ41pLDeyzlc+GKVJSSw9sJDioEikmvPU8KiBn+aWa4gDyKIZVyY+bJg9B8pW7gB4qLvMTcNIetywiIgAt/GaJXwBQTKmTmZFGO2EuEFc0FAwheU+BGMC0lLhbUcw9l2tJvg6Alay7qaDZW9KEuDaShfUCkLOm7veJqDI+v3f5GvfRLytbvli7OY4yQhUpm7afgjKdgaDd6bgkdHVvDoqBLGGRHVTgDMB71GuC5XIQ29iD3qfnwSuh936x567xS4AVCgcMPre13mSsmQNkQppgwEHa6JvYFXjDmBPQHoaxOvy4FW9dcpi7iruGlatdJgd3FsD0XAMetFwBq3FIFBSxFwDlYEjG6AXcZ3bMtwht1UoLmkwyyCM5nE3eUkFg0O1FFpw63CBjRPGE0FvijBq0PptFLm0HBaa12Zc96EszhPVuEsCv3+8Tw/alEvjIde/Yx1en2ZqQ/9By/Qq6mwU7SsBMSMRCKp3PlaOkruDZ069wbGYI08+R1LKhVLewO7zO5q8UiAOb8/z3Ibq8bQeHdnubbD+RogPhSJeOPu1UewVeWvfnp6xavRXsuK05IVqyUr54fKyrA7KSfN0i1U2PzGOzZVnU9Qmh+H1pRzbIwG3S8JfWbY5bDwYeDUh4Jj42c6R+0hjb5j29U+8szoG12N5MqqnAS7u8uWLwJd/eYeu8vBjs3l+I295QYigFn+/5N3h+XfaPbVPw== \ No newline at end of file diff --git a/docs/future/diagrams/core-cli-requirements.svg b/docs/future/diagrams/core-cli-requirements.svg deleted file mode 100644 index 628d414a5..000000000 --- a/docs/future/diagrams/core-cli-requirements.svg +++ /dev/null @@ -1,3 +0,0 @@ - - -
state
json
state...
expression
js
expression...
Module Name
Module Name
core
executable/node script
core...
final_state
json
final_state...
Module
ModuleText is not SVG - cannot display \ No newline at end of file diff --git a/docs/future/diagrams/core-compilation-steps.drawio b/docs/future/diagrams/core-compilation-steps.drawio deleted file mode 100644 index 04d2d0563..000000000 --- a/docs/future/diagrams/core-compilation-steps.drawio +++ /dev/null @@ -1 +0,0 @@ -7ZpLc5tADIB/jWfaQz08DNhH20n6mD4OOTQ5bkCGbReWLEuM++urhcWYkIfblEAzOQVptSAkfRIQT+x1XLwXJI2+8ADYxDKCYmKfTCzLNA0L/yjNTmsMy6k0oaCB1jWKc/oLakOtzWkAWctQcs4kTdtKnycJ+LKlI0Lwbdtsw1n7qikJoaM49wnrar/TQEaVdm55jf4D0DCqr2y6i2olJrWxvpMsIgHfHqjs04m9FpzL6igu1sBU9Oq4VPvO7lndOyYgkcds+Mi2gQfezPj2iQdOsrxerON35lw7J3f1HUOAAdAiFzLiIU8IO220K8HzJAB1WhOlxuYz56lW/gApdzqbJJccVZGMmV7d8ESekZgyVRVrngsKAp34ChieFSTBUiUOl64Y939WqjPK6t2ZFPznPhsYx1V1D8rxe2OjVRlezIcHAqLrVRIRgnzAbrbPINY+8Bik2OE+AYxIetP2g+gaDPd2TZrwQGfqD7KmnbwhLK+TVqQCsozypJPPbrgOMoFFmSq7uAgVwNMN41s/IkJOA+7nMYZPbdhGVMJ5SsrIbdFQ7aySa0wRaHt1A0JSpGbJaJigWqpKuFUo2mm0hOLhPHXjWm9wNEy6nVi2lrcNm2YNXHTApWv0lArTegWoFRD7SICcIQGyOwCRgKSSixdNjzM+ehZD0IPhEruLMvxOLV4qsRZOipa009KIqZsdSZ07JHWzLnU+F92J1U7wXfgcJJNcZZzlEpbC1wkvtY00uzP2D+ayh0llu0ey5vTGmv3K2j9hzTmStfmQrDkd1gRc51TAILRleEKahCh5z8De7Tk3Ava8QdgrqLw4OL5sMESpAU8JNXcNry1aG3j/P17dI3n1huTV7c7GTA7Cag/DbzY8gO6ww+/FwOT9D8PP68B0RpNAXancq16vLIPBTfmxcmK5TOqgqy+ERN+/e52rT3Ordvz3ajwKy2U85+n+60tWnw39rk5YWQ08dd1/w7Xt2FNnbGTPhhytzTi9PFh5bLS+mEfh+ZHdYDFkN5h3usE6AowUbtso5yNQvmCq1MA1aFIKKRZFNoJn5R4eh0fArDMos3/zOGxOXsoEXxzJrHlPVT0PtIsOtKucMjXDEzWFjc7YRY6J4peoV1zDx2UIyhrHGD55yL/JJJHwdpzDvac2sZcfaxOL3tqE0SmC7yp8ukk/MaeUbmBMCe3hvcvxhu70+39+v3b65+/0Zv2DiUdbvXF3WT1Pq6/d7AdzKMBHGN9Mp9NR9e9eXsf6BB7F5scq5drBb37s098= \ No newline at end of file diff --git a/docs/future/diagrams/core-compilation-steps.svg b/docs/future/diagrams/core-compilation-steps.svg deleted file mode 100644 index 249c1d463..000000000 --- a/docs/future/diagrams/core-compilation-steps.svg +++ /dev/null @@ -1,3 +0,0 @@ - - -
expression
expression
adaptor
adaptor
acorn
acorn
require
require
ast
ast
Find all top level CallExpressions
Find all top level...
Check if they exist in exports
Check if they exis...
Build new CallExpressions that are called with (state)
Build new CallExpr...
Wrap in iife
Wrap in iife
Wrap in execute(...)
Wrap in execute(...)Text is not SVG - cannot display \ No newline at end of file diff --git a/docs/future/diagrams/core-execution-steps.drawio b/docs/future/diagrams/core-execution-steps.drawio deleted file mode 100644 index d2940fa1b..000000000 --- a/docs/future/diagrams/core-execution-steps.drawio +++ /dev/null @@ -1 +0,0 @@ -7ZpZc5swEIB/jR+TAWFw+mg7STOdHjPJQ483GdagRiAihI376ysZiSM4rts4kdvpU9jVvbvf6nBG3jyt3nKcJx9YBHSEnKgaeZcjhFzXQfKP0my0xkF+rYk5ibSuVdyRH2Aqam1JIih6FQVjVJC8rwxZlkEoejrMOVv3qy0Z7Y+a4xgGirsQ06H2M4lEUmsv0KTV3wCJEzOyG7ypS1JsKuuVFAmO2Lqj8q5G3pwzJuqvtJoDVdYzdqnbXT9R2kyMQyYOafAtuz1b3ny6viuDKYsfPrwr08sz3UshNmbBEMn1a5FxkbCYZZhetdoZZ2UWgerVlVJb5z1juVZ+ByE22pm4FEyqEpFSXbpkmbjGKaEqKOas5AS4nMRHkNaZQRZNld9k0YKy8L5WXRNqWheCs/vGGdKMs3oNauJPmsasUw4Wwh576HAVmMcg9tQLGgfK0AeWguAb2Y4DxYKs+vPAOgTjpl7rJfmhHfUbTtOTXGFa6pFGKKByurOF/IjF1khYgNHKQZqCpiY3mu8FywZRMDRyx38yknNVL61iRf35krJ1mGAuziMWlqk0umqwToiAuxxv7b2WFVXLOiScc5kFvNkKuCAStSklcSbVQsXPo/DaGyvaDrIbqPa7fugq08DXeOoEhTwtr1vaXYNw0iE9cF7Iu95/JHv2GB+I5IVNJMcDJHGEc8H46ZJ1BHj8k4PHtwGPtBbffNla3zfiVyUa4bLqSRstnTB0wYHQuc7uAHkd6oIBdbeAIzVSFtULL2AQEH1372Kp41q8KBgtBUx5qN2/1bbSeKcnep69AboChePL7FhecCB0/ktBh6ycIhvq3B5zLYJ/Qt3Qlb/k8IjUXRxIHbJK3cWAOg4PJeF2UCtkhySLpTR5hePi4x3PPnxvrLBXEfGl890hT0oteEow3LW75BF5tbtLmkeKX2+TyCawZpodYqd5Dts90twHlZXV+wjWCw4eSvUwMesbvFE/dc+suzFXTUfGjTSj3I0XrLKSHV5gvx1bRx4Fe5kHumDr5+O+7UaK5i7xz5953UMff9yJVZqHzz+3ZaaMUOUcioKo5xyHZJbJa/fl4Dgcer537p8Yie5k4IvPnKiMKIOhKKlQFlCm32bJOh1GpLj/W3Oh5497HvAntj2A0Gvkwt2Jb2I18+3Iz0dMhujgo43VuwgaHm3mHGracJ0UISxFnRObdOiURQ3mM08/qxQ9efb5N1Ou/fsOGp/Q4eeYTw+Wf/jy/oqrjJlmh/eQpTmhED06AZ3qs3sLaQfZWw1O0Krew9JojkDy43eLg49Of/BSL8X2V+1tWeefA7yrnw== \ No newline at end of file diff --git a/docs/future/diagrams/core-execution-steps.svg b/docs/future/diagrams/core-execution-steps.svg deleted file mode 100644 index 77bf2d0df..000000000 --- a/docs/future/diagrams/core-execution-steps.svg +++ /dev/null @@ -1,3 +0,0 @@ - - -
state
json
state...
adaptor
adaptor
Read and parse
Read and parse
require
require
Append state to sandbox
Append state to sand...
Run expression in sandbox
Run expression in...
Write resulting state to disk
Write resulting stat...
Create an execution sandbox using vm2
Create an executio...
compiled expression
compiled expressionText is not SVG - cannot display \ No newline at end of file diff --git a/docs/future/diagrams/kit-components.drawio b/docs/future/diagrams/kit-components.drawio deleted file mode 100644 index 845a5079b..000000000 --- a/docs/future/diagrams/kit-components.drawio +++ /dev/null @@ -1 +0,0 @@ -7Vvdd+IoFP9rfKwnAROTx9bW6c7p7M45nd2Z2TdMMDJNQoZgrf3rlxhIzIc2Y6PonH0yXOAC9+MH94IDOIlePjCULD5RH4cDYPgvA3g7AMA0oCl+Mso6p4wdkBMCRnzZqCQ8klesekrqkvg4rTTklIacJFWiR+MYe7xCQ4zRVbXZnIbVURMU4Abh0UNhk/qV+HwhqabtlhX3mAQLObQDxnlFhFRjuZJ0gXy62iLBuwGcMEp5/hW9THCYCU/JJe833VFbTIzhmHfp8PjxJ/iLOl8+fjFfrX/+/fuVxPgK2DmbZxQu5YrlbPlaiYDRZezjjIs5gDerBeH4MUFeVrsSShe0BY9CWZ1yRp8KUYlF3iDmSbWKCYriLKXhkuPrgpx1k7PAjOOXneszC6kJc8M0wpytRRPVYSQFLS3NcmR5taU31WaxpTLTkkQkbSUoeJfiFB9Sor8iXfPo0u1BcCO3KjgIWwRntwgOuMcSnOk25IR94ZeySBlf0IDGKLwrqTdVSZZtHihNJPEH5nwtzQ4tOa1KF78Q/k18G8OxJYvft6puM3kZqrBWhVgs+Nt24fuGBbBUuey3KamOcxrzKYpImBHucfiMOfHQRsmI8esMu0RFTGOsaFOSSVGy8lWLWUi9p5wkG+yzlEyM++1ESJ0umYf3qUcCMWIB5m/af9PwGA4RJ8/VifTvfUbD+5hgTLAYWAzno4RTJr6G/pCnrfb2gGZiO6vYCApJEItvTwgNM0HIXE/oLbyWFRHx/dwccUpe0WzDL1NZQknMN2u0bgbW7T4L2OnUcpeTXMu9ZVuJexxqJwRcGUPDtp2cWWftSHafs4WVvOwqmlyNqwzofJ4Ko6krt5jTO0Cjoe47n2xU3C/kzoWXTWgoGGfcoOtOp67bDxQDqyo8022BYtACxfbRkBhqQeICVIdWBVYPxVS9eAkuAi/NUcOBxNHbY2RWwcszwklGuZALjWXPI8Am3OuuxhAawK3CHegFRC1Q5QrBqVAUNIxgIoI6QfmAY8xQvmUKPwnk8JcBrJZ5dsDqNAStA2m7nUIVZm4fQ80qrMqDaomqxrFRFXZEVXOsE1Wh5g20GpWcW1CiQi4VY+WzLMKv9pCrz525cyij14hasMIOuVRYltdCcpn2z2WWULqZiJUTnGH1n3hVksVXkP0iYUzr16w+ZyNmlXPKqxV5xuoUogifGX4eDjLIVLw9GiUk3PAsR1PcScnoyJvGZDKdHmfTsEbaNw2zIT6Np/Gi7gSncR+li2INPQLAqKP/Wzrdv3kwLxxUudUn6i+FtkuHm/2CL4sTNDDiJDqFv/bgly6s+aU6S2nzS+ttBd3TlAupAUOYOU7fo6hlnDwFQ4G2F6qukQo0talLT3a57yPZASjadorr8zTlXAKamloCrXNIaXWK6Xq0h67msCPLchpzaMbhExpFG6S9Lu4I/hA6oWmCvU26SYA48khIOMHNiwP9gFs/t0JLN+DCs3K5g8+tF3AzN+7ocnrjWTXN/gLaL+tkk6tO+M6Q9uz91B7rji9VDnJLL18pe5qHwp6BcUtQwFDUtySPmPY1a/dpI6BdwqP/kfDtdKCcpkoIDk6VDbS7oifQiZ6g93Qgw8jjV7mbnwQ9j/cKrObxlvaLHrA7OXCgtnD4dPUj/d00Zdu6NQX1ZFcPwLxLwfM+sbnzydbRis29n2y3rlVO4e9HvE8Z17FZu8eDZvBfxvwPFPkkDs492q9LVX96VT2J0fy04RwSru332+AM77ffi5o7HoM6NeMc1YwuX4HsVdpdg1H9EgE6oyqjfIUNRn09i1LDtz8m3uQEfRyh2G+19N/nITF48yGxCR05yDttR72cKd7A1UznkDdwolj+4yZvXv5vCd79Bw== \ No newline at end of file diff --git a/docs/future/diagrams/kit-components.svg b/docs/future/diagrams/kit-components.svg deleted file mode 100644 index 110930044..000000000 --- a/docs/future/diagrams/kit-components.svg +++ /dev/null @@ -1,3 +0,0 @@ - - -
retrieve adaptor .d.ts
retrieve adaptor .d.ts
Editor
Editor
describe adaptor
describe adaptor
Code Generator Widget
Code Generator Widget
analyzer
Prev. "compiler"
analyzer...
Module
on npm
Module...
Hosted Files
on unpkg.com
Hosted Files...
Common Adaptor Introspection Facilities
Common Adaptor Intro...
Typescript
Typescript
Workflow Diagram
Workflow Diagram
react-flow
react-flow
elk-js
elk-js
compiler
compiler
Adaptor Loading Facilities
Adaptor Loading Faci...
retrieve on demand
retrieve on demandText is not SVG - cannot display \ No newline at end of file diff --git a/docs/future/editor.md b/docs/future/editor.md deleted file mode 100644 index 216bd8eab..000000000 --- a/docs/future/editor.md +++ /dev/null @@ -1,25 +0,0 @@ -## Editor - -### Monaco - -**Adding libraries to Moanco** - -```ts -monaco.languages.typescript.typescriptDefaults.addExtraLib('const arr = [];') -``` - -**Hiding Editor Lines** - -https://github.com/microsoft/monaco-editor/issues/45#issuecomment-1159168677 - -```ts -interface IMyStandaloneCodeEditor extends monaco.editor.IStandaloneCodeEditor { - setHiddenAreas(range: monaco.IRange[]): void; -} - -... - -const casted = editor as IMyStandaloneCodeEditor; -const range = new monaco.Range(1, 0, 1, 0); -casted.setHiddenAreas([range]); -``` \ No newline at end of file diff --git a/docs/future/history.md b/docs/future/history.md deleted file mode 100644 index 99f1af56b..000000000 --- a/docs/future/history.md +++ /dev/null @@ -1,73 +0,0 @@ -# History of Core - -## How core works - -OpenFn [`core`](https://github.com/OpenFn/core) is an npm module, -that both transpiles and executes a job based on the following inputs: - -- A state file -- An adaptor path or module name -- An output path for "final state" - -![](diagrams/core-cli-requirements.svg) - -## Compilation - -The first thing that core does is try to compile an expression. -Since we allow users to write function calls -(without having to write require or import statements), -we have to transpile the code to be able to reference the adaptor functions. - -![](diagrams/core-compilation-steps.svg) - -The process outlined above would result in the transformation shown below: - -**Input** - -```js -fn((state) => { - return { ...state, counter: 1 }; -}); - -fn(({ counter, ...rest }) => { - return { ...rest, counter: counter + 1 }; -}); -``` - -**Output** - -```js -fn((state) => { - console.log(state); - return state; -})(function () { - return execute( - fn((state) => { - return { ...state, counter: 1 }; - }), - fn(({ counter, ...rest }) => { - return { ...rest, counter: counter + 1 }; - }), - fn((state) => { - console.log(state); - return state; - }) - )(state); -})(); -``` - -The execute function is an async reducer (using vanilla promises, -as async/await and generators were in Stage 0 at the time of implementation). -This pattern allows users to write code that ‘feels’ synchronous but is -executed asynchronously. - -## Execution - -The execution environment ties all the work together. - -![](diagrams/core-execution-steps.svg) - -It’s important to note that code is executed in a sandboxed environment, -where the vast majority of NodeJS root functionality is not available. -We also check in the compilation step that function calls that are not in -the ‘scope’ of our sandbox and throw errors in these cases. diff --git a/docs/future/kit-components.md b/docs/future/kit-components.md deleted file mode 100644 index 96bd0745f..000000000 --- a/docs/future/kit-components.md +++ /dev/null @@ -1,21 +0,0 @@ -Kit Components -============== - -![Draft Status](https://img.shields.io/badge/status-draft-red) - - -![](diagrams/kit-components.svg) - -## Analyzer - -See [here](../../packages/compiler/) - -## Compiler - -Produce transpiled/transformed expressions - -To be used from the worker to compile a job (or set of jobs) in the required -structure. - -Would contain all the transforms, and the helpers borne from creating the -transforms.