Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add no-super-linear-backtracking rule #242

Merged
merged 1 commit into from
Jun 12, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -109,6 +109,7 @@ The rules with the following star :star: are included in the `plugin:regexp/reco
| [regexp/no-lazy-ends](https://ota-meshi.github.io/eslint-plugin-regexp/rules/no-lazy-ends.html) | disallow lazy quantifiers at the end of an expression | |
| [regexp/no-optional-assertion](https://ota-meshi.github.io/eslint-plugin-regexp/rules/no-optional-assertion.html) | disallow optional assertions | |
| [regexp/no-potentially-useless-backreference](https://ota-meshi.github.io/eslint-plugin-regexp/rules/no-potentially-useless-backreference.html) | disallow backreferences that reference a group that might not be matched | |
| [regexp/no-super-linear-backtracking](https://ota-meshi.github.io/eslint-plugin-regexp/rules/no-super-linear-backtracking.html) | disallow exponential and polynomial backtracking | :wrench: |
| [regexp/no-useless-assertions](https://ota-meshi.github.io/eslint-plugin-regexp/rules/no-useless-assertions.html) | disallow assertions that are known to always accept (or reject) | |
| [regexp/no-useless-backreference](https://ota-meshi.github.io/eslint-plugin-regexp/rules/no-useless-backreference.html) | disallow useless backreferences in regular expressions | :star: |
| [regexp/no-useless-dollar-replacements](https://ota-meshi.github.io/eslint-plugin-regexp/rules/no-useless-dollar-replacements.html) | disallow useless `$` replacements in replacement string | |
Expand Down
1 change: 1 addition & 0 deletions docs/rules/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,7 @@ The rules with the following star :star: are included in the `plugin:regexp/reco
| [regexp/no-lazy-ends](./no-lazy-ends.md) | disallow lazy quantifiers at the end of an expression | |
| [regexp/no-optional-assertion](./no-optional-assertion.md) | disallow optional assertions | |
| [regexp/no-potentially-useless-backreference](./no-potentially-useless-backreference.md) | disallow backreferences that reference a group that might not be matched | |
| [regexp/no-super-linear-backtracking](./no-super-linear-backtracking.md) | disallow exponential and polynomial backtracking | :wrench: |
| [regexp/no-useless-assertions](./no-useless-assertions.md) | disallow assertions that are known to always accept (or reject) | |
| [regexp/no-useless-backreference](./no-useless-backreference.md) | disallow useless backreferences in regular expressions | :star: |
| [regexp/no-useless-dollar-replacements](./no-useless-dollar-replacements.md) | disallow useless `$` replacements in replacement string | |
Expand Down
92 changes: 92 additions & 0 deletions docs/rules/no-super-linear-backtracking.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,92 @@
---
pageClass: "rule-details"
sidebarDepth: 0
title: "regexp/no-super-linear-backtracking"
description: "disallow exponential and polynomial backtracking"
---
# regexp/no-super-linear-backtracking

> disallow exponential and polynomial backtracking

- :exclamation: <badge text="This rule has not been released yet." vertical="middle" type="error"> ***This rule has not been released yet.*** </badge>
- :wrench: The `--fix` option on the [command line](https://eslint.org/docs/user-guide/command-line-interface#fixing-problems) can automatically fix some of the problems reported by this rule.

## :book: Rule Details

This rule reports cases of exponential and polynomial backtracking.

These types of backtracking almost always cause an exponential or polynomial worst-case runtime. This super-linear worst-case runtime can be exploited by attackers in what is called [Regular expression Denial of Service - ReDoS][1].

<eslint-code-block fix>

```js
/* eslint regexp/no-super-linear-backtracking: "error" */

/* ✓ GOOD */
var foo = /a*b+a*$/;
var foo = /(?:a+)?/;

/* ✗ BAD */
var foo = /(?:a+)+$/;
var foo = /a*b?a*$/;
var foo = /(?:a|b|c+)*$/;
// not all cases can automatically be fixed
var foo = /\s*(.*?)(?=:)/;
var foo = /.+?(?=\s*=)/;
```

</eslint-code-block>

### Limitations

The rule only implements a very simplistic detection method and can only detect very simple cases of super-linear backtracking right now.

While the detection will improve in the future, this rule will never be able to perfectly detect all cases super-linear backtracking.


## :wrench: Options

```json
{
"regexp/no-super-linear-backtracking": ["error", {
"report": "certain"
}]
}
```

### `report`

Every input string that exploits super-linear worst-case runtime can be separated into 3 parts:

1. A prefix to leads to exploitable part of the regex.
2. A non-empty string that will be repeated to exploit the ambiguity.
3. A rejecting suffix that forces the regex engine to backtrack.

For some regexes it is not possible to find a rejecting suffix even though the regex contains exploitable ambiguity (e.g. `/(?:a+)+/`). These regexes are safe as long as they are used as is. However, regexes can also be used as building blocks to create more complex regexes. In this case, the ambiguity might cause super-linear backtracking in the composite regex.

This options control whether ambiguity that might cause super-linear backtracking will be reported.

- `report: "certain"` (_default_)

Only certain cases of super-linear backtracking will be reported.

This means that ambiguity will only be reported if this rule can prove that there exists a rejecting suffix.

- `report: "potential"`

All certain and potential cases of super-linear backtracking will be reported.

Potential cases are ones where a rejecting might be possible. Whether the reported potential cases are false positives or not has to be decided by the developer.

## :books: Further reading

- [Regular expression Denial of Service - ReDoS][1]
- [scslre]

[1]: https://owasp.org/www-community/attacks/Regular_expression_Denial_of_Service_-_ReDoS
[scslre]: https://github.com/RunDevelopment/scslre

## :mag: Implementation

- [Rule source](https://github.com/ota-meshi/eslint-plugin-regexp/blob/master/lib/rules/no-super-linear-backtracking.ts)
- [Test source](https://github.com/ota-meshi/eslint-plugin-regexp/blob/master/tests/lib/rules/no-super-linear-backtracking.ts)
166 changes: 166 additions & 0 deletions lib/rules/no-super-linear-backtracking.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,166 @@
import type { RegExpVisitor } from "regexpp/visitor"
import type { RegExpContext } from "../utils"
import { createRule, defineRegexpVisitor } from "../utils"
import { UsageOfPattern } from "../utils/get-usage-of-pattern"
import type { ParsedLiteral } from "scslre"
import { analyse } from "scslre"
import type { Position, SourceLocation } from "estree"

/**
* Returns the combined source location of the two given locations.
*/
function unionLocations(a: SourceLocation, b: SourceLocation): SourceLocation {
/** x < y */
function less(x: Position, y: Position): boolean {
if (x.line < y.line) {
return true
} else if (x.line > y.line) {
return false
}
return x.column < y.column
}

return {
start: { ...(less(a.start, b.start) ? a.start : b.start) },
end: { ...(less(a.end, b.end) ? b.end : a.end) },
}
}

/**
* Create a parsed literal object as required by the scslre library.
*/
function getParsedLiteral(context: RegExpContext): ParsedLiteral {
const { flags, flagsString, patternAst } = context

return {
pattern: patternAst,
flags: {
type: "Flags",
raw: flagsString ?? "",
parent: null,
start: NaN,
end: NaN,
dotAll: flags.dotAll ?? false,
global: flags.dotAll ?? false,
ignoreCase: flags.dotAll ?? false,
multiline: flags.dotAll ?? false,
sticky: flags.dotAll ?? false,
unicode: flags.dotAll ?? false,
},
}
}

export default createRule("no-super-linear-backtracking", {
meta: {
docs: {
description: "disallow exponential and polynomial backtracking",
category: "Possible Errors",
// TODO Switch to recommended in the major version.
// recommended: true,
recommended: false,
},
fixable: "code",
schema: [
{
type: "object",
properties: {
report: {
enum: ["certain", "potential"],
},
},
additionalProperties: false,
},
],
messages: {
self:
"This quantifier can reach itself via the loop '{{parent}}'." +
" Using any string accepted by {{attack}}, this can be exploited to cause at least polynomial backtracking." +
"{{exp}}",
trade:
"The quantifier '{{start}}' can exchange characters with '{{end}}'." +
" Using any string accepted by {{attack}}, this can be exploited to cause at least polynomial backtracking." +
"{{exp}}",
},
type: "problem",
},
create(context) {
const reportUncertain =
(context.options[0]?.report ?? "certain") === "potential"

/**
* Create visitor
*/
function createVisitor(
regexpContext: RegExpContext,
): RegExpVisitor.Handlers {
const {
node,
patternAst,
flags,
getRegexpLocation,
fixReplaceNode,
getUsageOfPattern,
} = regexpContext

const result = analyse(getParsedLiteral(regexpContext), {
reportTypes: { Move: false },
assumeRejectingSuffix:
reportUncertain &&
getUsageOfPattern() !== UsageOfPattern.whole,
})

for (const report of result.reports) {
const exp = report.exponential
? " This is going to cause exponential backtracking resulting in exponential worst-case runtime behavior."
: getUsageOfPattern() !== UsageOfPattern.whole
? " This might cause exponential backtracking."
: ""

const attack = `/${report.character.literal.source}+/${
flags.ignoreCase ? "i" : ""
}`

const fix = fixReplaceNode(
patternAst,
() => report.fix()?.source ?? null,
)

if (report.type === "Self") {
context.report({
node,
loc: getRegexpLocation(report.quant),
messageId: "self",
data: {
exp,
attack,
parent: report.parentQuant.raw,
},
fix,
})
} else if (report.type === "Trade") {
context.report({
node,
loc: unionLocations(
getRegexpLocation(report.startQuant),
getRegexpLocation(report.endQuant),
),
messageId: "trade",
data: {
exp,
attack,
start: report.startQuant.raw,
end: report.endQuant.raw,
},
fix,
})
}
}

return {}
}

return defineRegexpVisitor(context, {
createVisitor,
})
},
})
2 changes: 2 additions & 0 deletions lib/utils/rules.ts
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@ import noOctal from "../rules/no-octal"
import noOptionalAssertion from "../rules/no-optional-assertion"
import noPotentiallyUselessBackreference from "../rules/no-potentially-useless-backreference"
import noStandaloneBackslash from "../rules/no-standalone-backslash"
import noSuperLinearBacktracking from "../rules/no-super-linear-backtracking"
import noTriviallyNestedAssertion from "../rules/no-trivially-nested-assertion"
import noTriviallyNestedQuantifier from "../rules/no-trivially-nested-quantifier"
import noUnusedCapturingGroup from "../rules/no-unused-capturing-group"
Expand Down Expand Up @@ -87,6 +88,7 @@ export const rules = [
noOptionalAssertion,
noPotentiallyUselessBackreference,
noStandaloneBackslash,
noSuperLinearBacktracking,
noTriviallyNestedAssertion,
noTriviallyNestedQuantifier,
noUnusedCapturingGroup,
Expand Down
10 changes: 10 additions & 0 deletions package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

3 changes: 2 additions & 1 deletion package.json
Original file line number Diff line number Diff line change
Expand Up @@ -90,6 +90,7 @@
"jsdoctypeparser": "^9.0.0",
"refa": "^0.8.0",
"regexp-ast-analysis": "^0.2.2",
"regexpp": "^3.1.0"
"regexpp": "^3.1.0",
"scslre": "^0.1.5"
}
}
57 changes: 57 additions & 0 deletions tests/lib/rules/no-super-linear-backtracking.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,57 @@
import { RuleTester } from "eslint"
import rule from "../../../lib/rules/no-super-linear-backtracking"

const tester = new RuleTester({
parserOptions: {
ecmaVersion: 2020,
sourceType: "module",
},
})

tester.run("no-super-linear-backtracking", rule as any, {
valid: [
String.raw`/regexp/`,
String.raw`/a+b+a+b+/`,
String.raw`/\w+\b[\w-]+/`,
],
invalid: [
// self
{
code: String.raw`/b(?:a+)+b/`,
output: String.raw`/ba+b/`,
errors: [
"This quantifier can reach itself via the loop '(?:a+)+'. Using any string accepted by /a+/, this can be exploited to cause at least polynomial backtracking. This is going to cause exponential backtracking resulting in exponential worst-case runtime behavior.",
],
},
{
code: String.raw`/(?:ba+|a+b){2}/`,
output: null,
errors: [
"The quantifier 'a+' can exchange characters with 'a+'. Using any string accepted by /a+/, this can be exploited to cause at least polynomial backtracking. This might cause exponential backtracking.",
],
},

// trade
{
code: String.raw`/\ba+a+$/`,
output: String.raw`/\ba{2,}$/`,
errors: [
"The quantifier 'a+' can exchange characters with 'a+'. Using any string accepted by /a+/, this can be exploited to cause at least polynomial backtracking. This might cause exponential backtracking.",
],
},
{
code: String.raw`/\b\w+a\w+$/`,
output: String.raw`/\b\w[\dA-Z_b-z]*a\w+$/`,
errors: [
"The quantifier '\\w+' can exchange characters with '\\w+'. Using any string accepted by /a+/, this can be exploited to cause at least polynomial backtracking. This might cause exponential backtracking.",
],
},
{
code: String.raw`/\b\w+a?b{4}\w+$/`,
output: null,
errors: [
"The quantifier '\\w+' can exchange characters with '\\w+'. Using any string accepted by /b+/, this can be exploited to cause at least polynomial backtracking. This might cause exponential backtracking.",
],
},
],
})