Skip to content

Commit 36db48a

Browse files
authored
Merge pull request #12 from tom-lord/globally_configurable_results_limiters
Globally configurable results limiters
2 parents 7ae5278 + 67b1747 commit 36db48a

File tree

8 files changed

+227
-110
lines changed

8 files changed

+227
-110
lines changed

README.md

Lines changed: 25 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -12,7 +12,6 @@ Extends the `Regexp` class with the methods: `Regexp#examples` and `Regexp#rando
1212

1313
\* If the regex has an infinite number of possible strings that match it, such as `/a*b+c{2,}/`,
1414
or a huge number of possible matches, such as `/.\w/`, then only a subset of these will be listed.
15-
1615
For more detail on this, see [configuration options](#configuration-options).
1716

1817
If you'd like to understand how/why this gem works, please check out my [blog post](https://tom-lord.github.io/Reverse-Engineering-Regular-Expressions/) about it.
@@ -149,7 +148,9 @@ When generating examples, the gem uses 3 configurable values to limit how many e
149148

150149
`Rexexp#examples` makes use of *all* these options; `Rexexp#random_example` only uses `max_repeater_variance`, since the other options are redundant.
151150

152-
To use an alternative value, simply pass the configuration option as follows:
151+
### Defining custom configuration values
152+
153+
To use an alternative value, you can either pass the configuration option as a parameter:
153154

154155
```ruby
155156
/a*/.examples(max_repeater_variance: 5)
@@ -162,19 +163,41 @@ To use an alternative value, simply pass the configuration option as follows:
162163
#=> "A very unlikely result!"
163164
```
164165

166+
Or, set an alternative value *within a block*:
167+
168+
```ruby
169+
RegexpExamples::Config.with_configuration(max_repeater_variance: 5) do
170+
# ...
171+
end
172+
```
173+
174+
Or, globally set a different default value:
175+
176+
```ruby
177+
# e.g In a rails project, you may wish to place this in
178+
# config/initializers/regexp_examples.rb
179+
RegexpExamples::Config.max_repeater_variance = 5
180+
RegexpExamples::Config.max_group_results = 10
181+
RegexpExamples::Config.max_results_limit = 20000
182+
```
183+
165184
A sensible use case might be, for example, to generate all 1-5 digit strings:
166185

167186
```ruby
168187
/\d{1,5}/.examples(max_repeater_variance: 4, max_group_results: 10, max_results_limit: 100000)
169188
#=> ['0', '1', '2', ..., '99998', '99999']
170189
```
171190

191+
### Configuration Notes
192+
172193
Due to code optimisation, `Regexp#random_example` runs pretty fast even on very complex patterns.
173194
(I.e. It's typically a _lot_ faster than using `/pattern/.examples.sample(1)`.)
174195
For instance, the following takes no more than ~ 1 second on my machine:
175196

176197
`/.*\w+\d{100}/.random_example(max_repeater_variance: 1000)`
177198

199+
All forms of configuration mentioned above **are thread safe**.
200+
178201
## Bugs and TODOs
179202

180203
There are no known major bugs with this library. However, there are a few obscure issues that you *may* encounter:

lib/core_extensions/regexp/examples.rb

Lines changed: 6 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -5,19 +5,15 @@ module Regexp
55
# No core classes are extended in any way, other than the above two methods.
66
module Examples
77
def examples(**config_options)
8-
RegexpExamples::ResultCountLimiters.configure!(
9-
max_repeater_variance: config_options[:max_repeater_variance],
10-
max_group_results: config_options[:max_group_results],
11-
max_results_limit: config_options[:max_results_limit]
12-
)
13-
examples_by_method(:result)
8+
RegexpExamples::Config.with_configuration(config_options) do
9+
examples_by_method(:result)
10+
end
1411
end
1512

1613
def random_example(**config_options)
17-
RegexpExamples::ResultCountLimiters.configure!(
18-
max_repeater_variance: config_options[:max_repeater_variance]
19-
)
20-
examples_by_method(:random_result).sample(1).first
14+
RegexpExamples::Config.with_configuration(config_options) do
15+
examples_by_method(:random_result).sample
16+
end
2117
end
2218

2319
private

lib/regexp-examples/constants.rb

Lines changed: 41 additions & 25 deletions
Original file line numberDiff line numberDiff line change
@@ -1,9 +1,47 @@
11
# :nodoc:
22
module RegexpExamples
33
# Configuration settings to limit the number/length of Regexp examples generated
4-
class ResultCountLimiters
4+
class Config
5+
class << self
6+
def with_configuration(**new_config)
7+
original_config = config.dup
8+
9+
begin
10+
self.config = new_config
11+
result = yield
12+
ensure
13+
self.config = original_config
14+
end
15+
16+
result
17+
end
18+
19+
# Thread-safe getters and setters
20+
%i[max_repeater_variance max_group_results max_results_limit].each do |m|
21+
define_method(m) do
22+
config[m]
23+
end
24+
define_method("#{m}=") do |value|
25+
config[m] = value
26+
end
27+
end
28+
29+
private
30+
31+
def config=(**args)
32+
Thread.current[:regexp_examples_config].merge!(args)
33+
end
34+
35+
def config
36+
Thread.current[:regexp_examples_config] ||= {
37+
max_repeater_variance: MAX_REPEATER_VARIANCE_DEFAULT,
38+
max_group_results: MAX_GROUP_RESULTS_DEFAULT,
39+
max_results_limit: MAX_RESULTS_LIMIT_DEFAULT
40+
}
41+
end
42+
end
543
# The maximum variance for any given repeater, to prevent a huge/infinite number of
6-
# examples from being listed. For example, if @@max_repeater_variance = 2 then:
44+
# examples from being listed. For example, if self.max_repeater_variance = 2 then:
745
# .* is equivalent to .{0,2}
846
# .+ is equivalent to .{1,3}
947
# .{2,} is equivalent to .{2,4}
@@ -12,7 +50,7 @@ class ResultCountLimiters
1250
MAX_REPEATER_VARIANCE_DEFAULT = 2
1351

1452
# Maximum number of characters returned from a char set, to reduce output spam
15-
# For example, if @@max_group_results = 5 then:
53+
# For example, if self.max_group_results = 5 then:
1654
# \d is equivalent to [01234]
1755
# \w is equivalent to [abcde]
1856
MAX_GROUP_RESULTS_DEFAULT = 5
@@ -22,28 +60,6 @@ class ResultCountLimiters
2260
# /[ab]{30}/.examples
2361
# (Which would attempt to generate 2**30 == 1073741824 examples!!!)
2462
MAX_RESULTS_LIMIT_DEFAULT = 10_000
25-
class << self
26-
attr_reader :max_repeater_variance, :max_group_results, :max_results_limit
27-
def configure!(max_repeater_variance: nil,
28-
max_group_results: nil,
29-
max_results_limit: nil)
30-
@max_repeater_variance = (max_repeater_variance || MAX_REPEATER_VARIANCE_DEFAULT)
31-
@max_group_results = (max_group_results || MAX_GROUP_RESULTS_DEFAULT)
32-
@max_results_limit = (max_results_limit || MAX_RESULTS_LIMIT_DEFAULT)
33-
end
34-
end
35-
end
36-
37-
def self.max_repeater_variance
38-
ResultCountLimiters.max_repeater_variance
39-
end
40-
41-
def self.max_group_results
42-
ResultCountLimiters.max_group_results
43-
end
44-
45-
def self.max_results_limit
46-
ResultCountLimiters.max_results_limit
4763
end
4864

4965
# Definitions of various special characters, used in regular expressions.

lib/regexp-examples/helpers.rb

Lines changed: 5 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -10,10 +10,13 @@ module RegexpExamples
1010
# Edge case:
1111
# permutations_of_strings [ [] ] #=> nil
1212
# (For example, ths occurs during /[^\d\D]/.examples #=> [])
13-
def self.permutations_of_strings(arrays_of_strings, max_results_limiter = MaxResultsLimiterByProduct.new)
13+
def self.permutations_of_strings(arrays_of_strings,
14+
max_results_limiter = MaxResultsLimiterByProduct.new)
1415
partial_result = max_results_limiter.limit_results(arrays_of_strings.shift)
1516
return partial_result if arrays_of_strings.empty?
16-
partial_result.product(permutations_of_strings(arrays_of_strings, max_results_limiter)).map do |result|
17+
partial_result.product(
18+
permutations_of_strings(arrays_of_strings, max_results_limiter)
19+
).map do |result|
1720
join_preserving_capture_groups(result)
1821
end
1922
end

lib/regexp-examples/max_results_limiter.rb

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,6 @@
11
module RegexpExamples
2-
class MaxResultsLimiter # Base class
2+
# Abstract (base) class to assist limiting Regexp.examples max results
3+
class MaxResultsLimiter
34
def initialize(initial_results_count)
45
@results_count = initial_results_count
56
end
@@ -25,7 +26,8 @@ def cumulate_total(new_results_count, cumulator_method)
2526

2627
def results_allowed_from(partial_results, limiter_method)
2728
partial_results.first(
28-
RegexpExamples.max_results_limit.public_send(limiter_method, @results_count)
29+
RegexpExamples::Config.max_results_limit
30+
.public_send(limiter_method, @results_count)
2931
)
3032
end
3133
end

lib/regexp-examples/repeaters.rb

Lines changed: 11 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -10,7 +10,7 @@ def initialize(group)
1010
end
1111

1212
def result
13-
group_results = group.result.first(RegexpExamples.max_group_results)
13+
group_results = group.result.first(RegexpExamples::Config.max_group_results)
1414
results = []
1515
max_results_limiter = MaxResultsLimiterBySum.new
1616
min_repeats.upto(max_repeats) do |repeats|
@@ -51,7 +51,7 @@ class StarRepeater < BaseRepeater
5151
def initialize(group)
5252
super
5353
@min_repeats = 0
54-
@max_repeats = RegexpExamples.max_repeater_variance
54+
@max_repeats = RegexpExamples::Config.max_repeater_variance
5555
end
5656
end
5757

@@ -61,7 +61,7 @@ class PlusRepeater < BaseRepeater
6161
def initialize(group)
6262
super
6363
@min_repeats = 1
64-
@max_repeats = RegexpExamples.max_repeater_variance + 1
64+
@max_repeats = RegexpExamples::Config.max_repeater_variance + 1
6565
end
6666
end
6767

@@ -80,19 +80,14 @@ class RangeRepeater < BaseRepeater
8080
def initialize(group, min, has_comma, max)
8181
super(group)
8282
@min_repeats = min || 0
83-
if max # e.g. {1,100} --> Treat as {1,3} (by default max_repeater_variance)
84-
@max_repeats = smallest(max, @min_repeats + RegexpExamples.max_repeater_variance)
85-
elsif has_comma # e.g. {2,} --> Treat as {2,4} (by default max_repeater_variance)
86-
@max_repeats = @min_repeats + RegexpExamples.max_repeater_variance
87-
else # e.g. {3} --> Treat as {3,3}
88-
@max_repeats = @min_repeats
89-
end
90-
end
91-
92-
private
93-
94-
def smallest(x, y)
95-
x < y ? x : y
83+
@max_repeats = if !has_comma
84+
@min_repeats
85+
else
86+
[
87+
max,
88+
@min_repeats + RegexpExamples::Config.max_repeater_variance
89+
].compact.min
90+
end
9691
end
9792
end
9893
end

spec/config_spec.rb

Lines changed: 135 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,135 @@
1+
RSpec.describe RegexpExamples::Config do
2+
3+
describe 'max_repeater_variance' do
4+
context 'as a passed parameter' do
5+
it 'with low limit' do
6+
expect(/[A-Z]/.examples(max_results_limit: 5))
7+
.to match_array %w(A B C D E)
8+
end
9+
it 'with (default) high limit' do
10+
expect(/[ab]{14}/.examples.length)
11+
.to be <= 10000 # NOT 2**14 == 16384, because it's been limited
12+
end
13+
it 'with (custom) high limit' do
14+
expect(/[ab]{14}/.examples(max_results_limit: 20000).length)
15+
.to eq 16384 # NOT 10000, because it's below the limit
16+
end
17+
it 'for boolean or groups' do
18+
expect(/[ab]{3}|[cd]{3}/.examples(max_results_limit: 10).length)
19+
.to eq 10
20+
end
21+
it 'for case insensitive examples' do
22+
expect(/[ab]{3}/i.examples(max_results_limit: 10).length)
23+
.to be <= 10
24+
end
25+
it 'for range repeaters' do
26+
expect(/[ab]{2,3}/.examples(max_results_limit: 10).length)
27+
.to be <= 10 # NOT 4 + 8 = 12
28+
end
29+
it 'for backreferences' do
30+
expect(/([ab]{3})\1?/.examples(max_results_limit: 10).length)
31+
.to be <= 10 # NOT 8 * 2 = 16
32+
end
33+
it 'for a complex pattern' do
34+
expect(/(a|[bc]{2})\1{1,3}/.examples(max_results_limit: 14).length)
35+
.to be <= 14 # NOT (1 + 4) * 3 = 15
36+
end
37+
end
38+
39+
context 'as a global setting' do
40+
before do
41+
@original = RegexpExamples::Config.max_results_limit
42+
RegexpExamples::Config.max_results_limit = 5
43+
end
44+
after do
45+
RegexpExamples::Config.max_results_limit = @original
46+
end
47+
48+
it 'sets limit without passing explicitly' do
49+
expect(/[A-Z]/.examples)
50+
.to match_array %w(A B C D E)
51+
end
52+
end
53+
end # describe 'max_results_limit'
54+
55+
describe 'max_repeater_variance' do
56+
context 'as a passed parameter' do
57+
it 'with a larger value' do
58+
expect(/a+/.examples(max_repeater_variance: 5))
59+
.to match_array %w(a aa aaa aaaa aaaaa aaaaaa)
60+
end
61+
it 'with a lower value' do
62+
expect(/a{4,8}/.examples(max_repeater_variance: 0))
63+
.to eq %w(aaaa)
64+
end
65+
end
66+
67+
context 'as a global setting' do
68+
before do
69+
@original = RegexpExamples::Config.max_repeater_variance
70+
RegexpExamples::Config.max_repeater_variance = 5
71+
end
72+
after do
73+
RegexpExamples::Config.max_repeater_variance = @original
74+
end
75+
76+
it 'sets limit without passing explicitly' do
77+
expect(/a+/.examples)
78+
.to match_array %w(a aa aaa aaaa aaaaa aaaaaa)
79+
end
80+
end
81+
end # describe 'max_repeater_variance'
82+
83+
describe 'max_group_results' do
84+
context 'as a passed parameter' do
85+
it 'with a larger value' do
86+
expect(/\d/.examples(max_group_results: 10))
87+
.to match_array %w(0 1 2 3 4 5 6 7 8 9)
88+
end
89+
it 'with a lower value' do
90+
expect(/\d/.examples(max_group_results: 3))
91+
.to match_array %w(0 1 2)
92+
end
93+
end
94+
95+
context 'as a global setting' do
96+
before do
97+
@original = RegexpExamples::Config.max_group_results
98+
RegexpExamples::Config.max_group_results = 10
99+
end
100+
after do
101+
RegexpExamples::Config.max_group_results = @original
102+
end
103+
104+
it 'sets limit without passing explicitly' do
105+
expect(/\d/.examples)
106+
.to match_array %w(0 1 2 3 4 5 6 7 8 9)
107+
end
108+
end
109+
end # describe 'max_group_results'
110+
111+
describe 'thread safety' do
112+
it 'uses thread-local global config values' do
113+
thread = Thread.new do
114+
RegexpExamples::Config.max_group_results = 1
115+
expect(/\d/.examples).to eq %w(0)
116+
end
117+
sleep 0.1 # Give the above thread time to run
118+
expect(/\d/.examples).to eq %w(0 1 2 3 4)
119+
thread.join
120+
end
121+
122+
it 'uses thread-local block config values' do
123+
thread = Thread.new do
124+
RegexpExamples::Config.with_configuration(max_group_results: 1) do
125+
expect(/\d/.examples).to eq %w(0)
126+
sleep 0.2 # Give the below thread time to run while this block is open
127+
end
128+
end
129+
sleep 0.1 # Give the above thread time to run
130+
expect(/\d/.examples).to eq %w(0 1 2 3 4)
131+
thread.join
132+
end
133+
end # describe 'thread safety'
134+
135+
end

0 commit comments

Comments
 (0)