diff --git a/Gemfile b/Gemfile index 2890fd83..be07f2bc 100644 --- a/Gemfile +++ b/Gemfile @@ -2,7 +2,7 @@ source 'https://rubygems.org' gemspec name: 'recog-content' -gem 'recog', '~>3.0' +gem 'recog', '~>3.1' group :test do gem 'rake' diff --git a/README.md b/README.md index f9541f65..52bd4a64 100644 --- a/README.md +++ b/README.md @@ -127,8 +127,6 @@ At least one `example` element should be present, however multiple `example` ele tests that `RomSShell_4.62` matches the provided regular expression and that the value of `service.version` is 4.62. -The `param` elements contain a `pos` attribute, which indicates what capture field from the `pattern` should be extracted, or `0` for a static string. The `name` attribute is the key that will be reported in the case of a successful match and the `value` will either be a static string for `pos` values of `0` or missing and taken from the captured field. - The `example` string can be base64 encoded to permit the use of unprintable characters. To signal this to Recog an `_encoding` attribute with the value of `base64` is added to the `example` element. Based64 encoded text that is longer than 80 characters may be wrapped with newlines as shown below to aid in readability. ```xml @@ -155,6 +153,51 @@ They can then be loaded using the `_filename` attribute: This is useful for long examples. +The `param` elements contain a `pos` attribute, which indicates what capture field +from the `pattern` should be extracted, or `0` for a static string. The `name` attribute +is the key that will be reported in the case of a successful match and the `value` +will either be a static string for `pos` values of `0` or missing and taken from the +captured field. + +The `value` attribute supports interpolation of data from other fields. This is +often useful when capturing the value for `hw.product` via regex and re-using this +value in `os.product`. + +Here is an example from`http_servers.xml` where `hw.product` is captured and reused. + +```xml + + Eltex TAU model VoIP gateway + Eltex TAU-72 + Eltex TAU-1.IP + + + + + + + +``` + +There is special handling for temporary attributes that have a name starting with +`_tmp.`. These attributes can be used for interpolation but are not emitted in the +output. This is useful when a particular product name is inconsistent in various +banners, vendor marketing, or with NIST values when trying to generate CPEs. In +these cases the useful parts of the banner can be extracted and a new value +crafted without cluttering the data emitted by a match. + +```xml + + NetCorp NX series switches + foo baz switchThing-8200 + + + + +``` + +These temporary attributes are not tracked in the `identifiers/fields.txt`. + [^back to top](#recog-a-recognition-framework) ## Contributing diff --git a/bin/recog_standardize b/bin/recog_standardize index 4cb6f28a..99582c55 100755 --- a/bin/recog_standardize +++ b/bin/recog_standardize @@ -59,7 +59,7 @@ end # @param current [Hash] Indentifiers extracted from fingerprints # @param original [Hash] Indentifiers loaded from the existing identifiers file -# param msg [String] Context to include in messaging to user +# @param msg [String] Context to include in messaging to user # @param ident_type [String] Key used to get the identifier file path # @param write [Boolean] Indicate if changes should be written to disk def handle_changes(current, original, msg, ident_type, write) @@ -151,6 +151,8 @@ ARGV.each do |arg| ndb.fingerprints.each do |f| f.params.each do |k, v| + # Don't track temporary attributes. + next if k.start_with?("_tmp.") curr_fields[k] = true param_index, val = v diff --git a/features/data/successful_tests.xml b/features/data/successful_tests.xml index 23772467..85875320 100755 --- a/features/data/successful_tests.xml +++ b/features/data/successful_tests.xml @@ -15,4 +15,11 @@ + + test of temp params + foo sb-1.0 + + + + diff --git a/features/verify.feature b/features/verify.feature index 1675e372..843d9856 100644 --- a/features/verify.feature +++ b/features/verify.feature @@ -12,7 +12,7 @@ Feature: Verify When I run `recog_verify successful_tests.xml` Then it should pass with exactly: """ - successful_tests.xml: SUMMARY: Test completed with 4 successful, 0 warnings, and 0 failures + successful_tests.xml: SUMMARY: Test completed with 5 successful, 0 warnings, and 0 failures """ @no-clobber