parsers/base: make sure content always ends with an empty line #130
Conversation
For some whois output where the content does not end with an empty new line, the whois call was returning an 'Unexpected token: % (c) 2019 Canadian Internet Registration Authority, (http://www.cira.ca/)' error, for instance. This fixes that by forcing into existence that ending new line.
|
Looking at the build, looks like this is Ruby 2.5+ only because of |
|
Hah! Trying with |
|
Seems there are issues with other registrars now... Unrelated to what I changed |
|
@XaF I guess that as long as it doesn't introduce new test breaks, it's mergeable. (Interesting fixup technique by the way, I figure you show your fixups but offer squashing that right before the merge?) |
| @@ -363,7 +363,7 @@ def response_unavailable? | |||
| protected | |||
|
|
|||
| def content_for_scanner | |||
| @content_for_scanner ||= content.to_s.gsub(/\r\n/, "\n") | |||
| @content_for_scanner ||= "#{content.to_s.gsub(/\r\n/, "\n").gsub(/\n$/, '')}\n" | |||
There was a problem hiding this comment.
As a matter of readability, it's not immidiately clear what we're doing here and why we're doing it.
There was a problem hiding this comment.
What change would you suggest ?
Would that be acceptable ?
| @content_for_scanner ||= "#{content.to_s.gsub(/\r\n/, "\n").gsub(/\n$/, '')}\n" | |
| @content_for_scanner ||= begin | |
| content_str = content.to_s | |
| content_str.gsub!(/\r\n/, "\n") | |
| # We need a new line at the end, which exists in some cases, and does | |
| # not in others, we will thus remove the ending new line character if it | |
| # exists (or do nothing if it does not), then return the content appended | |
| # with a new line character | |
| content_str.gsub!(/\n$/, '') | |
| "#{content_str}\n" | |
| end |
| it "returns the part body with line feed normalized" do | ||
| instance = described_class.new(Whois::Record::Part.new(:body => "This is\r\nthe response.", :host => "whois.example.test")) | ||
| expect(instance.send(:content_for_scanner)).to eq("This is\nthe response.") | ||
| expect(instance.send(:content_for_scanner)).to eq("This is\nthe response.\n") |
There was a problem hiding this comment.
So I understand that it's better for the scanner to ingest whole lines (including linebreak) instead of just the string on it, thus rendering the line empty but still present, right?
Sorry for the delay in getting back to you for this @joallard (wow, and what a long delay...) |
For some whois output where the content does not end with an empty new line, the whois call was returning an
Unexpected token: % (c) 2019 Canadian Internet Registration Authority, (http://www.cira.ca/)error, for instance.This fixes that by forcing into existence that ending new line.
Note that this is going to be followed by another PR to support the last CIRA whois output, which is currently failing.