OSX HTML::Pipeline::MarkdownFilter Fails on Right Double Quotation Mark around email address about html-pipeline

[ericgoodwin]

cc/ @gjtorikian @mdiep

[ericgoodwin]

It looks like it is thinking the Right Double Quotation Mark is part of the email address when doing the autolink. This could be related to this bug. vmg/redcarpet#388

[jch]

@ericgoodwin do you have a backtrace of the error?

[ericgoodwin]

Not sure it's much help because there is only one line that isn't Pry, but here it is.

[2] pry(main)> renderer.to_html("“[email protected]”").split
ArgumentError: invalid byte sequence in UTF-8
from (pry):2:in `split'
[3] pry(main)> wtf
Exception: ArgumentError: invalid byte sequence in UTF-8
--
0: (pry):2:in `split'
1: (pry):2:in `__pry__'
2: /Users/ericgoodwin/.rbenv/versions/2.1.5/lib/ruby/gems/2.1.0/gems/pry-0.10.1/lib/pry/pry_instance.rb:355:in `eval'
3: /Users/ericgoodwin/.rbenv/versions/2.1.5/lib/ruby/gems/2.1.0/gems/pry-0.10.1/lib/pry/pry_instance.rb:355:in `evaluate_ruby'
4: /Users/ericgoodwin/.rbenv/versions/2.1.5/lib/ruby/gems/2.1.0/gems/pry-0.10.1/lib/pry/pry_instance.rb:323:in `handle_line'

The error can also be created by just using github-markdown as well.

[7] pry(main)> GitHub::Markdown.to_html("“[email protected]”", :gfm).split
ArgumentError: invalid byte sequence in UTF-8
from (pry):4:in `split'

[ericgoodwin]

Just found out it's an issue with Ruby versions as well.
Pasting the same text ("“[email protected]”") into irb Ruby 1.9 vs 2.1.3

Ruby 2.1.3

irb(main):001:0> "“[email protected]”"
=> "“[email protected]”"

Ruby 1.9

irb(main):001:0> "\[email protected]\U+FFE2"
=> "[email protected]"

It would appear that Ruby 1.9 is converting the characters automatically.

[jch]

@ericgoodwin 👍 thanks for digging in. I'm going to close this issue for now because I can't think of anything specific to take action on. If you find more details about the encoding, please continue to comment here. I'm happy to reopen if there is a fix that can be applied against the gem.

[ericgoodwin]

@jch Ok. Wasn't quite sure where to post this in the first place. I might keep on adding a few comments on here if that's ok with you so anyone else who runs into the issue can find some information on it. The issue seems to be with the github-markdown binary.

[jch]

👍 Yep keep 'em coming