pygments / pygments.rb

💎 Ruby wrapper for Pygments syntax highlighter

Welcome to Pygments

This is the source of Pygments. It is a generic syntax highlighter written in Python that supports over 500 languages and text formats, for use in code hosting, forums, wikis or other applications that need to prettify source code.

Installing

... works as usual, use pip install Pygments to get published versions, or pip install -e . to install from a checkout in editable mode.

Documentation

... can be found online at https://pygments.org/ or created with Sphinx by :

tox -e doc

By default, the documentation does not include the demo page, as it requires having Docker installed for building Pyodide. To build the documentation with the demo page, use :

tox -e web-doc

The initial build might take some time, but subsequent ones should be instant because of Docker caching.

To view the generated documentation, serve it using Python's http.server module (this step is required for the demo to work) :

python3 -m http.server --directory doc/_build/html

Development

... takes place on GitHub, where the Git repository, tickets and pull requests can be viewed.

Continuous testing runs on GitHub workflows:

Please read our Contributing instructions.

Security considerations

Pygments provides no guarantees on execution time, which needs to be taken into consideration when using Pygments to process arbitrary user inputs. For example, if you have a web service which uses Pygments for highlighting, there may be inputs which will cause the Pygments process to run "forever" and/or use significant amounts of memory. This can subsequently be used to perform a remote denial-of-service attack on the server if the processes are not terminated quickly.

Unfortunately, it's practically impossible to harden Pygments itself against those issues: Some regular expressions can result in "catastrophic backtracking", but other bugs like incorrect matchers can also cause similar problems, and there is no way to find them in an automated fashion (short of solving the halting problem.) Pygments has extensive unit tests, automated randomized testing, and is also tested by OSS-Fuzz, but we will never be able to eliminate all bugs in this area.

Our recommendations are:

• Ensure that the Pygments process is terminated after a reasonably short timeout. In general Pygments should take seconds at most for reasonably-sized input.
• Limit the number of concurrent Pygments processes to avoid oversubscription of resources.

The Pygments authors will treat any bug resulting in long processing times with high priority -- it's one of those things that will be fixed in a patch release. When reporting a bug where you suspect super-linear execution times, please make sure to attach an input to reproduce it.

The authors

Pygments is maintained by Georg Brandl, e-mail address [email protected], Matthäus Chajdas and Jean Abou-Samra.

Many lexers and fixes have been contributed by Armin Ronacher, the rest of the Pocoo team and Tim Hatch.

The code is distributed under the BSD 2-clause license. Contributors making pull requests must agree that they are able and willing to put their contributions under that license.

pygments.rb's Issues

Fails when python is installed in a dir with space (on windows)

When the code determines the path to the python executable i may contain a path a space in it on Windows (e.g. c:\program files (x86)\python27). The path is later used as a command without quotation marks.
For Windows the command should be quoted just in case.

A workaround is to change the python path to c:\progra~2\pyhton27.

Add Pygments::Popen

Present

Running a python VM inside the current ruby process continues to be problematic. There are reports of segfaults, problems with FFI, problems rubypython has finding libpython, and open bugs with multi-vm signal handling while inside python code.

Some of these issues are specific to Pygments::FFI and rubypython. But the alternative, Pygments::C, is too immature to use in production and would at a minimum require added exception handling code.

Past

pygments.rb's predecessors, albino and multipygmentize, suffered from a limited API and poor performance.

multipygmentize somewhat improved performance, but required additional work by the caller to make batch calls.

The benchmark isolates this performance problem to python startup and pygments library loading cost. In pygments.rb, we pay this startup cost only once in Pygments.start.

Future

The ideal implementation of pygments.rb then, is an API compatible interface that only pays startup cost once, but also provides isolation from the python code. Thus, Pygments::Popen.

Instead of a new process per invocation (like with albino), we keep a long-running python child and communicate with it over a pipe. To maintain the existing API and allow for future expansion, the protocol over the pipe can be simple bert-style RPC.

Alternatively, we could add Pygments::Socket and talk to a single pygments service over a tcp or unix socket. The advantages of this approach are limited, however, compared to the added complexity of packaging, scaling and deployment.

Add license information

Hello,

I couldn't find any information about the license status of your code. Could you please add some license information to make it redistributable?

Thank you in advance.

How to set custom python installation with rubypython 0.6+

How to set custom python installation with rubypython 0.6+? This doesn't work:

RubyPython.configure :python_exe => 'python2.7'

because configure method removed in 0.6.0

0.2.1 gem doesn't install C extension required for Pygments::C

% sudo gem install pygments.rb
Building native extensions.  This could take a while...
Successfully installed ffi-1.0.9
Successfully installed blankslate-2.1.2.4
Successfully installed rubypython-0.5.1
Successfully installed pygments.rb-0.2.1
4 gems installed
Installing ri documentation for ffi-1.0.9...
Installing ri documentation for blankslate-2.1.2.4...
Installing ri documentation for rubypython-0.5.1...
Installing ri documentation for pygments.rb-0.2.1...
Installing RDoc documentation for ffi-1.0.9...
Installing RDoc documentation for blankslate-2.1.2.4...
Installing RDoc documentation for rubypython-0.5.1...
Installing RDoc documentation for pygments.rb-0.2.1...
% irb
>> require 'pygments/c'
> true
>> Pygments::C.highlight(File.read('Gemfile'), :lexer => 'ruby')
LoadError: no such file to load -- pygments_ext
    from /usr/lib/ruby/1.8/rubygems/custom_require.rb:31:in `gem_original_require'
    from /usr/lib/ruby/1.8/rubygems/custom_require.rb:31:in `require'
    from /var/lib/gems/1.8/gems/pygments.rb-0.2.1/lib/pygments/c.rb:7:in `start'
    from /var/lib/gems/1.8/gems/pygments.rb-0.2.1/lib/pygments/c.rb:45:in `highlight'
    from (irb):2

Update to Pygments-1.6

https://bitbucket.org/birkenfeld/pygments-main/src/tip/CHANGES

PHP highlighting regression

Pygments.rb doesn't fully highlight php unless <? is included. The old lexer highlighted php as expected even without <?.

https://github.com/github/gollum/issues/554

require 'pygments'

fail = %(var_dump(array(1 => 'foo'));)
ok = %(<?
var_dump(array(1 => 'foo'));)

puts '-- ok'
puts Pygments.highlight(ok, :lexer => 'php')

puts '-- fail'
puts Pygments.highlight(fail, :lexer => 'php')

=begin
-- ok
<div class="highlight"><pre><span class="cp">&lt;?</span>
<span class="nb">var_dump</span><span class="p">(</span><span class="k">array</span><span class="p">(</span><span class="mi">1</span> <span class="o">=&gt;</span> <span class="s1">&#39;foo&#39;</span><span class="p">));</span>
</pre></div>
-- fail
<div class="highlight"><pre><span class="x">var_dump(array(1 =&gt; &#39;foo&#39;));</span>
</pre></div>
=end

No way to add custom styles

There is no way to add a custom style, without forking the code and dropping it into the pygments styles directory. This seems…irritating.

I'm not sure the best way to go about this, but some way to point to a python script in the filesystem that contains a style would be welcome

mentos.py uses /usr/bin/python, desires python2.

Not sure how many distros are doing this now, but I know a few are using "python2" to indicate python 2.x nowadays - and some have set /usr/bin/python to mean python 3.x.

0.4.1+ breaks gollum's tests

I updated pygments.rb from 0.3.7 and now there are test failures that go away when pygments is reverted. Were there breaking changes from 0.3 to 0.4?

Pygments.rb 0.5.1 causing an error with Jekyll on Windows.

When using Jekyll on Windows with the 0.5.1 version it gives a cryptic error message and fails to build. Reverting to version 0.5.0 eliminates the problem. Here is a stack overflow post about it. http://stackoverflow.com/questions/17364028/jekyll-on-windows-pygments-not-working

Please tag releases

Hi!

Please tag releases. It would greatly improve and help usage, inclusion and
packaging of pygments.rb.

I intend to package it for Debian.

A first step would be to just tag the latest release now and then continue
tagging succeeding releases.

Best,
Per

Could not open library 'lib.so': lib.so: cannot open shared object file: No such file or directory (LoadError)

I am using Octopress blog engine which internally uses pygments.rb. I am using Archlinux with python version 3.2.3. I am getting the below error within pygments. Any help to fix this issue would be very helpful.

I have raised a request to octopress also - imathis/octopress#704

Building site: source -> public/libvarnam
File "", line 1
import sys; print sys.executable
^
SyntaxError: invalid syntax
sh: - : invalid option
Usage: sh [GNU long option] [option] ...
sh [GNU long option] [option] script-file ...
GNU long options:
--debug
--debugger
--dump-po-strings
--dump-strings
--help
--init-file
--login
--noediting
--noprofile
--norc
--posix
--protected
--rcfile
--restricted
--verbose
--version
Shell options:
-irsD or -c command or -O shopt_option (invocation only)
-abefhkmnptuvxBCHP or -o option
sh: - : invalid option
Usage: sh [GNU long option] [option] ...
sh [GNU long option] [option] script-file ...
GNU long options:
--debug
--debugger
--dump-po-strings
--dump-strings
--help
--init-file
--login
--noediting
--noprofile
--norc
--posix
--protected
--rcfile
--restricted
--verbose
--version
Shell options:
-irsD or -c command or -O shopt_option (invocation only)
-abefhkmnptuvxBCHP or -o option
/home/nkn/.rvm/gems/ruby-1.9.3-p194/gems/ffi-1.0.11/lib/ffi/library.rb:121:in block in ffi_lib': Could not open library 'lib.so': lib.so: cannot open shared object file: No such file or directory (LoadError)
from /home/nkn/.rvm/gems/ruby-1.9.3-p194/gems/ffi-1.0.11/lib/ffi/library.rb:88:inmap'
from /home/nkn/.rvm/gems/ruby-1.9.3-p194/gems/ffi-1.0.11/lib/ffi/library.rb:88:in ffi_lib'
from /home/nkn/.rvm/gems/ruby-1.9.3-p194/gems/rubypython-0.5.3/lib/rubypython/python.rb:29:inmodule:Python'
from /home/nkn/.rvm/gems/ruby-1.9.3-p194/gems/rubypython-0.5.3/lib/rubypython/python.rb:21:in <top (required)>'
from /home/nkn/.rvm/gems/ruby-1.9.3-p194/gems/rubypython-0.5.3/lib/rubypython.rb:261:inload'
from /home/nkn/.rvm/gems/ruby-1.9.3-p194/gems/rubypython-0.5.3/lib/rubypython.rb:261:in reload_library'
from /home/nkn/.rvm/gems/ruby-1.9.3-p194/gems/rubypython-0.5.3/lib/rubypython.rb:104:instart'
from /home/nkn/.rvm/gems/ruby-1.9.3-p194/gems/pygments.rb-0.2.13/lib/pygments/ffi.rb:8:in start'
from /home/nkn/.rvm/gems/ruby-1.9.3-p194/gems/pygments.rb-0.2.13/lib/pygments/ffi.rb:82:inhighlight'
from /home/nkn/open_source/octopress/plugins/pygments_code.rb:24:in pygments'
from /home/nkn/open_source/octopress/plugins/pygments_code.rb:14:inhighlight'
from /home/nkn/open_source/octopress/plugins/backtick_code_block.rb:37:in block in render_code_block'
from /home/nkn/open_source/octopress/plugins/backtick_code_block.rb:13:ingsub'
from /home/nkn/open_source/octopress/plugins/backtick_code_block.rb:13:in render_code_block'
from /home/nkn/open_source/octopress/plugins/octopress_filters.rb:12:inpre_filter'
from /home/nkn/open_source/octopress/plugins/octopress_filters.rb:28:in pre_render'
from /home/nkn/open_source/octopress/plugins/post_filters.rb:112:inblock in pre_render'
from /home/nkn/open_source/octopress/plugins/post_filters.rb:111:in each'
from /home/nkn/open_source/octopress/plugins/post_filters.rb:111:inpre_render'
from /home/nkn/open_source/octopress/plugins/post_filters.rb:166:in do_layout'
from /home/nkn/.rvm/gems/ruby-1.9.3-p194/gems/jekyll-0.11.2/lib/jekyll/post.rb:189:inrender'
from /home/nkn/.rvm/gems/ruby-1.9.3-p194/gems/jekyll-0.11.2/lib/jekyll/site.rb:193:in block in render'
from /home/nkn/.rvm/gems/ruby-1.9.3-p194/gems/jekyll-0.11.2/lib/jekyll/site.rb:192:ineach'
from /home/nkn/.rvm/gems/ruby-1.9.3-p194/gems/jekyll-0.11.2/lib/jekyll/site.rb:192:in render'
from /home/nkn/.rvm/gems/ruby-1.9.3-p194/gems/jekyll-0.11.2/lib/jekyll/site.rb:40:inprocess'
from /home/nkn/.rvm/gems/ruby-1.9.3-p194/gems/jekyll-0.11.2/bin/jekyll:250:in <top (required)>'
from /home/nkn/.rvm/gems/ruby-1.9.3-p194/bin/jekyll:23:inload'
from /home/nkn/.rvm/gems/ruby-1.9.3-p194/bin/jekyll:23:in `'

Request: Update Pygments revision

Pygments finally merged a boatload of queued pull requests with new lexers several days ago. Could you update to a new revision so that we can add support to linguist for new languages?

Pull lexer changes in from pygments upstream

The Objective-C lexer, at least, has been updated upstream and should be pulled in; the @ signs around dictionary and array literals should no longer appear as errors, as they currently do on GitHub:

@"Collection literals:" @{ @"test": @[ @1, @3, @3 ] }

https://bitbucket.org/birkenfeld/pygments-main/annotate/7304e4759ae65343d89a51359ca538912519cc31/pygments/lexers/compiled.py?at=default#cl-1354

Added a Lexer for the Pike Programming Language (pending)

I added a Lexer for the Pike Programming Language and have a pull request pending.

I've added this issue here for when it has been merged.

pygments_ext not found

hi, when i try to run bench.rb i get this:

serialhex@mobilhex [~/.rvm/gems/ruby-1.9.2-p290/gems/pygments.rb-0.2.1]$ ruby bench.rb                                                                          [ruby-1.9.2-p290] 
/home/serialhex/.rvm/rubies/ruby-1.9.2-p290/lib/ruby/site_ruby/1.9.1/rubygems/custom_require.rb:36:in `require': no such file to load -- pygments_ext (LoadError)
        from /home/serialhex/.rvm/rubies/ruby-1.9.2-p290/lib/ruby/site_ruby/1.9.1/rubygems/custom_require.rb:36:in `require'
        from /home/serialhex/.rvm/gems/ruby-1.9.2-p290/gems/pygments.rb-0.2.1/lib/pygments/c.rb:7:in `start'
        from /home/serialhex/.rvm/gems/ruby-1.9.2-p290/gems/pygments.rb-0.2.1/lib/pygments/c.rb:45:in `highlight'
        from bench.rb:14:in `<main>'

i manually looked for the file and there is none. from what i understand this uses embedded python, so i shouldn't need python installed (though it is). just so you know, i'm running archlinux & rvm with ruby 1.9.2 if you need any other information let me know
hex

Misapplied comment highlighting in C

Example:

https://github.com/falconindy/cower/blob/master/cower.c#L72

Text after // is italicized even though it is part of a string.

(from github-linguist/linguist#432)

Should pygments.rb handle lexer not found errors differently?

Pygments is awesome, but I've noticed that as soon as I implemented this people trying to use backticks with lexers that don't exist or accidentally using backticks without any intention of adding a code block now get errors from mentos.rb about that lexer not being found.

get_lexer_by_name raise ClassNotFound('no lexer for alias %r found' % _alias) ClassNotFound: no lexer for alias 'Go' found

I was curious from your perspective, if it should be pygments.rb's responsibility to catch the class not found exception and just return the code block with some vanilla/default formatting instead of raising this exception up.

Or should this be an exception dealt with higher up the stack?

silence pkg_resources warnings

--- vendor/gems/ruby/1.8/gems/pygments.rb-0.1.3/lib/pygments/ffi.rb.orig        2011-08-18 12:37:48.000000000 -0700
+++ vendor/gems/ruby/1.8/gems/pygments.rb-0.1.3/lib/pygments/ffi.rb     2011-08-18 12:37:53.000000000 -0700
@@ -6,6 +6,7 @@

     def start(pygments_path = File.expand_path('../../../vendor/Pygments-1.4/', __FILE__))
       RubyPython.start
+      RubyPython.import('pkg_resources') rescue nil
       sys = RubyPython.import('sys')
       sys.path.insert(0, pygments_path)
       sys.path.insert(0, File.expand_path('../../../vendor/python2-chardet-2.0.1/', __FILE__))

Gem name ?

Why is the gem named pygments.rb instead of simply pygments, since there is no other gem of that name (currently)?

I guess it kinda makes sense out of ruby, but it'd be simpler to gem install pygments in order to require "pygments". Of course one could make a pygments gem with the pygments.rb dependency (don't please :p).

Just asking. :)

RubyPython::PyObject::FFI (NameError)

pygments.rb is breaking gollum on travis. Rubypython 0.6.1 is out so why not update pygments.rb gemspec?

http://travis-ci.org/#!/github/gollum/jobs/1423363

$ bundle exec rake
/home/vagrant/.rvm/rubies/ruby-1.8.7-p358/bin/ruby -I"lib:lib:test:." -I"/home/vagrant/.rvm/gems/ruby-1.8.7-p358/gems/rake-0.9.2.2/lib" "/home/vagrant/.rvm/gems/ruby-1.8.7-p358/gems/rake-0.9.2.2/lib/rake/rake_test_loader.rb" "test/**/test_*.rb" 
/home/vagrant/.rvm/gems/ruby-1.8.7-p358/gems/rake-0.9.2.2/lib/rake/ext/module.rb:36:in `const_missing': uninitialized constant RubyPython::PyObject::FFI (NameError)
    from /home/vagrant/.rvm/gems/ruby-1.8.7-p358/gems/rubypython-0.5.3/lib/rubypython/pyobject.rb:14
    from /home/vagrant/.rvm/gems/ruby-1.8.7-p358/gems/rubypython-0.5.3/lib/rubypython.rb:35:in `require'
    from /home/vagrant/.rvm/gems/ruby-1.8.7-p358/gems/rubypython-0.5.3/lib/rubypython.rb:35
    from /home/vagrant/.rvm/gems/ruby-1.8.7-p358/gems/pygments.rb-0.2.12/lib/pygments/ffi.rb:1:in `require'

Regexp error with Clojure Lexer

ValueError: uncompilable regex 'fn(?![\\w-!$%*+,<=>?/.-])|def(?![\\w-!$%*+,<=>?/.-])|defn(?![\\w-!$%*+,<=>?/.-])|defmacro(?![\\w-!$%*+,<=>?/.-])|defmethod(?![\\w-!$%*+,<=>?/.-])|defmulti(?![\\w-!$%*+,<=>?/.-])|defn\\-(?![\\w-!$%*+,<=>?/.-])|defstruct(?![\ 

https://github.com/tmm1/pygments.rb/blob/master/vendor/pygments-main/pygments/lexers/agile.py#L1433

No tags since 0.5.0, no notes in changelog since 0.5.1

Looks like 0.5.2 and 0.5.4 have been released but not much documentation as to what's changed, any halp? Hoping to audit before bumping the Pages dependency.

If it's helpful, this is what we've been using for the pages gem, which was "borrowed" from elsewhere, but auto tags / pushes releases.

Syntax Highlighting doesn't work on `<code>` tags inside `<pre>` tags

I may be doing something wrong, but the syntax is being highlighted on <code> tags that are not within <pre> tags in my app.

In other words, this works:

<code>std::sort(data, data + arraySize);</code>

But, this doesn't:

<pre class="lang-cpp prettyprint-override"> <code>#include &lt;algorithm&gt; #include &lt;ctime&gt; #include &lt;iostream&gt; int main()</code> </pre>

How do I fix this? Is this an issue you guys are aware of?

Is there a workaround?

Thanks.

Pygments 0.3.x broken on CentOS 5.8

All tests are failing on CentOS 5.8. This issue impacts gollum. The old ffi pygments.rb worked correctly.

https://github.com/github/gollum/issues/550

custom Formatter

I'm having trouble adding a custom formatter. Pygments documentation says to do it through setuptools entrypoints but I don't see how that would work here. I placed my formatter in vendor/custom_formatters/my_formatter.py and added a load_formatters task to the rakefile identical to the load_lexers task but with the appropriate paths switched but it fails with

Traceback (most recent call last):
  File "_mapping.py", line 16, in <module>
    from pygments.util import docstring_headline
ImportError: No module named pygments.util
rake aborted!
Command failed with status (1): [python _mapping.py...]

Tasks: TOP => vendor:update => vendor:load_formatters
(See full trace by running task with --trace)

if I install pygments via pip and try again it fails with

Traceback (most recent call last):
  File "_mapping.py", line 61, in <module>
    module = __import__(module_name, None, None, [''])
ImportError: No module named my_formatter
rake aborted!
Command failed with status (1): [python _mapping.py...]

Tasks: TOP => vendor:update => vendor:load_formatters
(See full trace by running task with --trace)

suggestions?

Edit: running python _mapping.py from inside vendor/pygments-main/pygments/formatters/ without my formatter preset also produces the first error.

Table output mode is missing a closing '>'

If you set linenos: 'table', the output is missing a closing > on the </table> tag

Rails crashes with 'trace trap' error

When attempting to run Pygments without the lexer set, I get a trace trap crash with Rails.

zsh: trace trap  rails s

Any of the following will cause the crash:

Pygments.highlight(code)
Pygments.lexer_name_for(code)

This will not crash and works properly:

Pygments.highlight(code, :lexer => 'javascript')

Using this input:

function makeExpandingArea(container) {
  var area = container.querySelector('textarea'),
      span = container.querySelector('span');

 if (area.addEventListener) {
   area.addEventListener('input', function() {
     span.textContent = area.value;
   }, false);
   span.textContent = area.value;
 } else if (area.attachEvent) {
   // IE8 compatibility
   area.attachEvent('onpropertychange', function() {
     span.innerText = area.value;
   });
   span.innerText = area.value;
 }

Running Lion on Mac with Ruby 1.9.2, Rails 3.2.0. Here's my gemfile.lock:

  remote: git://github.com/rsl/stringex.git
  revision: cdac489cb78de158dd52cfbfa388a3c9c3069e4e
  specs:
    stringex (1.3.2)

GEM
  remote: https://rubygems.org/
  specs:
    actionmailer (3.2.0)
      actionpack (= 3.2.0)
      mail (~> 2.4.0)
    actionpack (3.2.0)
      activemodel (= 3.2.0)
      activesupport (= 3.2.0)
      builder (~> 3.0.0)
      erubis (~> 2.7.0)
      journey (~> 1.0.0)
      rack (~> 1.4.0)
      rack-cache (~> 1.1)
      rack-test (~> 0.6.1)
      sprockets (~> 2.1.2)
    activemodel (3.2.0)
      activesupport (= 3.2.0)
      builder (~> 3.0.0)
    activerecord (3.2.0)
      activemodel (= 3.2.0)
      activesupport (= 3.2.0)
      arel (~> 3.0.0)
      tzinfo (~> 0.3.29)
    activeresource (3.2.0)
      activemodel (= 3.2.0)
      activesupport (= 3.2.0)
    activesupport (3.2.0)
      i18n (~> 0.6)
      multi_json (~> 1.0)
    addressable (2.2.7)
    arel (3.0.2)
    aws-sdk (1.3.9)
      httparty (~> 0.7)
      json (~> 1.4)
      nokogiri (>= 1.4.4)
      uuidtools (~> 2.1)
    bcrypt-ruby (3.0.1)
    bcrypt-ruby (3.0.1-x86-mingw32)
    blankslate (2.1.2.4)
    builder (3.0.0)
    capistrano (2.9.0)
      highline
      net-scp (>= 1.0.0)
      net-sftp (>= 2.0.0)
      net-ssh (>= 2.0.14)
      net-ssh-gateway (>= 1.1.0)
    cocaine (0.2.1)
    coffee-rails (3.2.2)
      coffee-script (>= 2.2.0)
      railties (~> 3.2.0)
    coffee-script (2.2.0)
      coffee-script-source
      execjs
    coffee-script-source (1.2.0)
    erubis (2.7.0)
    execjs (1.3.0)
      multi_json (~> 1.0)
    faraday (0.7.6)
      addressable (~> 2.2)
      multipart-post (~> 1.1)
      rack (~> 1.1)
    ffi (1.0.11)
    highline (1.6.11)
    hike (1.2.1)
    httparty (0.8.1)
      multi_json
      multi_xml
    i18n (0.6.0)
    journey (1.0.3)
    jquery-rails (2.0.1)
      railties (>= 3.2.0, < 5.0)
      thor (~> 0.14)
    json (1.6.5)
    kaminari (0.13.0)
      actionpack (>= 3.0.0)
      activesupport (>= 3.0.0)
      railties (>= 3.0.0)
    linguist (0.4.0)
      rails (>= 3.0.0, < 4.0.0)
      redis (~> 2.2.0)
    mail (2.4.4)
      i18n (>= 0.4.0)
      mime-types (~> 1.16)
      treetop (~> 1.4.8)
    mime-types (1.18)
    multi_json (1.1.0)
    multi_xml (0.4.2)
    multipart-post (1.1.5)
    net-scp (1.0.4)
      net-ssh (>= 1.99.1)
    net-sftp (2.0.5)
      net-ssh (>= 2.0.9)
    net-ssh (2.3.0)
    net-ssh-gateway (1.1.0)
      net-ssh (>= 1.99.1)
    nokogiri (1.5.2)
    nokogiri (1.5.2-x86-mingw32)
    oauth (0.4.5)
    oauth2 (0.5.2)
      faraday (~> 0.7)
      multi_json (~> 1.0)
    paperclip (3.0.1)
      activemodel (>= 3.0.0)
      activerecord (>= 3.0.0)
      activesupport (>= 3.0.0)
      cocaine (>= 0.0.2)
      mime-types
    pg (0.13.2)
    pg (0.13.2-x86-mingw32)
    polyglot (0.3.3)
    pygments.rb (0.2.8)
      rubypython (~> 0.5.3)
    rack (1.4.1)
    rack-cache (1.2)
      rack (>= 0.4)
    rack-ssl (1.3.2)
      rack
    rack-test (0.6.1)
      rack (>= 1.0)
    rails (3.2.0)
      actionmailer (= 3.2.0)
      actionpack (= 3.2.0)
      activerecord (= 3.2.0)
      activeresource (= 3.2.0)
      activesupport (= 3.2.0)
      bundler (~> 1.0)
      railties (= 3.2.0)
    railties (3.2.0)
      actionpack (= 3.2.0)
      activesupport (= 3.2.0)
      rack-ssl (~> 1.3.2)
      rake (>= 0.8.7)
      rdoc (~> 3.4)
      thor (~> 0.14.6)
    rake (0.9.2.2)
    rdoc (3.12)
      json (~> 1.4)
    redcarpet (2.1.1)
    redis (2.2.2)
    rubypython (0.5.3)
      blankslate (>= 2.1.2.3)
      ffi (~> 1.0.7)
    sass (3.1.15)
    sass-rails (3.2.5)
      railties (~> 3.2.0)
      sass (>= 3.1.10)
      tilt (~> 1.3)
    slim (1.2.0)
      temple (~> 0.4.0)
      tilt (~> 1.3.3)
    sorcery (0.7.7)
      bcrypt-ruby (~> 3.0.0)
      oauth (~> 0.4.4)
      oauth2 (~> 0.5.1)
      sorcery
    sprockets (2.1.2)
      hike (~> 1.2)
      rack (~> 1.0)
      tilt (~> 1.1, != 1.3.0)
    sqlite3 (1.3.5)
    sqlite3 (1.3.5-x86-mingw32)
    temple (0.4.0)
    thor (0.14.6)
    tilt (1.3.3)
    treetop (1.4.10)
      polyglot
      polyglot (>= 0.3.1)
    tzinfo (0.3.32)
    uglifier (1.2.3)
      execjs (>= 0.3.0)
      multi_json (>= 1.0.2)
    uuidtools (2.1.2)

PLATFORMS
  ruby
  x86-mingw32

DEPENDENCIES
  aws-sdk (~> 1.3.4)
  bcrypt-ruby (~> 3)
  capistrano
  coffee-rails (~> 3)
  jquery-rails (~> 2)
  kaminari (~> 0.13)
  linguist
  paperclip (~> 3.0)
  pg (~> 0.13)
  pygments.rb
  rails (~> 3)
  redcarpet
  sass-rails (~> 3)
  slim
  sorcery
  sqlite3 (~> 1)
  stringex (~> 1)!
  uglifier (~> 1)

Add Stylus lexer

It would be awesome to have support for Stylus. A duplicate issue has been opened on the Stylus issue page.

Occasional segmentation fault doing `rake test` on Linux

This is probably a RubyPython error, but thought I would bring it up anyway. Doing rake test, bundle exec rake test, or bundle exec ruby -I lib test/test_pygments.rb all trigger the issue. Here is a dump file.

$ ruby -v
ruby 1.9.3p0 (2011-10-30 revision 33570) [x86_64-linux]
$ uname -a
Linux abe-PC 3.0.0-16-generic #28-Ubuntu SMP Fri Jan 27 17:44:39 UTC 2012 x86_64 x86_64 x86_64 GNU/Linux
$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 11.10
Release:    11.10
Codename:   oneiric

Encoding error

pygments.rb is not respecting characters while converting code.

# -*- encoding: utf-8 -*-
require "pygments"
puts Pygments.highlight("puts 'áéíóú'", :lexer => "ruby")

Here's the current output:

<div class="highlight"><pre><span class="nb">puts</span> <span class="s1">&#39;áéíóú&#39;</span>
    </pre>
    </div>

The expected output is:

<div class="highlight"><pre><span class="nb">puts</span> <span class="s1">&#39;áéíóú&#39;</span>
    </pre>
    </div>

MentosError (Failed to get header.)

I checked issue #45 but it doesn't seem to be the same case. I'm running my Rails app in production using Passenger and Nginx and I'm trying to highlight a piece of Ruby code. When I submit the code, I get a 500 error from my app. Looking at the Nginx logs, I see:

App 17862 stdout: Completed 500 Internal Server Error in 13ms
App 17862 stdout: 
App 17862 stdout: MentosError (Failed to get header.):
App 17862 stdout:   config/initializers/markdown_renderer.rb:3:in `block_code'
App 17862 stdout:   app/models/article.rb:16:in `render'
App 17862 stdout:   app/models/article.rb:16:in `cache_makdown'
App 17862 stdout:   app/controllers/articles_controller.rb:15:in `create'

I have checked all the python executables available on my Ubuntu server. They're all Python 2.7.6, which I installed from source a few hours ago.

$ which python
/usr/local/bin/python
$ which python2
/usr/local/bin/python2
$ python --version
Python 2.7.6
$python2 --version
Python 2.7.6

There's also another weird issue, where if I try using pygments from a ruby console, everything works fine:

$ irb
irb(main):001:0> require "pygments"
=> true
irb(main):002:0> Pygments.highlight("1 + 1", lexer: "ruby")
=> "<div class=\"highlight\"><pre><span class=\"mi\">1</span> <span class=\"o\">+</span> <span class=\"mi\">1</span>\n</pre></div>"

And even from the Rails console of the actual application:

$ RAILS_ENV=production bundle exec rails c
Loading production environment (Rails 4.0.0)
irb(main):001:0> Pygments.highlight("1 + 1", lexer: "ruby")
=> "<div class=\"highlight\"><pre><span class=\"mi\">1</span> <span class=\"o\">+</span> <span class=\"mi\">1</span>\n</pre></div>"

I am running Ruby 2.1.0, I have Ubuntu and Python installed from source. Does anyone know what could be the cause?

Pygments.highlight returning nil

I've tried running this directly to no avail and through Jekyll [where nil is shown].

ruby -e "require 'pygments'; puts Pygments.css('.highlight')"

Returns just an empty line. When I run Jekyll I get the 'nil' error:

~/workspaces/web-skin (master): jekyll serve
Configuration file: /Users/johnbland/workspaces/web-skin/_config.yml
            Source: ./static
       Destination: ./style-guide
      Generating...   Liquid Exception: undefined method `sub' for nil:NilClass in components.html
~/.rvm/gems/ruby-2.0.0-p195/gems/jekyll-1.0.2/lib/jekyll/tags/highlight.rb:77:in `add_code_tags'
~/.rvm/gems/ruby-2.0.0-p195/gems/jekyll-1.0.2/lib/jekyll/tags/highlight.rb:56:in `render_pygments'
~/.rvm/gems/ruby-2.0.0-p195/gems/jekyll-1.0.2/lib/jekyll/tags/highlight.rb:45:in `render'
~/.rvm/gems/ruby-2.0.0-p195/gems/liquid-2.5.0/lib/liquid/block.rb:106:in `block in render_all'
~/.rvm/gems/ruby-2.0.0-p195/gems/liquid-2.5.0/lib/liquid/block.rb:93:in `each'
~/.rvm/gems/ruby-2.0.0-p195/gems/liquid-2.5.0/lib/liquid/block.rb:93:in `render_all'
~/.rvm/gems/ruby-2.0.0-p195/gems/liquid-2.5.0/lib/liquid/block.rb:82:in `render'
~/.rvm/gems/ruby-2.0.0-p195/gems/liquid-2.5.0/lib/liquid/template.rb:124:in `render'
~/.rvm/gems/ruby-2.0.0-p195/gems/liquid-2.5.0/lib/liquid/template.rb:132:in `render!'
~/.rvm/gems/ruby-2.0.0-p195/gems/jekyll-1.0.2/lib/jekyll/convertible.rb:77:in `render_liquid'
~/.rvm/gems/ruby-2.0.0-p195/gems/jekyll-1.0.2/lib/jekyll/convertible.rb:128:in `do_layout'
~/.rvm/gems/ruby-2.0.0-p195/gems/jekyll-1.0.2/lib/jekyll/page.rb:111:in `render'
~/.rvm/gems/ruby-2.0.0-p195/gems/jekyll-1.0.2/lib/jekyll/site.rb:235:in `block in render'
~/.rvm/gems/ruby-2.0.0-p195/gems/jekyll-1.0.2/lib/jekyll/site.rb:233:in `each'
~/.rvm/gems/ruby-2.0.0-p195/gems/jekyll-1.0.2/lib/jekyll/site.rb:233:in `render'
~/.rvm/gems/ruby-2.0.0-p195/gems/jekyll-1.0.2/lib/jekyll/site.rb:44:in `process'
~/.rvm/gems/ruby-2.0.0-p195/gems/jekyll-1.0.2/lib/jekyll/command.rb:18:in `process_site'
~/.rvm/gems/ruby-2.0.0-p195/gems/jekyll-1.0.2/lib/jekyll/commands/build.rb:23:in `build'
~/.rvm/gems/ruby-2.0.0-p195/gems/jekyll-1.0.2/lib/jekyll/commands/build.rb:7:in `process'
~/.rvm/gems/ruby-2.0.0-p195/gems/jekyll-1.0.2/bin/jekyll:83:in `block (2 levels) in <top (required)>'
~/.rvm/gems/ruby-2.0.0-p195/gems/commander-4.1.3/lib/commander/command.rb:180:in `call'
~/.rvm/gems/ruby-2.0.0-p195/gems/commander-4.1.3/lib/commander/command.rb:180:in `call'
~/.rvm/gems/ruby-2.0.0-p195/gems/commander-4.1.3/lib/commander/command.rb:155:in `run'
~/.rvm/gems/ruby-2.0.0-p195/gems/commander-4.1.3/lib/commander/runner.rb:402:in `run_active_command'
~/.rvm/gems/ruby-2.0.0-p195/gems/commander-4.1.3/lib/commander/runner.rb:66:in `run!'
~/.rvm/gems/ruby-2.0.0-p195/gems/commander-4.1.3/lib/commander/delegates.rb:7:in `run!'
~/.rvm/gems/ruby-2.0.0-p195/gems/commander-4.1.3/lib/commander/import.rb:10:in `block in <top (required)>'

Pygments bombs out on FreeBSD, because of ffi

This is not strictly pygment.rb issue, but since it depends on ffi gem... just for awareness in case somebody else comes across it.

$ ruby -v
ruby 1.9.2p290 (2011-07-09 revision 32553) [i386-freebsd8.2]
$ irb
ruby-1.9.2-p290 :001 > require 'pygments.rb'
 => true 
ruby-1.9.2-p290 :002 > Pygments.highlight("puts 'hi'", :lexer => :ruby)
TypeError: unable to resolve type 'ssize_t'
    from /usr/local/rvm/gems/ruby-1.9.2-p290@blurgh/gems/ffi-1.0.9/lib/ffi/types.rb:45:in `find_type'
    from /usr/local/rvm/gems/ruby-1.9.2-p290@blurgh/gems/ffi-1.0.9/lib/ffi/library.rb:287:in `find_type'
    from /usr/local/rvm/gems/ruby-1.9.2-p290@blurgh/gems/ffi-1.0.9/lib/ffi/library.rb:116:in `block in attach_function'
    from /usr/local/rvm/gems/ruby-1.9.2-p290@blurgh/gems/ffi-1.0.9/lib/ffi/library.rb:116:in `map!'
    from /usr/local/rvm/gems/ruby-1.9.2-p290@blurgh/gems/ffi-1.0.9/lib/ffi/library.rb:116:in `attach_function'
    from /usr/local/rvm/gems/ruby-1.9.2-p290@blurgh/gems/rubypython-0.5.1/lib/rubypython/python.rb:76:in `<module:Python>'
    from /usr/local/rvm/gems/ruby-1.9.2-p290@blurgh/gems/rubypython-0.5.1/lib/rubypython/python.rb:21:in `<top (required)>'
    from /usr/local/rvm/gems/ruby-1.9.2-p290@blurgh/gems/rubypython-0.5.1/lib/rubypython.rb:261:in `load'
    from /usr/local/rvm/gems/ruby-1.9.2-p290@blurgh/gems/rubypython-0.5.1/lib/rubypython.rb:261:in `reload_library'
    from /usr/local/rvm/gems/ruby-1.9.2-p290@blurgh/gems/rubypython-0.5.1/lib/rubypython.rb:104:in `start'
    from /usr/local/rvm/gems/ruby-1.9.2-p290@blurgh/gems/pygments.rb-0.1.2/lib/pygments/ffi.rb:8:in `start'
    from /usr/local/rvm/gems/ruby-1.9.2-p290@blurgh/gems/pygments.rb-0.1.2/lib/pygments/ffi.rb:82:in `highlight'
    from (irb):2
    from /usr/local/rvm/rubies/ruby-1.9.2-p290/bin/irb:16:in `<main>'
ruby-1.9.2-p290 :003 > 

Copying ffi's ffi-1.0.9/lib/ffi/platform/i386-openbsd to i386-freebsd resolves it.

ruby-1.9.2-p290 :001 > require 'pygments.rb'
 => true 
ruby-1.9.2-p290 :002 > Pygments.highlight("puts 'hi'", :lexer => :ruby)
 => "<div class=\"highlight\"><pre><span class=\"nb\">puts</span> <span class=\"s1\">&#39;hi&#39;</span>\n</pre>\n</div>\n" 
ruby-1.9.2-p290 :003 > 

Crashes Rails 3.2 Application server process

When using the latest version of the pygments.rb gem within a Rails 3.2.3 app it crashes the entire rails application server process with a Killed signal displayed on the console and an error of 324 empty response returned by the server in the browser

This has been replicated on the following ruby versions:

Ruby 1.9.2 p0

Ruby 1.9.2 p290

Ruby 1.9.3

The application servers tested are:

Mongrel

Webrick

The system in use is Snow Leopard 10.6.2

thanks!

License missing from gemspec

Some companies will only use gems with a certain license.
The canonical and easy way to check is via the gemspec
via e.g.

spec.license = 'MIT'
# or
spec.licenses = ['MIT', 'GPL-2']

There is even a License Finder to help companies ensure all gems they use
meet their licensing needs. This tool depends on license information being available in the gemspec.
Including a license in your gemspec is a good practice, in any case.

How did I find you?

I'm using a script to collect stats on gems, originally looking for download data, but decided to collect licenses too,
and make issues for missing ones as a public service :)
https://gist.github.com/bf4/5952053#file-license_issue-rb-L13 So far it's going pretty well

Lib.so issues when calling Pygments::Lexer

When running the Pyagments::Lexer class (as shown below) it complains about the lib.so file not being found. As the issue seemed similar to issue #7 I've tried following the steps there but this did not resolve the issue, the output below also contains an attempt at using the RubyPython configuration but neither the options 'python' nor 'python2' seem to help (which makes sense as python is available under the 'python' alias rather than python2).
The path suggested for 64-bit RHEL as suggested on http://www.dejaaugustine.com/2011/10/rubypython-on-64-bit-rhel5centos/ did not resolve this issue either.

 => {:python_exe=>"python"} 
ruby-1.9.2-p290 :002 > Pygments::Lexer[:bash]
LoadError: Could not open library 'lib.so': lib.so: cannot open shared object file: No such file or directory
        from /home/ruby/.rvm/gems/ruby-1.9.2-p290@gitlab/gems/ffi-1.0.10/lib/ffi/library.rb:121:in`block in ffi_lib'
        from /home/ruby/.rvm/gems/ruby-1.9.2-p290@gitlab/gems/ffi-1.0.10/lib/ffi/library.rb:88:in `map'
        from /home/ruby/.rvm/gems/ruby-1.9.2-p290@gitlab/gems/ffi-1.0.10/lib/ffi/library.rb:88:in`ffi_lib'
        from /home/ruby/.rvm/gems/ruby-1.9.2-p290@gitlab/gems/rubypython-0.5.2/lib/rubypython/python.rb:29:in `<module:Python>'
        from /home/ruby/.rvm/gems/ruby-1.9.2-p290@gitlab/gems/rubypython-0.5.2/lib/rubypython/python.rb:21:in`<top (required)>'
        from /home/ruby/.rvm/gems/ruby-1.9.2-p290@gitlab/gems/activesupport-3.1.0/lib/active_support/dependencies.rb:234:in `load'
        from /home/ruby/.rvm/gems/ruby-1.9.2-p290@gitlab/gems/activesupport-3.1.0/lib/active_support/dependencies.rb:234:in`block in load'
        from /home/ruby/.rvm/gems/ruby-1.9.2-p290@gitlab/gems/activesupport-3.1.0/lib/active_support/dependencies.rb:223:in `block in load_dependency'
        from /home/ruby/.rvm/gems/ruby-1.9.2-p290@gitlab/gems/activesupport-3.1.0/lib/active_support/dependencies.rb:640:in`new_constants_in'
        from /home/ruby/.rvm/gems/ruby-1.9.2-p290@gitlab/gems/activesupport-3.1.0/lib/active_support/dependencies.rb:223:in `load_dependency'
        from /home/ruby/.rvm/gems/ruby-1.9.2-p290@gitlab/gems/activesupport-3.1.0/lib/active_support/dependencies.rb:234:in`load'
        from /home/ruby/.rvm/gems/ruby-1.9.2-p290@gitlab/gems/rubypython-0.5.2/lib/rubypython.rb:261:in `reload_library'
        from /home/ruby/.rvm/gems/ruby-1.9.2-p290@gitlab/gems/rubypython-0.5.2/lib/rubypython.rb:104:in`start'
        from /home/ruby/.rvm/gems/ruby-1.9.2-p290@gitlab/gems/pygments.rb-0.2.3/lib/pygments/ffi.rb:8:in `start'
        from /home/ruby/.rvm/gems/ruby-1.9.2-p290@gitlab/gems/pygments.rb-0.2.3/lib/pygments/ffi.rb:42:in`lexers'
        from /home/ruby/.rvm/gems/ruby-1.9.2-p290@gitlab/gems/pygments.rb-0.2.3/lib/pygments/lexer.rb:147:in `<module:Pygments>'
        from /home/ruby/.rvm/gems/ruby-1.9.2-p290@gitlab/gems/pygments.rb-0.2.3/lib/pygments/lexer.rb:1:in`<top (required)>'
        from (irb):2
        from /home/ruby/.rvm/gems/ruby-1.9.2-p290@gitlab/gems/railties-3.1.0/lib/rails/commands/console.rb:45:in `start'
        from /home/ruby/.rvm/gems/ruby-1.9.2-p290@gitlab/gems/railties-3.1.0/lib/rails/commands/console.rb:8:in`start'
        from /home/ruby/.rvm/gems/ruby-1.9.2-p290@gitlab/gems/railties-3.1.0/lib/rails/commands.rb:40:in `<top (required)>'
        from script/rails:6:in`require'
        from script/rails:6:in `<main>'

This is on a machine with Debian 6 x64 with ia32 libs installed, Python 2.6.6 and Ruby 1.9.2. I suppose this should just be able to run or am I overlooking another requirement? Thanks in advance either way.

Problem passing options to formatter

I'm currently working on a feature for an octopress plugin (imathis/octopress#478). I need to pass the linenostart option to the formatter.

Calling Pygments with Pygments.highlight(code, :lexer => lang, :formatter => 'html', :options => {:encoding => 'utf-8', :linenostart => 2}) has no effect. Unfortunately I'm not very good at reading and understanding python code, but the option gets properly passed to Pygments::FFI.formatter_for

Any idea what's going on?

Need a way to instruct pygments which python to load

The solution stated on the index doesn't work anymore. It looks like you need to pass :python_exe as a parameter to RubyPython.start(:python_exe => 'python2.7')

RubyPython.configure :python_exe => 'python2.7'

CodeRay inadvertently used?

I've been using pygments.rb for a little while now, and as I was working on a project, all of a sudden I noticed syntax highlighting was kind of borked. I inspected the generated HTML and realized it was using CodeRay. I didn't think I'd added CodeRay to the bundle, but then I realized I'd inadvertently done so by adding pry, which depends on CodeRay for syntax highlighting. I didn't dive too deep, but I'm assuming that's not intended. Or at the very least, if it is, there should probably be a way to make it play nicer.

jRuby compatibility

POSIX::Spawn causes issues with the latest version of jRuby. Is there a way to remove this dependency or alter it for jRuby so that it is not needed?

MentosError header errors

1.9.3p286 :001 > require "pygments"
 => true 
1.9.3p286 :002 > Pygments.highlight("foo")
MentosError: Failed to get header.
    from /home/adam/.rvm/gems/ruby-1.9.3-p286/gems/pygments.rb-0.3.2/lib/pygments/popen.rb:351:in `rescue in get_header'
    from /home/adam/.rvm/gems/ruby-1.9.3-p286/gems/pygments.rb-0.3.2/lib/pygments/popen.rb:332:in `get_header'
    from /home/adam/.rvm/gems/ruby-1.9.3-p286/gems/pygments.rb-0.3.2/lib/pygments/popen.rb:229:in `block in mentos'
    from /home/adam/.rvm/rubies/ruby-1.9.3-p286/lib/ruby/1.9.1/timeout.rb:68:in `timeout'
    from /home/adam/.rvm/gems/ruby-1.9.3-p286/gems/pygments.rb-0.3.2/lib/pygments/popen.rb:203:in `mentos'
    from /home/adam/.rvm/gems/ruby-1.9.3-p286/gems/pygments.rb-0.3.2/lib/pygments/popen.rb:186:in `highlight'
    from (irb):2
    from /home/adam/.rvm/rubies/ruby-1.9.3-p286/bin/irb:16:in `<main>'

As far as I can tell, this is happening on a clean gem environment.

Curiously, installing python-pygments and calling Pygmentize.start("pygmentize") first gives, for one single subsequent call to highlight a slightly different message:

1.9.3p286 :003 > Pygments.start("pygmentize"); Pygments.highlight("foo")
MentosError: No header received back.
    from /home/adam/.rvm/gems/ruby-1.9.3-p286/gems/pygments.rb-0.3.2/lib/pygments/popen.rb:295:in `handle_header_and_return'
    from /home/adam/.rvm/gems/ruby-1.9.3-p286/gems/pygments.rb-0.3.2/lib/pygments/popen.rb:232:in `block in mentos'
    from /home/adam/.rvm/rubies/ruby-1.9.3-p286/lib/ruby/1.9.1/timeout.rb:68:in `timeout'
    from /home/adam/.rvm/gems/ruby-1.9.3-p286/gems/pygments.rb-0.3.2/lib/pygments/popen.rb:203:in `mentos'
    from /home/adam/.rvm/gems/ruby-1.9.3-p286/gems/pygments.rb-0.3.2/lib/pygments/popen.rb:186:in `highlight'
    from (irb):3
    from /home/adam/.rvm/rubies/ruby-1.9.3-p286/bin/irb:16:in `<main>'

1.9.3p286 :004 > Pygments.highlight("foo")
MentosError: Failed to get header.
    from /home/adam/.rvm/gems/ruby-1.9.3-p286/gems/pygments.rb-0.3.2/lib/pygments/popen.rb:351:in `rescue in get_header'
    from /home/adam/.rvm/gems/ruby-1.9.3-p286/gems/pygments.rb-0.3.2/lib/pygments/popen.rb:332:in `get_header'
    from /home/adam/.rvm/gems/ruby-1.9.3-p286/gems/pygments.rb-0.3.2/lib/pygments/popen.rb:229:in `block in mentos'
    from /home/adam/.rvm/rubies/ruby-1.9.3-p286/lib/ruby/1.9.1/timeout.rb:68:in `timeout'
    from /home/adam/.rvm/gems/ruby-1.9.3-p286/gems/pygments.rb-0.3.2/lib/pygments/popen.rb:203:in `mentos'
    from /home/adam/.rvm/gems/ruby-1.9.3-p286/gems/pygments.rb-0.3.2/lib/pygments/popen.rb:186:in `highlight'
    from (irb):4
    from /home/adam/.rvm/rubies/ruby-1.9.3-p286/bin/irb:16:in `<main>'

Please update to latest pygments

Please update to latest pygments, it has new useful langs i’d like to use in github.

Lexers are loaded in random order

It seems that the Lexers are loaded in random order. In my test, I get occasional failures when highlighting a .txt file, depending on which was loaded into Pygments.rb last: "Text only" (most of the time) or CMake (occasionally). This produces some truly bizarre behavior. Is it possible to make the lexer ordering deterministic?

rake/gempackagetask is obsolete in Rakefile

ERROR: 'rake/gempackagetask' is obsolete and no longer supported. Use 'rubygems/package_task' instead.

I believe the Rakefile should be updated to:

require 'rubygems/package_task'
Gem::PackageTask.new(GEMSPEC) do |pkg|
end

Of course, a shim will need to be provided for old versions of rake/rubygems.

Add LESS language support

What would I have to do to get this to happen?

EPIPE pygments.rb (0.5.4) lib/pygments/popen.rb:261:in `rescue in mentos'

Hi,

This has been something that's happened on my site since I started using pygments.rb a long time ago. It seemed to be less prominent before and I could deal with it as long as I occassionally 'pinged' my server with curl on a cron job, but now that I updated, it's happening many times per day and people can't read my blog articles that have syntax highlighting in them (they get 500 error).

After they visit the page and it 500's you can refresh and it magically will work for about 5 minutes on any article you go to. Then it starts 500'ing again. My curl trick no longer works it seems, as I get exception notifications all day long.

An ActionView::Template::Error occurred in discussions#show:

 EPIPE
 pygments.rb (0.5.4) lib/pygments/popen.rb:261:in `rescue in mentos'

and

-------------------------------
Backtrace:
-------------------------------

 pygments.rb (0.5.4) lib/pygments/popen.rb:261:in `rescue in mentos'
 pygments.rb (0.5.4) lib/pygments/popen.rb:213:in `mentos'
 pygments.rb (0.5.4) lib/pygments/popen.rb:202:in `highlight'
 lib/pineapple_markdown_renderer.rb:11:in `block_code'
 lib/pineapple_markdown.rb:22:in `render'
 lib/pineapple_markdown.rb:22:in `to_html'
 app/helpers/comments_helper.rb:3:in `markdown'
 app/views/discussion_comments/_comment.html.erb:10:in `block in _app_views_discussion_comments__comment_html_erb__3322690692533492810_295911540'
 actionpack (3.2.15) lib/action_view/helpers/capture_helper.rb:40:in `block in capture'
 actionpack (3.2.15) lib/action_view/helpers/capture_helper.rb:187:in `with_output_buffer'
 actionpack (3.2.15) lib/action_view/helpers/capture_helper.rb:40:in `capture'
 actionpack (3.2.15) lib/action_view/helpers/record_tag_helper.rb:104:in `content_tag_for_single_record'
 actionpack (3.2.15) lib/action_view/helpers/record_tag_helper.rb:89:in `content_tag_for'
 actionpack (3.2.15) lib/action_view/helpers/record_tag_helper.rb:34:in `div_for'
 app/views/discussion_comments/_comment.html.erb:9:in `_app_views_discussion_comments__comment_html_erb__3322690692533492810_295911540'
 actionpack (3.2.15) lib/action_view/template.rb:145:in `block in render'
 activesupport (3.2.15) lib/active_support/notifications.rb:125:in `instrument'
 actionpack (3.2.15) lib/action_view/template.rb:143:in `render'
 actionpack (3.2.15) lib/action_view/renderer/partial_renderer.rb:351:in `block in collection_with_template'
 actionpack (3.2.15) lib/action_view/renderer/partial_renderer.rb:348:in `each'
 actionpack (3.2.15) lib/action_view/renderer/partial_renderer.rb:348:in `collection_with_template'
 actionpack (3.2.15) lib/action_view/renderer/partial_renderer.rb:250:in `render_collection'
 actionpack (3.2.15) lib/action_view/renderer/partial_renderer.rb:234:in `block in render'
 actionpack (3.2.15) lib/action_view/renderer/abstract_renderer.rb:38:in `block in instrument'

Any idea how i can debug/remedy this?

Ruby 2.0 segfault (fixed on master)

0.5.1 segfaults on Ruby 2.0.0 (and 2.1.0). Master doesn't. I guess this means it must be caused by the duplicate Puppet lexer. Could you please release the current master as 0.5.2?

utf-8 on windows

Hi,

Jekyll on windows breaks. This has been tracked to pygments > 0.5.0 breaking, see here jekyll/jekyll#1487 (comment)

Any ideas or updates to this, since building github pages on windows is probably something many people want to do, and utf-8 (at least for me) has pretty much become standard as far as generating internationalized files go (actually i just utf-8 EVERYTHING ..html, js, txt, css).

Thanks,
Robert