imclab / the_silver_searcher

A code-searching tool similar to ack, but faster.

The Silver Searcher

A code searching tool similar to ack, with a focus on speed.

Note: I'm going to be busy with other things for a while. Don't take it personally if your pull request or issue languishes. I still want to improve Ag, but I have other priorities right now.

What's so great about Ag?

• It searches code about 3–5× faster than ack.
• It ignores file patterns from your .gitignore and .hgignore.
• If there are files in your source repo you don't want to search, just add their patterns to a .agignore file. *cough* extern *cough*
• The command name is 33% shorter than ack!

How is it so fast?

• Searching for literals (no regex) uses Boyer-Moore-Horspool strstr.
• Files are mmap()ed instead of read into a buffer.
• If you're building with PCRE 8.21 or greater, regex searches use the JIT compiler.
• Ag calls pcre_study() before executing the regex on a jillion files.
• Instead of calling fnmatch() on every pattern in your ignore files, non-regex patterns are loaded into an array and binary searched.
• Ag uses Pthreads to take advantage of multiple CPU cores and search files in parallel.

I've written several blog posts showing how I've improved performance. These include how I added pthreads, wrote my own scandir(), benchmarked every revision to find performance regressions, and profiled with gprof and Valgrind.

Installation

Gentoo:

emerge the_silver_searcher

OS X:

brew install the_silver_searcher

or

port install the_silver_searcher

ArchLinux:

pacman -S the_silver_searcher

FreeBSD:

Use either of the following commands depending on which package management tool your system is configured for:

pkg add the_silver_searcher

or

pkg_add the_silver_searcher

To build Ag from source on FreeBSD:

make -C /usr/ports/textproc/the_silver_searcher install clean

If you want a CentOS rpm or Ubuntu deb, take a look at Vikram Dighe's packages.

Building from source

1. Install dependencies (Automake, pkg-config, PCRE, LZMA):
   • Ubuntu: apt-get install -y automake pkg-config libpcre3-dev zlib1g-dev liblzma-dev
   • OS X:
     • Install homebrew, then brew install automake pkg-config pcre
     • Or install macports, then port install automake pkgconfig pcre
   • Windows: It's complicated. See this wiki page.
2. Run the build script (which just runs aclocal, automake, etc):
   • ./build.sh
3. Make install:
   • sudo make install

On windows:

mingw32-make -f Makefile.w32

Current development status

It's quite stable now. Most changes are new features, minor bug fixes, or performance improvements. It's much faster than Ack in my benchmarks.

ack blahblahblah ~/code  6.59s user 1.94s system 99% cpu 8.547 total

ag blahblahblah ~/code  1.39s user 1.81s system 229% cpu 1.396 total

Editor Integration

TextMate

TextMate users can use Ag with my fork of the popular AckMate plugin, which lets you use both Ack and Ag for searching. If you already have AckMate you just want to replace Ack with Ag, move or delete "~/Library/Application Support/TextMate/PlugIns/AckMate.tmplugin/Contents/Resources/ackmate_ack" and run ln -s /usr/local/bin/ag "~/Library/Application Support/TextMate/PlugIns/AckMate.tmplugin/Contents/Resources/ackmate_ack"

Vim

You can use Ag with [ack.vim][] by adding the following line to your .vimrc:

let g:ackprg = 'ag --nogroup --nocolor --column'

There's also a fork of ack.vim tailored for use with Ag: [ag.vim][] [ack.vim]: https://github.com/mileszs/ack.vim [ag.vim]: https://github.com/rking/ag.vim

Emacs

You can use use ag.el as an Emacs fronted to Ag.

Contributing

I like when people send pull requests. It validates my existence. If you want to help out, check the issue list or search the codebase for TODO. Don't worry if you lack experience writing C. If I think a pull request isn't ready to be merged, I'll give feedback in comments. Once everything looks good, I'll comment on your pull request with a cool animated gif and hit the merge button.

TODO

A special thanks goes out to Alex Davies. He has given me some excellent recommendations to improve Ag. Many of these things are still on my list:

• Optimizations
  • Profile read() against mmap(). Remember that's read() not fread().
  • Write a benchmarking script that tweaks various settings to find what's fastest.
• Features
  • Behave better when matching in files with really long lines.
  • Report "match found at position X of line N" if line is > 10k chars.
• Windows support
  • readdir() and stat() are much slower on Windows. Use FindNextFile() instead.
  • Support Visual Studio instead of autotools?
  • Need to use pthreads-win32 or something similar.

Other stuff you might like

• Ack - Better than grep
• AckMate - An ack-powered replacement for TextMate's slow built-in search.
• ack.vim
• ag.vim
• Exuberant Ctags - Faster than Ag, but it builds an index beforehand. Good for really big codebases.
• Git-grep - As fast as Ag but only works on git repos.
• Sack - A utility that wraps Ack and Ag. It removes a lot of repetition from searching and opening matching files.