roeun / engineroom

EngineRoom is a toolkit addressing basic, recurring needs when using C based languages. The most prominent (and mature) component is LogPoints, a logging facility designed to provide comfort to developers working on applications of all sizes. EngineRoom is primarily targeted at Objective-C developers on Mac OS X and iOS with some support for C on GNU/Linux and Solaris.

EngineRoom

A toolkit addressing basic, recurring needs when using C based languages.

The most prominent (and mature) component is LogPoints,
a logging facility designed to provide comfort to developers working on applications of all sizes.

Copyright (c) 2007-2011 Bjoern Kriews and contributors, BSD License.

The documentation is far from complete. Working on it. Real artists ship. :-)

State of the audience

EngineRoom is primarily targeted at Objective-C developers on Mac OS X and iOS with some support for C on GNU/Linux and Solaris.

The OS X version is maintained very actively. The iOS version is usable but needs more work.
Linux and Solaris are working but lack support code because I do not currently use them.
A windows port (and maintainer) would be very welcome, please contact me if you are interested.
I try to provide support for (Objective-)C++ but I don't use the language.

LogPoints design goals

• Creating a well formed log message should require less effort than a sloppy one.

  • what you type is what you get - no (mandatory) format strings, labels or casts, number of arguments and (in ObjC) types are detected:

      lpdebug( sender.title, sender.tag, self.bounds );
          HH:MM:SS.sss DEBUG 1 -[MyView myMethod:] sender.title: Action! ~ sender.tag: 100 $64 self.bounds: {{10,20},{30,40}} SubClassOfMyView:0xc0c0babe <MyView.m:42>
    

    (verbosity configurable at runtime) instead of

      NSLog(@"title: %@ tag: %ld bounds: %@", sender.title, (long)sender.tag, NSStringFromRect(self.bounds));
          YYYY-mm-dd HH:MM:SS.sss Application [pid:tid] title: Action! tag: 100 bounds: {{10,20},{30,40}}
    

• Log messages should be powerful tools to modern cavemen, complementing debuggers.

  • More is better, as long as you see only what you want right now.

  • Log levels are useful classifications but not sufficient.

  • LogPoints are selectively enabled using filter expressions which can target implicit metadata like file/class/function/line, explicit metadata (optional keywords, kind, label) or (static) parts of the message itself. This avoids code bureaucracy and still provides for much flexibility in selecting messages. Filters (on OSX) use NSPredicate syntax or convenient shorthand (see screenshot).
    Silence is golden - debug messages (by default :-) default to "off".

  • The GUI seen above is incorporated into an application simply by adding one object to MainMenu.xib.

  • Each team member selects her own set of messages, working sets can be selected using GUI tools (a rough implementation is provided), saved to user defaults and shared in form of predicates or shorthand.
    Default predicates can be stored in an applications Info.plist.
    Simplifies working with others, including beta testers and power users.

  • Runtime configuration can be used to selectively enable code paths:

        if( lpkswitch("showBounds") ) { NSFrameRect(self.bounds); }
    

    The Info.plist option provides for some fascinating abuses of this feature.

• Developers should be able (if desired) to ship fully instrumented builds without sacrificing performance.

  • The time required to decide which messages are enabled is spent once when the filter expression is set. Afterwards, the overhead is a test of one bit. On a 2.5GHz MacBook Pro this is ~1ns per disabled logpoint.

  • As an option, LogPoint passes can be counted even if disabled,
    browsing the LogPoint list (# column in screenshot) offers a quick
    impression on hotspots, complementing Instruments and other tools.

• Toolkits should be able to adapt to developers needs instead of forcing their way.

  • Large projects may require more metadata (i.e. keywords) to classify messages, small projects may want to avoid the overhead.

  • Developers tend to have strong preferences about logging macros and output.

  • Therefore, LogPoints strives to be adaptable. It doesn't enforce a particular macro style. While it offers a lot of convenience magic, you are free to ignore it and use it on a lower level. LogPoints provides a set of basic macros as an API. From there, the macros you actually use are generated by a heavily parameterized perl script. The recurrent aspects of creating (even the API) headers containing lots of variants are automated so you can easily experiment to find a style that suits your needs.
    A (hopefully) sensible default setting is provided. This is nonetheless strongly influenced by my taste, please let me know if you listen to a different drummer.

Implementation details

LogPoint macros are expanded to code that creates a static structure containing metadata and (besides others) an enabled flag.

These structures are placed in a separate linker segment and therefore locatable and manipulatable at runtime, allowing one to even treat them as objects (as seen in the screenshot above).

The logging core is compiler and object format dependent because it creates object file sections and analyzes the binary object file at runtime.
There is no need to keep the symbol table around and the activating code can even be loaded as a plugin if desired.

This mechanism is implemented for OS X / iOS Mach-O format and for ELF on Linux and Solaris, in both 32 and 64bit variants.

LogPoints are tested with gcc, clang and Sun Studio cc.

The OS X version was originally written for 10.3 and is generally targeted at 10.5 now. Some 10.6 features have crept in but this is easily fixed. With the advent of the AppStore and 10.7 at the horizon I currently do not plan to spend much time to support <10.6.

EngineRoom (mostly the LogPoints part) has support for embedding itself in a project. This is useful if you build a framework which profits from configurable logging, but you don't know if the final application will use EngineRoom too. You can build your framework so that it embeds a copy of LogPoints (using symbol prefixes). At runtime, it will check if the host application is linked against EngineRoom. If so, it will use that version (and its configuration), otherwise it falls back to its own copy.

This feature was just born and is not yet documented, for an example see AFCache, a feature-rich embeddable HTTP cache for iOS and OSX applications.
To use it with AFCache, checkout AFCache and EngineRoom at the same directory level, use a shared build directory, drag the AFCache Project into your project and then proceed as described under "Usage" but use the AFCache-EngineRoom target.

The Debug configuration has optimization enabled. This is because am a friend of -Wuninitialized and clang does not support it with -O0. They are working on it.

Usage

See HOWTO.txt for usage instructions.

Planned features

• interfacing to NSLogger, a very powerful logging tool on its own is one of the next steps.
  NSLogger has an impressive implementation of remote logging, a feature that was long planned for EngineRoom.

• explore possible interactions with Log4Cocoa, which provides a detailed backend.

• an improved GUI

Credits

• Steven Fuerst for publishing the technique to detect the number of arguments passed to a macro in an article on overloading in C.

• ancientbuho for unearthing how to make Xcode follow locations in log messages.

• Among many others, for generously taking time to share their knowledge and pieces of carefully crafted code:

  • Matt Gallagher @ Cocoa with love

  • Mike Ash @ NSBlog - Buy the Friday Q&A book!

  • Dave De Long @ Fun with Objective-C - Wonderful stuff!

  • Uli Kusterer @ Uli's Blog (I should visit more conferences to meet you...)

• Michael Markowski @ artifacts, fine software for input, enthusiasm and agreeing to test-drive in a large commercial project as well as in AFCache.

• Pezhman Givy, Carsten Müller and Kay Röpke for listening to and commenting my ramblings about this project since 2007.

• Sven Gohdes @ T42 for discussion, cunningly constructed all-night code-compatible playlists and for improving my writing.

• My family for loving someone who invests ridiculous amounts of time in stuff like this.

• Thanks for the ride!