workwithnano / ken

A Ruby API for accessing Freebase. It wraps the Metaweb Architecture to smart Ruby Objects.

Ken

Introduction

Ken is a Data Layer for Knowledge Representation.

It’s being built to access the Metaweb Services supplied by Freebase.com.
The project’s goals are the provision of a concise API for querying and writing.
Therefore it wraps the Metaweb Architecture to smart Ruby Objects.

You can navigate the Freebase Graph using a rubyish syntax.
Also you can use this library as a Data Layer (instead of or in addition to ActiveRecord/DataMapper) for your Web Framework of choice (Merb, Rails).

Installation

Use Gemcutter RubyGems:

  $ gem install gemcutter
  $ gem tumble
  $ gem install ken

In your Ruby files add:

  require 'rubygems'
  require 'ken'

Getting started

The first place to get started with Freebase is of course, Freebase. Try out their Browser at
http://www.freebase.com.

The Freebase Database can be thought of as a huge graph of interconnected nodes that represent
knowledge (in a much more structured way than wikipedia does).
That graph can be viewed at a higher level through an object-oriented lens which leads to easier interaction.
To understand the fundamental Metaweb Architecture please read the official
MQL Reference guide (with focus on Chapter 2)
provided by Freebase.

In addition, you can learn a lot by employing the Freebase Query Editor.

Fetching a Resource

Let’s ask Ken what he knows about the British Band New Order.

  resource = Ken.get('/en/new_order') # => <Resource id="/en/new_order" name="New Order">

Inspecting the Types

Every Resource can have multiple types.

  resource.types
  # => [ #<Type id="/film/music_contributor" name="Film music contributor">, #<Type id="/music/artist" name="Musical Artist">, 
         #<Type id="/common/topic" name="Topic">, #<Type id="/music/musical_group" name="Musical Group">,
         #<Type id="/broadcast/artist" name="Broadcast Artist">, #<Type id="/music/group_member" name="Musical Group Member"> ]

We can see that New Order is a member of Music Artist, Film Music Contributor, Broadcast Artist and other types.

Inspecting a Type’s properties

A type defines a set of properties to describe a Resource.

  resource.types.each do |type|
    type.properties # => e.g. [ #<Property id="/music/musical_group/member"> ]
  end

We get sets of Properties for each Type. The Type Musical Group has just one Property /music/musical_group/member
named Members Of Musical Group.

Listing all Attributes

After inspecting a Resource’s Types and Properties we now know what we could know. But actually we don’t know nothing :)
So it’s time to ask for the values of properties, the so called Attributes.

_{Note: In Ken’s terminology we differ between Properties and concrete Property instances, the Attributes, while
Freebase itself doesn’t.}

  resource.attributes.each do |att|
    att # => e.g. #<Attribute property="/music/artist/album">
    att.property.name # => e.g. "Albums"
    
    att.values
    # e.g. => [ #<Resource id="/guid/9202a8c04000641f8000000002fa2556" name="Ceremony">, 
                #<Resource id="/guid/9202a8c04000641f8000000002fa24d5" name="Procession">,
                #<Resource id="/guid/9202a8c04000641f8000000002fa20d3" name="Everything's Gone Green">, ... ]
    # e.g. => ["1980"]
  end
  
  # alternatively you can access them directly
  resource.attribute('/music/artist/album') # => #<Attribute property="/music/artist/album">

Attributes are slightly more complicated to handle compared to Types and Properties.

There are four kinds of Attributes.

• Unique Value Type
• Unique Object Type
• Non-unique Value Type
• Non-unique Object

In order to be able to use unique and non-unique Attributes in the same manner we always wrap the value of an Attribute
in a Collection, no matter if there’s one value or there are many.

Group Attributes by their Type using Views

  resource.views.each do |view|
    view # => e.g. #<View type="/music/artist">
    view.type # => e.g #<Type id="/music/artist" name="Musical Artist">
    view.attributes
    # => [#<Attribute property="/music/artist/home_page">, #<Attribute property="/music/artist/genre">,
          #<Attribute property="/music/artist/active_start">, #<Attribute property="/music/artist/similar_artist">,
          #<Attribute property="/music/artist/album">, #<Attribute property="/music/artist/label">,
          #<Attribute property="/music/artist/track">, #<Attribute property="/music/artist/origin">]
            
    view.attributes.each do |att|
      att.values
      # e.g. => [ #<Resource id="/en/alternative_dance" name="Alternative dance">,
                  #<Resource id="/en/synthpop" name="Synthpop">, 
                  #<Resource id="/en/house_music" name="House music">,
                  #<Resource id="/en/post-punk" name="Post-punk"> ]
      # e.g. => ["1980"]
    end
  end

Fetching multiple Resources using a query

As of now you can ask for multiple Resources by specifying a query.

  resources = Ken.all(:name => "Apple", :type => "/music/album")
  # => [#<Resource id="/guid/9202a8c04000641f80000000031dae7c" name="Apple">,
        #<Resource id="/guid/9202a8c04000641f8000000007ce31ec" name="Apple">]

Keep in mind that only the top level of the query is mapped to a Collection of Resource Objects.
So asking for values in a nested level does not make sense. Use nested statements just for
lowering the top level result.

However you can instead navigate the normal way to figure out that values.
But won’t that require another query to triggered? Certainly.

Let’s look at a nested query:

  query = {
    :directed_by => "George Lucas",
    :starring => [
      {
        :actor => "Harrison Ford"
      }
    ],
    :type => "/film/film"
  }
  
  resources = Ken.all(query)
  # => [#<Resource id="/en/star_wars_episode_iv_a_new_hope" name="Star Wars Episode IV: A New Hope">,
        #<Resource id="/en/american_graffiti" name="American Graffiti">,
        #<Resource id="/en/the_star_wars_holiday_special" name="The Star Wars Holiday Special">]

Access properties attributes directly

Ken is primarily designed for inspecting resources in a generic way, what’s ideal for domain independent browsing applications.
However, there are legitimate situations where you already know what you want to access.

That’s why I now added direct Property/Attribute access, but only on a Type/View level:

  resource = Ken.get('/en/new_order')
  type = resource.types[1] # => #<Type id="/music/artist" name="Musical Artist">
  # because we know _/music/artist_ has a _genre_ property we can access that directly
  type.genre # => #<Property id="/music/artist/genre" expected_type="/music/genre" unique="false" object_type="true">

The same works for views:

  resource = Ken.get('/en/new_order')
  view = resource.views[1] # => #<View type="/music/artist">
  # because we know _/music/artist_ has a _genre_ property we can access attribute directly as well
  view.genre # => #<Attribute property="/music/artist/genre">

If you rather want to query based on Types and access Properties/Attributes directly you can consider using
Chris Eppsteins Freebase Library as an alternative.

Low Level API

Sometimes you may want to do specific queries instead of inspecting Resources as a whole.
In such a case you would want to use Ken’s low level API.

mqlread works like the regular mqlread service, except that you are able to pass Ruby hashes instead of JSON.
And you don’t have to deal with HTTP, parameter encoding and parsing JSON.

  artists = Ken.session.mqlread([{
    :type => "/music/artist",
    :id => nil, 
    :"/common/topic/webpage" => [{:uri => nil}], 
    :home_page => [{:uri => nil}], 
    :limit => 2
  }])  
  
  # => [
         {"type"=>"/music/artist", "home_page"=>[{"uri"=>"http://www.massiveattack.co.uk/"}], "id"=>"/en/massive_attack", "/common/topic/webpage"=>[{"uri"=>"http://musicmoz.org/Bands_and_Artists/M/Massive_Attack/"}, {"uri"=>"http://www.discogs.com/artist/Massive+Attack"}, {"uri"=>"http://www.massiveattackarea.com/"}, {"uri"=>"http://www.massiveattack.co.uk/"}, {"uri"=>"http://www.massiveattack.com/"}]},
         {"type"=>"/music/artist", "home_page"=>[{"uri"=>"http://www.apartment26.com/"}], "id"=>"/en/apartment_26", "/common/topic/webpage"=>[{"uri"=>"http://www.discogs.com/artist/Apartment+26"}, {"uri"=>"http://musicmoz.org/Bands_and_Artists/A/Apartment_26/"}, {"uri"=>"http://www.apartment26.com/"}]}
       ]

Topic API

Please first have a look at the official Topic HTTP API documentation .

The API provides general meta-data such as name, description, links and images for a given topic,
as well as all properties directly related to that topic in the graph.
The API wraps a series of MQL queries that are needed to get this data, which otherwise must be performed separately.
So for gaining common interest information about a specific topic the Topic API is a way faster alternative to mqlread.

The latest update of Ken provides an easy way to access Freebase Topics using Ruby.
As usual Ken wraps the JSON result of the web service to convenient Ruby Objects.

For now Ken only returns simple properties. Support for so called mediator properties (aka ‘CVT’) will be added later.
To be honest, I just don’t know how to wrap them appropriately using the existing Ken object model. Any API ideas are welcome, btw! ;)
However, in the meanwhile you can access CVT’s by using the plain JSON result returned by the low level Ken.session.topic method.

The API for Topics is quite the same as for Resources.

t = Ken::Topic.get("/en/new_order")
  # => <Topic id="/en/new_order" name="New Order">
    
t.types
  # => [ #<Type id="/music/artist" name="Musical Artist">, #<Type id="/music/musical_group" name="Musical Group">, ... ]

t.views
  # => [ #<View type="/music/artist">, #<View type="/music/musical_group">, ... ]
    
t.properties
  # => [ #<Property id="/music/artist/similar_artist" expected_type="/music/artist">, ... ]

t.attributes
  # => [ #<Attribute property="/music/artist/similar_artist">, #<Attribute property="/music/artist/album">, ... ]

Additionally you can access some general meta-data, most importantly the topic’s description which otherwise would need an additional request to the raw service.

t.name # => "New Order"
t.description # => "New Order were an English musical group formed in 1980 by Bernard Sumner ... "
t.aliases # => [ "NewOrder", "Englandneworder" ]
t.webpages # => [ {"url"=>"http://en.wikipedia.org/wiki/index.html?curid=22146", "text"=>"Wikipedia"}, ... ]
t.url # => "http://www.freebase.com/view/en/new_order"
t.thumbnail => "http://api.freebase.com/api/trans/image_thumb/en/new_order"

Project Status

Features

• Fetching of single Resources
• Fetching of multiple Resources by specifying a query
• Accessing Properties/Attributes directly (on a type/view level)
• Type inspection
• Attribute inspection
• Low Level API (mqlread)
• Rails and Merb support
• Views on Resources to group Attributes based on the Resource’s types
• Accessing Topics using the new Freebase Topic API

Roadmap

1. More tests
2. Write-Support

Initial thoughts, obviously not up-to-date and not conforming to the current version, are available at http://wiki.github.com/michael/ken.