nkeyes / ok_hbase Goto Github PK

Hi All,

I'm relatively new to Hbase and the gem that connects to Hbase.
Currently I have a hadoop cluster set up remotely, but ruby on rails is on my local laptop.
My question is how to use ok-hbase to connect to hbase on remote server? I know the IP address for the server and the port number for the Thrift server on it.
Just want to make the connect work.

Thanks,

Fiona

As a result this causes gem install ok_hbase to fail. As a workaround I moved the thrift version up to 1.0.0 and it compiled, but I don't know if this will introduce any runtime side effects.

make "DESTDIR="
compiling binary_protocol_accelerated.c
compiling bytes.c
compiling compact_protocol.c
compiling memory_buffer.c
compiling protocol.c
compiling strlcpy.c
In file included from strlcpy.c:20:
./strlcpy.h:28:15: error: expected parameter declarator
extern size_t strlcpy(char *, const char *, size_t);
              ^
/usr/include/secure/_string.h:105:44: note: expanded from macro 'strlcpy'
  __builtin___strlcpy_chk (dest, src, len, __darwin_obsz (dest))
                                           ^
/usr/include/secure/_common.h:39:62: note: expanded from macro '__darwin_obsz'
#define __darwin_obsz(object) __builtin_object_size (object, _USE_FORTIFY_LEVEL > 1 ? 1 : 0)
                                                             ^
/usr/include/secure/_common.h:30:32: note: expanded from macro '_USE_FORTIFY_LEVEL'
#    define _USE_FORTIFY_LEVEL 2
                               ^
In file included from strlcpy.c:20:
./strlcpy.h:28:15: error: expected ')'
/usr/include/secure/_string.h:105:44: note: expanded from macro 'strlcpy'
  __builtin___strlcpy_chk (dest, src, len, __darwin_obsz (dest))
                                           ^
/usr/include/secure/_common.h:39:62: note: expanded from macro '__darwin_obsz'
#define __darwin_obsz(object) __builtin_object_size (object, _USE_FORTIFY_LEVEL > 1 ? 1 : 0)
                                                             ^
/usr/include/secure/_common.h:30:32: note: expanded from macro '_USE_FORTIFY_LEVEL'
#    define _USE_FORTIFY_LEVEL 2
                               ^
./strlcpy.h:28:15: note: to match this '('
/usr/include/secure/_string.h:105:44: note: expanded from macro 'strlcpy'
  __builtin___strlcpy_chk (dest, src, len, __darwin_obsz (dest))
                                           ^
/usr/include/secure/_common.h:39:53: note: expanded from macro '__darwin_obsz'
#define __darwin_obsz(object) __builtin_object_size (object, _USE_FORTIFY_LEVEL > 1 ? 1 : 0)
                                                    ^
In file included from strlcpy.c:20:
./strlcpy.h:28:15: error: type specifier missing, defaults to 'int' [-Werror,-Wimplicit-int]
extern size_t strlcpy(char *, const char *, size_t);
              ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/usr/include/secure/_string.h:105:44: note: expanded from macro 'strlcpy'
  __builtin___strlcpy_chk (dest, src, len, __darwin_obsz (dest))
                                           ^~~~~~~~~~~~~~~~~~~~
/usr/include/secure/_common.h:39:31: note: expanded from macro '__darwin_obsz'
#define __darwin_obsz(object) __builtin_object_size (object, _USE_FORTIFY_LEVEL > 1 ? 1 : 0)
                              ^~~~~~~~~~~~~~~~~~~~~
In file included from strlcpy.c:20:
./strlcpy.h:28:15: error: conflicting types for '__builtin___strlcpy_chk'
/usr/include/secure/_string.h:105:3: note: expanded from macro 'strlcpy'
  __builtin___strlcpy_chk (dest, src, len, __darwin_obsz (dest))
  ^
./strlcpy.h:28:15: note: '__builtin___strlcpy_chk' is a builtin with type 'unsigned long (char *, const char *, unsigned long, unsigned long)'
/usr/include/secure/_string.h:105:3: note: expanded from macro 'strlcpy'
  __builtin___strlcpy_chk (dest, src, len, __darwin_obsz (dest))
  ^
4 errors generated.
make: *** [strlcpy.o] Error 1

make failed, exit code 2

Gem files will remain installed in /Users/dennissoh/.rvm/gems/ruby-2.0.0-p247/gems/thrift-0.9.0 for inspection.
Results logged to /Users/dennissoh/.rvm/gems/ruby-2.0.0-p247/extensions/x86_64-darwin-13/2.0.0/thrift-0.9.0/gem_make.out
An error occurred while installing thrift (0.9.0), and Bundler cannot continue.
Make sure that `gem install thrift -v '0.9.0'` succeeds before bundling.

Hi,

I tried to implement row_start and row_stop parameters in scanning hbase table using the ok_hbase gem. It seems like its ignoring the filter and giving me all the data that is available in the table. Can you please help me regarding this?

table.scan(row_start:'2013-08-29 12:30:00', row_stop:'2013-08-29 13:00:00') do |row_key, columns|

Thanking You,
Yash Gunapati

There may already be a way to accomplish this, but I was hoping there may be a way to maintain connection to a remote Thrift server in-between requests. We're developing a RESTful API to provide basic reads of our HBase cluster, but are experiencing slow response times because (we think) a new connection is established for each read. Are you able to recommend a solution to this?

I envision this solution looking similar to the massive_record gem or active_record approach.

Hi nkeyes,
I have no idea how to run these examples, could you please show me the detail steps to run an example successfully in terminal?

thanks.

I was trying to add a hash as a value to a column, but was stopped by Thrift. Doesn't HBase support embedded data? Is there anyway to save that through Thrift?

Thanks,
Rockt

Do you have any plans to upgrade the Thrift client in the lib/thrift? There are new methods and signatures that I think would be great to have incorporated. For instance, the batch #incrementRows method.

http://svn.apache.org/viewvc/hbase/tags/0.95.2/hbase-server/src/main/resources/org/apache/hadoop/hbase/thrift/Hbase.thrift?view=markup

Hello, when i use ok_hbase gem i get many warning, i think solutions to this is only update thift gem from 0.9.0 to 0.9.1 Apache Thrift Jira

../.rvm/gems/ruby-2.1.0/gems/thrift-0.9.0/lib/thrift/protocol/json_protocol.rb:21: warning: class variable access from toplevel ../.rvm/gems/ruby-2.1.0/gems/thrift-0.9.0/lib/thrift/protocol/json_protocol.rb:22: warning: class variable access from toplevel ../.rvm/gems/ruby-2.1.0/gems/thrift-0.9.0/lib/thrift/protocol/json_protocol.rb:23: warning: class variable access from toplevel ../.rvm/gems/ruby-2.1.0/gems/thrift-0.9.0/lib/thrift/protocol/json_protocol.rb:24: warning: class variable access from toplevel ../.rvm/gems/ruby-2.1.0/gems/thrift-0.9.0/lib/thrift/protocol/json_protocol.rb:25: warning: class variable access from toplevel ../.rvm/gems/ruby-2.1.0/gems/thrift-0.9.0/lib/thrift/protocol/json_protocol.rb:26: warning: class variable access from toplevel ../.rvm/gems/ruby-2.1.0/gems/thrift-0.9.0/lib/thrift/protocol/json_protocol.rb:27: warning: class variable access from toplevel ../.rvm/gems/ruby-2.1.0/gems/thrift-0.9.0/lib/thrift/protocol/json_protocol.rb:28: warning: class variable access from toplevel ../.rvm/gems/ruby-2.1.0/gems/thrift-0.9.0/lib/thrift/protocol/json_protocol.rb:29: warning: class variable access from toplevel ../.rvm/gems/ruby-2.1.0/gems/thrift-0.9.0/lib/thrift/protocol/json_protocol.rb:31: warning: class variable access from toplevel ../.rvm/gems/ruby-2.1.0/gems/thrift-0.9.0/lib/thrift/protocol/json_protocol.rb:33: warning: class variable access from toplevel ../.rvm/gems/ruby-2.1.0/gems/thrift-0.9.0/lib/thrift/protocol/json_protocol.rb:34: warning: class variable access from toplevel ../.rvm/gems/ruby-2.1.0/gems/thrift-0.9.0/lib/thrift/protocol/json_protocol.rb:35: warning: class variable access from toplevel

RubyGems.org doesn't report a license for your gem. This is because it is not specified in the gemspec of your last release.

via e.g.

spec.license = 'MIT'
# or
spec.licenses = ['MIT', 'GPL-2']

Including a license in your gemspec is an easy way for rubygems.org and other tools to check how your gem is licensed. As you can imagine, scanning your repository for a LICENSE file or parsing the README, and then attempting to identify the license or licenses is much more difficult and more error prone. So, even for projects that already specify a license, including a license in your gemspec is a good practice. See, for example, how rubygems.org uses the gemspec to display the rails gem license.

There is even a License Finder gem to help companies/individuals ensure all gems they use meet their licensing needs. This tool depends on license information being available in the gemspec. This is an important enough issue that even Bundler now generates gems with a default 'MIT' license.

I hope you'll consider specifying a license in your gemspec. If not, please just close the issue with a nice message. In either case, I'll follow up. Thanks for your time!

Appendix:

If you need help choosing a license (sorry, I haven't checked your readme or looked for a license file), GitHub has created a license picker tool. Code without a license specified defaults to 'All rights reserved'-- denying others all rights to use of the code.
Here's a list of the license names I've found and their frequencies

p.s. In case you're wondering how I found you and why I made this issue, it's because I'm collecting stats on gems (I was originally looking for download data) and decided to collect license metadata,too, and make issues for gemspecs not specifying a license as a public service :). See the previous link or my blog post about this project for more information.

I am trying to pull in all the columnfamily names and column names available through OK_Hbase gem, is it possible through the current version?

Table#rows should return an array of (row_key, columns) tuples, but currently only returns an array of columns hashes

Hello,

After done getting data from HBase, I have to close connection? or How can I close connection pool?

Hi,

I am trying to use filters in table scan with ok_hbase gem. As, i didn't find an example i used similar syntax to one i used in python

Python Syntax:

for key, data in table.scan(row_start='2013-08-29 12:30:00', row_stop='2013-08-29 13:00:00',columns={'Solectria'},filter="ColumnPrefixFilter('Inv1')"):

But when i implement similar idea in ruby, It doesn't seem to work

OK_Hbase Implementation:

table.scan(start_row: '2013-08-29 13:00:00',stop_row: '2013-08-29 13:30:00', columns:['Solectria'], filter: { ColumnPrefixFilter: 'Inv1' })

Can you please help me regarding this?

Thanking You,
Yash Gunapati