juliadatabases / hive.jl Goto Github PK
View Code? Open in Web Editor NEWHive, Spark SQL, Impala client. Based on Thrift and HiveServer2 protocol.
License: Other
hive.jl's People
Contributors
Stargazers
Watchers
hive.jl's Issues
Automate tests on Travis
An approach similar to JuliaParallel/Elly.jl#25 should work for this as well.
Error on HDInsight
Running this on Azure HDInsight, I get the following
julia> session = HiveSession("hn0-myclust.3p0iyjauoc2e3faws152r5tm0e.cx.internal.cloudapp.net", 10000)
HiveSession: hive2://sshuser@hn0-myclust.3p0iyjauoc2e3faws152r5tm0e.cx.internal.cloudapp.net:10000
julia> rs = execute(session, "select * from hivesampletable";
async=true, config=Dict())
Hive.PendingResult(HiveSession: hive2://sshuser@hn0-myclust.3p0iyjauoc2e3faws152r5tm0e.cx.internal.cloudapp.net:10000
, Hive.HS2.TOperationHandle(Hive.HS2.THandleIdentifier(UInt8[0x3a, 0x6d, 0x38, 0x65, 0x90, 0xf4, 0x4e, 0x92, 0xad, 0x1e, 0x3d, 0xa4, 0xc5, 0x22, 0x3d, 0x3c], UInt8[0xe4, 0x41, 0xa3, 0x97, 0xb6, 0x42, 0x4d, 0xea, 0x8e, 0x12, 0xeb, 0x57, 0xa7, 0xfe, 0xdd, 0x34]), 0, true, 6.93851919494137e-310), Nullable{Hive.HS2.TGetOperationStatusResp}())
julia> while !isready(rs)
println("waiting...")
sleep(10)
end
ERROR: KeyError: key 10 not found
Stacktrace:
[1] getindex at ./dict.jl:474 [inlined]
[2] read_container(::Thrift.TBinaryProtocol, ::Hive.HS2.TGetOperationStatusResp) at /home/hadoop/.julia/v0.6/Thrift/src/base.jl:181
[3] read_container(::Thrift.TBinaryProtocol, ::Type{Hive.HS2.TGetOperationStatusResp}) at /home/hadoop/.julia/v0.6/Thrift/src/base.jl:168
[4] read_container(::Thrift.TBinaryProtocol, ::Hive.HS2.GetOperationStatus_result) at /home/hadoop/.julia/v0.6/Thrift/src/base.jl:190
[5] GetOperationStatus(::Hive.HS2.TCLIServiceClient, ::Hive.HS2.TGetOperationStatusReq) at /home/hadoop/.julia/v0.6/Hive/src/HS2/TCLIService.jl:607
[6] status(::Hive.PendingResult) at /home/hadoop/.julia/v0.6/Hive/src/resultset.jl:80
[7] isready(::Hive.PendingResult) at /home/hadoop/.julia/v0.6/Hive/src/resultset.jl:93
[8] anonymous at ./<missing>:?
julia> isready(rs)
ERROR: Thrift.TProtocolException(4, "No protocol version header")
Stacktrace:
[1] readMessageBegin(::Thrift.TBinaryProtocol) at /home/hadoop/.julia/v0.6/Thrift/src/protocols.jl:113
[2] GetOperationStatus(::Hive.HS2.TCLIServiceClient, ::Hive.HS2.TGetOperationStatusReq) at /home/hadoop/.julia/v0.6/Hive/src/HS2/TCLIService.jl:605
[3] status(::Hive.PendingResult) at /home/hadoop/.julia/v0.6/Hive/src/resultset.jl:80
[4] isready(::Hive.PendingResult) at /home/hadoop/.julia/v0.6/Hive/src/resultset.jl:93
julia> sample = result(rs)
ERROR: Thrift.TProtocolException(4, "No protocol version header")
Stacktrace:
[1] readMessageBegin(::Thrift.TBinaryProtocol) at /home/hadoop/.julia/v0.6/Thrift/src/protocols.jl:113
[2] GetOperationStatus(::Hive.HS2.TCLIServiceClient, ::Hive.HS2.TGetOperationStatusReq) at /home/hadoop/.julia/v0.6/Hive/src/HS2/TCLIService.jl:605
[3] status(::Hive.PendingResult) at /home/hadoop/.julia/v0.6/Hive/src/resultset.jl:80
[4] isready at /home/hadoop/.julia/v0.6/Hive/src/resultset.jl:93 [inlined]
[5] result(::Hive.PendingResult) at /home/hadoop/.julia/v0.6/Hive/src/resultset.jl:40
Seg fault in `show`ing a resultset after it is drained
julia> rs
Hive.ResultSet(HiveSession: hive2://sshuser@hn0-myclust.3p0iyjauoc2e3faws152r5tm0e.cx.internal.cloudapp.net:10000
, Hive.HS2.TOperationHandle(Hive.HS2.THandleIdentifier(UInt8[0xb5, 0xc9, 0xfd, 0x5e, 0xd0, 0xd0, 0x4c, 0x9d, 0xac, 0x5e, 0xe3, 0xfd, 0x52, 0x69, 0x1a, 0x1c], UInt8[0x01, 0x64, 0xd0, 0xb7, 0x4e, 0x8d, 0x43, 0x89, 0x83, 0x03, 0x03, 0xb8, 0x2b, 0xdc, 0xc1, 0x05]), 0, true, 6.92687530096267e-310), Nullable{Hive.HS2.TTableSchema}(), Nullable{Hive.HS2.TRowSet}(), 0, 1024, false)
julia> dataframe(rs)
WARNING: DataArrays.NA is deprecated.
likely near no file:0
WARNING: DataArrays.NA is deprecated.
likely near no file:0
WARNING: DataArrays.NA is deprecated.
likely near no file:0
in julia_type at /home/hadoop/.julia/v0.6/Hive/src/types.jl
WARNING: DataArrays.NA is deprecated.
likely near no file:0
in julia_type at /home/hadoop/.julia/v0.6/Hive/src/types.jl
1024×11 DataFrames.DataFrame. Omitted printing of 6 columns
│ Row │ hivesampletable.clientid │ hivesampletable.querytime │ hivesampletable.market │ hivesampletable.deviceplatform │ hivesampletable.devicemake │
├──────┼──────────────────────────┼───────────────────────────┼────────────────────────┼────────────────────────────────┼────────────────────────────┤
│ 1 │ 8 │ 18:54:20 │ en-US │ Android │ Samsung │
│ 2 │ 23 │ 19:19:44 │ en-US │ Android │ HTC │
│ 3 │ 23 │ 19:19:46 │ en-US │ Android │ HTC │
│ 4 │ 23 │ 19:19:47 │ en-US │ Android │ HTC │
│ 5 │ 28 │ 01:37:50 │ en-US │ Android │ Motorola │
│ 6 │ 28 │ 00:53:31 │ en-US │ Android │ Motorola │
│ 7 │ 28 │ 00:53:50 │ en-US │ Android │ Motorola │
│ 8 │ 28 │ 16:44:21 │ en-US │ Android │ Motorola │
│ 9 │ 28 │ 16:43:41 │ en-US │ Android │ Motorola │
│ 10 │ 28 │ 01:37:19 │ en-US │ Android │ Motorola │
│ 11 │ 30 │ 17:19:36 │ en-US │ RIM OS │ RIM │
│ 12 │ 30 │ 17:17:18 │ en-US │ RIM OS │ RIM │
│ 13 │ 30 │ 17:16:40 │ en-US │ RIM OS │ RIM │
│ 14 │ 43 │ 00:44:46 │ en-US │ RIM OS │ RIM │
│ 15 │ 43 │ 00:44:41 │ en-US │ RIM OS │ RIM │
│ 16 │ 45 │ 21:24:03 │ en-US │ Android │ Samsung │
│ 17 │ 45 │ 21:09:43 │ en-US │ Android │ Samsung │
│ 18 │ 45 │ 20:01:50 │ en-US │ Android │ Samsung │
⋮
│ 1006 │ 2053 │ 20:10:28 │ en-US │ iPhone OS │ Apple │
│ 1007 │ 2053 │ 14:31:48 │ en-US │ iPhone OS │ Apple │
│ 1008 │ 2053 │ 07:52:27 │ en-US │ iPhone OS │ Apple │
│ 1009 │ 2053 │ 07:51:15 │ en-US │ iPhone OS │ Apple │
│ 1010 │ 2053 │ 07:48:54 │ en-US │ iPhone OS │ Apple │
│ 1011 │ 2053 │ 05:19:33 │ en-US │ iPhone OS │ Apple │
│ 1012 │ 2053 │ 20:09:24 │ en-US │ iPhone OS │ Apple │
│ 1013 │ 2053 │ 04:06:34 │ en-US │ iPhone OS │ Apple │
│ 1014 │ 2068 │ 22:18:09 │ en-US │ Android │ LG │
│ 1015 │ 2068 │ 22:18:02 │ en-US │ Android │ LG │
│ 1016 │ 2068 │ 11:58:01 │ en-US │ Android │ LG │
│ 1017 │ 2068 │ 02:34:32 │ en-US │ Android │ LG │
│ 1018 │ 2068 │ 02:34:30 │ en-US │ Android │ LG │
│ 1019 │ 2074 │ 13:47:21 │ en-US │ iPhone OS │ Apple │
│ 1020 │ 2074 │ 13:46:12 │ en-US │ iPhone OS │ Apple │
│ 1021 │ 2100 │ 11:46:47 │ en-US │ iPhone OS │ Apple │
│ 1022 │ 2105 │ 03:07:43 │ en-US │ RIM OS │ RIM │
│ 1023 │ 2112 │ 20:08:05 │ en-US │ Android │ HTC │
│ 1024 │ 2112 │ 20:07:21 │ en-US │ Android │ HTC │
julia> rs
Hive.ResultSet(HiveSession: hive2://sshuser@hn0-myclust.3p0iyjauoc2e3faws152r5tm0e.cx.internal.cloudapp.net:10000
, Hive.HS2.TOperationHandle(Hive.HS2.THandleIdentifier(UInt8[0xb5, 0xc9, 0xfd, 0x5e, 0xd0, 0xd0, 0x4c, 0x9d, 0xac, 0x5e, 0xe3, 0xfd, 0x52, 0x69, 0x1a, 0x1c], UInt8[0x01, 0x64, 0xd0, 0xb7, 0x4e, 0x8d, 0x43, 0x89, 0x83, 0x03, 0x03, 0xb8, 0x2b, 0xdc, 0xc1, 0x05]), 0, true, 6.92687530096267e-310), Nullable{Hive.HS2.TTableSchema}(
signal (11): Segmentation fault
while loading no file, in expression starting on line 0
jl_new_structv at /buildworker/worker/package_linux64/build/src/datatype.c:679
Type at /home/hadoop/.julia/v0.6/DataArrays/src/dataarray.jl:68
dataframe at /home/hadoop/.julia/v0.6/Hive/src/util.jl:24
show at /home/hadoop/.julia/v0.6/Hive/src/util.jl:35 [inlined]
showcompact at ./show.jl:1730
show at ./nullable.jl:72
unknown function (ip: 0x7f8336f82766)
jl_call_fptr_internal at /buildworker/worker/package_linux64/build/src/julia_internal.h:339 [inlined]
jl_call_method_internal at /buildworker/worker/package_linux64/build/src/julia_internal.h:358 [inlined]
jl_apply_generic at /buildworker/worker/package_linux64/build/src/gf.c:1926
show_default at ./show.jl:140
unknown function (ip: 0x7f8336f70dc6)
jl_call_fptr_internal at /buildworker/worker/package_linux64/build/src/julia_internal.h:339 [inlined]
jl_call_method_internal at /buildworker/worker/package_linux64/build/src/julia_internal.h:358 [inlined]
jl_apply_generic at /buildworker/worker/package_linux64/build/src/gf.c:1926
display at ./REPL.jl:122
unknown function (ip: 0x7f8336f81ef6)
jl_call_fptr_internal at /buildworker/worker/package_linux64/build/src/julia_internal.h:339 [inlined]
jl_call_method_internal at /buildworker/worker/package_linux64/build/src/julia_internal.h:358 [inlined]
jl_apply_generic at /buildworker/worker/package_linux64/build/src/gf.c:1926
display at ./REPL.jl:125
unknown function (ip: 0x7f8336f81c46)
jl_call_fptr_internal at /buildworker/worker/package_linux64/build/src/julia_internal.h:339 [inlined]
jl_call_method_internal at /buildworker/worker/package_linux64/build/src/julia_internal.h:358 [inlined]
jl_apply_generic at /buildworker/worker/package_linux64/build/src/gf.c:1926
display at ./multimedia.jl:218
unknown function (ip: 0x7f8336f81b02)
jl_call_fptr_internal at /buildworker/worker/package_linux64/build/src/julia_internal.h:339 [inlined]
jl_call_method_internal at /buildworker/worker/package_linux64/build/src/julia_internal.h:358 [inlined]
jl_apply_generic at /buildworker/worker/package_linux64/build/src/gf.c:1926
do_call at /buildworker/worker/package_linux64/build/src/interpreter.c:75
eval at /buildworker/worker/package_linux64/build/src/interpreter.c:242
eval_body at /buildworker/worker/package_linux64/build/src/interpreter.c:539
jl_toplevel_eval_body at /buildworker/worker/package_linux64/build/src/interpreter.c:511
jl_toplevel_eval_flex at /buildworker/worker/package_linux64/build/src/toplevel.c:571
jl_toplevel_eval_in at /buildworker/worker/package_linux64/build/src/builtins.c:496
eval at ./boot.jl:235
unknown function (ip: 0x7f83498ccd2f)
jl_call_fptr_internal at /buildworker/worker/package_linux64/build/src/julia_internal.h:339 [inlined]
jl_call_method_internal at /buildworker/worker/package_linux64/build/src/julia_internal.h:358 [inlined]
jl_apply_generic at /buildworker/worker/package_linux64/build/src/gf.c:1926
print_response at ./REPL.jl:144
unknown function (ip: 0x7f8336f51b08)
jl_call_fptr_internal at /buildworker/worker/package_linux64/build/src/julia_internal.h:339 [inlined]
jl_call_method_internal at /buildworker/worker/package_linux64/build/src/julia_internal.h:358 [inlined]
jl_apply_generic at /buildworker/worker/package_linux64/build/src/gf.c:1926
print_response at ./REPL.jl:129
unknown function (ip: 0x7f8336f51508)
jl_call_fptr_internal at /buildworker/worker/package_linux64/build/src/julia_internal.h:339 [inlined]
jl_call_method_internal at /buildworker/worker/package_linux64/build/src/julia_internal.h:358 [inlined]
jl_apply_generic at /buildworker/worker/package_linux64/build/src/gf.c:1926
do_respond at ./REPL.jl:646
unknown function (ip: 0x7f8336f316b1)
jl_call_fptr_internal at /buildworker/worker/package_linux64/build/src/julia_internal.h:339 [inlined]
jl_call_method_internal at /buildworker/worker/package_linux64/build/src/julia_internal.h:358 [inlined]
jl_apply_generic at /buildworker/worker/package_linux64/build/src/gf.c:1926
do_call at /buildworker/worker/package_linux64/build/src/interpreter.c:75
eval at /buildworker/worker/package_linux64/build/src/interpreter.c:242
eval_body at /buildworker/worker/package_linux64/build/src/interpreter.c:539
jl_toplevel_eval_body at /buildworker/worker/package_linux64/build/src/interpreter.c:511
jl_toplevel_eval_flex at /buildworker/worker/package_linux64/build/src/toplevel.c:571
jl_toplevel_eval_in at /buildworker/worker/package_linux64/build/src/builtins.c:496
eval at ./boot.jl:235
unknown function (ip: 0x7f83498ccd2f)
jl_call_fptr_internal at /buildworker/worker/package_linux64/build/src/julia_internal.h:339 [inlined]
jl_call_method_internal at /buildworker/worker/package_linux64/build/src/julia_internal.h:358 [inlined]
jl_apply_generic at /buildworker/worker/package_linux64/build/src/gf.c:1926
run_interface at ./LineEdit.jl:1583
unknown function (ip: 0x7f83499447bf)
jl_call_fptr_internal at /buildworker/worker/package_linux64/build/src/julia_internal.h:339 [inlined]
jl_call_method_internal at /buildworker/worker/package_linux64/build/src/julia_internal.h:358 [inlined]
jl_apply_generic at /buildworker/worker/package_linux64/build/src/gf.c:1926
run_frontend at ./REPL.jl:945
run_repl at ./REPL.jl:180
unknown function (ip: 0x7f8336f1c282)
jl_call_fptr_internal at /buildworker/worker/package_linux64/build/src/julia_internal.h:339 [inlined]
jl_call_method_internal at /buildworker/worker/package_linux64/build/src/julia_internal.h:358 [inlined]
jl_apply_generic at /buildworker/worker/package_linux64/build/src/gf.c:1926
_start at ./client.jl:413
unknown function (ip: 0x7f834991ed28)
jl_call_fptr_internal at /buildworker/worker/package_linux64/build/src/julia_internal.h:339 [inlined]
jl_call_method_internal at /buildworker/worker/package_linux64/build/src/julia_internal.h:358 [inlined]
jl_apply_generic at /buildworker/worker/package_linux64/build/src/gf.c:1926
unknown function (ip: 0x401bc8)
unknown function (ip: 0x401612)
__libc_start_main at /lib/x86_64-linux-gnu/libc.so.6 (unknown line)
unknown function (ip: 0x4016bc)
Allocations: 9812451 (Pool: 9810530; Big: 1921); GC: 19
Segmentation fault (core dumped)
From the output, it looks to crash when trying to look inside a TableSchema?
Move `Tabular` to Tables.jl ?
error on HiveSession
Support transportMode: http
Support auth=noSasl
Is there a simple way to support auth=noSasl in this library? JDBC supports this: jdbc:hive2://impala_server:21050/db/;auth=noSasl
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.