There are some challenges with Queries using Rest. While this is laid out in an easy manner with Rest, we should consider the implications to using JDBC or ODBC with this as well.

I am getting Exception: Status Code: 500 - Error while trying to do '%Drill connect'. What am i missing.

Thanks,
Ganesh

Log below:

Please enter the password you wish to connect with:
Drill Connect Password: ········

Exception Traceback (most recent call last)
in
----> 1 get_ipython().run_line_magic('drill', 'connect')

~/jupyter/jupyter-env/lib/python3.5/site-packages/IPython/core/interactiveshell.py in run_line_magic(self, magic_name, line, _stack_depth)
2312 kwargs['local_ns'] = sys._getframe(stack_depth).f_locals
2313 with self.builtin_trap:
-> 2314 result = fn(*args, **kwargs)
2315 return result
2316

</home/linaro/jupyter/jupyter-env/lib/python3.5/site-packages/decorator.py:decorator-gen-126> in drill(self, line, cell)

~/jupyter/jupyter-env/lib/python3.5/site-packages/IPython/core/magic.py in (f, *a, **k)
185 # but it's overkill for just that one bit of state.
186 def magic_deco(arg):
--> 187 call = lambda f, *a, **k: f(*a, **k)
188
189 if callable(arg):

~/jupyter/jupyter-env/lib/python3.5/site-packages/drill_core/drill_base.py in drill(self, line, cell)
346 self.connectDrill(True)
347 elif line.lower() == "connect":
--> 348 self.connectDrill(False)
349 elif line.lower() .find('set ') == 0:
350 self.setvar(line)

~/jupyter/jupyter-env/lib/python3.5/site-packages/drill_core/drill_base.py in connectDrill(self, prompt)
210 requests.packages.urllib3.disable_warnings(InsecureRequestWarning)
211 result = -1
--> 212 result = self.authDrill()
213 if result == 0:
214 self.drill_connected = True

~/jupyter/jupyter-env/lib/python3.5/site-packages/drill_core/drill_base.py in authDrill(self)
293 result = 0
294 else:
--> 295 raise Exception("Status Code: %s - Error" % r.status_code)
296 return result
297

Exception: Status Code: 500 - Error

When trying to connect current IPython notebook to Drill installation (Dril=Drill(ipy,pd_use_beaker=False)), It gives an error

'Drill' object is not callable

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-10-dff484a38606> in <module>
----> 1 d=Drill(ipy,pd_use_beaker=False)

TypeError: 'Drill' object is not callable

What could be the issue here? Any pointers?

Sometimes, after your exploration with %%drill is done, you want to programmatically issue a query and get a data frame. There is no reason this jupyter_drill module can't do that as well.

Currently if you have a notebook open and don't issue a query in a certain timeframe, the Drill cluster will timeout your session, that is normal.

We should better trap when we try to issue a query and a drill connection is not active. We should be able to do that by return code, and

• Provide a prettier and more helpful error about what happened
• Reset the internal variable for the connection to status to false

The second point is important. Right now, if the connection times out, and try a query, it will error out, and when it does, you can go to your cell with %drill connect but it won't let you just connect because internally Drill thinks it's still connected. So you have to type %drill disconnect and then do the %drill connect. It's a small thing, but it's a pain and wastes a cell

I think it's possible to run a demo of this magic inside MyBinder.

Add:

• an apt.txt file that installs openjdk-8-jdk
• add wget -q -O apache-drill-1.15.0.tar.gz http://apache.mirrors.hoobly.com/drill/drill-1.15.0/apache-drill-1.15.0.tar.gz; tar xzf apache-drill-1.15.0.tar.gz to postBuild
• add jupyter-server-proxy to requirements.txt to let you see the Apache Drill Web UI via /proxy/8047.

You can start the server from a notebook terminal as:

apache-drill-1.15.0/bin/drill-embedded

or perhaps autostart it in MyBinder (I haven't tried this yet) by adding a start file containing something like:

#!/bin/bash

nohup apache-drill-1.15.0/bin/drill-embedded  > /dev/null 2>&1 &

exec "$@" 

The default Apache Drill install runs with auth, I think, so it would be handy if the jupyter_drill magic could connect without an auth requirement.