johnomernik / jupyter_drill_old Goto Github PK
View Code? Open in Web Editor NEWA module to help interaction with Jupyter Notebooks and Apache Drill
License: Apache License 2.0
A module to help interaction with Jupyter Notebooks and Apache Drill
License: Apache License 2.0
There are some challenges with Queries using Rest. While this is laid out in an easy manner with Rest, we should consider the implications to using JDBC or ODBC with this as well.
I am getting Exception: Status Code: 500 - Error while trying to do '%Drill connect'. What am i missing.
Thanks,
Ganesh
Please enter the password you wish to connect with:
Drill Connect Password: ········
Exception Traceback (most recent call last)
in
----> 1 get_ipython().run_line_magic('drill', 'connect')
~/jupyter/jupyter-env/lib/python3.5/site-packages/IPython/core/interactiveshell.py in run_line_magic(self, magic_name, line, _stack_depth)
2312 kwargs['local_ns'] = sys._getframe(stack_depth).f_locals
2313 with self.builtin_trap:
-> 2314 result = fn(*args, **kwargs)
2315 return result
2316
</home/linaro/jupyter/jupyter-env/lib/python3.5/site-packages/decorator.py:decorator-gen-126> in drill(self, line, cell)
~/jupyter/jupyter-env/lib/python3.5/site-packages/IPython/core/magic.py in (f, *a, **k)
185 # but it's overkill for just that one bit of state.
186 def magic_deco(arg):
--> 187 call = lambda f, *a, **k: f(*a, **k)
188
189 if callable(arg):
~/jupyter/jupyter-env/lib/python3.5/site-packages/drill_core/drill_base.py in drill(self, line, cell)
346 self.connectDrill(True)
347 elif line.lower() == "connect":
--> 348 self.connectDrill(False)
349 elif line.lower() .find('set ') == 0:
350 self.setvar(line)
~/jupyter/jupyter-env/lib/python3.5/site-packages/drill_core/drill_base.py in connectDrill(self, prompt)
210 requests.packages.urllib3.disable_warnings(InsecureRequestWarning)
211 result = -1
--> 212 result = self.authDrill()
213 if result == 0:
214 self.drill_connected = True
~/jupyter/jupyter-env/lib/python3.5/site-packages/drill_core/drill_base.py in authDrill(self)
293 result = 0
294 else:
--> 295 raise Exception("Status Code: %s - Error" % r.status_code)
296 return result
297
Exception: Status Code: 500 - Error
When trying to connect current IPython notebook to Drill installation (Dril=Drill(ipy,pd_use_beaker=False)), It gives an error
'Drill' object is not callable
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
<ipython-input-10-dff484a38606> in <module>
----> 1 d=Drill(ipy,pd_use_beaker=False)
TypeError: 'Drill' object is not callable
What could be the issue here? Any pointers?
Sometimes, after your exploration with %%drill is done, you want to programmatically issue a query and get a data frame. There is no reason this jupyter_drill module can't do that as well.
Currently if you have a notebook open and don't issue a query in a certain timeframe, the Drill cluster will timeout your session, that is normal.
We should better trap when we try to issue a query and a drill connection is not active. We should be able to do that by return code, and
The second point is important. Right now, if the connection times out, and try a query, it will error out, and when it does, you can go to your cell with %drill connect but it won't let you just connect because internally Drill thinks it's still connected. So you have to type %drill disconnect and then do the %drill connect. It's a small thing, but it's a pain and wastes a cell
I think it's possible to run a demo of this magic inside MyBinder.
Add:
apt.txt
file that installs openjdk-8-jdk
wget -q -O apache-drill-1.15.0.tar.gz http://apache.mirrors.hoobly.com/drill/drill-1.15.0/apache-drill-1.15.0.tar.gz; tar xzf apache-drill-1.15.0.tar.gz
to postBuild
jupyter-server-proxy
to requirements.txt
to let you see the Apache Drill Web UI via /proxy/8047
.You can start the server from a notebook terminal as:
apache-drill-1.15.0/bin/drill-embedded
or perhaps autostart it in MyBinder (I haven't tried this yet) by adding a start
file containing something like:
#!/bin/bash
nohup apache-drill-1.15.0/bin/drill-embedded > /dev/null 2>&1 &
exec "$@"
The default Apache Drill install runs with auth, I think, so it would be handy if the jupyter_drill
magic could connect without an auth requirement.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.