Comments (12)
@john-bodley note that cursor.description
should be None
in this case:
This attribute will be None for operations that do not return rows or if the cursor has not had an operation invoked via the .execute*() method yet. (https://www.python.org/dev/peps/pep-0249/#description)
from pydruid.
Maybe we're missing something in the SQLAlchemy dialect?
from pydruid.
@john-bodley note too that .fetchall()
should return an exception when no rows are returned:
An Error (or subclass) exception is raised if the previous call to .execute*() did not produce any result set or no call was issued yet. (https://www.python.org/dev/peps/pep-0249/#fetchall)
So I think this is behaving as expected.
from pydruid.
@betodealmeida the fetchall
isnโt throwing the exception, but instead the underlying cursor is None
because itโs prematurely closed.
from pydruid.
@john-bodley but my point is, you should always wrap your .fetchall()
in a try/except
block, in case zero rows were returned.
from pydruid.
@betodealmeida note's it's the ResultProxy fetchall
method that I'm calling which wraps the underlying DBAPI and has slightly different behavior (per the documentation).
Fetch all rows, just like DB-API cursor.fetchall().
After all rows have been exhausted, the underlying DBAPI cursor resource is released, and the object may be safely discarded.
Subsequent calls to ResultProxy.fetchall() will return an empty list. After the ResultProxy.close() method is called, the method will raise ResourceClosedError.
from pydruid.
@betodealmeida I created sqlalchemy/sqlalchemy#4523 to discuss how one should handle an empty result set associated with a SELECT statement when no description is available.
It seems pydruid
should provide a description even if no rows are returned, the issue is it may be non-trivial to determine the column names to add to the description given the lack of metadata provided by the Druid SQL REST API.
from pydruid.
@betodealmeida it seems that apache/druid#6191 will help remedy the solution as the SQL interface provides a mechanism for returning the column names via the X-Druid-Column-Names
header which will be included in version 0.13.0
(currently incubating).
from pydruid.
@john-bodley : I am also impacted by this issue. I am currently using Druid 0.13.0. But still I am not receiving X-Druid-Column-Names
header in my response. Any druid config file need to be updated for this ?
from pydruid.
As a part of this bug (apache/druid#6409,) , it looks like they moved the column info to response body which can be seen by setting "header" : true
in the request, which will never return empty response.
The below query will return 1 row with all columns
Query
{
"query":"SELECT delta, page, added, deleted FROM wikipedia WHERE 1=0 ",
"resultFormat" : "object",
"header" : true
}
Result
[
{
"delta": null,
"page": null,
"added": null,
"deleted": null
}
]
More:
Section "Response" under http://druid.io/docs/latest/querying/sql.html
from pydruid.
@Makesh-Gmak I can look into providing a fix for this.
@betodealmeida what are your thoughts on how to best proceed? It seems likes including the header
in the request for older versions doesn't break things, however one cannot differentiate between the responses whether the header is present, i.e., it merely inserts a new record at the beginning of the result set, .e.g.,
[
{
"delta": null,
"page": null,
"added": null,
"deleted": null
},
{
"delta": 36,
"page": "Talk:Oswald Tilghman",
"added": 36,
"deleted": 0
}
]
Checking whether the first records has all NULLs as values isn't a robust method as it's definitely plausible to have a result set which includes NULLs. I was thinking the only solutions include:
- Adding the
/status
API call to determine the version of the Druid broker prior to executing each query. - Modify the connection to include the Druid version or whether headers are supported (preferred).
Note (2) is more performant than (1) however it requires additional metadata.
from pydruid.
Thanks for the resolution. When will be the next release ?
from pydruid.
Related Issues (20)
- JSONDecodeError: Expecting ',' delimiter: line 1 column 7060430653 (char 7060430652)
- Evaluating query runtime without output HOT 1
- PyDruid for Data Ingestion in Druid HOT 2
- Version 0.6.3 is broken if you don't already have sqlalchemy installed
- Getting columns for a Druid datasource fails with a KeyError HOT 2
- Subquery not getting converted property HOT 2
- "do_ping" broken in last pydruid release, can't add a Druid as a new Database in Superset, need a pydruid release HOT 10
- Support newer Python versions
- ssl_client_cert passed is being assigned to proxies in Cursor object
- Support for multi-stage-query (sql-msq-task) HOT 1
- Is there a way to query for stats aggregator (STDDEV)?
- Pandas 2.0 dtype_backend="pyarrow"
- Column name not identified on certain special characters
- Querying list of data sources HOT 2
- pydruid not compatible with druid 27 HOT 2
- Is this project currently maintained?
- Error when calling the async client
- inadvertent hard dependency on sqlalchemy
- Row counter
- dependent on prompt-toolkit version 1 however prompt-toolkit version 3.0.30 is required HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. ๐๐๐
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google โค๏ธ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from pydruid.