Comments (13)
Thanks for reporting it, I'll fix it.
from heavydb.
I suspect you might be running an old version. Do you mind taking a look at the logs? They're in the mapd_log
sub-folder of the data folder; the tail of mapd_server.FATAL
is the interesting one. Also, a very small sample of the input data (CSV files, I assume) would help me investigate.
On master tip, I get an exception: "Columnar conversion not supported for variable length types". It should be relatively easy to make the query run, though. But let's check the logs first.
from heavydb.
@asuhan We're running the latest version as far as I know:
mapdql> \version
MapD Server Version: 3.1.1-20170626-45a6fa8
As for the logs, there's no mapd_server.FATAL file:
/mapd-storage/data/mapd_log# ll
total 12752
drwxr-xr-x 2 root root 4096 Jun 30 22:38 ./
drwxr-xr-x 6 root root 4096 Jun 28 20:59 ../
-rw-r--r-- 1 root root 43702 Jun 30 20:10 log4j.log
-rw-r--r-- 1 root root 4303 Jun 28 20:59 mapd_server.16966765e962.invalid-user.log.INFO.20170628-205953.10
-rw-r--r-- 1 root root 243 Jun 30 15:38 mapd_server.40b4a47303fa.invalid-user.log.ERROR.20170630-153826.8
-rw-r--r-- 1 root root 5133 Jun 30 15:38 mapd_server.40b4a47303fa.invalid-user.log.INFO.20170630-153729.8
-rw-r--r-- 1 root root 243 Jun 30 15:38 mapd_server.40b4a47303fa.invalid-user.log.WARNING.20170630-153826.8
-rw-r--r-- 1 root root 4265038 Jun 30 20:47 mapd_server.5318670aedde.invalid-user.log.ERROR.20170630-153940.8
-rw-r--r-- 1 root root 4314026 Jun 30 20:55 mapd_server.5318670aedde.invalid-user.log.INFO.20170630-153925.8
-rw-r--r-- 1 root root 4266468 Jun 30 20:47 mapd_server.5318670aedde.invalid-user.log.WARNING.20170630-153940.8
-rw-r--r-- 1 root root 269 Jun 28 21:10 mapd_server.5ebd079059b6.invalid-user.log.ERROR.20170628-211057.8
-rw-r--r-- 1 root root 5361 Jun 28 21:12 mapd_server.5ebd079059b6.invalid-user.log.INFO.20170628-210557.8
-rw-r--r-- 1 root root 269 Jun 28 21:10 mapd_server.5ebd079059b6.invalid-user.log.WARNING.20170628-211057.8
-rw-r--r-- 1 root root 7814 Jun 30 20:57 mapd_server.902ae0dc479e.invalid-user.log.INFO.20170630-205703.8
lrwxrwxrwx 1 root root 65 Jun 30 22:28 mapd_server.ERROR -> mapd_server.b52e1cddf3f3.invalid-user.log.ERROR.20170630-222827.8
lrwxrwxrwx 1 root root 64 Jun 30 22:38 mapd_server.INFO -> mapd_server.b52e1cddf3f3.invalid-user.log.INFO.20170630-223847.8
lrwxrwxrwx 1 root root 67 Jun 30 22:28 mapd_server.WARNING -> mapd_server.b52e1cddf3f3.invalid-user.log.WARNING.20170630-222827.8
-rw-r--r-- 1 root root 1459 Jun 30 15:30 mapd_server.a13f330deb9f.invalid-user.log.ERROR.20170629-190953.8
-rw-r--r-- 1 root root 16006 Jun 30 15:31 mapd_server.a13f330deb9f.invalid-user.log.INFO.20170629-190342.8
-rw-r--r-- 1 root root 1459 Jun 30 15:30 mapd_server.a13f330deb9f.invalid-user.log.WARNING.20170629-190953.8
-rw-r--r-- 1 root root 314 Jun 30 22:28 mapd_server.b52e1cddf3f3.invalid-user.log.ERROR.20170630-222827.8
-rw-r--r-- 1 root root 5851 Jun 30 22:28 mapd_server.b52e1cddf3f3.invalid-user.log.INFO.20170630-222146.8
-rw-r--r-- 1 root root 5655 Jun 30 22:38 mapd_server.b52e1cddf3f3.invalid-user.log.INFO.20170630-222926.8
-rw-r--r-- 1 root root 4303 Jun 30 22:38 mapd_server.b52e1cddf3f3.invalid-user.log.INFO.20170630-223847.8
-rw-r--r-- 1 root root 314 Jun 30 22:28 mapd_server.b52e1cddf3f3.invalid-user.log.WARNING.20170630-222827.8
-rw-r--r-- 1 root root 4303 Jun 29 13:52 mapd_server.be5c4364cc4d.invalid-user.log.INFO.20170629-135207.7
-rw-r--r-- 1 root root 7245 Jun 30 21:07 mapd_server.c706b41c7300.invalid-user.log.INFO.20170630-210150.8
-rw-r--r-- 1 root root 0 Jun 28 20:59 mapd_web_server.16966765e962.root.log.ACCESS.20170628-205953.11
-rw-r--r-- 1 root root 0 Jun 28 20:59 mapd_web_server.16966765e962.root.log.ALL.20170628-205953.11
-rw-r--r-- 1 root root 0 Jun 30 15:37 mapd_web_server.40b4a47303fa.root.log.ACCESS.20170630-153729.9
-rw-r--r-- 1 root root 0 Jun 30 15:37 mapd_web_server.40b4a47303fa.root.log.ALL.20170630-153729.9
-rw-r--r-- 1 root root 0 Jun 30 15:39 mapd_web_server.5318670aedde.root.log.ACCESS.20170630-153925.9
-rw-r--r-- 1 root root 0 Jun 30 15:39 mapd_web_server.5318670aedde.root.log.ALL.20170630-153925.9
-rw-r--r-- 1 root root 0 Jun 28 21:05 mapd_web_server.5ebd079059b6.root.log.ACCESS.20170628-210557.9
-rw-r--r-- 1 root root 0 Jun 28 21:05 mapd_web_server.5ebd079059b6.root.log.ALL.20170628-210557.9
-rw-r--r-- 1 root root 0 Jun 30 20:57 mapd_web_server.902ae0dc479e.root.log.ACCESS.20170630-205703.9
-rw-r--r-- 1 root root 0 Jun 30 20:57 mapd_web_server.902ae0dc479e.root.log.ALL.20170630-205703.9
-rw-r--r-- 1 root root 0 Jun 29 19:03 mapd_web_server.a13f330deb9f.root.log.ACCESS.20170629-190342.9
-rw-r--r-- 1 root root 0 Jun 29 19:03 mapd_web_server.a13f330deb9f.root.log.ALL.20170629-190342.9
-rw-r--r-- 1 root root 0 Jun 30 22:21 mapd_web_server.b52e1cddf3f3.root.log.ACCESS.20170630-222146.9
-rw-r--r-- 1 root root 0 Jun 30 22:29 mapd_web_server.b52e1cddf3f3.root.log.ACCESS.20170630-222926.9
-rw-r--r-- 1 root root 0 Jun 30 22:38 mapd_web_server.b52e1cddf3f3.root.log.ACCESS.20170630-223847.9
-rw-r--r-- 1 root root 0 Jun 30 22:21 mapd_web_server.b52e1cddf3f3.root.log.ALL.20170630-222146.9
-rw-r--r-- 1 root root 0 Jun 30 22:29 mapd_web_server.b52e1cddf3f3.root.log.ALL.20170630-222926.9
-rw-r--r-- 1 root root 0 Jun 30 22:38 mapd_web_server.b52e1cddf3f3.root.log.ALL.20170630-223847.9
-rw-r--r-- 1 root root 0 Jun 29 13:52 mapd_web_server.be5c4364cc4d.root.log.ACCESS.20170629-135206.8
-rw-r--r-- 1 root root 0 Jun 29 13:52 mapd_web_server.be5c4364cc4d.root.log.ALL.20170629-135206.8
-rw-r--r-- 1 root root 0 Jun 30 21:01 mapd_web_server.c706b41c7300.root.log.ACCESS.20170630-210150.9
-rw-r--r-- 1 root root 0 Jun 30 21:01 mapd_web_server.c706b41c7300.root.log.ALL.20170630-210150.9
Not sure if this is related, but this MapD instance is running in Docker (ref: https://community.mapd.com/t/mapd-docker-support/246/7). Perhaps when it's core dumping it's exiting the process and killing the container before it has a chance to write the mapd_server.FATAL
file?
from heavydb.
Ok, this makes sense, we did crash in 3.1.1. My testing shows we don't crash in master and throw the exception instead. But it shouldn't be too hard to make the query run, I'll give it a go next week.
from heavydb.
With 563e0b3, query now runs on my made-up data.
from heavydb.
Thanks for the quick turnaround @asuhan! Any idea when this will be released?
from heavydb.
@bploetz Most likely ~2 weeks from now. Also, how many rows do you have in profiles
and stories
, respectively? You might need the shard key feature too.
from heavydb.
@asuhan ok. Are there pre-release/beta prebuilt binaries available somewhere ahead of release, or will I need to build from source to get this change?
As for the number of rows in the tables, roughly 1B and 500M respectively. We plan on reaching out to you guys about the enterprise edition, but right now we're just in evaluation phase.
Thanks again.
from heavydb.
@andrewseidl might know more, although I don't know whether we have a well-defined process for sharing early builds.
Also, note that id
has to be unique within profiles
(we're working to remove this constraint) and you'll need to shard your tables for the full sizes you've mentioned. This patch alone should allow you to test on a sample, though.
from heavydb.
Thanks @asuhan. Yeah, we fully expected to use the sharded implementation for the full data set, we're just testing MapD out with a very small subset of the data.
from heavydb.
This will change soon, but for now you can find the latest open source build off master at https://jenkins-os.mapd.com/job/core-os/lastSuccessfulBuild/immerse=off,processor=cuda,sanitizer=off/
Note that this is a fully open source build: it does not include features like Immerse or rendering. Let me know if you need those.
from heavydb.
@andrewseidl Awesome, this is perfect as I don't need immerse or rendering. Thanks so much, I will give this a spin!
from heavydb.
@asuhan I downloaded the latest master build and the fix works like a charm!
mapdql> select count(p.id) from stories s, profiles p where s.dma = '501' and s.profile_id = p.id and 'interest:hispanic' = ANY p.audience_ids;
EXPR$0
2067
1 rows returned.
Execution time: 31 ms, Total time: 32 ms
from heavydb.
Related Issues (20)
- [GPU Error Bug] CAST(<column>+<column>(overflow) AS BOOLEAN) Brings Errors
- Evaluate using Profile-Guided Optimization (PGO) and Post-Link Optimization (PLO) HOT 1
- Intermitted SIGSEGV errors crashing heavyDB HOT 6
- Cannot import on an individual leaf. Please import from the Aggregator. HOT 1
- pinned memory HOT 2
- Failed to compile heavyDB; CUDA architecture not detected HOT 3
- Some demos on the website are not working or outdated HOT 1
- Error Running HeavyDB with Nvidia Nsight Compute: Broken Pipe in Thrift Connection HOT 7
- Heavydb has not been updated for more than 7 months, is there any plan to continue to update the open source code? HOT 4
- [GPU Logic Bug] SELECT /*+ keep_table_function_result */ <column> FROM <table> GROUP BY <column> Brings Errors HOT 1
- [GPU Logic Bug] SELECT /*+ keep_result */ DISTINCT <column> FROM <table> GROUP BY <column> LIMIT <number> Brings Errors HOT 1
- [GPU Logic Bug] SELECT DISTINCT <column> FROM <table> Brings Errors HOT 1
- [GPU Logic Bug] SELECT <column> FROM <table> JOIN <table> Brings Errors
- [GPU Logic Bug] SELECT /*+ keep_result */ <column> FROM <table> LIMIT <number> OFFSET <number> Brings Errors HOT 1
- [GPU Logic Bug] SELECT /*+ keep_table_function_result */ <column> FROM <table> LIMIT <number> OFFSET <number> Brings Errors HOT 1
- [GPU Logic Bug] SELECT <agg_func> FROM <table> Brings Errors HOT 1
- [GPU Logic Bug] SELECT <column> FROM <table> Brings Errors
- display a large amount of geometry HOT 2
- Too many active sessions HOT 1
- Heavyai-connector
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from heavydb.