Query causes server to core dump about heavydb

[asuhan]

Thanks for reporting it, I'll fix it.

[asuhan]

I suspect you might be running an old version. Do you mind taking a look at the logs? They're in the mapd_log sub-folder of the data folder; the tail of mapd_server.FATAL is the interesting one. Also, a very small sample of the input data (CSV files, I assume) would help me investigate.

On master tip, I get an exception: "Columnar conversion not supported for variable length types". It should be relatively easy to make the query run, though. But let's check the logs first.

[bploetz]

@asuhan We're running the latest version as far as I know:

mapdql> \version
MapD Server Version: 3.1.1-20170626-45a6fa8

As for the logs, there's no mapd_server.FATAL file:

/mapd-storage/data/mapd_log# ll
total 12752
drwxr-xr-x 2 root root    4096 Jun 30 22:38 ./
drwxr-xr-x 6 root root    4096 Jun 28 20:59 ../
-rw-r--r-- 1 root root   43702 Jun 30 20:10 log4j.log
-rw-r--r-- 1 root root    4303 Jun 28 20:59 mapd_server.16966765e962.invalid-user.log.INFO.20170628-205953.10
-rw-r--r-- 1 root root     243 Jun 30 15:38 mapd_server.40b4a47303fa.invalid-user.log.ERROR.20170630-153826.8
-rw-r--r-- 1 root root    5133 Jun 30 15:38 mapd_server.40b4a47303fa.invalid-user.log.INFO.20170630-153729.8
-rw-r--r-- 1 root root     243 Jun 30 15:38 mapd_server.40b4a47303fa.invalid-user.log.WARNING.20170630-153826.8
-rw-r--r-- 1 root root 4265038 Jun 30 20:47 mapd_server.5318670aedde.invalid-user.log.ERROR.20170630-153940.8
-rw-r--r-- 1 root root 4314026 Jun 30 20:55 mapd_server.5318670aedde.invalid-user.log.INFO.20170630-153925.8
-rw-r--r-- 1 root root 4266468 Jun 30 20:47 mapd_server.5318670aedde.invalid-user.log.WARNING.20170630-153940.8
-rw-r--r-- 1 root root     269 Jun 28 21:10 mapd_server.5ebd079059b6.invalid-user.log.ERROR.20170628-211057.8
-rw-r--r-- 1 root root    5361 Jun 28 21:12 mapd_server.5ebd079059b6.invalid-user.log.INFO.20170628-210557.8
-rw-r--r-- 1 root root     269 Jun 28 21:10 mapd_server.5ebd079059b6.invalid-user.log.WARNING.20170628-211057.8
-rw-r--r-- 1 root root    7814 Jun 30 20:57 mapd_server.902ae0dc479e.invalid-user.log.INFO.20170630-205703.8
lrwxrwxrwx 1 root root      65 Jun 30 22:28 mapd_server.ERROR -> mapd_server.b52e1cddf3f3.invalid-user.log.ERROR.20170630-222827.8
lrwxrwxrwx 1 root root      64 Jun 30 22:38 mapd_server.INFO -> mapd_server.b52e1cddf3f3.invalid-user.log.INFO.20170630-223847.8
lrwxrwxrwx 1 root root      67 Jun 30 22:28 mapd_server.WARNING -> mapd_server.b52e1cddf3f3.invalid-user.log.WARNING.20170630-222827.8
-rw-r--r-- 1 root root    1459 Jun 30 15:30 mapd_server.a13f330deb9f.invalid-user.log.ERROR.20170629-190953.8
-rw-r--r-- 1 root root   16006 Jun 30 15:31 mapd_server.a13f330deb9f.invalid-user.log.INFO.20170629-190342.8
-rw-r--r-- 1 root root    1459 Jun 30 15:30 mapd_server.a13f330deb9f.invalid-user.log.WARNING.20170629-190953.8
-rw-r--r-- 1 root root     314 Jun 30 22:28 mapd_server.b52e1cddf3f3.invalid-user.log.ERROR.20170630-222827.8
-rw-r--r-- 1 root root    5851 Jun 30 22:28 mapd_server.b52e1cddf3f3.invalid-user.log.INFO.20170630-222146.8
-rw-r--r-- 1 root root    5655 Jun 30 22:38 mapd_server.b52e1cddf3f3.invalid-user.log.INFO.20170630-222926.8
-rw-r--r-- 1 root root    4303 Jun 30 22:38 mapd_server.b52e1cddf3f3.invalid-user.log.INFO.20170630-223847.8
-rw-r--r-- 1 root root     314 Jun 30 22:28 mapd_server.b52e1cddf3f3.invalid-user.log.WARNING.20170630-222827.8
-rw-r--r-- 1 root root    4303 Jun 29 13:52 mapd_server.be5c4364cc4d.invalid-user.log.INFO.20170629-135207.7
-rw-r--r-- 1 root root    7245 Jun 30 21:07 mapd_server.c706b41c7300.invalid-user.log.INFO.20170630-210150.8
-rw-r--r-- 1 root root       0 Jun 28 20:59 mapd_web_server.16966765e962.root.log.ACCESS.20170628-205953.11
-rw-r--r-- 1 root root       0 Jun 28 20:59 mapd_web_server.16966765e962.root.log.ALL.20170628-205953.11
-rw-r--r-- 1 root root       0 Jun 30 15:37 mapd_web_server.40b4a47303fa.root.log.ACCESS.20170630-153729.9
-rw-r--r-- 1 root root       0 Jun 30 15:37 mapd_web_server.40b4a47303fa.root.log.ALL.20170630-153729.9
-rw-r--r-- 1 root root       0 Jun 30 15:39 mapd_web_server.5318670aedde.root.log.ACCESS.20170630-153925.9
-rw-r--r-- 1 root root       0 Jun 30 15:39 mapd_web_server.5318670aedde.root.log.ALL.20170630-153925.9
-rw-r--r-- 1 root root       0 Jun 28 21:05 mapd_web_server.5ebd079059b6.root.log.ACCESS.20170628-210557.9
-rw-r--r-- 1 root root       0 Jun 28 21:05 mapd_web_server.5ebd079059b6.root.log.ALL.20170628-210557.9
-rw-r--r-- 1 root root       0 Jun 30 20:57 mapd_web_server.902ae0dc479e.root.log.ACCESS.20170630-205703.9
-rw-r--r-- 1 root root       0 Jun 30 20:57 mapd_web_server.902ae0dc479e.root.log.ALL.20170630-205703.9
-rw-r--r-- 1 root root       0 Jun 29 19:03 mapd_web_server.a13f330deb9f.root.log.ACCESS.20170629-190342.9
-rw-r--r-- 1 root root       0 Jun 29 19:03 mapd_web_server.a13f330deb9f.root.log.ALL.20170629-190342.9
-rw-r--r-- 1 root root       0 Jun 30 22:21 mapd_web_server.b52e1cddf3f3.root.log.ACCESS.20170630-222146.9
-rw-r--r-- 1 root root       0 Jun 30 22:29 mapd_web_server.b52e1cddf3f3.root.log.ACCESS.20170630-222926.9
-rw-r--r-- 1 root root       0 Jun 30 22:38 mapd_web_server.b52e1cddf3f3.root.log.ACCESS.20170630-223847.9
-rw-r--r-- 1 root root       0 Jun 30 22:21 mapd_web_server.b52e1cddf3f3.root.log.ALL.20170630-222146.9
-rw-r--r-- 1 root root       0 Jun 30 22:29 mapd_web_server.b52e1cddf3f3.root.log.ALL.20170630-222926.9
-rw-r--r-- 1 root root       0 Jun 30 22:38 mapd_web_server.b52e1cddf3f3.root.log.ALL.20170630-223847.9
-rw-r--r-- 1 root root       0 Jun 29 13:52 mapd_web_server.be5c4364cc4d.root.log.ACCESS.20170629-135206.8
-rw-r--r-- 1 root root       0 Jun 29 13:52 mapd_web_server.be5c4364cc4d.root.log.ALL.20170629-135206.8
-rw-r--r-- 1 root root       0 Jun 30 21:01 mapd_web_server.c706b41c7300.root.log.ACCESS.20170630-210150.9
-rw-r--r-- 1 root root       0 Jun 30 21:01 mapd_web_server.c706b41c7300.root.log.ALL.20170630-210150.9

Not sure if this is related, but this MapD instance is running in Docker (ref: https://community.mapd.com/t/mapd-docker-support/246/7). Perhaps when it's core dumping it's exiting the process and killing the container before it has a chance to write the mapd_server.FATAL file?

[asuhan]

Ok, this makes sense, we did crash in 3.1.1. My testing shows we don't crash in master and throw the exception instead. But it shouldn't be too hard to make the query run, I'll give it a go next week.

[asuhan]

With 563e0b3, query now runs on my made-up data.

[bploetz]

Thanks for the quick turnaround @asuhan! Any idea when this will be released?

[asuhan]

@bploetz Most likely ~2 weeks from now. Also, how many rows do you have in profiles and stories, respectively? You might need the shard key feature too.

[bploetz]

@asuhan ok. Are there pre-release/beta prebuilt binaries available somewhere ahead of release, or will I need to build from source to get this change?

As for the number of rows in the tables, roughly 1B and 500M respectively. We plan on reaching out to you guys about the enterprise edition, but right now we're just in evaluation phase.

Thanks again.

[asuhan]

@andrewseidl might know more, although I don't know whether we have a well-defined process for sharing early builds.

Also, note that id has to be unique within profiles (we're working to remove this constraint) and you'll need to shard your tables for the full sizes you've mentioned. This patch alone should allow you to test on a sample, though.

[bploetz]

Thanks @asuhan. Yeah, we fully expected to use the sharded implementation for the full data set, we're just testing MapD out with a very small subset of the data.

[andrewseidl]

This will change soon, but for now you can find the latest open source build off master at https://jenkins-os.mapd.com/job/core-os/lastSuccessfulBuild/immerse=off,processor=cuda,sanitizer=off/

Note that this is a fully open source build: it does not include features like Immerse or rendering. Let me know if you need those.

[bploetz]

@andrewseidl Awesome, this is perfect as I don't need immerse or rendering. Thanks so much, I will give this a spin!

[bploetz]

@asuhan I downloaded the latest master build and the fix works like a charm!

mapdql> select count(p.id) from stories s, profiles p where s.dma = '501' and s.profile_id = p.id and 'interest:hispanic' = ANY p.audience_ids;
EXPR$0
2067
1 rows returned.
Execution time: 31 ms, Total time: 32 ms