I'm using Riak 1.3.1. The bucket in question has the following options enabled:
Sometimes I have about 200 siblings under a single key. If I try to write another sibling, try to merge siblings or try to delete the key I get a "500 Internal Server Error". For example:
<html><head><title>500 Internal Server Error</title></head><body><h1>Internal Server Error</h1>The server encountered an error while processing this request:<br><pre>{error,badarg,
[{erlang,iolist_to_binary,
[{hook_crashed,
{riak_search_kv_hook,precommit,error,
{badmatch,
[{{dict,3,16,16,8,80,48,
{[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]},
{{[],[],[],[],[],[],[],[],[],[],
[[<<"X-Riak-VTag">>,49,121,108,121,69,87,87,48,
122,67,86,120,82,50,118,55,118,90,84,80,120,
106]],
[[<<"index">>]],
[],
[[<<"X-Riak-Last-Modified">>|
{1366,994414,313405}]],
[],[]}}},
{riak_idx_doc,<<"scanner">>,
<<"3d7dc8ed1f828c727089033b2978a74ea7178833">>,
[{<<"envelope_from_add">>,
<<"[email protected]">>,
[{<<"name1">>,[0,2]},
{<<"name3">>,[3]},
{<<"name2">>,[1]},
{<<"com">>,[4]}]},
{<<"envelope_to_add">>,<<"[email protected]">>,
[{<<"name4">>,[0]},
{<<"name5">>,[1]},
{<<"com">>,[3]},
{<<"name6">>,[2]}]},
{<<"envelope_to_domain_add">>,<<"name6.com">>,
[{<<"com">>,[1]},{<<"name6">>,[0]}]},
{<<"files">>,
<<"file136699328784114516-unpacked:25999">>,
[{<<"file136699328784114516-unpacked:25999">>,
[0]}]},
{<<"host_add">>,<<"1.2.3.4">>,
[{<<"1">>,[0]},
{<<"3">>,[2]},
{<<"2">>,[1]},
{<<"4">>,[3]}]},
{<<"message_id_add">>,
<<"<[email protected]>">>,
[{<<"name1">>,[2]},
{<<"name7">>,[0]},
{<<"name3">>,[3]},
{<<"name2">>,[1]},
{<<"com">>,[4]}]},
{<<"qs_time_int">>,<<"479">>,
[{<<"0000000479">>,[0]}]},
{<<"relay_int">>,<<"0">>,
[{<<"0000000000">>,[0]}]},
{<<"result">>,<<"SPAM-REJECTED">>,
[{<<"SPAM-REJECTED">>,[0]}]},
{<<"scanner">>,<<"SA">>,[{<<"SA">>,[0]}]},
{<<"score">>,<<"16.6/5.0">>,
[{<<"16.6/5.0">>,[0]}]},
{<<"size_int">>,<<"25999">>,
[{<<"0000025999">>,[0]}]},
{<<"subject_txt">>,
<<"Don't miss our exclusive offers!">>,
[{<<"don">>,[0]},
{<<"offers">>,[4]},
{<<"our">>,[2]},
{<<"exclusive">>,[3]},
{<<"miss">>,[1]}]},
{<<"timestamp_int">>,<<"1366993287">>,
[{<<"1366993287">>,[0]}]}],
[],[],true}},
{{dict,3,16,16,8,80,48,
{[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]},
{{[],[],[],[],[],[],[],[],[],[],
[[<<"X-Riak-VTag">>,51,118,116,100,85,76,88,52,
81,70,118,122,78,66,111,112,55,55,114,102,101,
102]],
[[<<"index">>]],
[],
[[<<"X-Riak-Last-Modified">>|
{1366,994414,312370}]],
[],[]}}},
{riak_idx_doc,<<"scanner">>,
<<"3d7dc8ed1f828c727089033b2978a74ea7178833">>,
[{<<"envelope_from_add">>,
<<"[email protected]">>,
[{<<"name1">>,[0,2]},
{<<"name3">>,[3]},
{<<"name2">>,[1]},
{<<"com">>,[4]}]},
{<<"envelope_to_add">>,<<"[email protected]">>,
[{<<"name4">>,[0]},
{<<"name5">>,[1]},
{<<"com">>,[3]},
{<<"name6">>,[2]}]},
{<<"envelope_to_domain_add">>,<<"name6.com">>,
[{<<"com">>,[1]},{<<"name6">>,[0]}]},
{<<"files">>,
<<"file136699390984128832-unpacked:25998">>,
[{<<"file136699390984128832-unpacked:25998">>,
[0]}]},
{<<"host_add">>,<<"1.2.3.4">>,
[{<<"1">>,[0]},
{<<"3">>,[2]},
{<<"4">>,[3]},
{<<"2">>,[1]}]},
{<<"message_id_add">>,
<<"<[email protected]>">>,
[{<<"name1">>,[2]},
{<<"name7">>,[0]},
{<<"name3">>,[3]},
{<<"name2">>,[1]},
{<<"com">>,[4]}]},
{<<"qs_time_int">>,<<"303">>,
[{<<"0000000303">>,[0]}]},
{<<"relay_int">>,<<"0">>,
[{<<"0000000000">>,[0]}]},
{<<"result">>,<<"SPAM-REJECTED">>,
[{<<"SPAM-REJECTED">>,[0]}]},
{<<"scanner">>,<<"SA">>,[{<<"SA">>,[0]}]},
{<<"score">>,<<"16.6/5.0">>,
[{<<"16.6/5.0">>,[0]}]},
{<<"size_int">>,<<"25998">>,
[{<<"0000025998">>,[0]}]},
{<<"subject_txt">>,
<<"Don't miss our exclusive offers!">>,
[{<<"don">>,[0]},
{<<"offers">>,[4]},
{<<"our">>,[2]},
{<<"exclusive">>,[3]},
{<<"miss">>,[1]}]},
{<<"timestamp_int">>,<<"1366993909">>,
[{<<"1366993909">>,[0]}]}],
[],[],true}}]}}}],
[]},
{wrq,append_to_response_body,2,[{file,"src/wrq.erl"},{line,205}]},
{riak_kv_wm_object,handle_common_error,3,
[{file,"src/riak_kv_wm_object.erl"},{line,998}]},
{webmachine_resource,resource_call,3,
[{file,"src/webmachine_resource.erl"},{line,183}]},
{webmachine_resource,do,3,
[{file,"src/webmachine_resource.erl"},{line,141}]},
{webmachine_decision_core,resource_call,1,
[{file,"src/webmachine_decision_core.erl"},{line,48}]},
{webmachine_decision_core,decision,1,
[{file,"src/webmachine_decision_core.erl"},{line,439}]},
{webmachine_decision_core,handle_request,2,
[{file,"src/webmachine_decision_core.erl"},{line,33}]}]}</pre><P><HR><ADDRESS>mochiweb+webmachine web server</ADDRESS></body></html>
The setup is a fairly standard one, except that there is a custom text analyzer for fields ending with "_add".
If you want me to provide more details, let me please know what logs or files should I inspect. The logs I checked don't provide more details than this.
This error happens only sometimes, but when it does, it's impossible to delete the object. The only workaround I know is to turn off the search. It's possible to delete the object then, but the corrupted data stay in the index.