Benchmark for src-d/Engine about jgit-spark-connector

[bzz]

Goal is to determine expectations on the performance - how much is it possible to do in 1h?

Most common use-case:

• sub-set of repos, HEAD of ref, get all files
• filter for README or some other files on top of that

[ajnavarro]

Using ~330 siva files from the 10k most relevant github repositories, after ~30min of processing I got a out of space error:

Py4JJavaError: An error occurred while calling o73.showString.
: org.apache.spark.SparkException: Job aborted due to stage failure: Task 4 in stage 9.0 failed 1 times, most recent failure: Lost task 4.0 in stage 9.0 (TID 133, localhost, executor driver): java.io.IOException: No space left on device
	at java.io.FileOutputStream.writeBytes(Native Method)
	at java.io.FileOutputStream.write(FileOutputStream.java:326)
	at org.apache.spark.storage.TimeTrackingOutputStream.write(TimeTrackingOutputStream.java:58)
	at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
	at java.io.BufferedOutputStream.write(BufferedOutputStream.java:126)
	at net.jpountz.lz4.LZ4BlockOutputStream.flushBufferedData(LZ4BlockOutputStream.java:205)
	at net.jpountz.lz4.LZ4BlockOutputStream.write(LZ4BlockOutputStream.java:158)
	at java.io.BufferedOutputStream.write(BufferedOutputStream.java:122)
	at java.io.DataOutputStream.write(DataOutputStream.java:107)
	at org.apache.spark.sql.catalyst.expressions.UnsafeRow.writeToStream(UnsafeRow.java:554)
	at org.apache.spark.sql.execution.UnsafeRowSerializerInstance$$anon$1.writeValue(UnsafeRowSerializer.scala:69)
	at org.apache.spark.storage.DiskBlockObjectWriter.write(DiskBlockObjectWriter.scala:239)
	at org.apache.spark.shuffle.sort.BypassMergeSortShuffleWriter.write(BypassMergeSortShuffleWriter.java:151)
	at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:96)
	at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:53)
	at org.apache.spark.scheduler.Task.run(Task.scala:108)
	at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:335)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)

Before stop the container:

Filesystem      Size  Used Avail Use% Mounted on
udev            7,8G     0  7,8G   0% /dev
tmpfs           1,6G  9,9M  1,6G   1% /run
/dev/nvme0n1p2  454G  430G  1,2G 100% /

After stop the container:

Filesystem      Size  Used Avail Use% Mounted on
udev            7,8G     0  7,8G   0% /dev
tmpfs           1,6G  9,8M  1,6G   1% /run
/dev/nvme0n1p2  454G  168G  264G  39% /

We had ~263GB on suffle operation!

[ajnavarro]

Executed code: https://github.com/ajnavarro/spark-api/blob/b410479a55d1d47f142a69150269942d8826efe0/examples/Basic%2BExample.ipynb

List of siva files used:

002ae35061beef02453b7fb1045a50fa2f7f30f8.siva
00cf9644d8a85e98cf6eb6c0ebf32cf03046f0ee.siva
02de46a5464f182d3d64be5a7ee1087ae8be8646.siva
037ab4ae56cfd96cfd7f995acccb69ed5429048b.siva
03ff523bfe2f2a3848470ce0ca46f2ef7116453c.siva
042ce8780d989197c3d672807f54aa3e6baded3e.siva
0462cef5812ce31fe12f25596ff68dc614c708af.siva
04709fb6e5cb73060a962e85099ae704ce7bad45.siva
056055f316ae660880c8262feb29ccd4e2bc1191.siva
05ebf27d99375e846f4c6c8d29552382caf7876a.siva
07ae36e644cdca99eb5c2bbc5fbbf562512325a6.siva
080d38986c0745d953fc099c0b99b3d4615977e7.siva
091a319f5bef99eaf8b3566a8d1dec415ef5737e.siva
0b542c3aa496154c3efe4a00026dfc8c27a08c02.siva
0c084f803c50998d853e5b1920116e6685f8bd04.siva
0c08f4d2d6b64db491e255ab49949e8bb473519d.siva
0eb7d308615bae1ad4be1ca5112ac7b6b6cbfbaf.siva
1047940ca4363b04044c4963b9c88f7632746407.siva
1076024aa1deb4cabe02f240b7c2771ae5507d5e.siva
117a61cfd1bba9ad10c8b680a574fcf32f6f46be.siva
1193efb1ef4d4ac956f1e1a3342d3e1611b2df15.siva
11de50a83acb07ab3a4439b38f0153444a124381.siva
120877c56af49af18f80240d24b1df3bf5b45c59.siva
12612528e06b75c8621355192a99c8214eb5a45f.siva
1296064245b5ffdd5615442f3bc1f067f619a5e1.siva
131a927cc3ff3cdba7e426b580902c0656428d36.siva
146bae2f97f5d19a1d683a5087c2832cf832651a.siva
15319e86ebd4c7d36f38ae4dfe14b012596b08c0.siva
156a1882e158141e13ccd05d82672796646a0ae6.siva
15a96783ca2ac8c0eb2c400701e8eb335059c63b.siva
16aa0d001af4b590d9ae8753a45d6b9574b14436.siva
18fbe65523fe44bf95cca6ae72dc9aa52e9e933e.siva
1b495fab1d56c02d3f19930267101acf216a2596.siva
1b7ce7ad98dea38fa89114ae45e88355adb0f061.siva
1c8a9acc8a06fea922a218d3b8ca46b4f911ea59.siva
1d91b651ac002a221bea2e37fb72f205279264cf.siva
1edb98ccdf09859cdac87a496bc3ce51bdd4bab1.siva
209bc23d19e990a85655c4654a7190c585ca0733.siva
2300b8e2895667d360788d0077ac8ab2e43794fc.siva
245e4be029eec7ffe0e9bf0f08ccd7d2f27b63a5.siva
249e48bac66f1f6948819d8bf6320aab3fcf71a0.siva
261f28d833cb3df77dbc76852c85a98f61a643a3.siva
26832e92b40470bfefce1062110d02f6562d01b3.siva
270903c8657ce76bca7c3c33e712be0f7b843470.siva
2868b0be9865b5fcbf0280de5e0201875c2eda0b.siva
28ef08ceba8e6b03cbe79c885947eca21e3272b0.siva
2a178b53f90030b03cc82c53f8d56721b044baac.siva
2a63058a080da84f2eddd3cc8e041f2afe1e8918.siva
2b3fac174db42aa7944d6e606a17d5ca1ae66715.siva
2c4595e3278c4b4b4175c1dbf6d00a5b02fc65f2.siva
2c582787d67b754a703954f01e9fce6a0b75b9eb.siva
2c99eac2075dc992fc14b437f81bafdcf74da6f6.siva
2c9ed27ae772eb3e9756f6b63be4c7326d4b3b3f.siva
2d24f04aadda8e4c2ad1fa0a71be12a2ba1f31a5.siva
2d58138f24fa863c235b0c33158b870a40c79ee2.siva
2eb489e83543002270bcf4edbbf44507804e8c71.siva
2ee0bbd335b2fe6b2d038ebac8f881770ffd0358.siva
2ef252148f769d862397eefb94a922f5b058fee9.siva
2f3c86e962f4262f3c23a1360341accb3fc56533.siva
2ff6ac1f8bba78e2d608960d076b3090dd26e76e.siva
30733af1726cfbfbb0c764a6b695c16f0702ebb6.siva
314f13b1e4786ad7bc020dbad361e0d94c177833.siva
3275a2ac9688bf12bac0ac29b4977c52c3079dfc.siva
33850c0ebd23ae615e6823993d441f46d80b1ff0.siva
34a17a6af4162ab11fb7a907abecf0101b5e5cb1.siva
34b0f4d5c5326b287fcc31f09357fac365a1d51e.siva
34ea1ed8b7ec3188af378100dc43616e8d304856.siva
35c2d003470dc41fe1a438e43f345198127a920f.siva
3640eef214070825a8e84d0be524b74fbd56e231.siva
366676c9bbf8022001c88eb911d40870d662b593.siva
36dd99374a1d6857af7652a9fb832a712a066e16.siva
375faed5e47083d248516c15b5c2ab0af1d24845.siva
3a0ca9ed640c0bc3fd418ab37be1c14a325a2247.siva
3a305e16af9e274b89db2834e3b7cea9cea6806f.siva
3aae096ac28efc18a12b2a7bb8cb37aef42f9c62.siva
3b34c84efc543ec08fd55a341388ffe1171db5e3.siva
3b3a51b8d99218068c9174c8355d34700ed614e8.siva
3b8e8103e41ccf61902ee97a30106f1c98083bd5.siva
3c6cd485c6c51702661dcb42b3cbc1dc0a60c60a.siva
3c9076b3caaf7b870aa49ff22cf3d63877bd9507.siva
3d61eb00445c2706ba2a64abaffd8e1b90d4024c.siva
3dfc1865c6497e40949c554e3684442f0d830197.siva
3e97ebdc3422e2d55074a3ea728a54f1ef107884.siva
3f7fbf9db0f9a818140adb87731ba628ac6b8849.siva
3f90773a6694aaf472b1ad84e02dece87cf1df48.siva
3f977868de52848ffb5f435a6ecf7f3995c4e15e.siva
40181d504a6d3604ed08a3529b8f09a7d9da3553.siva
4022170d90bca1a5118fd813200087290c693b6f.siva
403dfa0a52772f21d1c8c62979f986a36ab47109.siva
4101de3daf91c6d35b92395a72bf84132ef48f7c.siva
4112c9f0b328cbe8895f56c4481a5b2492c10ff6.siva
418156debaf9f242f3c2bcc4badaa63e6630cb2a.siva
43d3a6289fa44ddaca718c210388f6a0890fa53e.siva
44326584e6423db445d59dd534492db29e336703.siva
444a50b78a3b20423bd47a51a759ba60b5e51ca3.siva
445265b4f7e8aeaa70ed6354eb2c0178f7448204.siva
45f236b5b01bb4035df69bd174cd3243e2d10285.siva
4777e8ba645b50a9c84bc19d9feb9111fdda498c.siva
47b25bd4da1dfac1e3339c3b4f1c90b58151d5cb.siva
48f7c8ec7a83ea97f954342bf1f58a575bf587ea.siva
48fac41b87acf318ab5b8cc8a7978ccea8c8f4ab.siva
4c4f4046e0446d3c2775b901a8934623a369a9b2.siva
4c86839a5fcc14594094aea52a5d5856c5e11e4a.siva
4d168dce42adf36f8cdd0d928fb8b12868d57773.siva
4d2f1c90d8288058631f10821bcf71db4b0cdbe2.siva
4d8d891c371b91abdda0f6933ec1ad75e26378c0.siva
4e693ffcfd12f597e39ec617fa8a8cbf686d14e5.siva
4f59fdcf5778917ce85ed9818f32a336b0b719d1.siva
4fa74b5252a23c2890ddee52b8ee5811b5bb2987.siva
4fd56bd35c27861337861233a7f362474157af57.siva
4fff4f5eac0b4e01066187be380cc0b2286110e0.siva
51607386c7609a483568ad935083c9668fe6241b.siva
52b950945fb1a399d171d42234b74afaba7eb579.siva
52ccda2b6d83b311f3099de852fbab6df7a4e6b4.siva
5473a6284c4bb6cdecdcee459da966e7b88e854f.siva
553bd93d537a47b7493e6d9fd47af17fbb311b64.siva
55c9ca1b404a76c08ffa65bf05ee483b00afad96.siva
5682a85075029a98e914a3d8b498307c8c44f37a.siva
5776fdfc70c038b8bcebd4b98c4d9deeae921a8d.siva
5813d22e0e84c41cc572ada4c014d591ee3f188b.siva
5aa20a1d0c988503dc1d5311d3508013c0a2cde8.siva
5c13be380afe5829b456197c01a611d7f204b3a8.siva
5c7359365f87a8032b9e91d9a50aa9990b9ebb46.siva
5fee47fc04f955f95e0abeebbe0cd2fae6c5e1e1.siva
6071ebef6f94c3cbb53be0799e7a70d728a73ee0.siva
60771351f981f5072e36c43159368854ccc8a331.siva
60f10041eac8f7c4760c006543a594ac2c6724ac.siva
6234a5d11d35d4a1b7fee9433461499f68a915d9.siva
62c30583534d52a03ac8451023b98a5edb85b29e.siva
62c5c676ffbe966e1130b076c2ab9fab92c81761.siva
6356f0f02c71c12bbe5a60a2d21a4a24a0c715ad.siva
650111dc8c0800e5b7d4c878c1d454657b68efca.siva
650ad87f35f4405a2ca8270d2b2835daa442e5f1.siva
653d60dfb2ab7320ae20a807b86b0b10a7fc99ca.siva
654bafd28ae2b1c2399d56b4adbee22f01a4a234.siva
669cebd58bedff32a613464040ac197cfb472475.siva
679f15811e4ffbde9df880daf012ffcafdddebb1.siva
681a22dba1bc01df8e08d9dd696ef21021d2e86e.siva
69988e50b1d6da6b1c35cfa8c12d8fc33ec65b31.siva
6ada7f885fb658bd11049eac63609777481bc598.siva
6c38e6c065e8a26add3126861e996fbcb03fb60d.siva
6ee52539e83d0452051fe08699b5d8436442f803.siva
6f3002ea9afc5fee0542b7ee2111823089932702.siva
6f45d4770551979fc4de326ed68ce03000ef5ff6.siva
6f6e812afb08cf794a4cb9771bf72b788d27d949.siva
6f9987dbc11f836245de8c29b65770c7d8dfa0c6.siva
6fd786557c637f9fb57fc28c9eaffce87e1a0c5b.siva
70358ca7bb862630c3b52961c702082177ffc46e.siva
710fab00beb7f1ec34da133d882bdf3913adb568.siva
71f1f4224b267711bd9cc372a2ec69902f423a01.siva
7240ff6b1cc93e20d6fca2e8726364d80bd041ee.siva
72cd034ad5a0327471e97d445ea9479c875eb72f.siva
731041cb6a12b6f46626e16a04ab11523fed1c49.siva
74f74bf9d3e3476307e5bae03747ae26704398dd.siva
75bc2768595ef186651096a52ee66229fd5754f5.siva
75bfbf09db6e6537558a653be15994b00943d0fb.siva
77807377146f26f2a204970fd39449107ffba868.siva
77ab9a63ea622bcf5780d715c5abc44f6f1efc2e.siva
784a6b1917e41aa16e7d660922a87def41d1bd48.siva
78cac2a9f1db133b30a7c5cb86e7ef1747692654.siva
7944cf83d12a8347053cfb7a7f13fc83e73705e4.siva
794d8d3118012d9f4988c9fe4df210af864eb6db.siva
794e1ef07eb72352f92b49f8ff63d42adf6f712e.siva
79f00579a96873945f4560144ef83c9d7df3ad88.siva
7a6df3308eb53f54945c30409cfe7411a6995399.siva
7b41a192d7a97ab2c7cc3bdccafce1b2dab54db6.siva
7ba9e7d50114f976282e6ffd52f05fa624eaa0c3.siva
7cfe62cee36dee34fe4d23aed5bdd00c4f42b3d0.siva
7d623455f45435ad351514dfd4abd064cac61aab.siva
7d7e03befa7bf867233ebc214970cdb8267499e3.siva
7dce3b19285e4da5d48da0051ef5bba588832d1d.siva
7ea0acc4846f8f39f1f5063e479b7bf0435ce1d2.siva
7eacef721cb9c7f1c120bff9ff03adb6e5c35b4d.siva
7eff46bf524e61fa093e7f3bdfd732c6c805b69a.siva
7f1879305e041b564b10dc082091a4cd217647d0.siva
7fb7b4bd1e03dfd1a0e9b9a5b1bcef2c95bbcbf0.siva
7fe9e833c012e0984aa0f91fd3751090a4fdafaf.siva
8094e443514a6d66bceba9cbe435bbf08fcb9136.siva
81de939cc88bd30595d98771b1121b77f28babee.siva
82672e171daf41b48c017d7457334c672a09059e.siva
8282fcc1e115ab1a1c48e86614c1ad5ab59c71b2.siva
82d8d02445a0d3674aea2bcfb7cc7a75e943bdc6.siva
830cb004d4421adcb34bb22577846c0fb47d299d.siva
843a07cff7ff594252a5b477ca9d6e3792a4de7c.siva
878ca065045543dee84baeb6837f5f9b7efc9bc6.siva
88635eeeee6ced6eea14491e56198152d4be2d8f.siva
88fb1fedbc53cc9d7b0e679c1273e0bb0ad0ed45.siva
89a0c9b701f85e88b1a1a2d1530205d04b2737b9.siva
8aae46224b163ae62041edc51090057fecb98242.siva
8e0e3da5fae48625e9229e4cc1fa3b350e196451.siva
8e3eb4551091373b631462caf66448dd71ec61ef.siva
8f2424d1289181ed36643aecef007ec0575418b5.siva
8f662a4337c055b78b788b9174b0412edef2b31f.siva
8f891b83b6beadf287f53fc3aa3f8beae22ff1b8.siva
90c46d9b7b9df5473fe271ae0fce33d431fdb798.siva
92f572f8489036f443fbe8a080868353d8718b65.siva
938bba32a4008bdde9c064dda6a0597987ddef54.siva
93b49db483c44711a310b8815c3351f7fb374802.siva
9456dad229d3c2a03af248f38f4e967d962c11f4.siva
9468d4c19626b5a861ec93902c50b2cff002b5c5.siva
94cc58fe987a9f30fdf505512808cd2794e81e54.siva
95051461df395ee68d32aedec2fcb4884246ce56.siva
95bcd74f701bdbd8df3c0b591a186a69ca70f078.siva
960311a075d5644c304410fe109862075206906d.siva
9661f84da9a28c77153f7024863e9967795bc1de.siva
96a608555bdaa2d5daf6c1d401193ba7dcdf3407.siva
976d90f37fc2a87eff4fba93db2db30788f67d83.siva
99b21ff58ad2e2ba83172512d7f513a7c37e50c3.siva
9ad9b952348ec7fc424ab056fe2c776c69cf9741.siva
9b2cd0a099ecdb888b582b8709e6989760b308c0.siva
9c38db000f3b26e737ba52185fc097e4ffe1fadb.siva
9d0080576446de475d34b0dbb58389b15cd4f529.siva
9e481dab6091e0438f571f1eba86555457b91005.siva
9f9630ac871138c04c8654eb46d66ca0a2f70b7c.siva
a07e36e1d1b6c5d948cbf490656f21e165154dc6.siva
a1222afa4e424ebd94480d126f334ff1eccfc325.siva
a22b763c81530acbc218342dbe2eb506967c4086.siva
a336a6e4f65f9b14ce51cb033989f7ba30672bf5.siva
a4465e3e9cc4c30e7f53e0b908f734a42ed32da4.siva
a4fef9c6d0f93d67297244fb2e96552827956a91.siva
a61d1c0eb5670f3f450229a4d49c1bff5b357dfe.siva
a8aa17fcf931b8f4bf22619bc4f87a9044d6e3dc.siva
ab06fbe6bf60be18a740547db957b5c44ae6b786.siva
abe496afe2d56ebfd1e4dd7681d1c4a7853adea3.siva
abeeb4d91a837b65743cf85f8b883bc2fdc6eb29.siva
ac6fbf21f870978484378fb8666837eb1650e5ec.siva
ad8d3d03279b74a3c5485e514f3cbfe4bc4a7713.siva
adb67e1e1f59b1a4ee1945c14f88b9874cc0d854.siva
ae7ccbd46471ad2d5c472d8c5a3a4ac8ee796146.siva
aeab7ecc3aa1aa60927858b04621ca8f933aae4a.siva
af3d0b8dbffac37bb242a28879e82bf6e0ca5f43.siva
b09e03fecf4f0790f4a06ace59c8b30d06620917.siva
b0cb0ccce2c347b406b84a538f06ef05be9a52da.siva
b1ed8502b7a276087cc1d1988895ee6427350085.siva
b2751f890cdece6b4d225ac2e658e0ed986bbecd.siva
b30e1cfc7ee8cd68a14979de9f9509674e40940f.siva
b3b2ea3f7793275d3b042e9bf19563d3b5fc6f68.siva
b4105be21e967f79d819749c44eff6ed4311f65d.siva
b47a462e47a49b9720652c90bea380c49a211c90.siva
b5326a1cd5bbc0d2ed3d3f74b4905b1185f4ff9b.siva
b558997fd9db8406b2a24a1831d06e283dbf35a6.siva
b61865e753688fb5965fd010d969a13673a39930.siva
b7311716519b8ae29d20a5bcd764a432724781a1.siva
b966efa17d837dc62c23d5f8064e184e22b14c2e.siva
ba0ba16dabdef85f628677a41c929b610e916c68.siva
bb3fdd63217cc9f582c46a4c11adc9ef9d84dea9.siva
bcc35f588bd8f7af3a5555b330d5cf50b5e752ef.siva
be01e422c4314e6cc6f470df6573e674784da573.siva
be4b70e646e6232834e9f9917fdc1adde2156f47.siva
bea62a7668e56bb2c5042f7063f4debcee9b4576.siva
bea6fb84bab0d6edef6acca79d4574f0391a5f26.siva
bf0740b3f5e3b5fa1814f9bd49e74b00eaff8def.siva
bf13d9878a670fea003fdcc1ea2d763119de674b.siva
bff3a4cc690f414dc92321800770c7084195da62.siva
c0bfb3b06288094cd82e9aef7b0116e8850e4e9c.siva
c1012878a40f7215c09f87a013aa685ceb104069.siva
c1ed60af98bd5f18ab0a32bf782260ac15954d8e.siva
c2000d7de68ef955a85cf8f5f6e78d4f25c10103.siva
c35d74add6d953a89b6be78b52d013df4c28c070.siva
c36e6770414efed78559185e1d63ada78f2f8a6c.siva
c46a1a491c8c4d69ac0a32d891079276e4b9f1b1.siva
c51d01228964c94f18cadc876c68b2046bb95b9b.siva
c6826f0f598eb974b662562b9966a7ec261b9b8b.siva
c6e27c48a3faeaf649e5801d06038aba173b0c54.siva
c7037656fdeeb46ebb23755f104627910247acef.siva
c86595aa283e57ca13a4c89adeffd7f09e9089e4.siva
c87853d17d759377596750cb905697a0f7034707.siva
ca9b71f294c48ab371b0b3f2d9cf4634caf7bb3e.siva
cb0e3287090786fad566feb67ac07b8ef361b2c3.siva
cbb3c11755b9d1ed773064d4b56a678ba963e4b4.siva
ccc8d434c5a47d1d5130d2cd11d097312c7e768b.siva
cd3f7de97bb2e07b54b776adf9da11c0db185a60.siva
cd79a570662117ebcb859df429f8f6b6a55c4608.siva
cf89a75eb8a6eb780cf6adfac4e3537843386777.siva
d0d6d182df526b43e135110d7964c2628fd95381.siva
d1ddf4804bd92674e425ba1560b74016246ac88c.siva
d3a20b3369b8438afd87b734758b9727cb182a2a.siva
d43a104503c6c186eb599524426453bab836f222.siva
d4a9b6215cead0e523bdd196d074f68d15a05ad3.siva
d4e323247e7b200f1b8bbc50979c31416525964f.siva
d55e4d53cccc9123d03f45c53441e7cbfc58e515.siva
d5c02164a6cbacee42d156253996ccbfc2687222.siva
d5edf20cee559050b70dd095bf243287bf2c2248.siva
d63fff30437e7e51ab643622651873c020eaec4e.siva
d813adedb7ed0b1ba2a1d2db568aacde261992d2.siva
d8d224a97c4611cab16cae33faf147a20419ff4d.siva
d917a9cb60c0d7f57ea80046359bc1456b058cee.siva
d9a61b1a6979ebbba2e5b999f3c434e1ff133177.siva
d9c6e7bc3cd6e2f20d33f60c91b4909fe95d0215.siva
d9fa05499b632bf6cdc8c0f94c47e8d4347b799f.siva
dc56b3f92fefcb00b7256b2a973c190d7c2d47c7.siva
dd5db41a55cd49f387dc9c5351ec8923b3cb1a19.siva
dd8d248c5a4b0f9c1684ca32c6551de50b778146.siva
ddd95b1d755bb24a84037837f1460b6bd7cb5067.siva
de36d1276f1da297920e4b9b5f594cd026109162.siva
de512348ac20d4334411dae691dd54bc747b8519.siva
e01d8f30c604bae1f499e4cb4d7a4c6b4c69d94c.siva
e256f9e622ffbf89e3b3694dc6bc813b7461c509.siva
e38a99f12cc4c0e7db99f675b4540ee9575c03d6.siva
e3fad681c5cdf88c002016ffd7a069c5f320b5bf.siva
e51b42b3360d39b9cfe6247799905404aeed6735.siva
e65d0da3d4db0cc71080e1b761f16bee183e91d0.siva
e7615cbc6b4af5985c4e0d4848a426e2d35f79c3.siva
e78bb438054338bb91ec5c79f87086ee037feebf.siva
e8d9ba406ab9e0ec5dd63a84ee805d67efc75632.siva
e8f29806033861f196cbc679788b985f4b40c4b7.siva
e907c422aab050d545983142194a9705fd17502f.siva
e90efc7c3ed67fd17413811a5d320ef489d74464.siva
e98b4029ee5d09f5261fa1961cb0e3129a7eb8cf.siva
ea43d2e40e59cb0240ea362fe30f9a435a4a2a1f.siva
ea7d02f9eb744ddc9cf85d957afa018243bc4293.siva
eb413b348902be82efc20c1e347e96c1b9e4a7fd.siva
ee442a7d98df9d43f3568e34bdf41b680109badb.siva
ef0c0cf7f8b4b16785e634c709ea29675c0eca5c.siva
ef84d3492412addbde0ee402af4e4df85268c221.siva
f16e88a9c0863bdd9c4c0d5e1448d4749998832e.siva
f29e387ef8a686d7ef99c674581b2100239e77e1.siva
f30272394d29a68d8f4f25355bd6b95e695fc81e.siva
f317cd8bb256411d802c754f5de5a56132374bfd.siva
f37c1a3fb0221be8bb1546b43d2baf0126d83fa5.siva
f382b5ffc445e45a110734f5396728da7914aeb6.siva
f52e3e297cb0daf513a9f548e603302e5b601770.siva
f6d85482031f14487b7f505eeecebef7a332ebc3.siva
f76c34b729e670e4185efd71f2d4af2448902f8b.siva
f7a8a237e6cb007d3ed6519a8f00a91edbab789b.siva
f9ab55e9dae56843701acf8b79b9253a54337903.siva
fa5a87982e32fc28cc6a5e667bc7ae965470058c.siva
fa5cb215933aaf872e402dfa7ddc708ce9053e6f.siva
fbe7952cce11e8073378b063bdae7ab277a96eb8.siva
fdeea359c6ca00033946a08a8293d13fbb076e9b.siva
feaa836a7fb62ea260690e351ea3990500d56e15.siva
ff3b47ce226e5e5972151b37fb64bb9e1ea6c632.siva
ff4e22fcec6d365bd1fbf8fc1fcf9d07aecf35fd.siva

[bzz]

Thank you for detained report on experiment! Very interesting.

Do you think it might be worth trying the same, but without .siva copy/unpack in spirit of #36 and check if that makes difference in performance?

[marnovo]

We should compare its performance vs Berserker baseline. I'm gathering more info with @eiso and @vmarkovtsev on what we should be benchmarking next.

[bzz]

Here is 100 siva files that we can test on https://drive.google.com/open?id=0BxNVgwtOUkMUaG5wbmpnRklmMEk

[marnovo]

Superseded by https://github.com/src-d/backlog/issues/1090.

[marnovo]

@bzz please close it.