miku / esbulk

Bulk indexing command line tool for elasticsearch.

esbulk

Fast parallel command line bulk loading utility for elasticsearch. Data is read from a newline delimited JSON file or stdin and indexed into elasticsearch in bulk and in parallel. The shortest command would be:

$ esbulk -index my-index-name < file.ldj

Caveat: If indexing pressure on the bulk API is too high (dozens or hundreds of parallel workers, large batch sizes, depending on you setup), esbulk will halt and report an error:

$ esbulk -index my-index-name -w 100 file.ldj
2017/01/02 16:25:25 error during bulk operation, try less workers (lower -w value) or
                    increase thread_pool.bulk.queue_size in your nodes

Please note that, in such a case, some documents are indexed and some are not. Your index will be in an inconsistent state, since there is no transactional bracket around the indexing process.

However, using defaults (parallelism: number of cores) on a single node setup will just work. For larger clusters, increase the number of workers until you see full CPU utilization. After that, more workers won't buy any more speed.

Currently, esbulk is tested against elasticsearch versions 2, 5, 6, 7 and 8 using testcontainers. Originally written for Leipzig University Library, project finc.

Installation

$ go install github.com/miku/esbulk/cmd/esbulk@latest

For deb or rpm packages, see: https://github.com/miku/esbulk/releases

Usage

$ esbulk -h
Usage of esbulk:
  -0    set the number of replicas to 0 during indexing
  -c string
        create index mappings, settings, aliases, https://is.gd/3zszeu
  -cpuprofile string
        write cpu profile to file
  -id string
        name of field to use as id field, by default ids are autogenerated
  -index string
        index name
  -mapping string
        mapping string or filename to apply before indexing
  -memprofile string
        write heap profile to file
  -optype string
        optype (index - will replace existing data,
                create - will only create a new doc,
                update - create new or update existing data)
        (default "index")
  -p string
        pipeline to use to preprocess documents
  -purge
        purge any existing index before indexing
  -purge-pause duration
        pause after purge (default 1s)
  -r string
        Refresh interval after import (default "1s")
  -server value
        elasticsearch server, this works with https as well
  -size int
        bulk batch size (default 1000)
  -skipbroken
        skip broken json
  -type string
        elasticsearch doc type (deprecated since ES7)
  -u string
        http basic auth username:password, like curl -u
  -v    prints current program version
  -verbose
        output basic progress
  -w int
        number of workers to use (default 8)
  -z    unzip gz'd file on the fly

To index a JSON file, that contains one document per line, just run:

$ esbulk -index example file.ldj

Where file.ldj is line delimited JSON, like:

{"name": "esbulk", "version": "0.2.4"}
{"name": "estab", "version": "0.1.3"}
...

By default esbulk will use as many parallel workers, as there are cores. To tweak the indexing process, adjust the -size and -w parameters.

You can index from gzipped files as well, using the -z flag:

$ esbulk -z -index example file.ldj.gz

Starting with 0.3.7 the preferred method to set a non-default server hostport is via -server, e.g.

$ esbulk -server https://0.0.0.0:9201

This way, you can use https as well, which was not possible before. Options -host and -port are gone as of esbulk 0.5.0.

Reusing IDs

Since version 0.3.8: If you want to reuse IDs from your documents in elasticsearch, you can specify the ID field via -id flag:

$ cat file.json
{"x": "doc-1", "db": "mysql"}
{"x": "doc-2", "db": "mongo"}

Here, we would like to reuse the ID from field x.

$ esbulk -id x -index throwaway -verbose file.json
...

$ curl -s http://localhost:9200/throwaway/_search | jq
{
  "took": 2,
  "timed_out": false,
  "_shards": {
    "total": 5,
    "successful": 5,
    "failed": 0
  },
  "hits": {
    "total": 2,
    "max_score": 1,
    "hits": [
      {
        "_index": "throwaway",
        "_type": "default",
        "_id": "doc-2",
        "_score": 1,
        "_source": {
          "x": "doc-2",
          "db": "mongo"
        }
      },
      {
        "_index": "throwaway",
        "_type": "default",
        "_id": "doc-1",
        "_score": 1,
        "_source": {
          "x": "doc-1",
          "db": "mysql"
        }
      }
    ]
  }
}

Nested ID fields

Version 0.4.3 adds support for nested ID fields:

$ cat fixtures/pr-8-1.json
{"a": {"b": 1}}
{"a": {"b": 2}}
{"a": {"b": 3}}

$ esbulk -index throwaway -id a.b < fixtures/pr-8-1.json
...

Concatenated ID

Version 0.4.3 adds support for IDs that are the concatenation of multiple fields:

$ cat fixtures/pr-8-2.json
{"a": {"b": 1}, "c": "a"}
{"a": {"b": 2}, "c": "b"}
{"a": {"b": 3}, "c": "c"}

$ esbulk -index throwaway -id a.b,c < fixtures/pr-8-1.json
...

      {
        "_index": "xxx",
        "_type": "default",
        "_id": "1a",
        "_score": 1,
        "_source": {
          "a": {
            "b": 1
          },
          "c": "a"
        }
      },

Using X-Pack

Since 0.4.2: support for secured elasticsearch nodes:

$ esbulk -u elastic:changeme -index myindex file.ldj

---

A similar project has been started for solr, called solrbulk.

Contributors

• klaubert
• sakshambathla
• mumoshu
• albertpastrana
• faultlin3
• gransy
• Christoph Kepper
• Christian Solomon
• Mikael Byström

and others.

Measurements

$ csvlook -I measurements.csv
| es    | esbulk | docs      | avg_b | nodes | cores | total_heap_gb | t_s   | docs_per_s | repl |
|-------|--------|-----------|-------|-------|-------|---------------|-------|------------|------|
| 6.1.2 | 0.4.8  | 138000000 | 2000  | 1     | 32    |  64           |  6420 |  22100     | 1    |
| 6.1.2 | 0.4.8  | 138000000 | 2000  | 1     |  8    |  30           | 27360 |   5100     | 1    |
| 6.1.2 | 0.4.8  |   1000000 | 2000  | 1     |  4    |   1           |   300 |   3300     | 1    |
| 6.1.2 | 0.4.8  |  10000000 |   26  | 1     |  4    |   8           |   122 |  81000     | 1    |
| 6.1.2 | 0.4.8  |  10000000 |   26  | 1     | 32    |  64           |    32 | 307000     | 1    |
| 6.2.3 | 0.4.10 | 142944530 | 2000  | 2     | 64    | 128           | 26253 |   5444     | 1    |
| 6.2.3 | 0.4.10 | 142944530 | 2000  | 2     | 64    | 128           | 11113 |  12831     | 0    |
| 6.2.3 | 0.4.13 |  15000000 | 6000  | 2     | 64    | 128           |  2460 |   6400     | 0    |

Why not add a row?

esbulk's Issues

Fail, if mapping cannot be applied

Currently, a failed mapping will be ignored.

More helpful error messages

Two issues:

• Parsing error responses seems to be broken. We need to support various ES versions and their errors.
• Make error messages more useful and nuanced.

Authentication problem

I secured my elastic (v5.0.1) with X-PACK and now it's not clear how to do the import...
Any thoughts?

json_parse_exception

Hi, I am trying to index some data using esbulk. But it is always throwing the error:

My command is
esbulk -server http://10.9.9.8:9200 -index epfo_ind -id name,memberOf.occurrence.identifier.transactionNumber -verbose -type epfo_typ -w 4 -size 1000 2017_09_13_10_20_01_epfo.json

indexing failed with 500 Internal Server Error: {"error":{"root_cause":[{"type":"json_parse_exception","reason":"Unrecognized character escape 'L' (code 76)\n at [Source: org.elasticsearch.transport.netty4.ByteBufStreamInput@7240ed3b; line: 1, column: 72]"}],"type":"json_parse_exception","reason":"Unrecognized character escape 'L' (code 76)\n at [Source: org.elasticsearch.transport.netty4.ByteBufStreamInput@7240ed3b; line: 1, column: 72]"},"status":500}

I thought there might be a problem in JSON file format. So I check by indexing data one by one using curl command. By this, all data got indexed but the same data when I am trying to index through esbulk it is always showing the above error.

I checked with another bulk submission https://github.com/xros/jsonpyes. With this data got indexed.

Why it's giving the error with esbulk. How can I resolve this?

Thanks,

Support for parent id

Hi,

Is it possible to bulk index child documents? E.g. the parent_id can be a part of the events to be submitted and "chosen" via command-line parameter, similar to id.

-parent-id string
          name of field to use as parent id field

Thanks,
Ivan

Provide docker image

Hey @miku, first of all, thanks a lot for your project, it's great and it's been proving really helpful to us :).

We were thinking it would be great if esbulk could be package inside a docker image as an easy way of "installing" and distributing it.

We've created a repo with a simple Dockerfile:
https://github.com/intenthq/esbulk-docker

And we've automated the build and published it as well:
https://hub.docker.com/r/intenthq/esbulk-docker/

If you want to include this in your repo (either the Dockerfile) or just a link to docker hub feel free to do it or ask for any help :)

AWS ElasticSearch bulk uploading

It'd be great if we could pre-sign requests to allow esbulk to be used easily with AWS ElasticSearch.

indexing failed with 504 Gateway Timeout

esbulk fails with the following error while indexing a file with around 850k JSON documents:

08:19:47.337 2018/07/06 07:01:38 indexing failed with 504 Gateway Timeout:

In this case, it could make sense to retry the HTTP request with a backoff until elasticsearch returns a 200 response. I noticed that currently the HTTP requests are not retried: https://github.com/miku/esbulk/blob/master/indexing.go#L174

Would it make sense to retry the HTTP requests N times with an exponential back-off?

deb for 0.3.9 says it's 0.3.8

Ran into this.

ubuntu@stage-rails:~$ sudo dpkg -i esbulk_0.3.9_amd64.deb
(Reading database ... 401180 files and directories currently installed.)
Preparing to unpack esbulk_0.3.9_amd64.deb ...
Unpacking esbulk (0.3.9) over (0.3.9) ...
Setting up esbulk (0.3.9) ...
ubuntu@stage-rails:~$ esbulk -v
0.3.8

Is it possible to identify certain field as document id

I'm using Elasticsearch to store data from MySql. I wanted to keep row ID in MySql and ES's Document ID, identical. Is it possible to do so with ESBulk?

support OpType

Hi,

please add support for Operation type: create / index - now you have fixed index (indexing.go:99).

With index - the data under same _id was replaced by new data
With create - the doc will not create if exists under same _id

(https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-bulk.html)

Its perfect if you dont want duplicity and also dont want replace current data in ES.

Jan

indexing failed with 400 Bad Request [resolved, issue with ES cluster]

Hi,

I am running into the following error when attempting to index using the following command:

esbulk -index osm_test -server https://vpc-xxx.awszone.es.amazonaws.com -u elastic:password -type _doc -0 -verbose -size 1000 -w 1 -id osm_id osm_data.jsonl
...
2021/02/11 21:52:30 [worker-0] @5227000
2021/02/11 21:52:30 message content-length will be 881947
2021/02/11 21:52:31 [worker-0] @5228000
2021/02/11 21:52:31 message content-length will be 884074
2021/02/11 21:52:31 indexing failed with 400 Bad Request: <html>
<head><title>400 Bad Request</title></head>
<body>
<center><h1>400 Bad Request</h1></center>
</body>
</html>

The data is open street map like data formatted in jsonl, and I run into the error after over 5M documents ingested, and I don't see any particular errors on the server side (OpenDistro 1.12 on AWS)

Any ideas on how to actually see the HTTP request in question that generates this error? I am assuming it is particular document, but no particular line number is provided in the error message.

Thank you in advance!

esbulk should not fail, if index already exists

This seems to be an issue on earlier elasticsearch version (like 1.4).

Check, if compressed content saves bandwidth or indexing time.

A 1Gb NIC is easily saturated with bulk requests. We might be able to save time and bandwidth through compression.

• https://www.elastic.co/guide/en/elasticsearch/reference/current/modules-http.html
• https://gist.github.com/bryfry/09a650eb8aac0fb76c24
• http://jijeshmohan.github.io/post/compress-http-request-in-go/

Add man page.

feature request: routing by _id

This elasticsearch blog post implies that doing batch indexing of documents all going to the same shard at a time improves performance: https://www.elastic.co/blog/how-kenna-security-speeds-up-elasticsearch-indexing-at-scale-part-1

The feature request for esbulk would be to somehow automate this speed-up, without users needing to re-sort or partition documents themselves. Some unstructured thoughts about this:

• probably control by a CLI flag. could esbulk fetch mapping info from the cluster, eg number of shareds?
• could require _routing field in documents, or fall back to _id or a key field if set
• esbulk could partition documents to the existing worker threads. I think this might "just work" even if the number of worker threads is not equal to the number of index shards, but it would probably work better if batches were all a single shard at a time
• or, esbulk could store per-shard caches internally, then when any individual shard cache reaches the bulk document size, send that batch to a worker thread. this would increase memory consumption, particularly with large documents and large number of shards, but that might be fine

Installing by "go install" not working because of replace in .mod

Hi,

I tried to install esbulk like documented in the readme:

go install github.com/miku/esbulk/cmd/esbulk@latest

that is not anymore because of the replace directive in the .mod file that was added two month ago.

The go.mod file for the module providing named packages contains one or  
        more replace directives. It must not contain directives that would cause 
        it to be interpreted differently than if it were the main module.

Docker is not possible for me. Is there another way to install the Tool? (new to go and its environment ;) )

migrate to official go client

• https://github.com/elastic/go-elasticsearch/tree/master/_examples/bulk

`go install` does not work because of mitchellh/osext

I'm trying to build an AMI that includes esbulk. However, it no longer can grab mitchellh/osext:

GOPATH=/tmp GOCACHE=/tmp GOBIN=/usr/local/sbin HOME=/root go install github.com/miku/esbulk/cmd/[email protected]


==> amazon-ebs: + GOPATH=/tmp
==> amazon-ebs: + GOCACHE=/tmp
==> amazon-ebs: + GOBIN=/usr/local/sbin
==> amazon-ebs: + HOME=/root
==> amazon-ebs: + go install github.com/miku/esbulk/cmd/[email protected]
==> amazon-ebs: go: downloading github.com/miku/esbulk v0.7.15
==> amazon-ebs: go: github.com/miku/esbulk/cmd/[email protected]: github.com/miku/[email protected] requires
==> amazon-ebs:         github.com/containerd/[email protected] requires
==> amazon-ebs:         github.com/Microsoft/[email protected] requires
==> amazon-ebs:         github.com/open-policy-agent/[email protected] requires
==> amazon-ebs:         oras.land/[email protected] requires
==> amazon-ebs:         github.com/distribution/distribution/[email protected] requires
==> amazon-ebs:         github.com/mitchellh/[email protected]: invalid version: git ls-remote -q origin in /tmp/pkg/mod/cache/vcs/94ed57c5b21c953d93b47487113db43a5c9b69fd990329ec70dc77348c4dd443: exit status 128:
==> amazon-ebs:         fatal: could not read Username for 'https://github.com': terminal prompts disabled
==> amazon-ebs: Confirm the import path was entered correctly.
==> amazon-ebs: If this is a private repository, see https://golang.org/doc/faq#git_https for additional information.

As far as I can tell, that repository has been made private, or deleted.

ES 6.1t

After upgrade ES to 6.1, the esbulk works incorrectly for me.

I start command with processes by count of cores, and with verbose i see how all processes doing bulk insert .. in few time a lot of processes ended without any error (i try put some logging) and thee is keep just one or two who fill finish insert. After last inserted data, the program is not finished and keep on this unknown state without any logging.

In total in ES is inserted total_docs-(10000*(processes-1)). In 24 processes i lost about 230.000 records.

Index failing with `connection reset by peer`

I twice attempted to import over 140 million documents into a local, single-node ES 6.8 cluster using a command like the following:

zcat /srv/fatcat/snapshots/release_export_expanded.json.gz |  pv -l | parallel -j20 --linebuffer --round-robin --pipe ./fatcat_transform.py elasticsearch-releases - - | esbulk -verbose -size 10000 -id ident -w 6 -index qa_release_v03b -type release

This is with esbulk 0.5.1. I will retry with the latest 0.6.0.

The index almost completed, but after more than 100m documents, failed with an error like:

2020/01/31 11:49:40 Post http://localhost:9200/_bulk: net/http: HTTP/1.x transport connection broken: write tcp [::1]:56970->[::1]:9200: write: connection reset by peer                                                                      
Warning: unable to close filehandle properly: Broken pipe during global destruction

(the "Warning" part might be one of the other pipeline commands)

I suspect this is actually a problem on the Elasticsearch side... maybe something like a GC pause? I looked in ES logs and see that there were garbage collects up until the time of failure, and none after, but no particularly large or noticeable GC right around the failure.

I would expect the esbulk HTTP retries to resolve any such issues; I assume in this case all the retries failed. Perhaps longer, more, or exponential back-offs would help. Unfortunately, I suspect that this failure may be difficult to reproduce reliably, as it has only occurred with these very large imports.

esbulk has been really useful, thank you for making it available and any maintenance time you can spare!

-host option broken

the -host option is broken in the actual version (0.4.14). esbulk always tries to use localhost instead. workaround: use -server option...

if -host is really deprecated and shouldn't be used anymore, why not remove it?

Appended two system call traces. first -host/-port and second, -server.

as you can see on the (bold) connect()-calls, when using -host/-port, esbulk tries to reach localhost (127.0.0.1).

user@host:~$ strace esbulk -host 192.168.0.1 -port 9200 -index bla
execve("/usr/sbin/esbulk", ["esbulk", "-host", "192.168.0.1", "-port", "9200", "-index", "bla", "-w", "1"], [/* 27 vars /]) = 0
brk(NULL) = 0x1f80000
access("/etc/ld.so.nohwcap", F_OK) = -1 ENOENT (No such file or directory)
access("/etc/ld.so.preload", R_OK) = -1 ENOENT (No such file or directory)
open("/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 3
fstat(3, {st_mode=S_IFREG|0644, st_size=39302, ...}) = 0
mmap(NULL, 39302, PROT_READ, MAP_PRIVATE, 3, 0) = 0x7f311581b000
close(3) = 0
access("/etc/ld.so.nohwcap", F_OK) = -1 ENOENT (No such file or directory)
open("/lib/x86_64-linux-gnu/libpthread.so.0", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\260`\0\0\0\0\0\0"..., 832) = 832
fstat(3, {st_mode=S_IFREG|0755, st_size=138696, ...}) = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f311581a000
mmap(NULL, 2212904, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7f31153e3000
mprotect(0x7f31153fb000, 2093056, PROT_NONE) = 0
mmap(0x7f31155fa000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x17000) = 0x7f31155fa000
mmap(0x7f31155fc000, 13352, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7f31155fc000
close(3) = 0
access("/etc/ld.so.nohwcap", F_OK) = -1 ENOENT (No such file or directory)
open("/lib/x86_64-linux-gnu/libc.so.6", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\3\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0P\t\2\0\0\0\0\0"..., 832) = 832
fstat(3, {st_mode=S_IFREG|0755, st_size=1868984, ...}) = 0
mmap(NULL, 3971488, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7f3115019000
mprotect(0x7f31151d9000, 2097152, PROT_NONE) = 0
mmap(0x7f31153d9000, 24576, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x1c0000) = 0x7f31153d9000
mmap(0x7f31153df000, 14752, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7f31153df000
close(3) = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f3115819000
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f3115818000
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f3115817000
arch_prctl(ARCH_SET_FS, 0x7f3115818700) = 0
mprotect(0x7f31153d9000, 16384, PROT_READ) = 0
mprotect(0x7f31155fa000, 4096, PROT_READ) = 0
mprotect(0x7f3115825000, 4096, PROT_READ) = 0
munmap(0x7f311581b000, 39302) = 0
set_tid_address(0x7f31158189d0) = 69326
set_robust_list(0x7f31158189e0, 24) = 0
rt_sigaction(SIGRTMIN, {0x7f31153e8b50, [], SA_RESTORER|SA_SIGINFO, 0x7f31153f4390}, NULL, 8) = 0
rt_sigaction(SIGRT_1, {0x7f31153e8be0, [], SA_RESTORER|SA_RESTART|SA_SIGINFO, 0x7f31153f4390}, NULL, 8) = 0
rt_sigprocmask(SIG_UNBLOCK, [RTMIN RT_1], NULL, 8) = 0
getrlimit(RLIMIT_STACK, {rlim_cur=81921024, rlim_max=RLIM64_INFINITY}) = 0
brk(NULL) = 0x1f80000
brk(0x1fa1000) = 0x1fa1000
sched_getaffinity(0, 8192, [ffffffffffffffff, ffffffffffffffff, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, ...]) = 64
mmap(0xc000000000, 65536, PROT_NONE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xc000000000
munmap(0xc000000000, 65536) = 0
mmap(NULL, 262144, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f31157d7000
mmap(0xc420000000, 1048576, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xc420000000
mmap(0xc41fff8000, 32768, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xc41fff8000
mmap(0xc000000000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xc000000000
mmap(NULL, 65536, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f31157c7000
mmap(NULL, 65536, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f31157b7000
rt_sigprocmask(SIG_SETMASK, NULL, [], 8) = 0
mmap(NULL, 262144, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f3115777000
sigaltstack(NULL, {ss_sp=NULL, ss_flags=SS_DISABLE, ss_size=0}) = 0
sigaltstack({ss_sp=0xc420002000, ss_flags=0, ss_size=32768}, NULL) = 0
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
gettid() = 69326
rt_sigaction(SIGHUP, NULL, {SIG_DFL, [], 0}, 8) = 0
rt_sigaction(SIGHUP, {0x45b610, ~[], SA_RESTORER|SA_STACK|SA_RESTART|SA_SIGINFO, 0x7f31153f4390}, NULL, 8) = 0
rt_sigaction(SIGINT, NULL, {SIG_DFL, [], 0}, 8) = 0
rt_sigaction(SIGINT, {0x45b610, ~[], SA_RESTORER|SA_STACK|SA_RESTART|SA_SIGINFO, 0x7f31153f4390}, NULL, 8) = 0
rt_sigaction(SIGQUIT, NULL, {SIG_DFL, [], 0}, 8) = 0
rt_sigaction(SIGQUIT, {0x45b610, ~[], SA_RESTORER|SA_STACK|SA_RESTART|SA_SIGINFO, 0x7f31153f4390}, NULL, 8) = 0
rt_sigaction(SIGILL, NULL, {SIG_DFL, [], 0}, 8) = 0
rt_sigaction(SIGILL, {0x45b610, ~[], SA_RESTORER|SA_STACK|SA_RESTART|SA_SIGINFO, 0x7f31153f4390}, NULL, 8) = 0
rt_sigaction(SIGTRAP, NULL, {SIG_DFL, [], 0}, 8) = 0
rt_sigaction(SIGTRAP, {0x45b610, ~[], SA_RESTORER|SA_STACK|SA_RESTART|SA_SIGINFO, 0x7f31153f4390}, NULL, 8) = 0
rt_sigaction(SIGABRT, NULL, {SIG_DFL, [], 0}, 8) = 0
rt_sigaction(SIGABRT, {0x45b610, ~[], SA_RESTORER|SA_STACK|SA_RESTART|SA_SIGINFO, 0x7f31153f4390}, NULL, 8) = 0
rt_sigaction(SIGBUS, NULL, {SIG_DFL, [], 0}, 8) = 0
rt_sigaction(SIGBUS, {0x45b610, ~[], SA_RESTORER|SA_STACK|SA_RESTART|SA_SIGINFO, 0x7f31153f4390}, NULL, 8) = 0
rt_sigaction(SIGFPE, NULL, {SIG_DFL, [], 0}, 8) = 0
rt_sigaction(SIGFPE, {0x45b610, ~[], SA_RESTORER|SA_STACK|SA_RESTART|SA_SIGINFO, 0x7f31153f4390}, NULL, 8) = 0
rt_sigaction(SIGUSR1, NULL, {SIG_DFL, [], 0}, 8) = 0
rt_sigaction(SIGUSR1, {0x45b610, ~[], SA_RESTORER|SA_STACK|SA_RESTART|SA_SIGINFO, 0x7f31153f4390}, NULL, 8) = 0
rt_sigaction(SIGSEGV, NULL, {SIG_DFL, [], 0}, 8) = 0
rt_sigaction(SIGSEGV, {0x45b610, ~[], SA_RESTORER|SA_STACK|SA_RESTART|SA_SIGINFO, 0x7f31153f4390}, NULL, 8) = 0
rt_sigaction(SIGUSR2, NULL, {SIG_DFL, [], 0}, 8) = 0
rt_sigaction(SIGUSR2, {0x45b610, ~[], SA_RESTORER|SA_STACK|SA_RESTART|SA_SIGINFO, 0x7f31153f4390}, NULL, 8) = 0
rt_sigaction(SIGPIPE, NULL, {SIG_DFL, [], 0}, 8) = 0
rt_sigaction(SIGPIPE, {0x45b610, ~[], SA_RESTORER|SA_STACK|SA_RESTART|SA_SIGINFO, 0x7f31153f4390}, NULL, 8) = 0
rt_sigaction(SIGALRM, NULL, {SIG_DFL, [], 0}, 8) = 0
rt_sigaction(SIGALRM, {0x45b610, ~[], SA_RESTORER|SA_STACK|SA_RESTART|SA_SIGINFO, 0x7f31153f4390}, NULL, 8) = 0
rt_sigaction(SIGTERM, NULL, {SIG_DFL, [], 0}, 8) = 0
rt_sigaction(SIGTERM, {0x45b610, ~[], SA_RESTORER|SA_STACK|SA_RESTART|SA_SIGINFO, 0x7f31153f4390}, NULL, 8) = 0
rt_sigaction(SIGSTKFLT, NULL, {SIG_DFL, [], 0}, 8) = 0
rt_sigaction(SIGSTKFLT, {0x45b610, ~[], SA_RESTORER|SA_STACK|SA_RESTART|SA_SIGINFO, 0x7f31153f4390}, NULL, 8) = 0
rt_sigaction(SIGCHLD, NULL, {SIG_DFL, [], 0}, 8) = 0
rt_sigaction(SIGCHLD, {0x45b610, ~[], SA_RESTORER|SA_STACK|SA_RESTART|SA_SIGINFO, 0x7f31153f4390}, NULL, 8) = 0
rt_sigaction(SIGURG, NULL, {SIG_DFL, [], 0}, 8) = 0
rt_sigaction(SIGURG, {0x45b610, ~[], SA_RESTORER|SA_STACK|SA_RESTART|SA_SIGINFO, 0x7f31153f4390}, NULL, 8) = 0
rt_sigaction(SIGXCPU, NULL, {SIG_DFL, [], 0}, 8) = 0
rt_sigaction(SIGXCPU, {0x45b610, ~[], SA_RESTORER|SA_STACK|SA_RESTART|SA_SIGINFO, 0x7f31153f4390}, NULL, 8) = 0
rt_sigaction(SIGXFSZ, NULL, {SIG_DFL, [], 0}, 8) = 0
rt_sigaction(SIGXFSZ, {0x45b610, ~[], SA_RESTORER|SA_STACK|SA_RESTART|SA_SIGINFO, 0x7f31153f4390}, NULL, 8) = 0
rt_sigaction(SIGVTALRM, NULL, {SIG_DFL, [], 0}, 8) = 0
rt_sigaction(SIGVTALRM, {0x45b610, ~[], SA_RESTORER|SA_STACK|SA_RESTART|SA_SIGINFO, 0x7f31153f4390}, NULL, 8) = 0
rt_sigaction(SIGPROF, NULL, {SIG_DFL, [], 0}, 8) = 0
rt_sigaction(SIGPROF, {0x45b610, ~[], SA_RESTORER|SA_STACK|SA_RESTART|SA_SIGINFO, 0x7f31153f4390}, NULL, 8) = 0
rt_sigaction(SIGWINCH, NULL, {SIG_DFL, [], 0}, 8) = 0
rt_sigaction(SIGWINCH, {0x45b610, ~[], SA_RESTORER|SA_STACK|SA_RESTART|SA_SIGINFO, 0x7f31153f4390}, NULL, 8) = 0
rt_sigaction(SIGIO, NULL, {SIG_DFL, [], 0}, 8) = 0
rt_sigaction(SIGIO, {0x45b610, ~[], SA_RESTORER|SA_STACK|SA_RESTART|SA_SIGINFO, 0x7f31153f4390}, NULL, 8) = 0
rt_sigaction(SIGPWR, NULL, {SIG_DFL, [], 0}, 8) = 0
rt_sigaction(SIGPWR, {0x45b610, ~[], SA_RESTORER|SA_STACK|SA_RESTART|SA_SIGINFO, 0x7f31153f4390}, NULL, 8) = 0
rt_sigaction(SIGSYS, NULL, {SIG_DFL, [], 0}, 8) = 0
rt_sigaction(SIGSYS, {0x45b610, ~[], SA_RESTORER|SA_STACK|SA_RESTART|SA_SIGINFO, 0x7f31153f4390}, NULL, 8) = 0
rt_sigaction(SIGRTMIN, NULL, {0x7f31153e8b50, [], SA_RESTORER|SA_SIGINFO, 0x7f31153f4390}, 8) = 0
rt_sigaction(SIGRTMIN, NULL, {0x7f31153e8b50, [], SA_RESTORER|SA_SIGINFO, 0x7f31153f4390}, 8) = 0
rt_sigaction(SIGRTMIN, {0x7f31153e8b50, [], SA_RESTORER|SA_STACK|SA_SIGINFO, 0x7f31153f4390}, NULL, 8) = 0
rt_sigaction(SIGRT_1, NULL, {0x7f31153e8be0, [], SA_RESTORER|SA_RESTART|SA_SIGINFO, 0x7f31153f4390}, 8) = 0
rt_sigaction(SIGRT_1, NULL, {0x7f31153e8be0, [], SA_RESTORER|SA_RESTART|SA_SIGINFO, 0x7f31153f4390}, 8) = 0
rt_sigaction(SIGRT_1, {0x7f31153e8be0, [], SA_RESTORER|SA_STACK|SA_RESTART|SA_SIGINFO, 0x7f31153f4390}, NULL, 8) = 0
rt_sigaction(SIGRT_2, NULL, {SIG_DFL, [], 0}, 8) = 0
rt_sigaction(SIGRT_2, {0x45b610, ~[], SA_RESTORER|SA_STACK|SA_RESTART|SA_SIGINFO, 0x7f31153f4390}, NULL, 8) = 0
rt_sigaction(SIGRT_3, NULL, {SIG_DFL, [], 0}, 8) = 0
rt_sigaction(SIGRT_3, {0x45b610, ~[], SA_RESTORER|SA_STACK|SA_RESTART|SA_SIGINFO, 0x7f31153f4390}, NULL, 8) = 0
rt_sigaction(SIGRT_4, NULL, {SIG_DFL, [], 0}, 8) = 0
rt_sigaction(SIGRT_4, {0x45b610, ~[], SA_RESTORER|SA_STACK|SA_RESTART|SA_SIGINFO, 0x7f31153f4390}, NULL, 8) = 0
rt_sigaction(SIGRT_5, NULL, {SIG_DFL, [], 0}, 8) = 0
rt_sigaction(SIGRT_5, {0x45b610, ~[], SA_RESTORER|SA_STACK|SA_RESTART|SA_SIGINFO, 0x7f31153f4390}, NULL, 8) = 0
rt_sigaction(SIGRT_6, NULL, {SIG_DFL, [], 0}, 8) = 0
rt_sigaction(SIGRT_6, {0x45b610, ~[], SA_RESTORER|SA_STACK|SA_RESTART|SA_SIGINFO, 0x7f31153f4390}, NULL, 8) = 0
rt_sigaction(SIGRT_7, NULL, {SIG_DFL, [], 0}, 8) = 0
rt_sigaction(SIGRT_7, {0x45b610, ~[], SA_RESTORER|SA_STACK|SA_RESTART|SA_SIGINFO, 0x7f31153f4390}, NULL, 8) = 0
rt_sigaction(SIGRT_8, NULL, {SIG_DFL, [], 0}, 8) = 0
rt_sigaction(SIGRT_8, {0x45b610, ~[], SA_RESTORER|SA_STACK|SA_RESTART|SA_SIGINFO, 0x7f31153f4390}, NULL, 8) = 0
rt_sigaction(SIGRT_9, NULL, {SIG_DFL, [], 0}, 8) = 0
rt_sigaction(SIGRT_9, {0x45b610, ~[], SA_RESTORER|SA_STACK|SA_RESTART|SA_SIGINFO, 0x7f31153f4390}, NULL, 8) = 0
rt_sigaction(SIGRT_10, NULL, {SIG_DFL, [], 0}, 8) = 0
rt_sigaction(SIGRT_10, {0x45b610, ~[], SA_RESTORER|SA_STACK|SA_RESTART|SA_SIGINFO, 0x7f31153f4390}, NULL, 8) = 0
rt_sigaction(SIGRT_11, NULL, {SIG_DFL, [], 0}, 8) = 0
rt_sigaction(SIGRT_11, {0x45b610, ~[], SA_RESTORER|SA_STACK|SA_RESTART|SA_SIGINFO, 0x7f31153f4390}, NULL, 8) = 0
rt_sigaction(SIGRT_12, NULL, {SIG_DFL, [], 0}, 8) = 0
rt_sigaction(SIGRT_12, {0x45b610, ~[], SA_RESTORER|SA_STACK|SA_RESTART|SA_SIGINFO, 0x7f31153f4390}, NULL, 8) = 0
rt_sigaction(SIGRT_13, NULL, {SIG_DFL, [], 0}, 8) = 0
rt_sigaction(SIGRT_13, {0x45b610, ~[], SA_RESTORER|SA_STACK|SA_RESTART|SA_SIGINFO, 0x7f31153f4390}, NULL, 8) = 0
rt_sigaction(SIGRT_14, NULL, {SIG_DFL, [], 0}, 8) = 0
rt_sigaction(SIGRT_14, {0x45b610, ~[], SA_RESTORER|SA_STACK|SA_RESTART|SA_SIGINFO, 0x7f31153f4390}, NULL, 8) = 0
rt_sigaction(SIGRT_15, NULL, {SIG_DFL, [], 0}, 8) = 0
rt_sigaction(SIGRT_15, {0x45b610, ~[], SA_RESTORER|SA_STACK|SA_RESTART|SA_SIGINFO, 0x7f31153f4390}, NULL, 8) = 0
rt_sigaction(SIGRT_16, NULL, {SIG_DFL, [], 0}, 8) = 0
rt_sigaction(SIGRT_16, {0x45b610, ~[], SA_RESTORER|SA_STACK|SA_RESTART|SA_SIGINFO, 0x7f31153f4390}, NULL, 8) = 0
rt_sigaction(SIGRT_17, NULL, {SIG_DFL, [], 0}, 8) = 0
rt_sigaction(SIGRT_17, {0x45b610, ~[], SA_RESTORER|SA_STACK|SA_RESTART|SA_SIGINFO, 0x7f31153f4390}, NULL, 8) = 0
rt_sigaction(SIGRT_18, NULL, {SIG_DFL, [], 0}, 8) = 0
rt_sigaction(SIGRT_18, {0x45b610, ~[], SA_RESTORER|SA_STACK|SA_RESTART|SA_SIGINFO, 0x7f31153f4390}, NULL, 8) = 0
rt_sigaction(SIGRT_19, NULL, {SIG_DFL, [], 0}, 8) = 0
rt_sigaction(SIGRT_19, {0x45b610, ~[], SA_RESTORER|SA_STACK|SA_RESTART|SA_SIGINFO, 0x7f31153f4390}, NULL, 8) = 0
rt_sigaction(SIGRT_20, NULL, {SIG_DFL, [], 0}, 8) = 0
rt_sigaction(SIGRT_20, {0x45b610, ~[], SA_RESTORER|SA_STACK|SA_RESTART|SA_SIGINFO, 0x7f31153f4390}, NULL, 8) = 0
rt_sigaction(SIGRT_21, NULL, {SIG_DFL, [], 0}, 8) = 0
rt_sigaction(SIGRT_21, {0x45b610, ~[], SA_RESTORER|SA_STACK|SA_RESTART|SA_SIGINFO, 0x7f31153f4390}, NULL, 8) = 0
rt_sigaction(SIGRT_22, NULL, {SIG_DFL, [], 0}, 8) = 0
rt_sigaction(SIGRT_22, {0x45b610, ~[], SA_RESTORER|SA_STACK|SA_RESTART|SA_SIGINFO, 0x7f31153f4390}, NULL, 8) = 0
rt_sigaction(SIGRT_23, NULL, {SIG_DFL, [], 0}, 8) = 0
rt_sigaction(SIGRT_23, {0x45b610, ~[], SA_RESTORER|SA_STACK|SA_RESTART|SA_SIGINFO, 0x7f31153f4390}, NULL, 8) = 0
rt_sigaction(SIGRT_24, NULL, {SIG_DFL, [], 0}, 8) = 0
rt_sigaction(SIGRT_24, {0x45b610, ~[], SA_RESTORER|SA_STACK|SA_RESTART|SA_SIGINFO, 0x7f31153f4390}, NULL, 8) = 0
rt_sigaction(SIGRT_25, NULL, {SIG_DFL, [], 0}, 8) = 0
rt_sigaction(SIGRT_25, {0x45b610, ~[], SA_RESTORER|SA_STACK|SA_RESTART|SA_SIGINFO, 0x7f31153f4390}, NULL, 8) = 0
rt_sigaction(SIGRT_26, NULL, {SIG_DFL, [], 0}, 8) = 0
rt_sigaction(SIGRT_26, {0x45b610, ~[], SA_RESTORER|SA_STACK|SA_RESTART|SA_SIGINFO, 0x7f31153f4390}, NULL, 8) = 0
rt_sigaction(SIGRT_27, NULL, {SIG_DFL, [], 0}, 8) = 0
rt_sigaction(SIGRT_27, {0x45b610, ~[], SA_RESTORER|SA_STACK|SA_RESTART|SA_SIGINFO, 0x7f31153f4390}, NULL, 8) = 0
rt_sigaction(SIGRT_28, NULL, {SIG_DFL, [], 0}, 8) = 0
rt_sigaction(SIGRT_28, {0x45b610, ~[], SA_RESTORER|SA_STACK|SA_RESTART|SA_SIGINFO, 0x7f31153f4390}, NULL, 8) = 0
rt_sigaction(SIGRT_29, NULL, {SIG_DFL, [], 0}, 8) = 0
rt_sigaction(SIGRT_29, {0x45b610, ~[], SA_RESTORER|SA_STACK|SA_RESTART|SA_SIGINFO, 0x7f31153f4390}, NULL, 8) = 0
rt_sigaction(SIGRT_30, NULL, {SIG_DFL, [], 0}, 8) = 0
rt_sigaction(SIGRT_30, {0x45b610, ~[], SA_RESTORER|SA_STACK|SA_RESTART|SA_SIGINFO, 0x7f31153f4390}, NULL, 8) = 0
rt_sigaction(SIGRT_31, NULL, {SIG_DFL, [], 0}, 8) = 0
rt_sigaction(SIGRT_31, {0x45b610, ~[], SA_RESTORER|SA_STACK|SA_RESTART|SA_SIGINFO, 0x7f31153f4390}, NULL, 8) = 0
rt_sigaction(SIGRT_32, NULL, {SIG_DFL, [], 0}, 8) = 0
rt_sigaction(SIGRT_32, {0x45b610, ~[], SA_RESTORER|SA_STACK|SA_RESTART|SA_SIGINFO, 0x7f31153f4390}, NULL, 8) = 0
rt_sigprocmask(SIG_SETMASK, ~[RTMIN RT_1], [], 8) = 0
mmap(NULL, 8392704, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS|MAP_STACK, -1, 0) = 0x7f3114818000
mprotect(0x7f3114818000, 4096, PROT_NONE) = 0
clone(child_stack=0x7f3115017ff0, flags=CLONE_VM|CLONE_FS|CLONE_FILES|CLONE_SIGHAND|CLONE_THREAD|CLONE_SYSVSEM|CLONE_SETTLS|CLONE_PARENT_SETTID|CLONE_CHILD_CLEARTID, parent_tidptr=0x7f31150189d0, tls=0x7f3115018700, child_tidptr=0x7f31150189d0) = 69327
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
rt_sigprocmask(SIG_SETMASK, ~[RTMIN RT_1], [], 8) = 0
mmap(NULL, 8392704, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS|MAP_STACK, -1, 0) = 0x7f3114017000
mprotect(0x7f3114017000, 4096, PROT_NONE) = 0
clone(child_stack=0x7f3114816ff0, flags=CLONE_VM|CLONE_FS|CLONE_FILES|CLONE_SIGHAND|CLONE_THREAD|CLONE_SYSVSEM|CLONE_SETTLS|CLONE_PARENT_SETTID|CLONE_CHILD_CLEARTID, parent_tidptr=0x7f31148179d0, tls=0x7f3114817700, child_tidptr=0x7f31148179d0) = 69328
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
rt_sigprocmask(SIG_SETMASK, ~[RTMIN RT_1], [], 8) = 0
mmap(NULL, 8392704, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS|MAP_STACK, -1, 0) = 0x7f310effe000
mprotect(0x7f310effe000, 4096, PROT_NONE) = 0
clone(child_stack=0x7f310f7fdff0, flags=CLONE_VM|CLONE_FS|CLONE_FILES|CLONE_SIGHAND|CLONE_THREAD|CLONE_SYSVSEM|CLONE_SETTLS|CLONE_PARENT_SETTID|CLONE_CHILD_CLEARTID, parent_tidptr=0x7f310f7fe9d0, tls=0x7f310f7fe700, child_tidptr=0x7f310f7fe9d0) = 69330
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
futex(0xc4200c4b90, FUTEX_WAKE, 1) = 1
futex(0xc4200fa110, FUTEX_WAKE, 1) = 1
futex(0x824590, FUTEX_WAIT, 0, NULL) = -1 EAGAIN (Resource temporarily unavailable)
readlinkat(AT_FDCWD, "/proc/self/exe", "/usr/sbin/esbulk", 128) = 16
futex(0xc4200c4b90, FUTEX_WAKE, 1) = 1
mmap(NULL, 262144, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f3115737000
openat(AT_FDCWD, "/proc/sys/net/core/somaxconn", O_RDONLY|O_CLOEXEC) = 3
epoll_create1(EPOLL_CLOEXEC) = 4
epoll_ctl(4, EPOLL_CTL_ADD, 3, {EPOLLIN|EPOLLOUT|EPOLLRDHUP|EPOLLET, {u32=359907184, u64=139848790032240}}) = 0
fcntl(3, F_GETFL) = 0x8000 (flags O_RDONLY|O_LARGEFILE)
fcntl(3, F_SETFL, O_RDONLY|O_NONBLOCK|O_LARGEFILE) = 0
mmap(NULL, 1439992, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f310ee9e000
read(3, "128\n", 4096) = 4
read(3, "", 4092) = 0
epoll_ctl(4, EPOLL_CTL_DEL, 3, 0xc4200e5c0c) = 0
close(3) = 0
epoll_wait(4, [], 128, 0) = 0
futex(0x823db8, FUTEX_WAKE, 1) = 1
futex(0x823cf0, FUTEX_WAKE, 1) = 1
rt_sigprocmask(SIG_SETMASK, ~[RTMIN RT_1], [], 8) = 0
mmap(NULL, 8392704, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS|MAP_STACK, -1, 0) = 0x7f310e69d000
mprotect(0x7f310e69d000, 4096, PROT_NONE) = 0
clone(child_stack=0x7f310ee9cff0, flags=CLONE_VM|CLONE_FS|CLONE_FILES|CLONE_SIGHAND|CLONE_THREAD|CLONE_SYSVSEM|CLONE_SETTLS|CLONE_PARENT_SETTID|CLONE_CHILD_CLEARTID, parent_tidptr=0x7f310ee9d9d0, tls=0x7f310ee9d700, child_tidptr=0x7f310ee9d9d0) = 69331
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
openat(AT_FDCWD, "/etc/nsswitch.conf", O_RDONLY|O_CLOEXEC) = 3
futex(0x824590, FUTEX_WAIT, 0, NULL) = -1 EAGAIN (Resource temporarily unavailable)
epoll_ctl(4, EPOLL_CTL_ADD, 3, {EPOLLIN|EPOLLOUT|EPOLLRDHUP|EPOLLET, {u32=359907184, u64=139848790032240}}) = -1 EPERM (Operation not permitted)
epoll_ctl(4, EPOLL_CTL_DEL, 3, 0xc4200d3adc) = -1 EPERM (Operation not permitted)
read(3, "# /etc/nsswitch.conf\n#\n# Example"..., 1024) = 497
read(3, "", 1024) = 0
close(3) = 0
openat(AT_FDCWD, "/etc/resolv.conf", O_RDONLY|O_CLOEXEC) = 3
epoll_ctl(4, EPOLL_CTL_ADD, 3, {EPOLLIN|EPOLLOUT|EPOLLRDHUP|EPOLLET, {u32=359907184, u64=139848790032240}}) = -1 EPERM (Operation not permitted)
epoll_ctl(4, EPOLL_CTL_DEL, 3, 0xc4200d39c4) = -1 EPERM (Operation not permitted)
fstat(3, {st_mode=S_IFREG|0644, st_size=200, ...}) = 0
read(3, "# Dynamic resolv.conf(5) file fo"..., 4096) = 200
read(3, "", 3896) = 0
read(3, "", 4096) = 0
close(3) = 0
stat("/etc/mdns.allow", 0xc420117218) = -1 ENOENT (No such file or directory)
stat("/etc/hosts", {st_mode=S_IFREG|0644, st_size=233, ...}) = 0
openat(AT_FDCWD, "/etc/hosts", O_RDONLY|O_CLOEXEC) = 3
epoll_ctl(4, EPOLL_CTL_ADD, 3, {EPOLLIN|EPOLLOUT|EPOLLRDHUP|EPOLLET, {u32=359907184, u64=139848790032240}}) = -1 EPERM (Operation not permitted)
epoll_ctl(4, EPOLL_CTL_DEL, 3, 0xc4200d3694) = -1 EPERM (Operation not permitted)
read(3, "127.0.0.1\tlocalhost\n194.95.145.2"..., 4096) = 233
read(3, "", 3863) = 0
read(3, "", 4096) = 0
close(3) = 0
socket(PF_INET, SOCK_DGRAM|SOCK_CLOEXEC|SOCK_NONBLOCK, IPPROTO_IP) = 3
setsockopt(3, SOL_SOCKET, SO_BROADCAST, [1], 4) = 0
connect(3, {sa_family=AF_INET, sin_port=htons(9), sin_addr=inet_addr("127.0.0.1")}, 16) = 0
epoll_ctl(4, EPOLL_CTL_ADD, 3, {EPOLLIN|EPOLLOUT|EPOLLRDHUP|EPOLLET, {u32=359907184, u64=139848790032240}}) = 0
getsockname(3, {sa_family=AF_INET, sin_port=htons(42324), sin_addr=inet_addr("127.0.0.1")}, [16]) = 0
getpeername(3, {sa_family=AF_INET, sin_port=htons(9), sin_addr=inet_addr("127.0.0.1")}, [16]) = 0
epoll_ctl(4, EPOLL_CTL_DEL, 3, 0xc4200d38dc) = 0
close(3) = 0
socket(PF_INET6, SOCK_DGRAM|SOCK_CLOEXEC|SOCK_NONBLOCK, IPPROTO_IP) = 3
setsockopt(3, SOL_IPV6, IPV6_V6ONLY, [0], 4) = 0
setsockopt(3, SOL_SOCKET, SO_BROADCAST, [1], 4) = 0
connect(3, {sa_family=AF_INET6, sin6_port=htons(9), inet_pton(AF_INET6, "::1", &sin6_addr), sin6_flowinfo=0, sin6_scope_id=0}, 28) = 0
epoll_ctl(4, EPOLL_CTL_ADD, 3, {EPOLLIN|EPOLLOUT|EPOLLRDHUP|EPOLLET, {u32=359907184, u64=139848790032240}}) = 0
getsockname(3, {sa_family=AF_INET6, sin6_port=htons(45407), inet_pton(AF_INET6, "::1", &sin6_addr), sin6_flowinfo=0, sin6_scope_id=0}, [28]) = 0
getpeername(3, {sa_family=AF_INET6, sin6_port=htons(9), inet_pton(AF_INET6, "::1", &sin6_addr), sin6_flowinfo=0, sin6_scope_id=0}, [28]) = 0
epoll_ctl(4, EPOLL_CTL_DEL, 3, 0xc4200d38dc) = 0
close(3) = 0
futex(0x823a20, FUTEX_WAKE, 1) = 1
socket(PF_INET6, SOCK_STREAM|SOCK_CLOEXEC|SOCK_NONBLOCK, IPPROTO_IP) = 3
setsockopt(3, SOL_IPV6, IPV6_V6ONLY, [0], 4) = 0
setsockopt(3, SOL_SOCKET, SO_BROADCAST, [1], 4) = 0
connect(3, {sa_family=AF_INET6, sin6_port=htons(9200), inet_pton(AF_INET6, "::1", &sin6_addr), sin6_flowinfo=0, sin6_scope_id=0}, 28) = -1 EINPROGRESS (Operation now in progress)
futex(0x824590, FUTEX_WAIT, 0, NULL) = 0
epoll_ctl(4, EPOLL_CTL_ADD, 3, {EPOLLIN|EPOLLOUT|EPOLLRDHUP|EPOLLET, {u32=359907184, u64=139848790032240}}) = 0
epoll_wait(4, [{EPOLLOUT, {u32=359907184, u64=139848790032240}}], 128, 0) = 1
getsockopt(3, SOL_SOCKET, SO_ERROR, [0], [4]) = 0
getpeername(3, {sa_family=AF_INET6, sin6_port=htons(9200), inet_pton(AF_INET6, "::1", &sin6_addr), sin6_flowinfo=0, sin6_scope_id=0}, [28]) = 0
getsockname(3, {sa_family=AF_INET6, sin6_port=htons(38822), inet_pton(AF_INET6, "::1", &sin6_addr), sin6_flowinfo=0, sin6_scope_id=0}, [28]) = 0
setsockopt(3, SOL_TCP, TCP_NODELAY, [1], 4) = 0
setsockopt(3, SOL_SOCKET, SO_KEEPALIVE, [1], 4) = 0
setsockopt(3, SOL_TCP, TCP_KEEPINTVL, [30], 4) = 0
setsockopt(3, SOL_TCP, TCP_KEEPIDLE, [30], 4) = 0
read(3, 0xc4201b3000, 4096) = -1 EAGAIN (Resource temporarily unavailable)
mmap(NULL, 262144, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f31156f7000
write(3, "GET /bla HTTP/1.1\r\nHost: localho"..., 130) = 130
epoll_wait(4, [], 128, 0) = 0
epoll_wait(4, [{EPOLLIN|EPOLLOUT, {u32=359907184, u64=139848790032240}}], 128, -1) = 1
futex(0x823db8, FUTEX_WAKE, 1) = 1
futex(0x823cf0, FUTEX_WAKE, 1) = 1
read(3, "HTTP/1.1 200 OK\r\ncontent-type: a"..., 4096) = 299
futex(0x823a20, FUTEX_WAKE, 1) = 1
socket(PF_INET, SOCK_DGRAM|SOCK_CLOEXEC|SOCK_NONBLOCK, IPPROTO_IP) = 5
setsockopt(5, SOL_SOCKET, SO_BROADCAST, [1], 4) = 0
connect(5, {sa_family=AF_INET, sin_port=htons(9), sin_addr=inet_addr("127.0.0.1")}, 16) = 0
epoll_ctl(4, EPOLL_CTL_ADD, 5, {EPOLLIN|EPOLLOUT|EPOLLRDHUP|EPOLLET, {u32=359906992, u64=139848790032048}}) = 0
getsockname(5, {sa_family=AF_INET, sin_port=htons(39628), sin_addr=inet_addr("127.0.0.1")}, [16]) = 0
getpeername(5, {sa_family=AF_INET, sin_port=htons(9), sin_addr=inet_addr("127.0.0.1")}, [16]) = 0
epoll_ctl(4, EPOLL_CTL_DEL, 5, 0xc4200d58dc) = 0
close(5) = 0
socket(PF_INET6, SOCK_DGRAM|SOCK_CLOEXEC|SOCK_NONBLOCK, IPPROTO_IP) = 5
setsockopt(5, SOL_IPV6, IPV6_V6ONLY, [0], 4) = 0
setsockopt(5, SOL_SOCKET, SO_BROADCAST, [1], 4) = 0
connect(5, {sa_family=AF_INET6, sin6_port=htons(9), inet_pton(AF_INET6, "::1", &sin6_addr), sin6_flowinfo=0, sin6_scope_id=0}, 28) = 0
epoll_ctl(4, EPOLL_CTL_ADD, 5, {EPOLLIN|EPOLLOUT|EPOLLRDHUP|EPOLLET, {u32=359906992, u64=139848790032048}}) = 0
getsockname(5, {sa_family=AF_INET6, sin6_port=htons(41298), inet_pton(AF_INET6, "::1", &sin6_addr), sin6_flowinfo=0, sin6_scope_id=0}, [28]) = 0
getpeername(5, {sa_family=AF_INET6, sin6_port=htons(9), inet_pton(AF_INET6, "::1", &sin6_addr), sin6_flowinfo=0, sin6_scope_id=0}, [28]) = 0
epoll_ctl(4, EPOLL_CTL_DEL, 5, 0xc4200d58dc) = 0
close(5) = 0
socket(PF_INET6, SOCK_STREAM|SOCK_CLOEXEC|SOCK_NONBLOCK, IPPROTO_IP) = 5
setsockopt(5, SOL_IPV6, IPV6_V6ONLY, [0], 4) = 0
setsockopt(5, SOL_SOCKET, SO_BROADCAST, [1], 4) = 0
connect(5, {sa_family=AF_INET6, sin6_port=htons(9200), inet_pton(AF_INET6, "::1", &sin6_addr), sin6_flowinfo=0, sin6_scope_id=0}, 28) = -1 EINPROGRESS (Operation now in progress)
epoll_ctl(4, EPOLL_CTL_ADD, 5, {EPOLLIN|EPOLLOUT|EPOLLRDHUP|EPOLLET, {u32=359906992, u64=139848790032048}}) = 0
epoll_ctl(4, EPOLL_CTL_DEL, 3, 0xc4200d39bc) = 0
close(3) = 0
futex(0xc4200c4f10, FUTEX_WAKE, 1) = 1
futex(0x823a20, FUTEX_WAIT, 0, {0, 299542828}) = 0
futex(0x824590, FUTEX_WAIT, 0, NULL) = 0
epoll_wait(4, [], 128, 0) = 0
epoll_wait(4, ^Cstrace: Process 69326 detached
<detached ...>

user@host:~$ strace esbulk -server http://192.168.0.1:9200 -index bla
execve("/usr/sbin/esbulk", ["esbulk", "-server", "http://192.168.0.1:9200", "-index", "bla"], [/* 27 vars /]) = 0
brk(NULL) = 0x22ed000
access("/etc/ld.so.nohwcap", F_OK) = -1 ENOENT (No such file or directory)
access("/etc/ld.so.preload", R_OK) = -1 ENOENT (No such file or directory)
open("/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 3
fstat(3, {st_mode=S_IFREG|0644, st_size=39302, ...}) = 0
mmap(NULL, 39302, PROT_READ, MAP_PRIVATE, 3, 0) = 0x7fc004952000
close(3) = 0
access("/etc/ld.so.nohwcap", F_OK) = -1 ENOENT (No such file or directory)
open("/lib/x86_64-linux-gnu/libpthread.so.0", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\260`\0\0\0\0\0\0"..., 832) = 832
fstat(3, {st_mode=S_IFREG|0755, st_size=138696, ...}) = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fc004951000
mmap(NULL, 2212904, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7fc00451a000
mprotect(0x7fc004532000, 2093056, PROT_NONE) = 0
mmap(0x7fc004731000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x17000) = 0x7fc004731000
mmap(0x7fc004733000, 13352, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7fc004733000
close(3) = 0
access("/etc/ld.so.nohwcap", F_OK) = -1 ENOENT (No such file or directory)
open("/lib/x86_64-linux-gnu/libc.so.6", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\3\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0P\t\2\0\0\0\0\0"..., 832) = 832
fstat(3, {st_mode=S_IFREG|0755, st_size=1868984, ...}) = 0
mmap(NULL, 3971488, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7fc004150000
mprotect(0x7fc004310000, 2097152, PROT_NONE) = 0
mmap(0x7fc004510000, 24576, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x1c0000) = 0x7fc004510000
mmap(0x7fc004516000, 14752, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7fc004516000
close(3) = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fc004950000
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fc00494f000
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fc00494e000
arch_prctl(ARCH_SET_FS, 0x7fc00494f700) = 0
mprotect(0x7fc004510000, 16384, PROT_READ) = 0
mprotect(0x7fc004731000, 4096, PROT_READ) = 0
mprotect(0x7fc00495c000, 4096, PROT_READ) = 0
munmap(0x7fc004952000, 39302) = 0
set_tid_address(0x7fc00494f9d0) = 68406
set_robust_list(0x7fc00494f9e0, 24) = 0
rt_sigaction(SIGRTMIN, {0x7fc00451fb50, [], SA_RESTORER|SA_SIGINFO, 0x7fc00452b390}, NULL, 8) = 0
rt_sigaction(SIGRT_1, {0x7fc00451fbe0, [], SA_RESTORER|SA_RESTART|SA_SIGINFO, 0x7fc00452b390}, NULL, 8) = 0
rt_sigprocmask(SIG_UNBLOCK, [RTMIN RT_1], NULL, 8) = 0
getrlimit(RLIMIT_STACK, {rlim_cur=81921024, rlim_max=RLIM64_INFINITY}) = 0
brk(NULL) = 0x22ed000
brk(0x230e000) = 0x230e000
sched_getaffinity(0, 8192, [ffffffffffffffff, ffffffffffffffff, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, ...]) = 64
mmap(0xc000000000, 65536, PROT_NONE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xc000000000
munmap(0xc000000000, 65536) = 0
mmap(NULL, 262144, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fc00490e000
mmap(0xc420000000, 1048576, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xc420000000
mmap(0xc41fff8000, 32768, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xc41fff8000
mmap(0xc000000000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xc000000000
mmap(NULL, 65536, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fc0048fe000
mmap(NULL, 65536, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fc0048ee000
rt_sigprocmask(SIG_SETMASK, NULL, [], 8) = 0
mmap(NULL, 262144, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fc0048ae000
sigaltstack(NULL, {ss_sp=NULL, ss_flags=SS_DISABLE, ss_size=0}) = 0
sigaltstack({ss_sp=0xc420002000, ss_flags=0, ss_size=32768}, NULL) = 0
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
gettid() = 68406
rt_sigaction(SIGHUP, NULL, {SIG_DFL, [], 0}, 8) = 0
rt_sigaction(SIGHUP, {0x45b610, ~[], SA_RESTORER|SA_STACK|SA_RESTART|SA_SIGINFO, 0x7fc00452b390}, NULL, 8) = 0
rt_sigaction(SIGINT, NULL, {SIG_DFL, [], 0}, 8) = 0
rt_sigaction(SIGINT, {0x45b610, ~[], SA_RESTORER|SA_STACK|SA_RESTART|SA_SIGINFO, 0x7fc00452b390}, NULL, 8) = 0
rt_sigaction(SIGQUIT, NULL, {SIG_DFL, [], 0}, 8) = 0
rt_sigaction(SIGQUIT, {0x45b610, ~[], SA_RESTORER|SA_STACK|SA_RESTART|SA_SIGINFO, 0x7fc00452b390}, NULL, 8) = 0
rt_sigaction(SIGILL, NULL, {SIG_DFL, [], 0}, 8) = 0
rt_sigaction(SIGILL, {0x45b610, ~[], SA_RESTORER|SA_STACK|SA_RESTART|SA_SIGINFO, 0x7fc00452b390}, NULL, 8) = 0
rt_sigaction(SIGTRAP, NULL, {SIG_DFL, [], 0}, 8) = 0
rt_sigaction(SIGTRAP, {0x45b610, ~[], SA_RESTORER|SA_STACK|SA_RESTART|SA_SIGINFO, 0x7fc00452b390}, NULL, 8) = 0
rt_sigaction(SIGABRT, NULL, {SIG_DFL, [], 0}, 8) = 0
rt_sigaction(SIGABRT, {0x45b610, ~[], SA_RESTORER|SA_STACK|SA_RESTART|SA_SIGINFO, 0x7fc00452b390}, NULL, 8) = 0
rt_sigaction(SIGBUS, NULL, {SIG_DFL, [], 0}, 8) = 0
rt_sigaction(SIGBUS, {0x45b610, ~[], SA_RESTORER|SA_STACK|SA_RESTART|SA_SIGINFO, 0x7fc00452b390}, NULL, 8) = 0
rt_sigaction(SIGFPE, NULL, {SIG_DFL, [], 0}, 8) = 0
rt_sigaction(SIGFPE, {0x45b610, ~[], SA_RESTORER|SA_STACK|SA_RESTART|SA_SIGINFO, 0x7fc00452b390}, NULL, 8) = 0
rt_sigaction(SIGUSR1, NULL, {SIG_DFL, [], 0}, 8) = 0
rt_sigaction(SIGUSR1, {0x45b610, ~[], SA_RESTORER|SA_STACK|SA_RESTART|SA_SIGINFO, 0x7fc00452b390}, NULL, 8) = 0
rt_sigaction(SIGSEGV, NULL, {SIG_DFL, [], 0}, 8) = 0
rt_sigaction(SIGSEGV, {0x45b610, ~[], SA_RESTORER|SA_STACK|SA_RESTART|SA_SIGINFO, 0x7fc00452b390}, NULL, 8) = 0
rt_sigaction(SIGUSR2, NULL, {SIG_DFL, [], 0}, 8) = 0
rt_sigaction(SIGUSR2, {0x45b610, ~[], SA_RESTORER|SA_STACK|SA_RESTART|SA_SIGINFO, 0x7fc00452b390}, NULL, 8) = 0
rt_sigaction(SIGPIPE, NULL, {SIG_DFL, [], 0}, 8) = 0
rt_sigaction(SIGPIPE, {0x45b610, ~[], SA_RESTORER|SA_STACK|SA_RESTART|SA_SIGINFO, 0x7fc00452b390}, NULL, 8) = 0
rt_sigaction(SIGALRM, NULL, {SIG_DFL, [], 0}, 8) = 0
rt_sigaction(SIGALRM, {0x45b610, ~[], SA_RESTORER|SA_STACK|SA_RESTART|SA_SIGINFO, 0x7fc00452b390}, NULL, 8) = 0
rt_sigaction(SIGTERM, NULL, {SIG_DFL, [], 0}, 8) = 0
rt_sigaction(SIGTERM, {0x45b610, ~[], SA_RESTORER|SA_STACK|SA_RESTART|SA_SIGINFO, 0x7fc00452b390}, NULL, 8) = 0
rt_sigaction(SIGSTKFLT, NULL, {SIG_DFL, [], 0}, 8) = 0
rt_sigaction(SIGSTKFLT, {0x45b610, ~[], SA_RESTORER|SA_STACK|SA_RESTART|SA_SIGINFO, 0x7fc00452b390}, NULL, 8) = 0
rt_sigaction(SIGCHLD, NULL, {SIG_DFL, [], 0}, 8) = 0
rt_sigaction(SIGCHLD, {0x45b610, ~[], SA_RESTORER|SA_STACK|SA_RESTART|SA_SIGINFO, 0x7fc00452b390}, NULL, 8) = 0
rt_sigaction(SIGURG, NULL, {SIG_DFL, [], 0}, 8) = 0
rt_sigaction(SIGURG, {0x45b610, ~[], SA_RESTORER|SA_STACK|SA_RESTART|SA_SIGINFO, 0x7fc00452b390}, NULL, 8) = 0
rt_sigaction(SIGXCPU, NULL, {SIG_DFL, [], 0}, 8) = 0
rt_sigaction(SIGXCPU, {0x45b610, ~[], SA_RESTORER|SA_STACK|SA_RESTART|SA_SIGINFO, 0x7fc00452b390}, NULL, 8) = 0
rt_sigaction(SIGXFSZ, NULL, {SIG_DFL, [], 0}, 8) = 0
rt_sigaction(SIGXFSZ, {0x45b610, ~[], SA_RESTORER|SA_STACK|SA_RESTART|SA_SIGINFO, 0x7fc00452b390}, NULL, 8) = 0
rt_sigaction(SIGVTALRM, NULL, {SIG_DFL, [], 0}, 8) = 0
rt_sigaction(SIGVTALRM, {0x45b610, ~[], SA_RESTORER|SA_STACK|SA_RESTART|SA_SIGINFO, 0x7fc00452b390}, NULL, 8) = 0
rt_sigaction(SIGPROF, NULL, {SIG_DFL, [], 0}, 8) = 0
rt_sigaction(SIGPROF, {0x45b610, ~[], SA_RESTORER|SA_STACK|SA_RESTART|SA_SIGINFO, 0x7fc00452b390}, NULL, 8) = 0
rt_sigaction(SIGWINCH, NULL, {SIG_DFL, [], 0}, 8) = 0
rt_sigaction(SIGWINCH, {0x45b610, ~[], SA_RESTORER|SA_STACK|SA_RESTART|SA_SIGINFO, 0x7fc00452b390}, NULL, 8) = 0
rt_sigaction(SIGIO, NULL, {SIG_DFL, [], 0}, 8) = 0
rt_sigaction(SIGIO, {0x45b610, ~[], SA_RESTORER|SA_STACK|SA_RESTART|SA_SIGINFO, 0x7fc00452b390}, NULL, 8) = 0
rt_sigaction(SIGPWR, NULL, {SIG_DFL, [], 0}, 8) = 0
rt_sigaction(SIGPWR, {0x45b610, ~[], SA_RESTORER|SA_STACK|SA_RESTART|SA_SIGINFO, 0x7fc00452b390}, NULL, 8) = 0
rt_sigaction(SIGSYS, NULL, {SIG_DFL, [], 0}, 8) = 0
rt_sigaction(SIGSYS, {0x45b610, ~[], SA_RESTORER|SA_STACK|SA_RESTART|SA_SIGINFO, 0x7fc00452b390}, NULL, 8) = 0
rt_sigaction(SIGRTMIN, NULL, {0x7fc00451fb50, [], SA_RESTORER|SA_SIGINFO, 0x7fc00452b390}, 8) = 0
rt_sigaction(SIGRTMIN, NULL, {0x7fc00451fb50, [], SA_RESTORER|SA_SIGINFO, 0x7fc00452b390}, 8) = 0
rt_sigaction(SIGRTMIN, {0x7fc00451fb50, [], SA_RESTORER|SA_STACK|SA_SIGINFO, 0x7fc00452b390}, NULL, 8) = 0
rt_sigaction(SIGRT_1, NULL, {0x7fc00451fbe0, [], SA_RESTORER|SA_RESTART|SA_SIGINFO, 0x7fc00452b390}, 8) = 0
rt_sigaction(SIGRT_1, NULL, {0x7fc00451fbe0, [], SA_RESTORER|SA_RESTART|SA_SIGINFO, 0x7fc00452b390}, 8) = 0
rt_sigaction(SIGRT_1, {0x7fc00451fbe0, [], SA_RESTORER|SA_STACK|SA_RESTART|SA_SIGINFO, 0x7fc00452b390}, NULL, 8) = 0
rt_sigaction(SIGRT_2, NULL, {SIG_DFL, [], 0}, 8) = 0
rt_sigaction(SIGRT_2, {0x45b610, ~[], SA_RESTORER|SA_STACK|SA_RESTART|SA_SIGINFO, 0x7fc00452b390}, NULL, 8) = 0
rt_sigaction(SIGRT_3, NULL, {SIG_DFL, [], 0}, 8) = 0
rt_sigaction(SIGRT_3, {0x45b610, ~[], SA_RESTORER|SA_STACK|SA_RESTART|SA_SIGINFO, 0x7fc00452b390}, NULL, 8) = 0
rt_sigaction(SIGRT_4, NULL, {SIG_DFL, [], 0}, 8) = 0
rt_sigaction(SIGRT_4, {0x45b610, ~[], SA_RESTORER|SA_STACK|SA_RESTART|SA_SIGINFO, 0x7fc00452b390}, NULL, 8) = 0
rt_sigaction(SIGRT_5, NULL, {SIG_DFL, [], 0}, 8) = 0
rt_sigaction(SIGRT_5, {0x45b610, ~[], SA_RESTORER|SA_STACK|SA_RESTART|SA_SIGINFO, 0x7fc00452b390}, NULL, 8) = 0
rt_sigaction(SIGRT_6, NULL, {SIG_DFL, [], 0}, 8) = 0
rt_sigaction(SIGRT_6, {0x45b610, ~[], SA_RESTORER|SA_STACK|SA_RESTART|SA_SIGINFO, 0x7fc00452b390}, NULL, 8) = 0
rt_sigaction(SIGRT_7, NULL, {SIG_DFL, [], 0}, 8) = 0
rt_sigaction(SIGRT_7, {0x45b610, ~[], SA_RESTORER|SA_STACK|SA_RESTART|SA_SIGINFO, 0x7fc00452b390}, NULL, 8) = 0
rt_sigaction(SIGRT_8, NULL, {SIG_DFL, [], 0}, 8) = 0
rt_sigaction(SIGRT_8, {0x45b610, ~[], SA_RESTORER|SA_STACK|SA_RESTART|SA_SIGINFO, 0x7fc00452b390}, NULL, 8) = 0
rt_sigaction(SIGRT_9, NULL, {SIG_DFL, [], 0}, 8) = 0
rt_sigaction(SIGRT_9, {0x45b610, ~[], SA_RESTORER|SA_STACK|SA_RESTART|SA_SIGINFO, 0x7fc00452b390}, NULL, 8) = 0
rt_sigaction(SIGRT_10, NULL, {SIG_DFL, [], 0}, 8) = 0
rt_sigaction(SIGRT_10, {0x45b610, ~[], SA_RESTORER|SA_STACK|SA_RESTART|SA_SIGINFO, 0x7fc00452b390}, NULL, 8) = 0
rt_sigaction(SIGRT_11, NULL, {SIG_DFL, [], 0}, 8) = 0
rt_sigaction(SIGRT_11, {0x45b610, ~[], SA_RESTORER|SA_STACK|SA_RESTART|SA_SIGINFO, 0x7fc00452b390}, NULL, 8) = 0
rt_sigaction(SIGRT_12, NULL, {SIG_DFL, [], 0}, 8) = 0
rt_sigaction(SIGRT_12, {0x45b610, ~[], SA_RESTORER|SA_STACK|SA_RESTART|SA_SIGINFO, 0x7fc00452b390}, NULL, 8) = 0
rt_sigaction(SIGRT_13, NULL, {SIG_DFL, [], 0}, 8) = 0
rt_sigaction(SIGRT_13, {0x45b610, ~[], SA_RESTORER|SA_STACK|SA_RESTART|SA_SIGINFO, 0x7fc00452b390}, NULL, 8) = 0
rt_sigaction(SIGRT_14, NULL, {SIG_DFL, [], 0}, 8) = 0
rt_sigaction(SIGRT_14, {0x45b610, ~[], SA_RESTORER|SA_STACK|SA_RESTART|SA_SIGINFO, 0x7fc00452b390}, NULL, 8) = 0
rt_sigaction(SIGRT_15, NULL, {SIG_DFL, [], 0}, 8) = 0
rt_sigaction(SIGRT_15, {0x45b610, ~[], SA_RESTORER|SA_STACK|SA_RESTART|SA_SIGINFO, 0x7fc00452b390}, NULL, 8) = 0
rt_sigaction(SIGRT_16, NULL, {SIG_DFL, [], 0}, 8) = 0
rt_sigaction(SIGRT_16, {0x45b610, ~[], SA_RESTORER|SA_STACK|SA_RESTART|SA_SIGINFO, 0x7fc00452b390}, NULL, 8) = 0
rt_sigaction(SIGRT_17, NULL, {SIG_DFL, [], 0}, 8) = 0
rt_sigaction(SIGRT_17, {0x45b610, ~[], SA_RESTORER|SA_STACK|SA_RESTART|SA_SIGINFO, 0x7fc00452b390}, NULL, 8) = 0
rt_sigaction(SIGRT_18, NULL, {SIG_DFL, [], 0}, 8) = 0
rt_sigaction(SIGRT_18, {0x45b610, ~[], SA_RESTORER|SA_STACK|SA_RESTART|SA_SIGINFO, 0x7fc00452b390}, NULL, 8) = 0
rt_sigaction(SIGRT_19, NULL, {SIG_DFL, [], 0}, 8) = 0
rt_sigaction(SIGRT_19, {0x45b610, ~[], SA_RESTORER|SA_STACK|SA_RESTART|SA_SIGINFO, 0x7fc00452b390}, NULL, 8) = 0
rt_sigaction(SIGRT_20, NULL, {SIG_DFL, [], 0}, 8) = 0
rt_sigaction(SIGRT_20, {0x45b610, ~[], SA_RESTORER|SA_STACK|SA_RESTART|SA_SIGINFO, 0x7fc00452b390}, NULL, 8) = 0
rt_sigaction(SIGRT_21, NULL, {SIG_DFL, [], 0}, 8) = 0
rt_sigaction(SIGRT_21, {0x45b610, ~[], SA_RESTORER|SA_STACK|SA_RESTART|SA_SIGINFO, 0x7fc00452b390}, NULL, 8) = 0
rt_sigaction(SIGRT_22, NULL, {SIG_DFL, [], 0}, 8) = 0
rt_sigaction(SIGRT_22, {0x45b610, ~[], SA_RESTORER|SA_STACK|SA_RESTART|SA_SIGINFO, 0x7fc00452b390}, NULL, 8) = 0
rt_sigaction(SIGRT_23, NULL, {SIG_DFL, [], 0}, 8) = 0
rt_sigaction(SIGRT_23, {0x45b610, ~[], SA_RESTORER|SA_STACK|SA_RESTART|SA_SIGINFO, 0x7fc00452b390}, NULL, 8) = 0
rt_sigaction(SIGRT_24, NULL, {SIG_DFL, [], 0}, 8) = 0
rt_sigaction(SIGRT_24, {0x45b610, ~[], SA_RESTORER|SA_STACK|SA_RESTART|SA_SIGINFO, 0x7fc00452b390}, NULL, 8) = 0
rt_sigaction(SIGRT_25, NULL, {SIG_DFL, [], 0}, 8) = 0
rt_sigaction(SIGRT_25, {0x45b610, ~[], SA_RESTORER|SA_STACK|SA_RESTART|SA_SIGINFO, 0x7fc00452b390}, NULL, 8) = 0
rt_sigaction(SIGRT_26, NULL, {SIG_DFL, [], 0}, 8) = 0
rt_sigaction(SIGRT_26, {0x45b610, ~[], SA_RESTORER|SA_STACK|SA_RESTART|SA_SIGINFO, 0x7fc00452b390}, NULL, 8) = 0
rt_sigaction(SIGRT_27, NULL, {SIG_DFL, [], 0}, 8) = 0
rt_sigaction(SIGRT_27, {0x45b610, ~[], SA_RESTORER|SA_STACK|SA_RESTART|SA_SIGINFO, 0x7fc00452b390}, NULL, 8) = 0
rt_sigaction(SIGRT_28, NULL, {SIG_DFL, [], 0}, 8) = 0
rt_sigaction(SIGRT_28, {0x45b610, ~[], SA_RESTORER|SA_STACK|SA_RESTART|SA_SIGINFO, 0x7fc00452b390}, NULL, 8) = 0
rt_sigaction(SIGRT_29, NULL, {SIG_DFL, [], 0}, 8) = 0
rt_sigaction(SIGRT_29, {0x45b610, ~[], SA_RESTORER|SA_STACK|SA_RESTART|SA_SIGINFO, 0x7fc00452b390}, NULL, 8) = 0
rt_sigaction(SIGRT_30, NULL, {SIG_DFL, [], 0}, 8) = 0
rt_sigaction(SIGRT_30, {0x45b610, ~[], SA_RESTORER|SA_STACK|SA_RESTART|SA_SIGINFO, 0x7fc00452b390}, NULL, 8) = 0
rt_sigaction(SIGRT_31, NULL, {SIG_DFL, [], 0}, 8) = 0
rt_sigaction(SIGRT_31, {0x45b610, ~[], SA_RESTORER|SA_STACK|SA_RESTART|SA_SIGINFO, 0x7fc00452b390}, NULL, 8) = 0
rt_sigaction(SIGRT_32, NULL, {SIG_DFL, [], 0}, 8) = 0
rt_sigaction(SIGRT_32, {0x45b610, ~[], SA_RESTORER|SA_STACK|SA_RESTART|SA_SIGINFO, 0x7fc00452b390}, NULL, 8) = 0
rt_sigprocmask(SIG_SETMASK, ~[RTMIN RT_1], [], 8) = 0
mmap(NULL, 8392704, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS|MAP_STACK, -1, 0) = 0x7fc00394f000
mprotect(0x7fc00394f000, 4096, PROT_NONE) = 0
clone(child_stack=0x7fc00414eff0, flags=CLONE_VM|CLONE_FS|CLONE_FILES|CLONE_SIGHAND|CLONE_THREAD|CLONE_SYSVSEM|CLONE_SETTLS|CLONE_PARENT_SETTID|CLONE_CHILD_CLEARTID, parent_tidptr=0x7fc00414f9d0, tls=0x7fc00414f700, child_tidptr=0x7fc00414f9d0) = 68407
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
rt_sigprocmask(SIG_SETMASK, ~[RTMIN RT_1], [], 8) = 0
mmap(NULL, 8392704, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS|MAP_STACK, -1, 0) = 0x7fc00314e000
mprotect(0x7fc00314e000, 4096, PROT_NONE) = 0
clone(child_stack=0x7fc00394dff0, flags=CLONE_VM|CLONE_FS|CLONE_FILES|CLONE_SIGHAND|CLONE_THREAD|CLONE_SYSVSEM|CLONE_SETTLS|CLONE_PARENT_SETTID|CLONE_CHILD_CLEARTID, parent_tidptr=0x7fc00394e9d0, tls=0x7fc00394e700, child_tidptr=0x7fc00394e9d0) = 68408
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
rt_sigprocmask(SIG_SETMASK, ~[RTMIN RT_1], [], 8) = 0
mmap(NULL, 8392704, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS|MAP_STACK, -1, 0) = 0x7fc00294d000
mprotect(0x7fc00294d000, 4096, PROT_NONE) = 0
clone(child_stack=0x7fc00314cff0, flags=CLONE_VM|CLONE_FS|CLONE_FILES|CLONE_SIGHAND|CLONE_THREAD|CLONE_SYSVSEM|CLONE_SETTLS|CLONE_PARENT_SETTID|CLONE_CHILD_CLEARTID, parent_tidptr=0x7fc00314d9d0, tls=0x7fc00314d700, child_tidptr=0x7fc00314d9d0) = 68410
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
futex(0xc4200c4b90, FUTEX_WAKE, 1) = 1
futex(0xc4200fa110, FUTEX_WAKE, 1) = 1
readlinkat(AT_FDCWD, "/proc/self/exe", "/usr/sbin/esbulk", 128) = 16
futex(0xc4200c4b90, FUTEX_WAKE, 1) = 1
mmap(NULL, 262144, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fc00486e000
openat(AT_FDCWD, "/proc/sys/net/core/somaxconn", O_RDONLY|O_CLOEXEC) = 3
epoll_create1(EPOLL_CLOEXEC) = 4
epoll_ctl(4, EPOLL_CTL_ADD, 3, {EPOLLIN|EPOLLOUT|EPOLLRDHUP|EPOLLET, {u32=75968368, u64=140462686416752}}) = 0
fcntl(3, F_GETFL) = 0x8000 (flags O_RDONLY|O_LARGEFILE)
fcntl(3, F_SETFL, O_RDONLY|O_NONBLOCK|O_LARGEFILE) = 0
read(3, "128\n", 4096) = 4
read(3, "", 4092) = 0
epoll_ctl(4, EPOLL_CTL_DEL, 3, 0xc4200e5c0c) = 0
close(3) = 0
futex(0xc4200fa110, FUTEX_WAKE, 1) = 1
futex(0xc4200fa110, FUTEX_WAKE, 1) = 1
socket(PF_INET, SOCK_STREAM|SOCK_CLOEXEC|SOCK_NONBLOCK, IPPROTO_IP) = 3
setsockopt(3, SOL_SOCKET, SO_BROADCAST, [1], 4) = 0
connect(3, {sa_family=AF_INET, sin_port=htons(9200), sin_addr=inet_addr("192.168.0.1")}, 16) = -1 EINPROGRESS (Operation now in progress)
epoll_ctl(4, EPOLL_CTL_ADD, 3, {EPOLLIN|EPOLLOUT|EPOLLRDHUP|EPOLLET, {u32=75968368, u64=140462686416752}}) = 0
futex(0x823a20, FUTEX_WAKE, 1) = 1
futex(0x823a00, FUTEX_WAKE, 1) = 1
futex(0xc4200c4b90, FUTEX_WAKE, 1) = 1
epoll_wait(4, [], 128, 0) = 0
futex(0x824590, FUTEX_WAIT, 0, NULL) = 0
futex(0x824590, FUTEX_WAIT, 0, NULL2018/06/18 10:46:04 Get http://192.168.0.1:9200/bla: dial tcp 192.168.0.1:9200: i/o timeout
<unfinished ...>
+++ exited with 1 +++

Support combined mappings and settings at index creation time

• https://www.elastic.co/guide/en/elasticsearch/reference/6.3/indices-create-index.html#mappings

Support ES 7.x-style mappings with no document type

When trying to index documents into an Elasticsearch 6.x index with no document type (eg, created with include_type_name=false and mappings is directly the schema, not a JSON object with key _doc), I got the following esbulk error:

2020/05/20 13:46:39 using 1 servers
2020/05/20 13:46:39 {[http://localhost:9200] dev_scholar_fulltext default 200 true key http  }
panic: interface conversion: interface {} is nil, not map[string]interface {}

goroutine 1 [running]:
main.main()
        /home/tir/go/src/github.com/miku/esbulk/cmd/esbulk/esbulk.go:189 +0x1aed

This might be an issue with using ES 6.x in this way, but from the panic text I suspect this is due to esbulk not supporting this newer single-document-type mapping format.

ES 6.2 Not working

I was using esbulk, then upgraded ES to 6.2. Now, esbulk shows an error of "mapper_parsing_exception". The index that it attempts to write to has an index pattern applied to it.

Losing documents

Hi again,

We've been doing some tests using esbulk and it works really well in most cases, but we've found some issues when loading millions of documents.

The scenario is the following, we have a line delimited json with 50 million documents (we've tested with 20M too and we get the same results), each of the document is something like this:

• 40 attributes of type short
• 2 attributes of type integer[] that contain an average of 50 values/document
• 4 attributes of type short[] that contain an average of 5 values/document
• 1 attribute of type geo_point

For the purposes of this test, we have a cluster with 8 machines, each of them with SSD disks and 16 cores.

From one of the nodes of the cluster we run the following:

1. Create the index

$ curl -XPUT "http://localhost:9200/index_name" -d'
{
  "settings": {
    "number_of_shards": 256,
    "number_of_replicas": 0
  },
  "mappings": {
     "default": {
        "properties": {
           "p1": {
              "type": "short"
           },
           "p2": {
              "type": "short"
           },
...
}

1. Load the data

$ esbulk -server http://localhost:9200 -size 5000 -w 16 -index index_name -verbose /data-file

We've done the tests using different configurations (size of 1000, 5000, 10000, 20000 and number of workers 1, 4, 8, 16) and we always get similar results.

The thing is that we get a message like

50000000 docs in Xs at Y docs/s with Z workers

But when we check the size of the index (either by querying GET /index_name/_search or GET /_cat/indices?v) we get a smaller number of documents in it. Sometimes it's only a few hundreds and sometimes some thousands, it looks like the higher the number of workers, the more documents are lost.

We've also checked the logs of all the nodes in the cluster and we haven't seen anything unusual, actually, the only messages are the following:

updating refresh_interval from [1s] to [-1]
updating refresh_interval from [-1] to [1s]

Which would indicate a successful load (if we, for example, load an incorrect document, we can see the error log).

It's a very strange behavior and we wonder why we don't see any log either in ES or in esbulk indicating the problem with some documents.

Any help will be much appreciated, thanks in advance!

ps1: we've tested with smaller batch sizes (1M docs) and it works well, it looks like it only happens with big loads.
ps2: if you need a sample file we can provide it although it's pretty big (>10G compressed) or we could provide the code that we've used to generate the data

Workers

I'm curious about the -w option. I have a 40GB file to import but it doesn't seem to matter how much CPU or mem I give the system. I have 192 cores and ~300GB ram but can't get -w to go past -w 7. I have 6 nodes in this cluster.

2020/06/22 16:20:36 error during bulk operation, check error details; maybe try fewer workers (-w) or increase thread_pool.bulk.queue_size in your nodes

FWIW, htop shows barely any utilization while I am running the import.
Am I doing something wrong?

[feature request] index name logstash pattern.

Hello.

Thanks for develop such a cool utility.
I have moved from logstash to this esbulk.
Btw, I have a small leak of function with this utility.

We usually have log files which contains date field. and we create index with logstash index pattern. (e.g logstash-2016.05.30)
But In some(or many) case dates of single file can be spreaded over several days, particularly local date based rolling strategy forced.

For example
event_20160530.json may have these lines

...
{"time":"2016-05-30T00:00:00.000+0900"} // <--- log 1 ( 05/29 15:00 in UTC )
{"time":"2016-05-30T10:00:00.000+0900"} // <--- log 2 ( 05/30 01:00 in UTC )
...

However, elasticsearch and kibana forces UTC convert.
So log 1 have to logstash-2016.05.29 and log 2 have to logstash-2016.05.30.

I know it is not a simple problem.
But could you please consider feature something like this?

-index logstash-{yyyy.MM.dd} -date_field time -date_field_pattern yyyy-MM-dd'T'hh:mm:ss.SSSZ

Allow mapping to be passed for index creation.

If building an index from scratch, we could just pass a mapping definition file, e.g.

$ esbulk -mapping mymapping.json -index test -type default file.ldj

Related to this, a -purge flag could be added, to always delete an existing index before reapplying the mapping and indexing the file, e.g.

$ esbulk -purge -mapping mymapping.json -index test -type default file.ldj

Implement round robin requests for cluster.