GithubHelp home page GithubHelp logo

bekt / logspout-logstash Goto Github PK

View Code? Open in Web Editor NEW
41.0 3.0 24.0 5 KB

Hosted Logstash adapter for logspout

Home Page: https://hub.docker.com/r/bekt/logspout-logstash/

License: MIT License

Shell 68.27% Go 26.75% Dockerfile 4.98%
logstash logspout-logstash logspout logging

logspout-logstash's Issues

ELK has all been up, logspout run through the docker run

docker run -d bekt/logspout-logstash -e ROUTE_URIS: logstash://ip:5000 -v /var/run/docker.sock:/var/run/docker.sock

root@docker:~# docker  logs  714959c7b1c7
# logspout v3.2-dev-custom by gliderlabs
# adapters: tcp logstash raw udp
# options : persist:/mnt/routes
!! bad adapter: 

What should I do ?

too many open files error

Too many open files error appeared when I was trying to pull any image.
Also the docker images output was empty even though there were many images in the system.

# docker pull bekt/logspout-logstash:latest
Error response from daemon: open /etc/docker/certs.d/docker.io: too many open files
# docker images
REPOSITORY          TAG                 IMAGE ID            CREATED             SIZE

With lsof I found out that there were too many open files of the same json.log file of the container with id a9c24dfd9963.

Running logspout-logstash with -e DEBUG=true indicated that there was a start/stop loop for the same container a9c24dfd9963.

2017/07/25 11:14:04 pump.pumpLogs(): a9c24dfd9963 stopped
2017/07/25 11:14:04 pump.pumpLogs(): a9c24dfd9963 started

and this behavior after some time is consuming all the allowed file descriptors of the docker engine process.

Which means that there is a leak on close function.

There is a similar bug in gliderlabs/logspout repository marked as Closed. gliderlabs/logspout#115

Any chance of updating the image with the latest logspout version?

connection refused

I know, this is not a fault made by you, but maybe you can help me with that. Hopefully, because I'm stuck for hours! Thx

logspout_1       | # logspout v3.2-dev-custom by gliderlabs
logspout_1       | # adapters: tcp logstash raw udp
logspout_1       | # options : persist:/mnt/routes
logspout_1       | # jobs    : http[]:80 pump routes
logspout_1       | # routes  :
logspout_1       | #   ADAPTER	ADDRESS		CONTAINERS	SOURCES	OPTIONS
logspout_1       | #   logstash	logstash:5000				map[]
logspout_1       | 2016/12/02 04:07:10 logstash: could not write:write udp 172.23.0.5:49301->172.23.0.4:5000: write: connection refused
elk2_logspout_1 exited with code 1

I am getting a connection refused, when the logspout-container is trying to connect to my logstash-container
docker-compose.yml

version: '2'
services:
  

  elasticsearch:
    build: elasticsearch/
    ports:
      - "127.0.0.1:9200:9200"
      - "127.0.0.1:9300:9300"
    environment:
      ES_JAVA_OPTS: "-Xms1g -Xmx1g"
    networks:
      - docker_elk


  logstash:
    build: logstash/
    command: -f /etc/logstash/conf.d/
    volumes:
      - ./logstash/config:/etc/logstash/conf.d
    ports:
      - "127.0.0.1:5000:5000/udp"
    networks:
      - docker_elk
    depends_on:
      - elasticsearch

  logspout:
    image: bekt/logspout-logstash
    volumes:
    - /var/run/docker.sock:/var/run/docker.sock
    environment:
      ROUTE_URIS: logstash://logstash:5000
    networks:
      - docker_elk
    depends_on:
      - logstash
    

  kibana:
    build: kibana/
    volumes:
      - ./kibana/config/:/opt/kibana/config/
    ports:
      - "127.0.0.1:5601:5601"
    networks:
      - docker_elk
    depends_on:
      - elasticsearch

networks:
  docker_elk:
    driver: bridge

and my logstash.conf

input {
	tcp {
		port => 5000
		codec => json
	}
	udp {
		port => 5000
		codec => json
	}
}

## Add your filters / logstash plugins configuration here

output {
	elasticsearch {
		hosts => "elasticsearch:9200"
	}
}

Tag releases

Would it be possible to create official releases based on gliderlabs/logspout releases?
That would be tags in this github repo and tags in https://hub.docker.com/r/bekt/logspout-logstash/tags/

Then we could use the official releases instead of master or latest, e.g.:
docker pull bekt/logspout-logstash:v3.2.5

This will give us the possibility to roll back and also to use official releases instead of last commit.
Thanks

License

Is there a license associated with this project? If so, would you please add it here, or if not, would you please consider choosing one?

Thank you.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.