higkoo / nginx-sflow-module Goto Github PK
View Code? Open in Web Editor NEWAutomatically exported from code.google.com/p/nginx-sflow-module
License: Other
Automatically exported from code.google.com/p/nginx-sflow-module
License: Other
Copyright (c) 2011-2014 InMon Corp. Licensed under the terms of the InMon sFlow licence: http://www.inmon.com/technology/sflowlicense.txt nginx-sflow-module ================== A binary, random-sampling nginx module designed for: lightweight, centralized, continuous, real-time monitoring of very large and very busy web farms. Installation ============ First download the nginx sources, then tell the "./configure" script that you want to add this module: ./configure ... --add-module=/absolute/path/to/nginx-sflow-module make make install Configuration ============= The module picks up it's sFlow config from the file /etc/hsflowd.auto, so the easiest way to configure it is to install that daemon: http://host-sflow.sourceforge.net Alternatively, you could type something like this into /etc/hsflowd.auto: rev_start=1 sampling=100 header=128 polling=10 agentIP=<ip-address> collector=127.0.0.1 6343 rev_end=1 Where <ip-address> is the local IP address that best represents this host. Output ====== The sFlow output goes to a UDP port on your sFlow collector host. There you can examine it using a number of tools, including the freeware "sflowtool", which can be downloaded as source code from: http://www.inmon.com/technology/sflowTools.php For example, to log all the fields in ASCII: $ sflowtool Or to log the HTTP samples in common-log-file format: $ sflowtool -H Example output from sflowtool: startDatagram ================================= datagramSourceIP 10.0.0.150 datagramSize 192 unixSecondsUTC 1294168545 datagramVersion 5 agentSubId 32576 agent 10.0.0.150 packetSequenceNo 7 sysUpTime 25000 samplesInPacket 1 startSample ---------------------- sampleType_tag 0:1 sampleType FLOWSAMPLE sampleSequenceNo 1 sourceId 3:65537 meanSkipCount 400 samplePool 124 dropEvents 0 inputPort 0 outputPort 1073741823 flowBlock_tag 0:2100 extendedType socket4 socket4_ip_protocol 6 socket4_local_ip 10.0.0.150 socket4_remote_ip 10.0.0.70 socket4_local_port 80 socket4_remote_port 63023 flowBlock_tag 0:2201 flowSampleType http http_method 2 http_protocol 1001 http_uri /membase.php http_host 10.0.0.150 http_useragent Java/1.6.0_22 http_bytes 3487 http_duration_uS 24278 http_status 200 endSample ---------------------- endDatagram ================================= startDatagram ================================= datagramSourceIP 10.0.0.150 datagramSize 116 unixSecondsUTC 1294168501 datagramVersion 5 agentSubId 32576 agent 10.0.0.150 packetSequenceNo 3 sysUpTime 42000 samplesInPacket 1 startSample ---------------------- sampleType_tag 0:2 sampleType COUNTERSSAMPLE sampleSequenceNo 3 sourceId 3:65537 counterBlock_tag 0:2201 http_method_option_count 0 http_method_get_count 113 http_method_head_count 0 http_method_post_count 0 http_method_put_count 0 http_method_delete_count 0 http_method_trace_count 0 http_methd_connect_count 0 http_method_other_count 0 http_status_1XX_count 0 http_status_2XX_count 112 http_status_3XX_count 0 http_status_4XX_count 1 http_status_5XX_count 0 http_status_other_count 0 endSample ---------------------- endDatagram =================================
Compiled with nginx 1.0.2.
First if you dont have default configfile on filesystem, you are getting this
error message from every http request:
[error] 3711#0: *1 stat(/etc/hsflowd.auto) failed while logging request
If you make 0 byte /etc/hsflowd.auto, nginx will segault on every http request
Then if trying with example configuration i am getting error:
[error] 3935#0: *2755 line=rev_start tokc=^@rev_start while logging request
So it seems sflow module (0.9.3) does not work atm.
Original issue reported on code.google.com by [email protected]
on 18 May 2011 at 12:24
What steps will reproduce the problem?
1. set up centOS 7 minimal on hyper-v environment
2. make install nginx with the mod and dependencies
3. rpm install sflowtool
What is the expected output?
0:0 < counterBlock_tag < 0:5 or 0:1000 < counterBlock_tag < 0:1013
What do you see instead?
1012 < counterBlock_tag
What version of the product are you using? On what operating system?
latest as of Jan 7th 2015 on centOS 7 minimal (upgraded to latest as of Jan 7th
2015
Please provide any additional information below.
Original issue reported on code.google.com by [email protected]
on 8 Jan 2015 at 5:07
What steps will reproduce the problem?
1. Nginx (1.4.3) compiled with sflow module nginx-sflow-module-0.9.7
2. host sflow agent hsflowd-1.23.4.tar.gz
3.
What is the expected output? What do you see instead?
I see only counter samples, no flow samples.
All the fields releated to sampleType FLOWSAMPLE is missing.
What version of the product are you using? On what operating system?
I am running on Ubbuntu
Please provide any additional information below.
startDatagram =================================
datagramSourceIP 10.0.1.1
datagramSize 448
unixSecondsUTC 1388048293
datagramVersion 5
agentSubId 100000
agent 10.0.1.1
packetSequenceNo 283
sysUpTime 8470000
samplesInPacket 1
startSample ----------------------
sampleType_tag 0:2
sampleType COUNTERSSAMPLE
sampleSequenceNo 283
sourceId 2:1
counterBlock_tag 0:2001
adaptor_0_ifIndex 4
adaptor_0_MACs 1
adaptor_0_MAC_0 0a0027000000
adaptor_1_ifIndex 2
adaptor_1_MACs 1
adaptor_1_MAC_0 28d2441cf152
adaptor_2_ifIndex 1
adaptor_2_MACs 1
adaptor_2_MAC_0 000000000000
adaptor_3_ifIndex 3
adaptor_3_MACs 1
adaptor_3_MAC_0 48d22440bd8c
counterBlock_tag 0:2005
disk_total 98844752896
disk_free 76965779456
disk_partition_max_used 34.44
disk_reads 129884
disk_bytes_read 4901825536
disk_read_time 1542304
disk_writes 45740
disk_bytes_written 2375642112
disk_write_time 599256
counterBlock_tag 0:2004
mem_total 16538497024
mem_free 10718740480
mem_shared 0
mem_buffers 93036544
mem_cached 2841763840
swap_total 4194299904
swap_free 4194299904
page_in 2404447
page_out 1159981
swap_in 0
swap_out 0
counterBlock_tag 0:2003
cpu_load_one 0.710
cpu_load_five 0.400
cpu_load_fifteen 0.350
cpu_proc_run 0
cpu_proc_total 601
cpu_num 8
cpu_speed 2401
cpu_uptime 8650
cpu_user 1542000
cpu_nice 2110
cpu_system 424060
cpu_idle 66219210
cpu_wio 386600
cpuintr 0
cpu_sintr 19110
cpuinterrupts 16062097
cpu_contexts 35214887
counterBlock_tag 0:2006
nio_bytes_in 58593324
nio_pkts_in 65095
nio_errs_in 0
nio_drops_in 0
nio_bytes_out 10417578
nio_pkts_out 54820
nio_errs_out 0
nio_drops_out 0
counterBlock_tag 0:2000
hostname AVNI
UUID 10fcbc9807f4e211bf9428d2441cf152
machine_type 3
os_name 2
os_release 3.8.0-33-generic
endSample ----------------------
endDatagram =================================
startDatagram =================================
datagramSourceIP 10.0.1.1
datagramSize 132
unixSecondsUTC 1388048301
datagramVersion 5
agentSubId 8000
agent 10.0.1.1
packetSequenceNo 15
sysUpTime 0
samplesInPacket 1
startSample ----------------------
sampleType_tag 0:2
sampleType COUNTERSSAMPLE
sampleSequenceNo 15
sourceId 3:8000
counterBlock_tag 0:2002
parent_dsClass 2
parent_dsIndex 1
counterBlock_tag 0:2201
http_method_option_count 0
http_method_get_count 0
http_method_head_count 0
http_method_post_count 0
http_method_put_count 0
http_method_delete_count 0
http_method_trace_count 0
http_methd_connect_count 0
http_method_other_count 0
http_status_1XX_count 0
http_status_2XX_count 0
http_status_3XX_count 0
http_status_4XX_count 0
http_status_5XX_count 0
http_status_other_count 0
endSample ----------------------
endDatagram =================================
startDatagram =================================
datagramSourceIP 10.0.1.1
datagramSize 132
unixSecondsUTC 1388048301
datagramVersion 5
agentSubId 8000
agent 10.0.1.1
packetSequenceNo 15
sysUpTime 0
samplesInPacket 1
startSample ----------------------
sampleType_tag 0:2
sampleType COUNTERSSAMPLE
sampleSequenceNo 15
sourceId 3:8000
counterBlock_tag 0:2002
parent_dsClass 2
parent_dsIndex 1
counterBlock_tag 0:2201
http_method_option_count 0
http_method_get_count 0
http_method_head_count 0
http_method_post_count 0
http_method_put_count 0
http_method_delete_count 0
http_method_trace_count 0
http_methd_connect_count 0
http_method_other_count 0
http_status_1XX_count 0
http_status_2XX_count 0
http_status_3XX_count 0
http_status_4XX_count 0
http_status_5XX_count 0
http_status_other_count 0
endSample ----------------------
endDatagram =================================
startDatagram =================================
datagramSourceIP 10.0.1.1
datagramSize 132
unixSecondsUTC 1388048301
datagramVersion 5
agentSubId 8000
agent 10.0.1.1
packetSequenceNo 15
sysUpTime 0
samplesInPacket 1
startSample ----------------------
sampleType_tag 0:2
sampleType COUNTERSSAMPLE
sampleSequenceNo 15
sourceId 3:8000
counterBlock_tag 0:2002
parent_dsClass 2
parent_dsIndex 1
counterBlock_tag 0:2201
http_method_option_count 0
http_method_get_count 0
http_method_head_count 0
http_method_post_count 0
http_method_put_count 0
http_method_delete_count 0
http_method_trace_count 0
http_methd_connect_count 0
http_method_other_count 0
http_status_1XX_count 0
http_status_2XX_count 0
http_status_3XX_count 0
http_status_4XX_count 0
http_status_5XX_count 0
http_status_other_count 0
endSample ----------------------
endDatagram =================================
startDatagram =================================
datagramSourceIP 10.0.1.1
datagramSize 132
unixSecondsUTC 1388048302
datagramVersion 5
agentSubId 8000
agent 10.0.1.1
packetSequenceNo 15
sysUpTime 0
samplesInPacket 1
startSample ----------------------
sampleType_tag 0:2
sampleType COUNTERSSAMPLE
sampleSequenceNo 15
sourceId 3:8000
counterBlock_tag 0:2002
parent_dsClass 2
parent_dsIndex 1
counterBlock_tag 0:2201
http_method_option_count 0
http_method_get_count 16
http_method_head_count 0
http_method_post_count 0
http_method_put_count 0
http_method_delete_count 0
http_method_trace_count 0
http_methd_connect_count 0
http_method_other_count 0
http_status_1XX_count 0
http_status_2XX_count 15
http_status_3XX_count 1
http_status_4XX_count 0
http_status_5XX_count 0
http_status_other_count 0
endSample ----------------------
endDatagram =================================
Original issue reported on code.google.com by [email protected]
on 26 Dec 2013 at 8:59
Compiled new 0.9.4 version and no more segfaults, etc. Which is great.
BUT:
- i am only getting COUNTERSSAMPLE but i am missing FLOWSAMPLE.
How do i activate nginx to send FLOWSAMPLE, cant figure it out...
Original issue reported on code.google.com by [email protected]
on 1 Jun 2011 at 6:54
What steps will reproduce the problem?
1. Run an sflow-enabled nginx without an existing, valid /etc/hsflowd.auto
2. Lots of noise to error logs
What is the expected output? What do you see instead?
Less noise :)
What version of the product are you using? On what operating system?
0.9.4 with nginx 1.1.0 and 1.1.1
Please provide any additional information below.
I run a lot of different types of servers with the same nginx build. It's
really easy to generate a mountain of log entries to nginx error log on boxes
that don't expect to be running sflow and /etc/hsflowd.auto doesn't exist (and
if it does exist but is empty, I get piles of errors like "syntax error :
agentIP=<IP address>|<IPv6 address> (on line 0) while logging request...").
I think it would be less surprising to default to 'sflow off;' and have it
activated with 'sflow on;', rather than defaulting to on. Or alternatively, if
/etc/hsflowd.auto exists but is empty, then disable sflow logging.
Love the module btw!
Original issue reported on code.google.com by [email protected]
on 31 Aug 2011 at 9:02
I am trying to use nginx-sflow module with Ganglia-3.4.0 I cannot get it to
work.
Looking at ganglia code it looks like it expects HID to be passed to it.
However, the nginx-sflow module never passes that.
Original issue reported on code.google.com by [email protected]
on 7 Sep 2012 at 11:52
What steps will reproduce the problem?
1. "nginx -s quit" will put worker into "shutting down" state which never
finishes
2. "nginx -s stop" stops nginx successfully
What is the expected output? What do you see instead?
I would expect nginx workers shut down. Instead they hang in "shutting down"
status
What version of the product are you using? On what operating system?
0.9.7, Centos 6
Please provide any additional information below.
Original issue reported on code.google.com by [email protected]
on 5 Jan 2014 at 2:45
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.