[Ntop] using ntop to see flows report

Ricky Charlet rcharlet at mudynamics.com
Sat Sep 10 01:05:59 CEST 2011 

Gary,
	Your suggestions are extremely helpful. I bumped into a project deadline here; I'm gonna still try and implement with your suggestion and see what happens. But I may not get to results and replying until after next week.

--
Live strong,
Ricky Charlet






On Sep 9, 2011, at 3:03 PM, Gary Gatten wrote:

> PS: Careful with sticky hosts - stuff NEVER goes away which means you need LOTS of RAM and/or restart ntop often.  But, in your scenario it may work perfect, especially if you don't have many hosts and/or they are fairly stable IP's and such.  Some people enable sticky and monitor internet traffic for their org.  It doesn't take long before ntop is trying to track and store info about 200K hosts - or more.  And it just keeps growing and growing until you get a malloc abend or fill up your disk, whichever comes first.  Not good.
> 
> G
> 
> 
> -----Original Message-----
> From: ntop-bounces at listgateway.unipi.it [mailto:ntop-bounces at listgateway.unipi.it] On Behalf Of Gary Gatten
> Sent: Friday, September 09, 2011 4:54 PM
> To: 'ntop at unipi.it'
> Subject: Re: [Ntop] using ntop to see flows report
> 
> Ok....  Couple options.
> 
> 1.) Check out sticky hosts.  If you enable that the "global" tcp/udp stats MAY stay around "forever" as well, not 100% sure on that.  No recompile needed here.
> 
>        - Each host report also has a similar table (TCP/UDP Service/Port Usage), perhaps that would be useful if it was persistent?
> 
> 2.) Check out options in globals-defines.h
>        - idle purge timers; may need to tweak for 48 hours.
>        - max numbers defs for various things
> 
> 3.) Tracking of tcp/udp ports > 1023.  This is disabled by default, but not sure if it's client and server side, or just one.  I only played with it a bit.
> 
> 4.) "wget"  Some people have used wget to fetch a page every n secs / mins and save that info to create the history type reports.
> 
> 
> 
> -----Original Message-----
> From: ntop-bounces at listgateway.unipi.it [mailto:ntop-bounces at listgateway.unipi.it] On Behalf Of Ricky Charlet
> Sent: Friday, September 09, 2011 4:39 PM
> To: <ntop at unipi.it>
> Subject: Re: [Ntop] using ntop to see flows report
> 
> Thanks again Gary,
> 
>        I appreciate the suggestion that ntop can hold stats for longer based on recompiles across edits of something or other in globals-defines.h. I will start seeking. Any more specific advice would be appreciated.
> 
> 
>        However, in the meantime... not sure why I did not see this before, but the"TCP/UDP Traffic Port Distribution: Last Minute View" section of the Summary::Traffic report is (almost) exactly what I want. I just need it to persist a little longer. Can I get a "last day" view or a "last two days" view?
> 
> 
> --
> Live strong,
> Ricky Charlet
> 
> 
> 
> 
> 
> 
> On Sep 9, 2011, at 12:33 PM, Gary Gatten wrote:
> 
>> I can't comment on the dev part - I'm just a lowly user!
>> 
>> Ntop CAN hold some stats for 24+ hours, but requires some tweaks via startup args and/or globals-defines.h (which requires a recompile).  Ie:, sticky hosts, idle session purge time, idle host purge time, etc.
>> 
>> FWIW I think Wireshark will do what you want more easily.  Are you familiar with it?  Most just use it as a simple packet capture tool, but it tracks all kinds of stats and has flow/ conversation "reports" - such as this flow has x Bytes TX and y bytes RX.
>> 
>> G
>> 
>> 
>> -----Original Message-----
>> From: ntop-bounces at listgateway.unipi.it [mailto:ntop-bounces at listgateway.unipi.it] On Behalf Of Ricky Charlet
>> Sent: Friday, September 09, 2011 2:04 PM
>> To: <ntop at unipi.it>
>> Cc: ntop at listgateway.unipi.it
>> Subject: Re: [Ntop] using ntop to see flows report
>> 
>> Thanks Gary,
>> 
>>      I have a traffic generator application. It simulates many thousands of clients and servers -- lots and lots of unique flows, somewhat distinguised by dest-port (app) but mosty distinguished by source port. I what to know what it did for the last run of (1 ~ 60*24) minutes.
>> 
>>      During my traffic generator testrun, I do see "Active Sessions" at the bottom of a Hosts::<click-an-ip> report.
>> 
>>      I need bytes sent/received per interface  per flow. I would also appreciate tcp retransmission counts, flow counts, interface error counts. I am willing to go with the roll-my-own rdd-querries with few helpful hints from the community if that is what it takes.
>> 
>>      Please let me know if you see a path to victory here and if ntop-dev team would be willing to make it so.
>> 
>> --
>> Live strong,
>> Ricky Charlet
>> 
>> 
>> 
>> 
>> 
>> 
>> On Sep 9, 2011, at 11:28 AM, Gary Gatten wrote:
>> 
>>> Hello,
>>> 
>>> I don't THINK the report you seek exists.  First, ignore "Summary -> Network Flows".  It has nothing to do with anything - see the FAQ.
>>> 
>>> Next, check to see if whatever version of ntop you're using is actually tracking flows; aka tcp/udp sessions.  Select a busy host and scroll to the bottom of the report.  If you don't see a bunch of active sessions, you're screwed.  If they ARE there AND you have rrd configured, you MAY be able to get what you want with rrd queries, but I doubt it....
>>> 
>>> If you can tell me what problem you're trying to solve I can maybe recommend an alternative view / report.  That said, ntop is TYPICALLY best at "real-time" reporting and not so good at reporting on historical stuff, especially detailed history such as flow/conversation info.
>>> 
>>> HOWEVER  :)  There are subsets of ntop that are exposed via Python, snmp, http, etc. - it's possible to create the reports you want - but I really don't think it's possible with shipping code.
>>> 
>>> G
>>> 
>>> 
>>> -----Original Message-----
>>> From: ntop-bounces at listgateway.unipi.it [mailto:ntop-bounces at listgateway.unipi.it] On Behalf Of Ricky Charlet
>>> Sent: Friday, September 09, 2011 12:44 PM
>>> To: ntop at listgateway.unipi.it
>>> Subject: [Ntop] using ntop to see flows report
>>> 
>>> Howdy,
>>> 
>>>     I'm new to ntop (for about 20 hours so far). But I know my way around compiling/unixOS/networking very well.
>>> 
>>>     I can't quite figure out how to find a report in ntop which shows a historic list of flows. I do have several nifty reports like Summary::Traffic, Summary::Hosts, Summary::NetworkLoad, AllProtocols::Traffic and more. But some of the reports are either missing or empty. In particular, I very much want to see a flows report (that is sort of the reason why I started experimenting with an ipfix probe/collector).
>>> 
>>>     So, it turns out that bothSummary:NetworkFlows and Utils::Datadump::NetworkFlows are empty. Just judging by the name, I think those are the reports I'm interested in.
>>> 
>>>     Note that I have already found my way into Pluggins::RDD::Configure and enabled DataToDump=(flows, subnets, hosts, interfaces).
>>> 
>>>     I'm not sure if I'm chasing the 'right' reports and if so, if I have correct or incorrect config. Please help. For reasons beyond my control, I need a project answer here within a few hours :-(
>>> 
>>>     I can post any config, log upon request.
>>> 
>>> 
>>> --
>>> Live strong,
>>> Ricky Charlet
>>> 
>>> 
>>> 
>>> 
>>> 
>>> 
>>> _______________________________________________
>>> Ntop mailing list
>>> Ntop at listgateway.unipi.it
>>> http://listgateway.unipi.it/mailman/listinfo/ntop
>>> 
>>> 
>>> 
>>> 
>>> 
>>> <font size="1">
>>> <div style='border:none;border-bottom:double windowtext 2.25pt;padding:0in 0in 1.0pt 0in'>
>>> </div>
>>> "This email is intended to be reviewed by only the intended recipient
>>> and may contain information that is privileged and/or confidential.
>>> If you are not the intended recipient, you are hereby notified that
>>> any review, use, dissemination, disclosure or copying of this email
>>> and its attachments, if any, is strictly prohibited.  If you have
>>> received this email in error, please immediately notify the sender by
>>> return email and delete this email from your system."
>>> </font>
>>> 
>>> _______________________________________________
>>> Ntop mailing list
>>> Ntop at listgateway.unipi.it
>>> http://listgateway.unipi.it/mailman/listinfo/ntop
>> 
>> _______________________________________________
>> Ntop mailing list
>> Ntop at listgateway.unipi.it
>> http://listgateway.unipi.it/mailman/listinfo/ntop
>> 
>> 
>> 
>> 
>> 
>> <font size="1">
>> <div style='border:none;border-bottom:double windowtext 2.25pt;padding:0in 0in 1.0pt 0in'>
>> </div>
>> "This email is intended to be reviewed by only the intended recipient
>> and may contain information that is privileged and/or confidential.
>> If you are not the intended recipient, you are hereby notified that
>> any review, use, dissemination, disclosure or copying of this email
>> and its attachments, if any, is strictly prohibited.  If you have
>> received this email in error, please immediately notify the sender by
>> return email and delete this email from your system."
>> </font>
>> 
>> _______________________________________________
>> Ntop mailing list
>> Ntop at listgateway.unipi.it
>> http://listgateway.unipi.it/mailman/listinfo/ntop
> 
> _______________________________________________
> Ntop mailing list
> Ntop at listgateway.unipi.it
> http://listgateway.unipi.it/mailman/listinfo/ntop
> 
> 
> 
> 
> 
> <font size="1">
> <div style='border:none;border-bottom:double windowtext 2.25pt;padding:0in 0in 1.0pt 0in'>
> </div>
> "This email is intended to be reviewed by only the intended recipient
> and may contain information that is privileged and/or confidential.
> If you are not the intended recipient, you are hereby notified that
> any review, use, dissemination, disclosure or copying of this email
> and its attachments, if any, is strictly prohibited.  If you have
> received this email in error, please immediately notify the sender by
> return email and delete this email from your system."
> </font>
> 
> _______________________________________________
> Ntop mailing list
> Ntop at listgateway.unipi.it
> http://listgateway.unipi.it/mailman/listinfo/ntop
> 
> 
> 
> 
> 
> <font size="1">
> <div style='border:none;border-bottom:double windowtext 2.25pt;padding:0in 0in 1.0pt 0in'>
> </div>
> "This email is intended to be reviewed by only the intended recipient
> and may contain information that is privileged and/or confidential.
> If you are not the intended recipient, you are hereby notified that
> any review, use, dissemination, disclosure or copying of this email
> and its attachments, if any, is strictly prohibited.  If you have
> received this email in error, please immediately notify the sender by
> return email and delete this email from your system."
> </font>
> 
> _______________________________________________
> Ntop mailing list
> Ntop at listgateway.unipi.it
> http://listgateway.unipi.it/mailman/listinfo/ntop

More information about the Ntop mailing list