[Ntop] Ntop NetFlow Question

Fri Feb 13 01:52:07 CET 2009

Post or email me all details of your env and conf and ill try to help 

________________________________

From: ntop-bounces at unipi.it 
To: ntop at unipi.it 
Sent: Thu Feb 12 17:51:32 2009
Subject: Re: [Ntop] Ntop NetFlow Question 

I would normally agree, and although I dont have all the answers I do know our network is somewhat different. in this environment the heaviest talkers are machines that send tons, and tons of tiny packets (basically exactly what you describe below :) ). What are you using as a collector? As i mentioned maybe ntop can handle this, but using flow-tools and flowscan/cuflow definately couldnt hang, which is all perl based.

On Thu, Feb 12, 2009 at 3:42 PM, Gary Gatten <Ggatten at waddell.com> wrote:

	Something is not right! 500Mb/s is "nothing" if the traffic is typical and not 500Mb/s of 64byte packets all with unique source and destination info – Ie: some sort of DoS or test environment.

	The POS box I have sees peaks of 700Mb/s and still only uses maybe 60% cpu.  What are your rrd configs?  Maybe that's what's slowing everything down?  If you're using "full" and saving hosts, interfaces, etc. etc. – that could be it.  You're 2.4GHz system should EASILY handle 500Mb/s using netflow and not even break a sweat.

	G

________________________________

	From: ntop-bounces at unipi.it [mailto:ntop-bounces at unipi.it] On Behalf Of Kyle McLerren
	Sent: Thursday, February 12, 2009 5:30 PM

	To: ntop at unipi.it
	Subject: Re: [Ntop] Ntop NetFlow Question

	Well hopefully I can look forward to such a feature in the future. I'll play more with tweaking some settings, but in an environment like ours, processing flows on over 500Mbps worth of traffic is... interesting :) Even on a quad-core 2.4ghz collector with 4gb of ram and 10k SAS disks the flows were taking 10 minutes a piece to process with sampling turned off. 

	Might just have to bite the bullet and move to some expensive but robust solution :) thanks again.

	On Thu, Feb 12, 2009 at 3:20 PM, Gary Gatten <Ggatten at waddell.com> wrote:

	Seems like it would be simple – multiply rx stats by sample rate before storing/graphing?  I don't do much development so who knows.

	I would maybe try without sampling, but maybe set your active/inactive timers to 120/60?  It's not as real-time as it could be, but if you have a ton of dynamic traffic it will slow down the flow rate.

	I'm only seeing traffic of around 100Kpps, but I'm on an old P-III and it's only using 25% during peaks.  Surely a fast box can keep up with netflow exports from really high util – unless every packet is a different "flow" – like during some sort of DoS attack.

	G

________________________________

	From: ntop-bounces at unipi.it [mailto:ntop-bounces at unipi.it] On Behalf Of Kyle McLerren
	Sent: Thursday, February 12, 2009 4:03 PM

	To: ntop at unipi.it
	Subject: Re: [Ntop] Ntop NetFlow Question

	Thanks for the response. We are doing a boat load of traffic. We had to enable sampling because our previous collector could no longer cope with the sheer volume of flows it was trying to process. I do admit I dont know if ntop can process the flows better as I just started to use it. Previous we were using flow-tools and flowscan/cuflow. I should not have said "accurate," as they are prefectly accurate. It just others get confuse when looking at them and it gets old telling people to keep in mind the data is accurate, its just the numbers are "smaller" then they actually really are :)

	so it looks like the answer is no, there isnt a way to set the sample rate.. seems like a really basic and easy feature to implement, would be great to see it!

	thanks again.

	On Thu, Feb 12, 2009 at 1:54 PM, Gary Gatten <Ggatten at waddell.com> wrote:

	How'd anyone ever function without Google?

	http://www.mail-archive.com/ntop@unipi.it/msg11605.html

________________________________

	From: ntop-bounces at unipi.it [mailto:ntop-bounces at unipi.it] On Behalf Of Gary Gatten
	Sent: Thursday, February 12, 2009 3:49 PM
	To: ntop at unipi.it
	Subject: Re: [Ntop] Ntop NetFlow Question

	I have seen this asked and I think answered a couple times, but since I don't do sampling I wasn't paying much attention.  I'm sure if you searched the list you'd see some answers from Luca and/or Burton.  I searched the FAQ and man page and couldn't find anything.  Also checked the "Preferences" and couldn't see anything there either.  MAYBE check the globals-define.h; there's lots of cool stuff in there but you have to recompile with changes :-(

	Of course the easy answer is to NOT sample.  Unless you have a $HIT load of traffic it will be OK, especially if you set the active/inactive export timers to something reasonable.

	BTW: What is "not accurate"?

________________________________

	From: ntop-bounces at unipi.it [mailto:ntop-bounces at unipi.it] On Behalf Of Kyle McLerren
	Sent: Thursday, February 12, 2009 3:13 PM
	To: ntop at unipi.it
	Subject: [Ntop] Ntop NetFlow Question

	Hi All,

	Im sure this has been answered before, but I couldnt find an answer anywhere. I use sampled netflow, and I simply wanted to know if theres an option with the ntop netflow plugin to configure the sample rate? Otherwise, my results arent acurate. Im sending 1 out of 100 from my router.

	thanks!

	"This email is intended to be reviewed by only the intended recipient and may contain information that is privileged and/or confidential. If you are not the intended recipient, you are hereby notified that any review, use, dissemination, disclosure or copying of this email and its attachments, if any, is strictly prohibited. If you have received this email in error, please immediately notify the sender by return email and delete this email from your system." 

	"This email is intended to be reviewed by only the intended recipient and may contain information that is privileged and/or confidential. If you are not the intended recipient, you are hereby notified that any review, use, dissemination, disclosure or copying of this email and its attachments, if any, is strictly prohibited. If you have received this email in error, please immediately notify the sender by return email and delete this email from your system." 

	_______________________________________________
	Ntop mailing list
	Ntop at unipi.it
	http://listgateway.unipi.it/mailman/listinfo/ntop

	"This email is intended to be reviewed by only the intended recipient and may contain information that is privileged and/or confidential. If you are not the intended recipient, you are hereby notified that any review, use, dissemination, disclosure or copying of this email and its attachments, if any, is strictly prohibited. If you have received this email in error, please immediately notify the sender by return email and delete this email from your system." 

	_______________________________________________
	Ntop mailing list
	Ntop at unipi.it
	http://listgateway.unipi.it/mailman/listinfo/ntop

	"This email is intended to be reviewed by only the intended recipient and may contain information that is privileged and/or confidential. If you are not the intended recipient, you are hereby notified that any review, use, dissemination, disclosure or copying of this email and its attachments, if any, is strictly prohibited. If you have received this email in error, please immediately notify the sender by return email and delete this email from your system." 

	_______________________________________________
	Ntop mailing list
	Ntop at unipi.it
	http://listgateway.unipi.it/mailman/listinfo/ntop

<font size="1">
<div style='border:none;border-bottom:double windowtext 2.25pt;padding:0in 0in 1.0pt 0in'>
</div>
"This email is intended to be reviewed by only the intended recipient
 and may contain information that is privileged and/or confidential.
 If you are not the intended recipient, you are hereby notified that
 any review, use, dissemination, disclosure or copying of this email
 and its attachments, if any, is strictly prohibited.  If you have
 received this email in error, please immediately notify the sender by
 return email and delete this email from your system."
</font>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://listgateway.unipi.it/pipermail/ntop/attachments/20090212/eab86951/attachment-0001.html 