[Ntop-misc] why is PF_RING limited to only allocating up to
4MBofring slots
Rob Willis
rob.willis at quest.com
Tue Jun 27 18:30:51 CEST 2006
That would be great if you could send the source. There is definitely a limit as far as that fucntion goes, so would love to see your implementation.
Thanks,
-Rob
-----Original Message-----
From: Amitava Biswas [mailto:solutions_amit at hotmail.com]
Sent: Tue 6/27/2006 11:38 AM
To: ntop-misc at listgateway.unipi.it
Cc: Rob Willis
Subject: Re: [Ntop-misc] why is PF_RING limited to only allocating up to 4MBofring slots
Re: [Ntop-misc] why is PF_RING limited to only allocating up to 4MB ofring slotsWell I realized you are right about that 4Gb limit.
during writing it slipped my mind the units, now a days I am away from kernel coding
well you can used my own kernel routine i used instead of _get_free...
if you want I can send that code and the design explanation, if it helps. I get the pages in chunks of non contiguous segments, and therefore is less likely to hit any limit. however the chunks are fairly non contiguous to ensure that a buffer has 1.5Kb space. you can tune the parameters, you can experiment with that code to see whether its the lack of big non- contiguous segment availibility that is causing any problem, or is it some other limit.
i traced the existing kernel API long back, but i hardly remember whether there was such limit there or not, may be you can trace the source code also to verify.
----- Original Message -----
From: Rob Willis
To: ntop-misc at listgateway.unipi.it
Sent: Tuesday, June 27, 2006 8:51 AM
Subject: Re: [Ntop-misc] why is PF_RING limited to only allocating up to 4MBofring slots
> I think you may be speaking of the 4/gb/ limit instead of /mb/.
Nope. This kernel has the 64 GB option enabled.
I'm definitely refering to the max amount of memory that will
be allocated for ring slots. I've modified the # of slots via
modprobe.conf. That definitely works for me. However, after a
certain point, the amount of memory (and hence the # of slots
available for buffering) is capped at 4 MB due to __get_free_pages
not returning anything larger than 4 MB.
Since that is a low level kernel call, probably not much you can do
to change its behaviour. However, PF_RING could probably be modified
to create multiple large chunks of memory. Rings slots within each chunk
would be contiguous but at chunk boundaries there would be a gap.
This would mean pf_ring would make multiple calls to __get_free_pages and
have to manage these multiple chunks - tracking which ring slots are on
which chunks. Not sure if this is even possible since the device driver
is writing to this memory... Hummm... Maybe not so easy unless it is
possible to make the device driver aware of multiple blocks of memory.
Anyways, I'm not a kernel hacker... Hoping for some guidance on this
point (can it be done?). If someone with more knowledge thinks it can
be done, I might try my hand at it...
-Rob
------------------------------------------------------------------
Re: [Ntop-misc] why is PF_RING limited to only allocating up to 4MB ofring slots
Nick Whitehill
Mon, 26 Jun 2006 19:50:27 -0700
Hi,
I think you may be speaking of the 4/gb/ limit instead
of /mb/.
You can add additional ring slots in modprobe.conf. I
cannot remember the syntax but will report back when I
get to work tomorrow. I get the slotsize vs ring size
confused but I know that once we moved to 16384k
ringsize we no longer dropped packets. We are now
running multiple instances of snort on several
interfaces so we were able to remove the 16384 entry
in modprobe.conf as each instance of a libpcap'd
program calls an additional "ring buffer".
Luca, please correct me if the above statement is off
the mark. I have been using PF_RING with tcpdump and
snort for over a year now and I am only reporting my
experience.
Thanks again for the wiki and thanks to Luca for
PF_RING!!
Cheers,
Nick
--- Amitava Biswas <[EMAIL PROTECTED]> wrote:
> why is PF_RING limited to only allocating up to 4MB
> of ring slotsWell I do not know much about 2.6
> kernels,
> but a few observations which might be the cause-
> 1) The get free pages routine has non deterministic
> behavior, I mean its a best effort process, when I
> wanted to get very big chunks of contiguous blocks,
> the process failed because there wasnt any such
> bloocks left. however it runs ok for smaller chunks,
> its expected as bigger contiguos blocks may not be
> available at all,
>
> 2) There may be a limit programed in the process
> itself, because 4 Mb was the kernel memory limit for
> older kernels, though it should be taken care off if
> you enable larger memory support option during
> kernel compile.
>
> and by the way you can use a different option to
> reduce memory requirement, try my posting on DMA
> ring, i wrote couple of papers on it and notified
> the user group about that approach, this approach
> cut down the memory requirement drasticaly and
> improved the performance on the other hand.
>
> allocating big memory blocks might create problem to
> other processes or kernel activities, so should take
> care about it.
> ----- Original Message -----
> From: Rob Willis
> To: ntop-misc at listgateway.unipi.it
> Sent: Monday, June 26, 2006 6:24 PM
> Subject: [Ntop-misc] why is PF_RING limited to
> only allocating up to 4MB ofring slots
>
>
>
>
> (Hopefully, this doesnt turn into a double post
> since I sent an earlier email via my home email
> account by accident).
>
> I've been playing with PF_RING (with positive
> results). However it seems to be limited to only
> allocating enough slots up to 4MB of memory. I'm on
> FedoraCore4, 2.6.12.
>
> I took a look at the RF_RING code and it is
> calling __get_free_pages(priority,order) where order
> represents the number of kernel memory pages to
> return: 2^order. So pages returned:
>
> order=0: 1 page
> order=1: 2 pages
> order=2: 4 pages
> ...
> And in my case, PF_RING decides on an order of 10:
> order=10: 1024 pages
>
> Since each page is 4kb, that means it returns 4 MB
> of memory for use as ring slots. My ring slot size
> is set to 1518 (just big enough for an 802.1q
> packet).
>
> Anyways, even when I request 8MB or 16 MB, this
> function always returns 4 MB (ie it always uses an
> order of 10). Is there another option to get
> PF_RING to allocate more memory? I've got 8 GB on
> this system and need to do some fairly indepth
> analysis of packets. So would like to have a very
> large buffer allocated.
>
> If there is some reason that __get_free_pages can
> only return 4 MB blocks, perhaps PF_RING could be
> reworked to manage multiple large blocks. Then
> could call __get_free_pages multiple times and put
> all of the blocks in a array. Would mean an extra
> level of indirection when referencing a ring slot,
> but that doesnt seem like that much more overhead.
>
> I might even be willing to dive in and come up
> with a patch to do just that if someone told me that
> was even possible. And if they said there wasnt an
> easier way to get PF_RING to allocate more memory.
>
> -Rob
>
>
>
>
>
------------------------------------------------------------------------------
>
>
> _______________________________________________
> Ntop-misc mailing list
> Ntop-misc at listgateway.unipi.it
>
>
http://listgateway.unipi.it/mailman/listinfo/ntop-misc
>
> > _______________________________________________
> Ntop-misc mailing list
> Ntop-misc at listgateway.unipi.it
>
http://listgateway.unipi.it/mailman/listinfo/ntop-misc
>
__________________________________________________
Do You Yahoo!?
Tired of spam? Yahoo! Mail has the best spam protection around
http://mail.yahoo.com
_______________________________________________
Ntop-misc mailing list
Ntop-misc at listgateway.unipi.it
http://listgateway.unipi.it/mailman/listinfo/ntop-misc
------------------------------------------------------------------------------
_______________________________________________
Ntop-misc mailing list
Ntop-misc at listgateway.unipi.it
http://listgateway.unipi.it/mailman/listinfo/ntop-misc
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://listgateway.unipi.it/pipermail/ntop-misc/attachments/20060627/3d66a24f/attachment.htm
More information about the Ntop-misc
mailing list