[Ntop] packet dropped

Thu Oct 6 11:16:44 CEST 2005

Hi

I installed ntop and when it runs it claims that libpcap drops about 80% of
the packets on a 100Mbit full duplex interface. I'm trying to find out why
but can't. Can anyone give me pointers on what to look for?
Until then here is some information on the environment (hardware and
software) that ntop is running in:
1. CPU: Pentium 4 (2.8 GHz): The load on the CPU is always 1 (or close to
it) with ntop taking 99.9% of the CPU (should it be like this?)
2. RAM: 512MB: of which about half is used and half is free (swap isn't
used).
3. Linux: Fedora Core 4. running the minumum processes needed (kernel, ntop,
ssh).
3. NTOP version is: 3.1.50 running with following command line:
/usr/local/bin/ntop -u ntopuser -w
192.168.240.3:80<http://192.168.240.3:80>-i eth1 -P/var/ntop

HD (from dmesg):
ide0: BM-DMA at 0x24c0-0x24c7, BIOS settings: hda:DMA, hdb:pio
hda: ST340014A, ATA DISK drive
hda: max request size: 1024KiB
hda: 78165360 sectors (40020 MB) w/2048KiB Cache, CHS=16383/255/63,
UDMA(100)
hda: cache flushes supported
hda: hda1 hda2 hda3 hda4
SELinux: initialized (dev hda1, type ext3), uses xattr
EXT3 FS on hda1, internal journal
EXT3 FS on hda2, internal journal
SELinux: initialized (dev hda2, type ext3), uses xattr

ntop configuration:
 ntop Configuration

 Basic Information ntop Version 3.1.50 Configured on Aug 4 2005 11:32:47 Built
on Aug 4 2005 11:35:48 OS i686-pc-linux-gnu libpcap version libpcap version
0.8.3 Running from /usr/local/bin Libraries in /usr/local/lib Process Id
26604 Command line Started as.... /usr/local/bin/ntop -u ntopuser -w
192.168.240.3:80 <http://192.168.240.3:80> -i eth1 -P/var/ntop Resolved
to.... /usr/local/bin/ntop -u ntopuser -w
192.168.240.3<http://192.168.240.3>-i eth1 -P/var/ntop Preferences
used NOTE: * (effective) means that this is the value after ntop has
processed the parameter.(default) means this is the default value, usually
(but not always) set by a #define in globals-defines.h.*  -a |
--access-log-file (default) (nil) -b | --disable-decoders (default) No -c |
--sticky-hosts (default) No -d | --daemon No -e | --max-table-rows (default)
128 -f | --traffic-dump-file (default) (nil) -g | --track-local-hosts (default)
Track all hosts -o | --no-mac (default) Trust MAC Addresses -i | --interface
(effective) eth1 -j | --create-other-packets (default) Disabled -l |
--pcap-log (default) (nil) -m | --local-subnets (effective) (default) (nil) -n
| --numeric-ip-addresses (default) No -p | --protocols (default) internal
list -q | --create-suspicious-packets (default) Disabled -r |
--refresh-time (default)
120 -s | --no-promiscuous (default) No -t | --trace-level (default) 3 -u |
--user ntopuser (uid=80, gid=503) -w | --http-server Active, address
192.168.240.3 <http://192.168.240.3>, port 80 -z | --disable-sessions (default)
No -B | --filter-expression (default) none -D | --domain
hot.net.il<http://hot.net.il> -F
| --flow-spec (default) none -K | --enable-debug (default) No -L |
--use-syslog daemon -M | --no-interface-merge (effective) (default) (Merging
Interfaces) Yes -N | --wwn-map (default) (nil) -O | --pcap-file-path (default)
/var/ntop -P | --db-file-path (default) /var/ntop -Q |
--spool-file-path (default)
/var/ntop -U | --mapper (default) (nil) -W | --https-server Uninitialized
--disable-schedYield Yes --disable-instantsessionpurge Yes
--disable-mutexextrainfo Yes --disable-stopcap Yes --fc-only (default) No
--instance (default) (nil) --no-fc (default) No --no-invalid-lun (default)
No --p3p-cp (default) none --p3p-uri (default) none --pcap-nonblocking
(default)
No --skip-version-check Yes --ssl-watchdog (default) No --w3c Yes NOTE: *The
--w3c flag makes the generated html MORE compatible with the w3c
recommendations, but it in no way addresses all of the compatibility and
markup issues. We would like to make ntop more compatible, but some basic
issues of looking decent on real-world browsers mean it will never be 100%.
If you find any issues, please report them to
ntop-dev<http://lists.ntop.org/mailman/listinfo/ntop-dev>.
*  Run time/Internal Web server URL http://192.168.240.3:80 SSL Web server
(https://) Not Active GDBM version This is GDBM version 1.8.0, as of May 19,
1999. OpenSSL Version OpenSSL 0.9.7f 22 Mar 2005 zlib version
1.2.2.2<http://1.2.2.2> gd
version (guess) 2.x Protocol Decoders Enabled Fragment Handling
Enabled Tracking
only local hosts No # IP Protocols Being Monitored 20 # Protocol slots 978 #
IP Ports Being Monitored 177 # IP Ports slots 354 WebServer Request
Queue 10 Devices
(Network Interfaces) 1 Domain name (short) il IP to country flag table
(entries) 52395 Total Hash Collisions (Vendor/Special) (lookup) 0 ntop Web
Server Item http:// https:// # Handled Requests 133 - # Successful requests
(200) 132 - # Bad (We don't want to talk with you) requests 0 - # Invalid
requests - 403 FORBIDDEN 0 - # Invalid requests - 404 NOT FOUND 0 - NOTE:

   - *Counts may not total because of in-process requests.*
   - *Each request to the ntop web server - page, chart, etc. is counted
   separately*

  # SSI Requests 0 # Bad SSI Requests 0 # Handled SSI Requests 0 # Handled
SIGPIPE Errors 0 Memory allocation - data segment arena limit,
getrlimit(RLIMIT_DATA, ...) -1 Allocated blocks (ordblks) 317 Allocated
(arena) 16392192 Used (uordblks) 16260824 Free (fordblks) 131368 Memory
allocation - mmapped Allocated blocks (hblks) 6 Allocated bytes (hblkhd)
5287936 Host Memory Cache Limit #define MAX_HOSTS_CACHE_LEN 512 Current Size
0 Maximum Size 0 # Entries Reused 0 Packets Received 3,076,586 Processed
Immediately 3,076,586 (100.0 %) Queued 0 (0.0 %) Current Queue 0 Maximum
Queue 0 (Limit 2048) Packet Processing Queue (pre-process) Processing
Minimum 0.024183 0.000013 Average 0.027177 0.000091 Maximum 0.030567
0.000348 Standard Deviation 0.001352 0.000037 Maximum ever 0.400866
0.203713 Min
Estimated Thpt (pps) 0.000000 3271.501953 Average Estimated Thpt (pps)
0.000000 11020.000000 NOTE: *'Queue' time is the elapsed time between the
packet arrival (libpcap) and the gettimeofday() value as the packet starts
processPacket(). For a queued packet, this includes the time in queue.

'Processing' time is the elapsed time between starting and finishing
processPacket(). Errors and/or unrecognized packets may cause processing to
be abandoned and those packets are not counted in the 'processing' averages.
This means that the 1024 packets for the 'queue' and 'processing'
calculations are not necessarily the same physical packets, and may lead to
over estimation of the per-packet 'processing' time.

Small averages are good, especially if the standard deviation is small
(standard deviation is a measurement of the variability of the actual values
around the average). The computations are based only on the most recent 1024
packets processed.

Maximum ever ignores the first 100 packets for each device - this lets
ntopget over startup agony.

What does this mean? Not much. Still, 1/(queue-average+process-average) (i.e.
36.7) gives a very rough indication of the packet per second rate this
instance of ntop can handle.*  Host/Session counts - global Purged
Hosts 0 Multi-VLANed
Hosts 1067 Terminated Sessions 0 Host/Session counts - Device 0 (eth1) Hash
Bucket Size 1.9 KB Actual Host Hash Size 32768 Stored hosts 3190 Host Bucket
List Length [min 1][max 9][avg 1.1] Max host lookup 8 Session Bucket Size
264 Session Actual Hash Size 65535 Sessions 0 Max Num. Sessions 0 Session
Bucket List Length [min 4294967295][max 0][avg 1.1] ----- Address Resolution
----- DNS Sniffing (other hosts requests) DNS Packets sniffed 4081 DNS
Packets processed 668 Stored in cache (includes aliases) 757 Queued -
dequeueAddress() Total Queued 3835 Not queued (duplicate) 0 Maximum
Queued 1 Current
Queue 0 DNS Lookup Calls: DNS resolution attempts 3835 ....Success: Resolved
1 ....Failed 3834 DNS lookups stored in cache 1 Host addresses kept numeric
3834 NOTE: *'DNS lookups stored in cache' includes HOST_NOT_FOUND replies.
Thus it may be larger than the number of 'Success: Resolved' queries. *  Thread
counts Active 8 Dequeue 1 Children (active) 28 Directory (search) order Data
Files .
/usr/local/share/ntop Config Files .
/etc/ntop
/etc Plugins ./plugins
/usr/local/lib/ntop/plugins NOTE: *REMEMBER that the . (current working
directory) value will be different when you run ntop from the command line
vs. a cron job or startup script!*  Compile Time: ./configure ./configure
parameters --sysconfdir=/etc --localstatedir=/var +
configureextra/LINUXfedora Built on (Host) i686-pc-linux-gnu Built
for(Target) i686-pc-linux-gnu preprocessor (CPPFLAGS) gcc -E -DLINUX
-I/usr/local/include compiler (CFLAGS) gcc -g -O2 -I/usr/local/include
-Wshadow -Wpointer-arith -Wmissing-prototypes -Wmissing-declarations
-Wnested-externs -fPIC -DPIC -DHAVE_CONFIG_H include path (nil) system
libraries -L/usr/local/lib -lxml2 -lpthread -lresolv -lnsl -lcrypt -lc -lssl
-lcrypto -lpcap -lgdbm -lgd -lpng -lz install path /usr/local GNU C (gcc)
version 4.0.0 20050519 (Red Hat 4.0.0-8) (4.0.0) uname data sysname(Linux)
release(2.6.11-1.1369_FC4) version(#1 Thu Jun 2 22:55:56 EDT 2005)
machine(i686) Internationalization (i18n) i18n enabled No

[ Click here <http://192.168.240.3/textinfo.html> for a more extensive, text
version of this page, suitable for inclusion into a bug report ]
 ------------------------------
* Report created on Thu Oct 6 12:15:15 2005 [ntop uptime: 6:06]
Generated by ntop <http://www.ntop.org/> v.3.1.50 [i686-pc-linux-gnu]
(c) 1998-2005 by Luca Deri <deri at ntop.org>, built: Aug 4 2005 11:35:48.
Listening on [eth1] for all packets (i.e. without a filtering expression)
Web reports include all interfaces (merged)*
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://listgateway.unipi.it/pipermail/ntop/attachments/20051006/b06aae24/attachment-0001.htm