Inconsistency between statistics.AcceptedPackets and communicator.ReceivePackets

Aug 12, 2013 at 5:18 PM
Edited Aug 12, 2013 at 5:27 PM
Hello,

I have a problem with this code:
class Program
{
    static void Main(string[] args)
    {
        LivePacketDevice device = LivePacketDevice.AllLocalMachine[0];
        ProcessPackets(device);
    }

    private static void ProcessPackets(IPacketDevice device)
    {
        Thread statisticsThread = new Thread(StatisticsCallback);
        statisticsThread.Start(device);

        Thread receivePacketsThread = new Thread(ReceivePacketsCallback);
        receivePacketsThread.Start(device);
    }

    private static void StatisticsCallback(object state)
    {
        IPacketDevice device = (IPacketDevice)state;

        using (PackeCommunicator communicator = device,Open(65536, PacketDeviceOpenAttributes.Promiscuous, 1000))
        {
            communicator.Mode = PacketCommunicatorMode.Statistics;
            communicator.ReceiveStatistics(-1, StatisticsHandler);
        }
    }

    private static void ReceivePacketsCallback(object state)
    {
        IPacketDevice device = (IPacketDevice)state;

        using (PackeCommunicator communicator = device,Open(65536, PacketDeviceOpenAttributes.Promiscuous, 1000))
        {
            communicator.ReceivePackets(-1, PacketHandler);
        }
    }

    private static uint _AcceptedPacketsCounter = 0;
    private static void StatisticsHandler(PacketSampleStatistics statistics)
    {
        _AcceptedPacketsCounter += statistics.AcceptedPackets;

        Console.WriteLine("Received Packets: {0} - Accepted Packets: {1}", _ReceivedPacketsCounter, _AcceptedPacketsCounter );
    }

    private static uint _ReceivedPacketsCounter = 0;
    private static void PacketHandler(Packet packet)
    {
        _ReceivedPacketsCounter++;
    }
}
The problem is that the ReceivedPackets are not the same as AcceptedPackets.
On low traffic everything works, but when you starting to raise pps you can see that ReceivedPackets are smaller than AcceptedPackets.
note: I'm not talking about the difference in few packets that are still processed, but when you completely stop all the traffic you see the difference.

Value of AcceptedPackets always the same as in Wireshark.

I tried to use all overloads of ReceivePackets and all of them having the same problem.

Can you help me with this issue?

Thanks
Coordinator
Aug 16, 2013 at 9:35 AM
  1. Which one is bigger?
  2. By how much?
  3. What is the packets per seconds you're using?
  4. Is your CPU fully utilized (100% on a single core)?
  5. How often is the StatisticsHandler called? Have you tried lowering its rate?
Aug 22, 2013 at 10:43 AM
Sorry it tooked me long time to answer, I was on vacation.
I wrote the exect code with which I reproduce the problem (first post).
For simple scenario I'm opennig IIS home page on near by computer on the same network and just refreshing it.
"1. Which one is bigger?"
Always, AcceptedPackets are bigger then ReceivedPackets.
"2. By how much?"
The amount is not constant (depend on the traffic), its about ~2% loss for low traffic
"3. What is the packets per seconds you're using?"
To reproduce the problem enough <10k pps. My goal is to reach 100k - 200k.
"Is your CPU fully utilized (100% on a single core)?"
I have 4 cores and no one reaches 60%.
"How often is the StatisticsHandler called? Have you tried lowering its rate?"
In the original test i used 1 sec.
Every other tests giving me the same results, when I'm raising or lowering the StatisticsHandler and PacketHandler.

Thanks
Coordinator
Aug 23, 2013 at 12:29 PM
Can you also check the PacketCommunicator.TotalStatistics properties?
You might get more information from that.
Let me know what you see there.
Aug 25, 2013 at 9:05 AM
I checked both PacketCommunicator.TotalStatistics (Statistics.TotalStatistics and Packets.TotalStatistics).

In Statistics.TotalStatistics, all the fields are 0 except for PacketsReceived which is the same as AcceptedPackets.

In Packets.TotalStatistics, PacketCaptured filed is equal to ReceivedPackets, PacketsDroppedByDriver field holds the missing packets, PacketsDroppedByInterface field is 0, PacketsReceived field is the same as AcceptedPackets.

I see where was my mistake in the original tests, I checked drops only in Statistics.TotalStatistics.

What does PacketsDroppedByDriver mean's?

Will increasing the buffer size solve this problem? How can I do it with Pcap.Net?

Thanks
Coordinator
Aug 30, 2013 at 8:11 AM
PacketsDroppedByDriver is the number of packets dropped by the WinPcap driver.
It is the same as WinPcap's ps_drop field: http://www.winpcap.org/docs/docs_40_2/html/structpcap__stat.html#2cfc89d84a0ba404cacfa59f6b112bc1

If you manage to read the packets fast enough, and the problem isn't because over time the packets are received faster than you manage to read them, it makes sense that increasing the buffer size would help.
You can do that by calling SetKernelBufferSize().
Sep 15, 2013 at 4:14 PM
The change of the buffer size fixed my problem.

Thanks for help.