HTTPLayer: How the relationship between Request and Response ?

Feb 12, 2011 at 12:19 PM
Edited Feb 12, 2011 at 12:21 PM

hello, I just discovered your library, she looks really powerful.

I would like to retrieve the HTTP requests (Request and Response) is a bit like the Fiddler, but how the relationship between HttpRequestLayer and HttpResponseLayer (in which I Do not be the referring URL)?

I start like this, this is the right classes to use?

        private static void PacketHandler(Packet packet)
        {
            try
            {
                var http2 =((PcapDotNet.Packets.Http.HttpLayer)((((packet.Ethernet).IpV4).Tcp).Http.ExtractLayer()));
                if (http2.Header != null && http2.Header.BytesLength > 0)

but how tied after the two datagrams ?

Thank.

 

Feb 12, 2011 at 2:15 PM

 

HTTP packets are mainly of two types

1- a request , where a device will request a web page (for example) from a server or any other device

2- a reply , the reply will contain data that a device asked for by sending a request , so a reply will be made only after receiving a request

in Pcap.NET you can use (HttpRequestLayer) to create an HTTP request packet , and you can use (HttpReplyLayer) to create an HTTP reply packet

 

note that the class (HtttpLayer) is for read only , its inherited by (HttpRequestLayer) and (HttpReplyLayer) , so if you wanna create an HTTP packet use only one of those two

i hope this answers your question , if you wanna learn more just ask :)

Feb 12, 2011 at 6:13 PM
Edited Feb 12, 2011 at 9:16 PM
No this is not what I do, I already done with classes. NET framework (HttpWebRequest / HttpWebResponse).

I want to trace (sniffing a WebBrowser) requests (HTTP Request / Response).

With pcap.net I get datagrams each separately, but I do not know how to be sure that the response corresponds to the request
 

Should I use the sequence of datagram IP with source/destination ?   

var ack = (((packet.Ethernet).IpV4).Tcp).AcknowledgmentNumber;
var nextseq = (((packet.Ethernet).IpV4).Tcp).NextSequenceNumber




Feb 13, 2011 at 1:22 PM

Yes you r right, u have to find the relationship between the acknowledgement  and sequence number.

 

the acknowledgement no of the packet in which your HTTP request was sent should match the sequence number of the arriving packet. From the sniffer (my own sniffer made from pcap.net) I think there is no need of next sequence number. 

But the best technique is to parse the payload and look for the string ---> HTTP/1.1 200 OK (where 1.1 is the version, and 200 is the status code).

Feb 13, 2011 at 1:41 PM

please also check the following discussion

 

http://pcapdotnet.codeplex.com/discussions/228056

 

especially the commenst of *donuts

Feb 13, 2011 at 1:41 PM

please also check the following discussion

 

http://pcapdotnet.codeplex.com/discussions/228056

 

especially the commenst of *donuts

Feb 16, 2011 at 8:56 AM

Thank you, but I use the new classes HTTPDatagram / HTTPLayer that all cleanliness (see last post on Pcap.Net Brickner 0.8.0)

I proceeded as follows:

        private static Queue<IpV4Datagram> requestQueue = new Queue<IpV4Datagram>(30); 
        private static void PacketHandler(Packet packet)
        {
            try
            {
                if (
                    packet.Ethernet.EtherType == EthernetType.IpV4 &&
                    packet.Ethernet.IpV4.Protocol == IpV4Protocol.Tcp &&
                    packet.Ethernet.IpV4.Tcp.Http != null)
                {
                    var http2 = ((PcapDotNet.Packets.Http.HttpLayer)((((packet.Ethernet).IpV4).Tcp).Http.ExtractLayer()));
                    if (http2 != null && http2.Header != null && http2.Header.BytesLength > 0)
                    {
                        if (http2 is HttpRequestLayer)
                        {
                            requestQueue.Enqueue((packet.Ethernet).IpV4);
                        }
                        else if (http2 is HttpResponseLayer && http2.Header.ContentType != null && http2.Header.ContentType.MediaSubtype != null
                            && (/*http2.Header.ContentType.MediaSubtype == "json" ||*/ http2.Header.ContentType.MediaSubtype == "html"))
                        {
                            if ((packet.Ethernet.IpV4.Tcp.Http as HttpResponseDatagram).StatusCode == 200)
                            {
                                var lst = requestQueue.Where(p =>
                                    p.Source.Equals(packet.Ethernet.IpV4.Destination) &&
                                    p.Destination.Equals(packet.Ethernet.IpV4.Source) &&
                                    p.Tcp.DestinationPort.Equals(packet.Ethernet.IpV4.Tcp.SourcePort) &&
                                    p.Tcp.SourcePort.Equals(packet.Ethernet.IpV4.Tcp.DestinationPort) &&
                                    p.Tcp.NextSequenceNumber.Equals(packet.Ethernet.IpV4.Tcp.AcknowledgmentNumber)).ToList();

                                var request = lst.FirstOrDefault();

                                if (request != null)
                                    Console.WriteLine(((HttpRequestLayer)((request.Tcp.Http as HttpDatagram).ExtractLayer())).Uri);

                                HttpTranscode trans = new HttpTranscode(http2.Header, http2.Body.ToMemoryStream().ToArray());
                                string test2 = trans.DecodeBody();
                                if (!string.IsNullOrEmpty(test2))
                                    Console.WriteLine(test2);
                                
                            }
                        }
                    }
                }
            }
            catch (Exception ex)
            {
                Console.WriteLine("Error " + ex.Message);
            }
        }


But I still have some problems in my decoding function of the body of the message when the message is Transfer-Encoding: chunked

 

Coordinator
Feb 19, 2011 at 7:50 AM

Hi dauphinus,

 

I think the best way to know what HTTP request each HTTP reply belongs to is to just count the requests and replies of the same TCP connection and to match them by order.

Of course, you have to look at the TCP parameters to make sure you don't count the same request or reply twice due to retransmissions.

 

Please remember, Pcap.Net currently only parse packets independently. Since HTTP is over TCP, HTTP requests or replies can be longer than one packet so Pcap.Net will try to parse well only the first packet of each request or reply (when they start at the beginning of the TCP payload packet).

In other words, without TCP reconstruction you can't expect to parse fully HTTP requests or replies that are longer than one packet.

 

If you want to have TCP reconstruction in Pcap.Net, you are welcome to vote for this feature in the Issue Tracker.

 

Boaz.

Feb 20, 2011 at 3:47 PM

Hi brickner,

After analysis of TCP and HTTP packets, I saw that it was necessary recontruct the TCP session. I am trying to code a class that will allow me to reconstruct the HTTP session(Request/Response).

It is true that it would be great if Pcap.net would recontruct the HTTP message as in Wireshark.

Thank you

Coordinator
Feb 23, 2011 at 12:30 PM

Hi,

 

I will try to continue expanding the capabilities of Pcap.Net according to users requests.

Of course, this is a volunteering project so far, so I can't give a deadline for this feature. This is a non-trivial feature and is a bit out of the scope of Pcap.Net (the scope is flexible, though).

 

Boaz.