Get Referer url

Oct 21, 2011 at 5:10 PM

Hi,

I need to find a way to get the original url that has a request or response packet. Example if i navigate to http://www.codeplex.com  and i sniff some packets and pick random request to find that this request came from page www.codeplex.com. Is this possible?

Coordinator
Oct 21, 2011 at 5:27 PM

I suggest looking in Wireshark and see which field you need.

This might work:

Console.WriteLine(packet.Ethernet.IpV4.Tcp.Http.Header["Referer"].ValueString);

 

Boaz.

Oct 21, 2011 at 5:38 PM

Hi,

thanks for reply.

Unfortunately this field is completed only in chrome, for firefox is empty. Is there another way of getting this info?

Coordinator
Oct 21, 2011 at 6:03 PM

I'm pretty sure this field shouldn't be empty in Firefox, unless you have some specific settings for this browser.

I can see it in Firefox.

If it is not in the packet, there's no way to get it. I suggest looking at the packets in Wireshark and decide what are the fields you need.

Oct 21, 2011 at 6:24 PM

I've checked with Wireshark and you're right in Wireshark the field exists, but in my sniffed requests header in app is empty. I'm doing something wrong or do i have to specify some option to pcapdotnet in order to get this field.

In my code i use this in order to get the Referer info. In Chrome it works perfectly, but in Firefox i get this field empty. In wireshark works perfectly. i use the dame url in both Chrome and Firefox.

        private string GetHeaderInfo(PcapDotNet.Packets.Http.HttpRequestLayer reqlayer, string Field)
        {
            if (reqlayer == null) return string.Empty;
            var info = reqlayer.Header[Field];
            return info == null ? string.Empty : info.ValueString.Trim();
        }

Coordinator
Oct 21, 2011 at 6:39 PM

Why do you take the field from the HTTP layer and not from the HTTP datagram?

 

It is possible that the HTTP request is split to two packets and that field isn't entirely in the first packet. This is why you might not get it using Pcap.Net.

You check if it splitted or not in Wireshark.

You're welcome to refer me to a pcap file so I can see for myself.

 

Boaz.

Oct 28, 2011 at 5:24 PM

Hi Boaz, 

Is there a way to increase packet size? I can't read the referer  filed because is split into 3 several packets and i need a way to read entire header of a request. For ex. if launch a youtube page the header is readed in first packet, but incomplete. I've tried with http datagram and it reads the same way splitted in 3 packets. Do you have any ideas on how to get the entire header?

Coordinator
Oct 29, 2011 at 7:53 AM

Hi so3,

 

In order to read the 3 packets together, you'll need to do TCP reconstruction.

This means looking at the 3 packets together and form there sequence number and acknowledge number build the full payload.

 

Boaz.

Oct 31, 2011 at 6:56 AM
Edited Oct 31, 2011 at 6:57 AM

Can you please show me an example? I can't figure it.

Oct 31, 2011 at 4:20 PM

Hi Boaz,

I use this code to build the packet requests. Please tell me if it's correct. And also this code doesn't work for responses packets, any ideas why not?

 

 

			List<byte> RecRequests = new List<byte>();
                        RecRequests.AddRange(ipV4.Tcp.Http.ToArray<byte>());
                        if (ipV4.Tcp.IsAcknowledgment)
                        {
                            uint nextPacketNr = ipV4.Tcp.NextSequenceNumber;
                            IpV4Datagram NextPacket = null;
                            do
                            {
                                lock (((IList)PRequests).SyncRoot)
                                {
                                    NextPacket = PRequests.Where(c => c.Tcp.AcknowledgmentNumber == nextPacketNr).FirstOrDefault();
                                    if (NextPacket != null) PRequests.Remove(NextPacket);
                                }
                                if (NextPacket != null)
                                {
                                    nextPacketNr = NextPacket.Tcp.NextSequenceNumber;
                                    RecRequests.AddRange(NextPacket.Tcp.Http.ToArray<byte>());
                                }
                            } while (NextPacket != null);

 

 

Coordinator
Nov 2, 2011 at 6:54 PM

I don't see the input (for example - what is PRequests).

 

Anyways, there are two issues here:

1. How to do TCP reconstruction in general - which is something you better describe how you plan to do without showing me a code snippet so I guess what is right or not.

2. How to do each of the operation you need for TCP reconstruction using Pcap.Net. This is better to separate to different questions.

 

I hope this helps,

 

Boaz.

Nov 3, 2011 at 8:51 AM

Hi,

I needed to unify requests and responses so to read entire body, but i think i figure it out (by using AcknowledgmentNumber). I read all packets that contained the same ack number.

Initially i've separated requests and response which was not correctly because if a request is splited in 3 packets i got first packet as a request and the next packets as response.

So i have one more question please. Is this correct to do the reconstruction based by ack number?. It will always the packets contains the same ack number?

Thanks 

Coordinator
Nov 3, 2011 at 9:34 PM

No, you should basically reconstruct the packets according to the sequence number (seq).

The acknowledgment number (ack) is the number that was received, while the sequence number is the offset (with some random initial number) in the stream