libpcap problem (hdr.len vs tcpdump file size)?

libpcap problem (hdr.len vs tcpdump file size)?

Post by c00l.. » Mon, 27 Oct 2003 18:44:39



Hi,

I'm trying to use libpcap to output tpcdump raw data that I've
captured in a file.
here's the info of the tcpdump file:
-rw-r--r--    1 david    david        1042 Oct 26 01:36 tcpdump.out

now, when I use my code to capture the packets in that file and
directly printing them out, the output of this code is somehow larger
than the actual dump file.
Here's my (C++) code:

-------------
#include <iostream>
#include <netinet/in.h>
#include <linux/inet.h>
#include <netinet/if_ether.h>
#include <pcap.h>
#include <fstream>

int main(){

        using namespace std;
        ofstream fout;
        fout.open("pcaptest.out");
        char errbuf[PCAP_ERRBUF_SIZE];
       pcap_t *descr= pcap_open_offline("/home/david/tcpdump.out",
errbuf);
        const u_char * packet;
        struct pcap_pkthdr hdr;

        packet = pcap_next(descr, &hdr);
        while (packet){

                unsigned packetSize= hdr.len;
                for (unsigned i=0; i< packetSize;i++){
                        fout<<packet[i];
                }

                packet = pcap_next(descr, &hdr);

        }

        return 0;

Quote:}

--------------------------------

and here's the info on the output file:
-rw-r--r--    1 david    david        1508 Oct 26 01:36 pcaptest.out

How come the size of the output file does not match the actual dump
file at all? And furthermore, how come the it's larger in size?

any help/comment is appreciated.

thanks in advanced!
-D

 
 
 

libpcap problem (hdr.len vs tcpdump file size)?

Post by Michael Fu » Tue, 28 Oct 2003 02:52:14



> I'm trying to use libpcap to output tpcdump raw data that I've
> captured in a file.
> here's the info of the tcpdump file:
> -rw-r--r--    1 david    david        1042 Oct 26 01:36 tcpdump.out

> now, when I use my code to capture the packets in that file and
> directly printing them out, the output of this code is somehow larger
> than the actual dump file.
> Here's my (C++) code:

> -------------
> #include <iostream>
> #include <netinet/in.h>
> #include <linux/inet.h>
> #include <netinet/if_ether.h>
> #include <pcap.h>
> #include <fstream>

You shouldn't need to include <netinet/in.h>, <netinet/if_ether.h>,
and the OS-specific <linux/inet.h>, and you're not using anything
from <iostream>.  This reduced set of headers works for me, and on
several different platforms:

#include <fstream>

extern "C" {
#include <pcap.h>

Quote:}
> int main(){

>         using namespace std;
>         ofstream fout;
>         fout.open("pcaptest.out");

You could also do this:

    ofstream fout("pcaptest.out");

I'll assume you omitted checking whether the open succeeded to
simplify your example.  A real program should check the result
of any operation on which it depends.

Quote:>         char errbuf[PCAP_ERRBUF_SIZE];
>        pcap_t *descr= pcap_open_offline("/home/david/tcpdump.out", errbuf);

Again, I'll assume you omitted error checking to keep the example
as minimal as possible.

Quote:>         const u_char * packet;
>         struct pcap_pkthdr hdr;

>         packet = pcap_next(descr, &hdr);
>         while (packet){

>                 unsigned packetSize= hdr.len;

This is part of the problem.  You're using the length of the packet
off the wire (hdr.len) instead of the captured length (hdr.caplen).
By default tcpdump captures only the first 68 bytes of each packet,
so you could end up writing more data for each packet than is present
in the dump file.  You're also risking a segmentation fault (SIGSEGV)
and a core dump by reading memory beyond bounds.

Quote:>                 for (unsigned i=0; i< packetSize;i++){
>                         fout<<packet[i];
>                 }

Another part of the problem is that the dump file contains a file
header (struct pcap_file_header) and each packet in the file is
prepended with a packet header (struct pcap_pkthdr); your program
copies only the packet data.  This alone would result in an output
file smaller than the dump file -- the opposite of what you're
seeing -- but it's still contributing to the difference in sizes.

Quote:>                 packet = pcap_next(descr, &hdr);

>         }

>         return 0;
> }
> --------------------------------

> and here's the info on the output file:
> -rw-r--r--    1 david    david        1508 Oct 26 01:36 pcaptest.out

> How come the size of the output file does not match the actual dump
> file at all? And furthermore, how come the it's larger in size?

If you want to duplicate the dump file exactly, you'll need to copy
the file header and each packet header in addition to the packet
data.  For the packet data, make sure you copy the captured length
and not the off-the-wire length.

Hope this helps.

--
Michael Fuhr
http://www.fuhr.org/~mfuhr/

 
 
 

libpcap problem (hdr.len vs tcpdump file size)?

Post by c00l.. » Fri, 31 Oct 2003 11:50:52


Ah, yes. It works better now. I'm using tcpdump with -s 0, which I've
read should capture the whole frame; however it seems like that data
that I output from my code still differs from the actual tcpdump
offline content. I've output the pcap_pkthdr bytes before starting to
output the rest of the packet.

Quote:> If you want to duplicate the dump file exactly, you'll need to copy
> the file header and each packet header in addition to the packet
> data.  For the packet data, make sure you copy the captured length
> and not the off-the-wire length

I believe that the packet data that I receive from the call to
pcap_next should include the TCP/IP headers (from what I've read from
the tcpdump/libpcap tutorial pages that I found on www.tcpdump.org).

Any comment?

Thanks a lot!
-D



> > I'm trying to use libpcap to output tpcdump raw data that I've
> > captured in a file.
> > here's the info of the tcpdump file:
> > -rw-r--r--    1 david    david        1042 Oct 26 01:36 tcpdump.out

> > now, when I use my code to capture the packets in that file and
> > directly printing them out, the output of this code is somehow larger
> > than the actual dump file.
> > Here's my (C++) code:

> > -------------
> > #include <iostream>
> > #include <netinet/in.h>
> > #include <linux/inet.h>
> > #include <netinet/if_ether.h>
> > #include <pcap.h>
> > #include <fstream>

> You shouldn't need to include <netinet/in.h>, <netinet/if_ether.h>,
> and the OS-specific <linux/inet.h>, and you're not using anything
> from <iostream>.  This reduced set of headers works for me, and on
> several different platforms:

> #include <fstream>

> extern "C" {
> #include <pcap.h>
> }

> > int main(){

> >         using namespace std;
> >         ofstream fout;
> >         fout.open("pcaptest.out");

> You could also do this:

>     ofstream fout("pcaptest.out");

> I'll assume you omitted checking whether the open succeeded to
> simplify your example.  A real program should check the result
> of any operation on which it depends.

> >         char errbuf[PCAP_ERRBUF_SIZE];
> >        pcap_t *descr= pcap_open_offline("/home/david/tcpdump.out", errbuf);

> Again, I'll assume you omitted error checking to keep the example
> as minimal as possible.

> >         const u_char * packet;
> >         struct pcap_pkthdr hdr;

> >         packet = pcap_next(descr, &hdr);
> >         while (packet){

> >                 unsigned packetSize= hdr.len;

> This is part of the problem.  You're using the length of the packet
> off the wire (hdr.len) instead of the captured length (hdr.caplen).
> By default tcpdump captures only the first 68 bytes of each packet,
> so you could end up writing more data for each packet than is present
> in the dump file.  You're also risking a segmentation fault (SIGSEGV)
> and a core dump by reading memory beyond bounds.

> >                 for (unsigned i=0; i< packetSize;i++){
> >                         fout<<packet[i];
> >                 }

> Another part of the problem is that the dump file contains a file
> header (struct pcap_file_header) and each packet in the file is
> prepended with a packet header (struct pcap_pkthdr); your program
> copies only the packet data.  This alone would result in an output
> file smaller than the dump file -- the opposite of what you're
> seeing -- but it's still contributing to the difference in sizes.

> >                 packet = pcap_next(descr, &hdr);

> >         }

> >         return 0;
> > }
> > --------------------------------

> > and here's the info on the output file:
> > -rw-r--r--    1 david    david        1508 Oct 26 01:36 pcaptest.out

> > How come the size of the output file does not match the actual dump
> > file at all? And furthermore, how come the it's larger in size?

> If you want to duplicate the dump file exactly, you'll need to copy
> the file header and each packet header in addition to the packet
> data.  For the packet data, make sure you copy the captured length
> and not the off-the-wire length.

> Hope this helps.

 
 
 

libpcap problem (hdr.len vs tcpdump file size)?

Post by Michael Fu » Fri, 31 Oct 2003 13:35:54



> > If you want to duplicate the dump file exactly, you'll need to copy
> > the file header and each packet header in addition to the packet
> > data.  For the packet data, make sure you copy the captured length
> > and not the off-the-wire length

> I believe that the packet data that I receive from the call to
> pcap_next should include the TCP/IP headers (from what I've read from
> the tcpdump/libpcap tutorial pages that I found on www.tcpdump.org).

> Any comment?

The packet data you get should include link-, network-, and transport-
layer headers, but in the dump file each packet is prepended with
an additional pcap packet header (struct pcap_pkthdr) -- that's
where the timestamp, captured length, and off-the-wire length for
each packet come from.  The format of the dump file is:

struct pcap_file_header
struct pcap_pkthdr
packet
struct pcap_pkthdr
packet
struct pcap_pkthdr
packet
 .
 .
 .
etc.

See pcap.h for the structure definitions.  Hope this helps.

--
Michael Fuhr
http://www.fuhr.org/~mfuhr/

 
 
 

1. libpcap problem (hdr.len vs tcpdump file size)?

Hi,

I'm trying to use libpcap to output tpcdump raw data that I've
captured in a file.
here's the info of the tcpdump file:
-rw-r--r--    1 david    david        1042 Oct 26 01:36 tcpdump.out

now, when I use my code to capture the packets in that file and
directly printing them out, the output of this code is somehow larger
than the actual dump file.
Here's my (C++) code:

-------------
#include <iostream>
#include <netinet/in.h>
#include <linux/inet.h>
#include <netinet/if_ether.h>
#include <pcap.h>
#include <fstream>

int main(){

        using namespace std;
        ofstream fout;
        fout.open("pcaptest.out");
        char errbuf[PCAP_ERRBUF_SIZE];
       pcap_t *descr= pcap_open_offline("/home/david/tcpdump.out",
errbuf);
        const u_char * packet;
        struct pcap_pkthdr hdr;

        packet = pcap_next(descr, &hdr);
        while (packet){

                unsigned packetSize= hdr.len;
                for (unsigned i=0; i< packetSize;i++){
                        fout<<packet[i];
                }

                packet = pcap_next(descr, &hdr);

        }

        return 0;
--------------------------------

and here's the info on the output file:
-rw-r--r--    1 david    david        1508 Oct 26 01:36 pcaptest.out

How come the size of the output file does not match the actual dump
file at all? And furthermore, how come the it's larger in size?

any help/comment is appreciated.

thanks in advanced!
-D

2. trying to jumpstart Netra X1, but get interactive install

3. tcpdump, SW [bad hdr length], what does it mean?

4. QDI R2000 AGP 3D Graphics Card

5. File size problem, what is the biggest size of a Linux file?

6. Xterm forwarding

7. tcpdump-3.0.2 and libpcap-0.0.6

8. Connecting intranet to internet with dynamic (modem) IP - how?

9. libpcap and tcpdump

10. tcpdump, libpcap

11. libpcap, tcpdump,..

12. tcpdump & libpcap