3

Is it possible to capture only the first 10k of data in a TCP flow and discard the rest at capture time? I have a large number of TCP sessions where each session starts with some metadata and then proceeds with an enormous binary dump. I suspect an occasional problem in the metadata portion of the flow and want to capture it for study, but don't have enough resources to keep all of the bulk data segments around.

How would you set up such a capture?

I've found filters for segment length and total reassembled length, but the former is effectively just packet size and the latter doesn't increment each time the window is updated, it's calculated once for the entire flow. Maybe there is a way to do the math for the sequence numbers? I don't know how to do that in a capture filter though, not even in Wireshark's more powerful display filters. Maybe there is a better tool? I'm working on a Linux machine.

Improve this question
2
  • wireshark captures all packets. it does not capture TCP streams. That's the job of an IDS/IPS, or other expensive network product.
    – Ricky
    Mar 18, 2015 at 20:16
  • Did any answer help you? if so, you should accept the answer so that the question doesn't keep popping up forever, looking for an answer. Alternatively, you could provide and accept your own answer.
    – Ron Maupin ♦
    Aug 11, 2017 at 14:42

2 Answers 2

Reset to default
2

Wireshark and other capture tools are not aware of TCP streams during capture. They would have to save tcp stream state in memory to do that which would reduce troughput.

That being said there might be a way. If your packets have metadata there might be some sort of identifier (header, string, etc.) that is in all the packets with metadata? If so you could filter these with iptables and feed them to a NFLOG target which can be captured with dumpcap, a tool that is shipped with wireshark.

For example, if all your metadata packets have a "X-Metadata" string in them, and you capture them on eth0 you could do:

iptables -A INPUT -i eth0 -m string --algo bm --string "X-Metadata" -j NFLOG --nflog-group 1
dumpcap -i nflog:1 -w test.pcap

This will save all packets with "X-Metadata" in them in the test.pcap file. Mind you, if there is some other way to identify packets with metadata, iptables might be able to do that. There are many filters/extensions.

Improve this answer
4
  • Seems like the kernel should theoretically be able to support this using the nf_conntrack module, but I don't see this specific feature.
    – Jason Andresen
    Mar 20, 2015 at 14:54
  • Where are you not seeing this feature? Do you get an error?
    – Sebastian Wiesinger
    Mar 30, 2015 at 15:28
  • I don't see it on the manpage. It's something that nf_conntrack could do, and almost does already, but i don't think this particular functionality is implemented.
    – Jason Andresen
    Mar 31, 2015 at 16:25
  • @Sebastian I think you are on the right track, why not use the Linux/netfilter connbytes to only log initial number of bytes from each flow. Match by how many bytes or packets a connection (or one of the two flows constituting the connection) has transferred so far, or by average bytes per packet.
    – Pieter
    Jul 17, 2015 at 21:14
-2

The display and capture filters from Wireshark will only display or capture packets that match a certain size (or threshold), as you discoverred. What you want, is to capture only the first N bytes from each packet.

Ideally, you want to catch just the headers, which will typically be the first 54 bytes: 18 for Ethernet, 20 for IP, 20 for TCP. (If you're receiving VLAN tagged packets, add another 4 for the 802.1q header, making a total of 58 bytes).

The way to do that in tcpdump is to use the -s option:

-s snaplen
--snapshot-length=snaplen

Snarf snaplen bytes of data from each packet rather than the default of 65535 bytes. Packets truncated because of a limited snapshot are indicated in the output with ``[|proto]'', where proto is the name of the protocol level at which the truncation has occurred. Note that taking larger snapshots both increases the amount of time it takes to process packets and, effectively, decreases the amount of packet buffering. This may cause packets to be lost. You should limit snaplen to the smallest number that will capture the protocol information you're interested in. Setting snaplen to 0 sets it to the default of 65535, for backwards compatibility with recent older versions of tcpdump.

This way, you capture all the "important stuff", and avoid capturing the long streams of binary data.

You might also find additional useful information on this post over at Server Fault.

Improve this answer
2
  • 1
    I'm sorry, what he wants is "Is it possible to capture only the first 10k of data in a TCP flow and discard the rest at capture time?" In other words, he has very large TCP flows and needs to reject the rest of a flow after capturing the first 10KB of it... this is not a snaplen solution. 10KB TCP flows have at least seven 1460-byte TCP payloads in them. The OP wants to capture the "first seven" 1460-byte TCP payloads for each TCP socket, and then forget about any other packets in that specific TCP socket
    – Mike Pennington
    Mar 19, 2015 at 3:37
  • @MikePennington I see. I misunderstood it. I thought he wanted to capture packets without the overhead of having to store a byte for byte copy of every bit that came across his network. Technically, the snaplen option would work so long as the meta data was in the first couple hundred bytes of the packet... it would at least significantly decrease the size of the capture compared to storing all 1500~ bytes of each packet. But if the meta data spreads across multiple packets, then you're right, snaplen wouldn't be useful in this case.
    – Eddie
    Mar 19, 2015 at 6:52

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.