How to use SFLOW for per-flow throughput?

Question

I have a working sFlow collector on my network (vFlow, available as an opensource solution on GitHub, very awesome). The collector is running, successfully receiving sFlow data from my network devices, and then exporting it off to a MySQL database. Now that I have my pipeline working, its time to make sense of the data.

The bad news is, what I really need is per-flow network throughput. For every sampled flow from Host A to Host B, I’d like to know: How many bytes were sent? This seems like a simple and reasonable question.

But when I look at the statistics sFlow is giving me, no “BytesPerSecond” or “PacketsPerSecond” or “TotalLoad” stat jumps out at me. I’ve read documents on sflow.org, and there doesn’t seem to be a magic statistic which fits my needs.

There is, however, the following (from here):

unsigned int sample_pool :: Total number of packets that could have been sampled

This seems to be the total number of packets in a sampled flow? Or the total number of packets that the network device has seen across all flows?

I also see a “TotalLen” statistic in my Packet.L3 sflow datagrams. In my test traffic, this is almost always 1500, which I assume means 1500 bytes of payload in the L3 packet header. Which seems accurate.

So if one sFlow datagram message reports information for one flow, would the total bytes in that flow be:

• ( sample_pool packets/flow ) * ( TotalLen bytes/packet ) = X bytes/flow

On paper, it looks logical.

Another thing I’ve been wondering: Am I not getting the Throughput statistic because that particular statistic needs to be enabled on my sampling network devices? Or does my collector need to turn this on? Or something?

I’ve been Googling for a solution to using sFlow to generate basic network throughput statistics per flow. All I seem to pull up are ads for commercial sFlow products or white papers that don’t seem to address this problem. Has anyone solved this issue? Any advice is appreciated!

An example of one of my sFlow records is below:

{
"Header":
    {"Version":5,
     "IPVersion":1,
     "AgentSubID":1,
     "SequenceNo":416389,
     "SysUpTime":2118000,
     "SamplesNo":6,
     "IPAddress":"10.10.20.10"
    },
"ExtSWData":
    {"SrcVlan":0,
     "SrcPriority":0,
     "DstVlan":0,
     "DstPriority":0
    },
"Sample":
    {"SequenceNo":2488018,
     "SourceID":2,
     "SamplingRate":64,
     "SamplePool":159233152,
     "Drops":0,
     "Input":7004843,
     "Output":4882507,
     "RecordsNo":2
    },
"Packet":{
    "L2":{
        "SrcMAC":"52:54:00:38:43:f3",
        "DstMAC":"52:54:00:62:32:cc",
        "Vlan":0,
        "EtherType":2048
        },
    "L3":{
        "Version":4,
        "TOS":0,
        "TotalLen":1500,
        "ID":30758,
        "Flags":0,
        "FragOff":0,
        "TTL":64,
        "Protocol":6,
        "Checksum":34707,
        "Src":"10.10.10.10",
        "Dst":"10.10.10.11"
        },
    "L4":{
        "SrcPort":47346,
        "DstPort":5201,
        "DataOffset":0,
        "Reserved":0,
        "Flags":128
        }
    }
}

Answer 1

In the sample, you can see the source/destination IP addresses, transport protocol and its ports - that is your flow.

You should check out https://sflow.org/developers/specifications.php where the data structure is broken down (and pretty much everything else).