Likelihood-based inference for modelling packet transit from thinned flow summaries

Abstract

Network traffic speeds and volumes present practical challenges to analysis. Packet thinning and flow aggregation protocols provide smaller structured data summaries, but conversely impede statistical inference. Methodswhich model traffic propagation typically do not account for the packet thinning and aggregation in their analysis and are of limited practical use. We introduce a likelihood-based analysis which fully incorporates packet thinning and flow aggregation. Inferences can hence be made for models on the level of individual packets while only observing thinned flow summaries. We establish consistency of the resulting maximum likelihood estimator, derive bounds on the volume of traffic which should be observed to achieve a desired degree of efficiency, and identify an ideal family of models. The robust performance of the estimator is examined through simulated analyses and an application on a publicly accessible trace which captured in excess of 36 m packets over a 1 minute period.