Cumulant generating functions and distribution of empirical averages


Common wisdom tells us that "the sum of a large number of uncorrelated variables is Gaussian distributed". We investigate here conditions for this Central Limit Theorem (CLT) to hold, and discuss its interpretation in terms of the scaling of the fluctutations of such sums.

Distribution with power-law tails

A law that may be used as a benchmark for distributions whose momenta are not all defined is the (normalized) power law:

For α > 2, one checks that the mean μ and the variance σ2 are well defined:

One checks however that for 1 < α < 2, the mean μ is still defined but the variance σ is infinite.

Cumulant generating function

For any real random variable ξ of distribution π(ξ) one defines the cumulant generating function

If existing, its Taylor expansion around 0 defines the cumulants

as follows:

Even if all the cumulants do not exist, the function ψξ(s) may still be well defined around 0 (it can be non-analytic) if π(ξ) decreases fast enough at large ξ. The first and second cumulant are easy to express in terms of the momenta:

It is possible to determine the higher order cumulants but there is not direct interpretation of the result.

There are two simple situations where the cumulant generating function is easy to compute:
  • If the distribution is a Dirac delta of mean μ:

  • If the distribution is a normalized Gaussian of mean μ and variance σ2:

and this is a characterization of the Gaussian distributions (all cumulants of order >2 are zero).

The cumulant generating function verify an important property of linearity: for two independent random variables ξ1 and ξ2, one has

which implies the linearity property of the cumulant generating function

Distribution of the empirical average

One defines the empirical average XN of N independent instances of a random variable ξ drawn from the same distribution π(ξ) as

A simple case of the Central Limit Theorem determines the fluctuations of the empirical average XN.

Case 1: when the second moment exists

We assume that the first and second cumulants (or equivalently, moments) of ξ exist. This correspond to the case α > 2 for the power law distribution πα(ξ) defined above. The cumulant generating function of ξ is defined at least up to order 2 in s:

This shows that at the infinite size limit,

and the probability distribution function of XN is a delta function around μ. The fluctuations around this average are determined from the next order of the expansion. The quadratic form of

show that to the next order, XN has a Gaussian distribution of mean μ and variance σ2/N. This is the Central Limit Theorem.

Case 2: when the second moment does not exist but the first does

In that situation the function ψξ (s) can't be expanded as previously. We focus on the case when the distribution is the power law πα(ξ) with 1 < α < 2. Standard asymptotic analysis shows in that case that

where the generalised cumulant of order α is

and Γ(z) is the Euler Gamma function. For another distribution π(ξ) with different form but same power law tail, the expansion is the same with different value for κα. In any case, by the linearity property, one has

Interpretation in terms of the scaling of fluctuations

In the Gaussian case one sees that, in the large N limit, the empirical average XN takes the form

where μ is constant and Y has fluctuations of order 1. Indeed the distribution of

is Gaussian of zero mean and variance σ2, a distribution independent of N.

We now would like to determine the scaling of the fluctuations ofXN around μ in the case 1 < α < 2, which are not of order N-1/2. To do so, one remarks that

In other words the dependence of the non-linear terms in

in s and N is made only through a function of sN −γ. In the case 1 < α < 2, by writing

one finds that γ = 1−1/α and thus the scaling of XN is

The distribution of Y is not trivial. It belongs to the class of Lévy alpha-stable distributions. Thanks to the last program of Class Session 04: Errors and fluctuations , one can evaluate it numerically as follows:

[Print this page]