How to use `OnlineLogBinning`

First, construct an empty [BinningAccumulator] with of T <: Number parametric type. Let's take the default T = Float64 as an example.

Initialization

To start, initialize a BinningAccumulator{T}:

julia> bacc = BinningAccumulator()
BinningAccumulator{Float64} with 0 binning levels.
0th Binning Level (unbinned data):
LevelAccumulator{Float64} with online fields:
    level    = 0
    num_bins = 0
    Taccum   = 0.0
    Saccum   = 0.0
    Paccum   = PairAccumulator{Float64}(true, [0.0, 0.0])

    Calculated Level Statistics:
    Current Mean             = NaN
    Current Variance         = -0.0
    Current Std. Deviation   = -0.0
    Current Var. of the Mean = NaN
    Current Std. Error       = NaN

Note

We currently only support Float types, i.e. T <: AbstractFloat or T is a Complex{Float#}. The tested types are listed in OLB_tested_numbers.

Alternatively, if one has an anticipated data stream size in mind, one may pre-allocate a completely empty BinningAccumulator as


julia> stream_length = 1212
julia> bacc_prealloc = BinningAccumulator(stream_length)   # using the default `Float64` typeBinningAccumulator{Float64} with 3 binning levels.
0th Binning Level (unbinned data):
LevelAccumulator{Float64} with online fields:
    level    = 0
    num_bins = 0
    Taccum   = 0.0
    Saccum   = 0.0
    Paccum   = PairAccumulator{Float64}(true, [0.0, 0.0])

    Calculated Level Statistics:
    Current Mean             = NaN
    Current Variance         = -0.0
    Current Std. Deviation   = -0.0
    Current Var. of the Mean = NaN
    Current Std. Error       = NaN

1th Binning Level:
LevelAccumulator{Float64} with online fields:
    level    = 1
    num_bins = 0
    Taccum   = 0.0
    Saccum   = 0.0
    Paccum   = PairAccumulator{Float64}(true, [0.0, 0.0])

    Calculated Level Statistics:
    Current Mean             = NaN
    Current Variance         = -0.0
    Current Std. Deviation   = -0.0
    Current Var. of the Mean = NaN
    Current Std. Error       = NaN

2th Binning Level:
LevelAccumulator{Float64} with online fields:
    level    = 2
    num_bins = 0
    Taccum   = 0.0
    Saccum   = 0.0
    Paccum   = PairAccumulator{Float64}(true, [0.0, 0.0])

    Calculated Level Statistics:
    Current Mean             = NaN
    Current Variance         = -0.0
    Current Std. Deviation   = -0.0
    Current Var. of the Mean = NaN
    Current Std. Error       = NaN

3th Binning Level:
LevelAccumulator{Float64} with online fields:
    level    = 3
    num_bins = 0
    Taccum   = 0.0
    Saccum   = 0.0
    Paccum   = PairAccumulator{Float64}(true, [0.0, 0.0])

    Calculated Level Statistics:
    Current Mean             = NaN
    Current Variance         = -0.0
    Current Std. Deviation   = -0.0
    Current Var. of the Mean = NaN
    Current Std. Error       = NaN

Compat

The pre-allocated functionality requires at least version 0.2.2.

Once initialized, then one push!es either a single value or a data stream (sequence of values of itr type) to the BinningAccumulator. The online analysis will be taken care of automatically.

The highest binning level will typically yield useless NaN statistics, but that just reflects the fact that the num_bins, Taccum, and Saccum accumulators are only updated once the level's PairAccumulator is full.

Available online statistics

One can then calculate the following statistics from the BinningAccumulator at any binning level = lvl:

mean(bacc::BinningAccumulator; level = lvl)           # arithmetic mean
var(bacc::BinningAccumulator; level = lvl)            # sample variance 
std(bacc::BinningAccumulator; level = lvl)            # sample standard deviation 
var_of_mean(bacc::BinningAccumulator; level = lvl)    # variance of the mean 
std_error(bacc::BinningAccumulator; level = lvl)      # standard error of the mean

The binning level is optional. By default, the binning level is set to level = 0. This level, accessed by bacc[level = 0], represents the unbinnned statistics from of the original data stream. The LevelAccumulators from any binning level can also be extracted using the overloaded [] notation as

julia> bacc[level = 0]
LevelAccumulator{Float64} with online fields:
    level    = 0
    num_bins = 8
    Taccum   = 17.0
    Saccum   = 8.875
    Paccum   = PairAccumulator{Float64}(true, [0.0, 0.0])

    Calculated Level Statistics:
    Current Mean             = 2.125
    Current Variance         = 1.2678571428571428
    Current Std. Deviation   = 1.1259916264596033
    Current Var. of the Mean = 0.15848214285714285
    Current Std. Error       = 0.3980981573144277

julia> bacc[level = 1]
LevelAccumulator{Float64} with online fields:
    level    = 1
    num_bins = 4
    Taccum   = 8.5
    Saccum   = 3.6875
    Paccum   = PairAccumulator{Float64}(true, [0.0, 0.0])

    Calculated Level Statistics:
    Current Mean             = 2.125
    Current Variance         = 1.2291666666666667
    Current Std. Deviation   = 1.1086778913041726
    Current Var. of the Mean = 0.3072916666666667
    Current Std. Error       = 0.5543389456520863

Perform the Binning Analysis

Once a sufficient amount of data has been binned, one can employ the BinningAnalysis routines found in BinningAnalysis.jl. To show how this works, we make use of a pre-prepared random telegraph signal generated with the TelegraphNoise.jl package. The signal is stored as a binary file in docs/src/assets.

The simplest one to use is fit_RxValues that takes in a single required BinningAccumulator as an argument.

julia> signal = zeros(Float64, Int(2^18));

julia> read!( joinpath("build", "assets", "telegraph_plateau.bin"), signal);

julia> bacc = BinningAccumulator();

julia> push!(bacc, signal);

julia> result = fit_RxValues(bacc)
Binning Analysis Result:
    Plateau Present:             true
    Fitted Rx Plateau:           14.611315366653367
    Autocorrelation time τₓ:     6.805657683326683
    Effective Datastream Length: 17941
    Binning Analysis Mean:       0.00440216064453125
    Binning Analysis Error:      0.007465747493169594

The Plateau Present flag indicates whether a sigmoid fit to the RxValues is reasonable so as to take its plateau seriously. (See _plateau_found for details.) The value of the fitted plateau is also returned. If Plateau Present == false, then the plateau is set to be the size of the data stream. This is because the effective number of uncorrelated values in the data stream of size $M$ is given by $M_{\rm eff} = M / R_X$.

Other quantities can be extracted from the BinningAnalysisResult, for example, the autocorrelation_time and the effective_uncorrelated_values in the data stream.

julia> autocorrelation_time(result)
6.805657683326683

julia> effective_uncorrelated_values(result)
17941

Additionally, as of v0.4, one can also export a BinningAnalysisResult as a Measurement from Measurements.jl:

julia> measurement(result)
0.0044 ± 0.0075

and then one can fully take advantage of propagated errors thanks to that wonderful package!

How to use OnlineLogBinning

Initialization

Accumulate data

Available online statistics

Perform the Binning Analysis

How to use `OnlineLogBinning`