movingStandardDeviationBy()

Calculate the moving standard deviation of BigDecimal objects mapped from a Stream<INPUT> via a mappingFunction and looking back windowSize elements, as either a population or a sample.

Implementation Notes

This implementation is suitable for mapping an arbitrary Stream<INPUT> to BigDecimal via a mappingFunction; for a version that operates directly on a Stream<BigDecimal>, see movingStandardDeviation(). By default, nulls are ignored and play no part in calculations, see treatNullAs() and treatNullAsZero() below for ways to change this behavior. The default MathContext for all calculations is MathContext.DECIMAL64, but this can be overridden (see withMathContext(), below).

Use Dataset.Population when the elements are the entire dataset (the variance is divided by n), and Dataset.Sample when they are a sample of a larger population (the variance is divided by n - 1). The sample standard deviation of a single value is undefined, so when using Dataset.Sample nothing is emitted until at least two values have been seen.

Signatures

movingStandardDeviationBy(Dataset dataset, int windowSize, Function<INPUT, BigDecimal> mappingFunction)

dataset - Either Population or Sample
windowSize - How many trailing elements to calculate the standard deviation over at any given point in the stream, must be greater than 1
mappingFunction - A non-null function to map stream INPUT elements into BigDecimal for calculation

Additional Methods

Method	Purpose
`excludePartialValues()`	When calculating the moving standard deviation, and the full size of the window has not yet been reached, the gatherer should suppress emitting values until the lookback window is full. See example.
`treatNullAsZero()`	When encountering a `null` value in a stream, treat it as `BigDecimal.ZERO` instead. See example.
`treatNullAs(BigDecimal replacement)`	When encountering a `null` value in a stream, treat it as the given `replacement` value instead. See example.
`treatNullAsOne()`	When encountering a `null` value in a stream, treat it as `BigDecimal.ONE` instead.
`withMathContext(MathContext mathContext)`	Replace the `MathContext` used for all mathematical operations performed by this gatherer. See example.
`withOriginal()`	Include the original input value from the stream in addition to the calculated value in a `WithOriginal` record. See example.

Examples

Moving standard deviation (population) of window size 3, mapped from an object

record NamedValue(String name, BigDecimal value) {}

Stream
    .of(
        new NamedValue("first",  new BigDecimal("1.0")),
        new NamedValue("second", new BigDecimal("2.0")),
        new NamedValue("third",  new BigDecimal("10.0")),
        new NamedValue("fourth", new BigDecimal("20.0")),
        new NamedValue("fifth",  new BigDecimal("30.0"))
    )
    .gather(Gatherers4j.movingStandardDeviationBy(Dataset.Population, 3, NamedValue::value))
    .toList();

// [
//   BigDecimal("0.0")
//   BigDecimal("0.5")
//   BigDecimal("4.02768199119819")
//   BigDecimal("7.363574011458174")
//   BigDecimal("8.164965809277261")
// ]

Moving standard deviation (sample) of window size 3, mapped from an object

record NamedValue(String name, BigDecimal value) {}

Stream
    .of(
        new NamedValue("first",  new BigDecimal("1.0")),
        new NamedValue("second", new BigDecimal("2.0")),
        new NamedValue("third",  new BigDecimal("10.0")),
        new NamedValue("fourth", new BigDecimal("20.0")),
        new NamedValue("fifth",  new BigDecimal("30.0"))
    )
    .gather(Gatherers4j.movingStandardDeviationBy(Dataset.Sample, 3, NamedValue::value))
    .toList();

// [
//   BigDecimal("0.7071067811865475")
//   BigDecimal("4.932882862316247")
//   BigDecimal("9.018499505645788")
//   BigDecimal("10.0")
// ]

Excluding partial values

Showing that in-process moving standard deviation values are not emitted for each element until the lookback window has been filled.

record NamedValue(String name, BigDecimal value) {}

Stream
    .of(
        new NamedValue("first",  new BigDecimal("1.0")),
        new NamedValue("second", new BigDecimal("2.0")),
        new NamedValue("third",  new BigDecimal("10.0")),
        new NamedValue("fourth", new BigDecimal("20.0")),
        new NamedValue("fifth",  new BigDecimal("30.0"))
    )
    .gather(Gatherers4j.movingStandardDeviationBy(Dataset.Population, 3, NamedValue::value).excludePartialValues())
    .toList();

// [
//   BigDecimal("4.02768199119819")
//   BigDecimal("7.363574011458174")
//   BigDecimal("8.164965809277261")
// ]

Showing nulls are ignored by default

record NamedValue(String name, BigDecimal value) {}

Stream
    .of(
        new NamedValue("first",  null),
        new NamedValue("second", null),
        new NamedValue("third",  new BigDecimal("10.0")),
        new NamedValue("fourth", new BigDecimal("20.0")),
        new NamedValue("fifth",  new BigDecimal("30.0"))
    )
    .gather(Gatherers4j.movingStandardDeviationBy(Dataset.Population, 3, NamedValue::value))
    .toList();

// [
//   BigDecimal("0.0")
//   BigDecimal("5.0")
//   BigDecimal("8.164965809277261")
// ]

Treating null as zero

record NamedValue(String name, BigDecimal value) {}

Stream
    .of(
        new NamedValue("first",  null),
        new NamedValue("second", null),
        new NamedValue("third",  new BigDecimal("10.0")),
        new NamedValue("fourth", new BigDecimal("20.0")),
        new NamedValue("fifth",  new BigDecimal("30.0"))
    )
    .gather(Gatherers4j.movingStandardDeviationBy(Dataset.Population, 3, NamedValue::value).treatNullAsZero())
    .toList();

// [
//   BigDecimal("0")
//   BigDecimal("0")
//   BigDecimal("4.714045207910317")
//   BigDecimal("8.164965809277261")
//   BigDecimal("8.164965809277261")
// ]

Replacing null with another `BigDecimal`

record NamedValue(String name, BigDecimal value) {}

Stream
    .of(
        new NamedValue("first",  null),
        new NamedValue("second", null),
        new NamedValue("third",  new BigDecimal("10.0")),
        new NamedValue("fourth", new BigDecimal("20.0")),
        new NamedValue("fifth",  new BigDecimal("30.0"))
    )
    .gather(Gatherers4j.movingStandardDeviationBy(Dataset.Population, 3, NamedValue::value).treatNullAs(BigDecimal.TWO))
    .toList();

// [
//   BigDecimal("0")
//   BigDecimal("0")
//   BigDecimal("3.771236166328253")
//   BigDecimal("7.363574011458174")
//   BigDecimal("8.164965809277261")
// ]

Specifying a new `MathContext`

record NamedValue(String name, BigDecimal value) {}

Stream
    .of(
        new NamedValue("first",  new BigDecimal("1.0")),
        new NamedValue("second", new BigDecimal("2.0")),
        new NamedValue("third",  new BigDecimal("10.0")),
        new NamedValue("fourth", new BigDecimal("20.0")),
        new NamedValue("fifth",  new BigDecimal("30.0"))
    )
    .gather(Gatherers4j
        .movingStandardDeviationBy(Dataset.Population, 3, NamedValue::value)
        .withMathContext(new MathContext(3, RoundingMode.DOWN))
    )
    .toList();

// [
//   BigDecimal("0.0")
//   BigDecimal("0.5")
//   BigDecimal("4.02")
//   BigDecimal("7.36")
//   BigDecimal("8.16")
// ]

Emitting a record containing the original and calculated values

record NamedValue(String name, BigDecimal value) {}

Stream
    .of(
        new NamedValue("first",  new BigDecimal("1.0")),
        new NamedValue("second", new BigDecimal("2.0")),
        new NamedValue("third",  new BigDecimal("10.0")),
        new NamedValue("fourth", new BigDecimal("20.0")),
        new NamedValue("fifth",  new BigDecimal("30.0"))
    )
    .gather(Gatherers4j.movingStandardDeviationBy(Dataset.Population, 3, NamedValue::value).withOriginal())
    .toList();

// [
//   WithOriginal[original=NamedValue[name=first, value=1.0], calculated=0.0]
//   WithOriginal[original=NamedValue[name=second, value=2.0], calculated=0.5]
//   WithOriginal[original=NamedValue[name=third, value=10.0], calculated=4.02768199119819]
//   WithOriginal[original=NamedValue[name=fourth, value=20.0], calculated=7.363574011458174]
//   WithOriginal[original=NamedValue[name=fifth, value=30.0], calculated=8.164965809277261]
// ]