Tutorial :Standard Deviation in LINQ



Question:

Does LINQ model the aggregate SQL function STDDEV() (standard deviation)?

If not, what is the simplest / best-practices way to calculate it?

Example:

  SELECT test_id, AVERAGE(result) avg, STDDEV(result) std       FROM tests  GROUP BY test_id  


Solution:1

You can make your own extension calculating it

public static class Extensions  {      public static double StdDev(this IEnumerable<double> values)      {         double ret = 0;         int count = values.Count();         if (count  > 1)         {            //Compute the Average            double avg = values.Average();              //Perform the Sum of (value-avg)^2            double sum = values.Sum(d => (d - avg) * (d - avg));              //Put it all together            ret = Math.Sqrt(sum / count);         }         return ret;      }  }  

If you have a sample of the population rather than the whole population, then you should use ret = Math.Sqrt(sum / (count - 1));.

Transformed into extension from Adding Standard Deviation to LINQ by Chris Bennett.


Solution:2

Dynami's answer works but makes multiple passes through the data to get a result. This is a single pass method that calculates the sample standard deviation:

public static double StdDev(this IEnumerable<double> values)  {      // ref: http://warrenseen.com/blog/2006/03/13/how-to-calculate-standard-deviation/      double mean = 0.0;      double sum = 0.0;      double stdDev = 0.0;      int n = 0;      foreach (double val in values)      {          n++;          double delta = val - mean;          mean += delta / n;          sum += delta * (val - mean);      }      if (1 < n)          stdDev = Math.Sqrt(sum / (n - 1));        return stdDev;  }  

This is the sample standard deviation since it divides by n - 1. For the normal standard deviation you need to divide by n instead.

This uses Welford's method which has higher numerical accuracy compared to the Average(x^2)-Average(x)^2 method.


Solution:3

This converts David Clarke's answer into an extension that follows the same form as the other aggregate LINQ functions like Average.

Usage would be: var stdev = data.StdDev(o => o.number)

public static class Extensions  {      public static double StdDev<T>(this IEnumerable<T> list, Func<T, double> values)      {          // ref: https://stackoverflow.com/questions/2253874/linq-equivalent-for-standard-deviation          // ref: http://warrenseen.com/blog/2006/03/13/how-to-calculate-standard-deviation/           var mean = 0.0;          var sum = 0.0;          var stdDev = 0.0;          var n = 0;          foreach (var value in list.Select(values))          {              n++;              var delta = value - mean;              mean += delta / n;              sum += delta * (value - mean);          }          if (1 < n)              stdDev = Math.Sqrt(sum / (n - 1));            return stdDev;         }  }   


Solution:4

var stddev = Math.Sqrt(data.Average(z=>z*z)-Math.Pow(data.Average(),2));  


Solution:5

public static double StdDev(this IEnumerable<int> values, bool as_sample = false)  {      var count = values.Count();      if (count > 0) // check for divide by zero      // Get the mean.      double mean = values.Sum() / count;        // Get the sum of the squares of the differences      // between the values and the mean.      var squares_query =          from int value in values          select (value - mean) * (value - mean);      double sum_of_squares = squares_query.Sum();      return Math.Sqrt(sum_of_squares / (count - (as_sample ? 1 : 0)))  }  

Note:If u also have question or solution just comment us below or mail us on toontricks1994@gmail.com
Previous
Next Post »