Standard Deviation is a powerful, evergreen topic that bridges statistics, data science, AI/ML, finance, and education. In the following sections, we’ll explore Standard Deviation Explained: Formula, Examples & Python Code.
What is standard deviation?
Standard deviation is a number that tells us how spread out the values in a dataset are around the mean (average). In simple terms, it shows how much the values vary from the average value. In machine learning, understanding this spread helps us get a clearer picture of the data’s distribution.
To calculate standard deviation, we first find the variance, which is the average of the squared differences between each value and the mean. Then, we take the square root of the variance to get the standard deviation. A small standard deviation means the data points are close to the mean, while a large one means they are more spread out.
In this example, we will be using the NumPy library to calculate the standard deviation
|
1 2 3 4 5 6 |
import numpy as np data = np.array([1, 2, 3, 4, 5, 6]) std_dev = np.std(data) print('Standard deviation:', std_dev) |
Let’s see another example in which we will calculate the standard deviation of each column in the Iris flower dataset using Python and the Pandas library −
|
1 2 3 4 5 6 7 8 9 10 11 12 |
import pandas as pd # load the iris dataset iris_df = pd.read_csv('https://archive.ics.uci.edu/ml/machine-learningdatabases/iris/iris.data', names=['sepal length', 'sepal width', 'petal length', 'petal width', 'class']) # calculate the standard deviation of each column std_devs = iris_df.std() # print the standard deviations print('Standard deviations:') print(std_devs) |
Why Standard Deviation is Important
Standard deviation is important because it measures how spread out the values in a dataset are. A low standard deviation means the data points are close to the mean, while a high standard deviation indicates more variability. This helps in understanding consistency, risk, and reliability in fields like finance, quality control, and data analysis. This helps in:
- Understanding data variability
- Making informed decisions
- Identifying outliers
- Comparing datasets
- Measuring risk in finance and performance in ML
Variance
Variance is a number that tells us how much the values in a dataset differ from the mean (average). It helps us understand how spread out the data is. If the variance is low, it means most of the values are close to the mean, and the data is tightly packed. If the variance is high, it means the values are more spread out and vary more from the mean.
So, a higher variance shows more variability in the data, while a lower variance means the data is more consistent. Understanding variance is important in machine learning because it gives us insight into the data’s behavior and how stable or noisy it might be.
|
1 2 3 4 5 6 7 8 9 10 11 12 |
import statistics data = [10, 20, 30, 40, 50] # Sample variance sample_variance = statistics.variance(data) # Population variance population_variance = statistics.pvariance(data) print("Sample Variance:", sample_variance) print("Population Variance:", population_variance) |
Leave a Comment