Correlation and covariance are two fundamental concepts used to measure the relationship between two variables in statistics. Understanding these helps us describe how variables move together—whether they rise or fall at the same time, and how strongly they do so.
1. Covariance
Covariance measures the **joint variability** of two variables. If two variables tend to increase together, the covariance is positive. If one increases while the other decreases, the covariance is negative.
Cov(X, Y) = Σ[(xᵢ - x̄)(yᵢ - ȳ)] / (n - 1)
- A positive covariance means the variables move in the same direction.
- A negative covariance means they move in opposite directions.
- Covariance is not scaled, so its value is hard to interpret directly.
2. Correlation
Correlation measures both the **strength and direction** of a **linear relationship** between two variables. It's a standardized version of covariance.
r = Cov(X, Y) / (σₓ * σᵧ)
where σₓ and σᵧ are the standard deviations of X and Y.
- Correlation values range from -1 to +1.
- +1 indicates a perfect positive linear relationship.
- -1 indicates a perfect negative linear relationship.
- 0 means no linear relationship.
3. Pearson’s Correlation Coefficient
Pearson's correlation coefficient is the most common method to calculate correlation. It assumes the relationship between variables is linear and that the data is normally distributed.
r = Σ[(xᵢ - x̄)(yᵢ - ȳ)] / √[Σ(xᵢ - x̄)² * Σ(yᵢ - ȳ)²]
- Widely used in fields like psychology, economics, and machine learning.
- Only detects linear relationships; nonlinear associations require other methods.
4. Summary
- Covariance: Tells whether variables vary together (positive/negative).
- Correlation: Standardized form, shows strength and direction (-1 to +1).
- Pearson's r: Most commonly used correlation coefficient for linear relationships.