Finite Difference Approximations for Derivative Estimation

A practical example is estimating acceleration from velocity data. If we have a set of discrete velocity measurements over time, we can use finite-difference approximations to estimate the acceleration (the derivative of velocity) at specific points.

Deriving Approximations via Weighted Averages

To understand how these formulas are built, imagine we have three uniformly spaced data points—$y_0, y_1,$ and $y_2$—with a spacing of $h$. We want to approximate the first derivative at the central point, $x_1$.

We can postulate that the derivative is a weighted average of these three points:

$$\frac{dy}{dx}\bigg|_{x_1} \approx c_0 y_0 + c_1 y_1 + c_2 y_2$$

To find the unknown coefficients $c_0, c_1,$ and $c_2$, we use Taylor series expansions for $y_0$ and $y_2$ centered at $x_1$. By substituting these expansions into our weighted average equation and matching the terms for the derivative, we can solve a system of linear equations.

This process yields the central-difference approximation:

$$\frac{dy}{dx}\bigg|_{x_1} \approx \frac{y_2 - y_0}{2h}$$

where $h$ is the spacing between points.

Exam Tip: The "order of accuracy" tells you how quickly the error vanishes as you decrease the spacing $h$. For a $p$-order scheme, the error is $O(h^p)$. If you halve the spacing ($h/2$) in a second-order scheme ($h^2$), your error drops by a factor of 4.

Accuracy and Boundary Issues

While the central-difference formula is highly efficient, it has a significant limitation: it cannot be used at the boundaries of an interval. Because it requires a data point on both the left and the right, it fails at the very first and very last points of a dataset.

To solve this, we use one-sided difference formulas at the boundaries:

1. Forward-difference approximation (used at the start of an interval): $$\frac{dy}{dx}\bigg|_{x_i} \approx \frac{y_{i+1} - y_i}{h}$$
2. Backward-difference approximation (used at the end of an interval): $$\frac{dy}{dx}\bigg|_{x_i} \approx \frac{y_i - y_{i-1}}{h}$$

Note that these one-sided formulas are only first-order accurate, $O(h)$, meaning they are significantly less accurate than the central-difference method as $h$ gets smaller.

Generalizing the Method: Stencils and Automation

Manually deriving Taylor series for every new problem is tedious. However, we can automate the process by treating it as a linear algebra problem. To find the coefficients for any $q^{th}$ derivative using $m$ data points, we can set up a system of equations $A\mathbf{c} = q!\mathbf{e}_q$, where $A$ is a matrix derived from the Taylor expansions and $\mathbf{c}$ is the vector of coefficients we need to find.

The collection of points and their corresponding weights used to calculate a derivative is known as a stencil. A stencil can be visualized as the relative importance of each data point to the calculation.

By mastering stencils and the underlying discretization process, engineers can move from simple approximations to highly accurate, high-order numerical simulations capable of solving the most complex differential equations.