Introduction to Jupyter Notebooks and Least Squares Fitting

Exam Tip: Managing the Kernel If your code enters an infinite loop or if you encounter a strange error that doesn't seem to make sense, you may need to Restart the Kernel. This clears all variables from the computer's memory and starts the computational engine from a fresh state.

Markdown for Documentation

Markdown is a lightweight markup language that makes it easy to write formatted text using a plain-text editor. In a scientific context, Markdown is essential because it allows you to integrate LaTeX equations directly into your notebook, making your reports look professional and easy to follow.

The Method of Least Squares

Once you are comfortable with the environment, you can apply mathematical optimization techniques. A common task is Least Squares Fitting, where we attempt to find the best parameters for a model given a set of data points $(x_i, y_i)$ for $i = 1, \dots, N$.

Suppose we want to fit a two-parameter model that has no constant term: $y = ax + bx^2$

Defining the Objective Function

To find the "best" values for $a$ and $b$, we must minimize the discrepancy between our model and the actual data. This discrepancy is measured by the Sum of Squared Residuals, denoted as $S(a, b)$. A residual is the difference between the observed value $y_i$ and the value predicted by our model $ax_i + bx_i^2$.

The sum of these squared differences is expressed as:

$$\begin{aligned} S(a, b) = \sum_{i=1}^{N} (y_i - (ax_i + bx_i^2))^2 \end{aligned}$$

Minimization via Calculus

To minimize $S(a, b)$, we take the partial derivatives with respect to our parameters $a$ and $b$ and set them equal to zero:

1. Partial derivative with respect to $a$: $\frac{\partial S}{\partial a} = \sum_{i=1}^{N} 2(y_i - ax_i - bx_i^2)(-x_i) = 0$

2. Partial derivative with respect to $b$: $\frac{\partial S}{\partial b} = \sum_{i=1}^{N} 2(y_i - ax_i - bx_i^2)(-x_i^2) = 0$

By simplifying these equations (dividing by $-2$ and distributing the summation), we transform the problem into a linear system of two equations:

$$\begin{aligned} a \sum x_i^2 + b \sum x_i^3 &= \sum x_i y_i \\ a \sum x_i^3 + b \sum x_i^4 &= \sum x_i^2 y_i \end{aligned}$$

Matrix Representation

In computational programming, it is much more efficient to solve such systems using Matrix Form. We can rewrite the linear system above as:

$$\begin{bmatrix} \sum x_i^2 & \sum x_i^3 \\ \sum x_i^3 & \sum x_i^4 \end{bmatrix} \begin{bmatrix} a \\ b \end{bmatrix} = \begin{bmatrix} \sum x_i y_i \\ \sum x_i^2 y_i \end{bmatrix}$$

By solving this matrix equation (using tools like numpy.linalg.solve in Python), you can find the optimal values for $a$ and $b$ that provide the best fit for your experimental data.

Note on Solvability For a unique solution to exist, the matrix on the left-hand side must be non-singular (invertible). This generally requires that the data points $x_i$ are not all the same and provide enough variation to distinguish between the $x$ and $x^2$ terms.