Differences

I though I'd start this post with some updates about new things - I'm now in my final year of secondary school (Y13) and a few things are quite different than last year. The most notable change is my timetable, having now dropped Biology I find myself with much more free time whilst at school and now have to make sure that I use those 10 free periods a week (45 min each, 7.5 hours total) productively. Not that I didn't last year, of course, but I had half that number and was doing much of my work at home anyway.

Additionally, this term is quite frantic compared to last year. My subject spread has become more maths oriented, with C4, D1 and M3 in maths at the moment and further mechanics in Physics making up all but my chemistry lessons. Add to that personal statements and UCAS forms (and PAT applications and practice) and you have something to fill in those extra five lessons a week!

With that in mind, I want to aknowledge that I have neglected this site recently and want to start updating it a bit more often. In particular I want to start posting blog posts more casually every time I learn something cool. So here's today's cool thing: numerical differentiation.

Hopefully you know what a derivative is - it's the rate of change of a function at a point (specifically the instantaneous rate of change, or graphically it's the gradient of the tangent at a point). This concept is used loads in physics where the velocity is the derivative of position wrt time, and thus you can find it by drawing tangent lines on a position/time graph. You probably also know what an integral is - it's the total change in a function - and how to get it from the area of a graph. We also do this loads in A Level Physics, although since we aren't expected to be able to integrate for some unknown reason, we instead tend to calculate the areas of lots of triangles, rectangles, trapeziums and yes: we count squares.

Leaving aside the ridiculousness of that fact (and the fabled 'weighing the paper' exercise) for a moment, what all that amounts to is numerical integration. In a way, you can view the different methods as successively better ways of integrating numerically: rectangles are 'Riemann sums', trapeziums give the 'trapezium rule', triangles are a special case of trapeziums and ultimately the overall method of cleverly picking out shapes that are easy is similar to Gaussian quadrature. The point is, everyone gets taught how to integrate numerically.

What about differentiation? Well, they're not as useful numerically since for any function that's reasonable it turns out to be pretty easy to differentiate it analytically (i.e. with symbols). But maybe you have a data set of values every 0.1 seconds and want to calculate derivatives without drawing the graph. What then? Well, we can take our cue from integration:

Riemann sums are how integrals are defined, so the equvalent method is the definition of the derivative. This is the part where maths formatting would help, but doesn't exist on WiX:

hf'(x)= f(x+h)-f(x)

As h gets smaller, you get a derivative. But maybe if h isn't quite that small, it will still be pretty good? Well, yes and no. Some analysis will reveal that this systematically undershoots or overshoots depending on whether the curve is convex or concave (i.e. the error term has to do with the second derivative). Very small h will work, but otherwise you're going to start having problems.

The reason is clear graphically: imaging a steep concave curve like e^x. If you connect two points, one lower and one upper, the chord joining them is going to be very steep, whereas the tangent to the lower point will be less steep. Doing the same thing the other way (i.e. to find the tangent of the upper one) will clearly undershoot for the same reason. I think that a very natural way to think of this problem is that it's a lack of symmetry. When h is small, the chord is going to be parallel to the tangent, but at all other points it's always too high.

Maybe a more symmetric option will work?

2hf'(x) = f(x+h) - (fx-h)

This is equivalent to drawing a tangent at one point, then drawing a chord linking two points on either side on the curve. This has a much better chance to be parallel. What's really interesting here is the proof:

consider the two points f(x+h) and f(x-h). There's a formula we can use to approximate the function at these points called a Taylor Expansion. They look like this:

f(x+h) = f(x) + f'(x)h + (1/2) f''(x)h^2 + (1/6) f'''(x)h^3 + ...

f(x -h) = f(x) - f'(x)h + (1/2) f''(x)h^2 - (1/6) f'''(x)h^3 + ...

Rearranging the first equation for f'(x) will give something resembling the original approximation we gave that was not symmetrical. That will show that the error is ~(1/2)hf''(x) for small h. That's okay, but we'd like a higher power on that, because higher powers of h^n go to zero quicker as h goes to zero, and we'd really like our error to go to zero quickly for obvious reasons.

So here's the trick: subtract the two equations. The h^2 terms go away and you get:

f(x+h)-f(x-h) = 2f'(x)h +(1/3)f'''(x)h^3

Dividing through by 2h gives the approximate derivative and shows the error to be O(h^2). The O notation is a handy shorthand which lets you ignore everything but the important fact that the error is proprtional to the square of the 'step size'. This means that if we halve our step size, our accuracy will quadruple. That's very good.

This method will of course generalise to be more accurate, but at the expense of computing loads of functions and differences. The symmetrical difference, however, is so simple and elegant and I can actually remember it which is always nice! Just remember to divide by (2h) not (h).

Oh, and one last thing: in those Taylor expansions, all of the even powers were elimanted when we did the subtraction. That means that if we get rid of one more error term, our error will go from O(h^2) all the way to O(h^4) which is really very accurate indeed. One way to do that would be to write out the error terms explicitly and find the error in close-by points such as f(x+2h). Then it's possible to add a linear combination of function values together such that the O(h^2) term cancels out when you subtract and you end up doubling the accuracy for very little extra work.

Happy Physics.

#meta #mathematics #text