This TedEd video explains a few common sorting methods used in computer science. Sorting can be a challenging computational problem because of the enormous number of comparisons between items that can be involved, so computer scientists has spent a lot of time looking into it.
The video below shows the Quick Sort method using Hungarian folk dance.
Previously, I showed how to solve a simple problem of motion at a constant velocity analytically and numerically. Because of the nature of the problem both solutions gave the same result. Now we’ll try a constant acceleration problem which should highlight some of the key differences between the two approaches, particularly the tradeoffs you must make when using numerical approaches.
The Problem
A ball starts at the origin and moves horizontally with an acceleration of 0.2 m/s2. Print out a table of the ball’s position (in x) with time (every second) for the first 20 seconds.
Analytical Solution
We know that acceleration (a) is the change in velocity with time (t):
so if we integrate acceleration we can find the velocity. Then, as we saw before, velocity (v) is the change in position with time:
which can be integrated to find the position (x) as a function of time.
So, to summarize, to find position as a function of time given only an acceleration, we need to integrate twice: first to get velocity then to get x.
For this problem where the acceleration is a constant 0.2 m/s2 we start with acceleration:
which integrates to give the general solution,
To find the constant of integration we refer to the original question which does not say anything about velocity, so we assume that the initial velocity was 0: i.e.:
at t = 0 we have v = 0;
which we can substitute into the velocity equation to find that, for this problem, c is zero:
making the specific velocity equation:
we replace v with dx/dt and integrate:
This constant of integration can be found since we know that the ball starts at the origin so
at t = 0 we have x = 0, so;
Therefore our final equation for x is:
Summarizing the Analytical
To summarize the analytical solution:
These are all a function of time so it might be more proper to write them as:
Velocity and acceleration represent rates of change which so we could also write these equations as:
or we could even write acceleration as the second differential of the position:
or, if we preferred, we could even write it in prime notation for the differentials:
The Numerical Solution
As we saw before we can determine the position of a moving object if we know its old position (xold) and how much that position has changed (dx).
where the change in position is determined from the fact that velocity (v) is the change in position with time (dx/dt):
which rearranges to:
So to find the new position of an object across a timestep we need two equations:
In this problem we don’t yet have the velocity because it changes with time, but we could use the exact same logic to find velocity since acceleration (a) is the change in velocity with time (dv/dt):
which rearranges to:
and knowing the change in velocity (dv) we can find the velocity using:
Therefore, we have four equations to find the position of an accelerating object (note that in the third equation I’ve replaced v with vnew which is calculated in the second equation):
These we can plug into a python program just so:
motion-01-both.py
from visual import *
# Initialize
x = 0.0
v = 0.0
a = 0.2
dt = 1.0
# Time loop
for t in arange(dt, 20+dt, dt):
# Analytical solution
x_a = 0.1 * t**2
# Numerical solution
dv = a * dt
v = v + dv
dx = v * dt
x = x + dx
# Output
print t, x_a, x
Here, unlike the case with constant velocity, the two methods give slightly different results. The analytical solution is the correct one, so we’ll use it for reference. The numerical solution is off because it does not fully account for the continuous nature of the acceleration: we update the velocity ever timestep (every 1 second), so the velocity changes in chunks.
To get a better result we can reduce the timestep. Using dt = 0.1 gives final results of:
which is much closer, but requires a bit more runtime on the computer. And this is the key tradeoff with numerical solutions: greater accuracy requires smaller timesteps which results in longer runtimes on the computer.
Post Script
To generate a graph of the data use the code:
from visual import *
from visual.graph import *
# Initialize
x = 0.0
v = 0.0
a = 0.2
dt = 1.0
analyticCurve = gcurve(color=color.red)
numericCurve = gcurve(color=color.yellow)
# Time loop
for t in arange(dt, 20+dt, dt):
# Analytical solution
x_a = 0.1 * t**2
# Numerical solution
dv = a * dt
v = v + dv
dx = v * dt
x = x + dx
# Output
print t, x_a, x
analyticCurve.plot(pos=(t, x_a))
numericCurve.plot(pos=(t,x))
We’ve started working on the physics of motion in my programming class, and really it boils down to solving differential equations using numerical methods. Since the class has a calculus co-requisite I thought a good way to approach teaching this would be to first have the solve the basic equations for motion (velocity and acceleration) analytically–using calculus–before we took the numerical approach.
Constant velocity
Question 1. A ball starts at the origin and moves horizontally at a speed of 0.5 m/s. Print out a table of the ball’s position (in x) with time (t) (every second) for the first 20 seconds.
Analytical Solution:
Well, we know that speed is the change in position (in the x direction in this case) with time, so a constant velocity of 0.5 m/s can be written as the differential equation:
To get the ball’s position at a given time we need to integrate this differential equation. It turns out that my calculus students had not gotten to integration yet. So I gave them the 5 minute version, which they were able to pick up pretty quickly since integration’s just the reverse of differentiation, and we were able to move on.
Integrating gives:
which includes a constant of integration (c). This is the general solution to the differential equation. It’s called the general solution because we still can’t use it since we don’t know what c is. We need to find the specific solution for this particular problem.
In order to find c we need to know the actual position of the ball is at one point in time. Fortunately, the problem states that the ball starts at the origin where x=0 so we know that:
at t = 0, x = 0
So we plug these values into the general solution to get:
solving for c gives:
Therefore our specific solution is simply:
And we can write a simple python program to print out the position of the ball every second for 20 seconds:
Numerical Solution:
Finding the numerical solution to the differential equation involves not integrating, which is particularly good if the differential equation can’t be integrated.
We start with the same differential equation for velocity:
but instead of trying to solve it we’ll just approximate a solution by recognizing that we use dx/dy to represent when the change in x and t are really, really small. If we were to assume they weren’t infinitesimally small we would rewrite the equations using deltas instead of d’s:
now we can manipulate this equation using algebra to show that:
so the change in the position at any given moment is just the velocity (0.5 m/s) times the timestep. Therefore, to keep track of the position of the ball we need to just add the change in position to the old position of the ball:
Now we can write a program to calculate the position of the ball using this numerical approximation.
motion-01-numeric.py
from visual import *
# Initialize
x = 0.0
dt = 1.0
# Time loop
for t in arange(dt, 21, dt):
v = 0.5
dx = v * dt
x = x + dx
print t, x
I’m sure you’ve noticed a couple inefficiencies in this program. Primarily, that the velocity v, which is a constant, is set inside the loop, which just means it’s reset to the same value every time the loop loops. However, I’m putting it in there because when we get working on acceleration the velocity will change with time.
I also import the visual library (vpython.org) because it imports the numpy library and we’ll be creating and moving 3d balls in a little bit as well.
Finally, the two statements for calculating dx and x could easily be combined into one. I’m only keeping them separate to be consistent with the math described above.
A Program with both Analytical and Numerical Solutions
For constant velocity problems the numerical approach gives the same results as the analytical solution, but that’s most definitely not going to be the case in the future, so to compare the two results more easily we can combine the two programs into one:
motion-01.py
from visual import *
# Initialize
x = 0.0
dt = 1.0
# Time loop
for t in arange(dt, 21, dt):
v = 0.5
# Analytical solution
x_a = v * t
# Numerical solution
dx = v * dt
x = x + dx
# Output
print t, x_a, x
I’m teaching a numerical methods class that’s partly an introduction to programming, and partly a survey of numerical solutions to different types of problems students might encounter in the wild. I thought I’d look into doing a session on genetic algorithms, which are an important precursor to things like networks that have been found to be useful in a wide variety of fields including image and character recognition, stock market prediction and medical diagnostics.
The ai-junkie, bare-essentials page on genetic algorithms seemed a reasonable place to start. The site is definitely readable and I was able to put together a code to try to solve its example problem: to figure out what series of four mathematical operations using only single digits (e.g. +5*3/2-7) would give target number (42 in this example).
The procedure is as follows:
Initialize: Generate several random sets of four operations,
Test for fitness: Check which ones come closest to the target number,
Select: Select the two best options (which is not quite what the ai-junkie says to do, but it worked better for me),
Mate: Combine the two best options semi-randomly (i.e. exchange some percentage of the operations) to produce a new set of operations
Mutate: swap out some small percentage of the operations randomly,
Repeat: Go back to the second step (and repeat until you hit the target).
And this is the code I came up with:
genetic_algorithm2.py
''' Write a program to combine the sequence of numbers 0123456789 and
the operators */+- to get the target value (42 (as an integer))
'''
'''
Procedure:
1. Randomly generate a few sequences (ns=10) where each sequence is 8
charaters long (ng=8).
2. Check how close the sequence's value is to the target value.
The closer the sequence the higher the weight it will get so use:
w = 1/(value - target)
3. Chose two of the sequences in a way that gives preference to higher
weights.
4. Randomly combine the successful sequences to create new sequences (ns=10)
5. Repeat until target is achieved.
'''
from visual import *
from visual.graph import *
from random import *
import operator
# MODEL PARAMETERS
ns = 100
target_val = 42 #the value the program is trying to achieve
sequence_length = 4 # the number of operators in the sequence
crossover_rate = 0.3
mutation_rate = 0.1
max_itterations = 400
class operation:
def __init__(self, operator = None, number = None, nmin = 0, nmax = 9, type="int"):
if operator == None:
n = randrange(1,5)
if n == 1:
self.operator = "+"
elif n == 2:
self.operator = "-"
elif n == 3:
self.operator = "/"
else:
self.operator = "*"
else:
self.operator = operator
if number == None:
#generate random number from 0-9
self.number = 0
if self.operator == "/":
while self.number == 0:
self.number = randrange(nmin, nmax)
else:
self.number = randrange(nmin, nmax)
else:
self.number = number
self.number = float(self.number)
def calc(self, val=0):
# perform operation given the input value
if self.operator == "+":
val += self.number
elif self.operator == "-":
val -= self.number
elif self.operator == "*":
val *= self.number
elif self.operator == "/":
val /= self.number
return val
class gene:
def __init__(self, n_operations = 5, seq = None):
#seq is a sequence of operations (see class above)
#initalize
self.n_operations = n_operations
#generate sequence
if seq == None:
#print "Generating sequence"
self.seq = []
self.seq.append(operation(operator="+")) # the default operation is + some number
for i in range(n_operations-1):
#generate random number
self.seq.append(operation())
else:
self.seq = seq
self.calc_seq()
#print "Sequence: ", self.seq
def stringify(self):
seq = ""
for i in self.seq:
seq = seq + i.operator + str(i.number)
return seq
def calc_seq(self):
self.val = 0
for i in self.seq:
#print i.calc(self.val)
self.val = i.calc(self.val)
return self.val
def crossover(self, ingene, rate):
# combine this gene with the ingene at the given rate (between 0 and 1)
# of mixing to create two new genes
#print "In 1: ", self.stringify()
#print "In 2: ", ingene.stringify()
new_seq_a = []
new_seq_b = []
for i in range(len(self.seq)):
if (random() < rate): # swap
new_seq_a.append(ingene.seq[i])
new_seq_b.append(self.seq[i])
else:
new_seq_b.append(ingene.seq[i])
new_seq_a.append(self.seq[i])
new_gene_a = gene(seq = new_seq_a)
new_gene_b = gene(seq = new_seq_b)
#print "Out 1:", new_gene_a.stringify()
#print "Out 2:", new_gene_b.stringify()
return (new_gene_a, new_gene_b)
def mutate(self, mutation_rate):
for i in range(1, len(self.seq)):
if random() < mutation_rate:
self.seq[i] = operation()
def weight(target, val):
if val <> None:
#print abs(target - val)
if abs(target - val) <> 0:
w = (1. / abs(target - val))
else:
w = "Bingo"
print "Bingo: target, val = ", target, val
else:
w = 0.
return w
def pick_value(weights):
#given a series of weights randomly pick one of the sequence accounting for
# the values of the weights
# sum all the weights (for normalization)
total = 0
for i in weights:
total += i
# make an array of the normalized cumulative totals of the weights.
cum_wts = []
ctot = 0.0
cum_wts.append(ctot)
for i in range(len(weights)):
ctot += weights[i]/total
cum_wts.append(ctot)
#print cum_wts
# get random number and find where it occurs in array
n = random()
index = randrange(0, len(weights)-1)
for i in range(len(cum_wts)-1):
#print i, cum_wts[i], n, cum_wts[i+1]
if n >= cum_wts[i] and n < cum_wts[i+1]:
index = i
#print "Picked", i
break
return index
def pick_best(weights):
# pick the top two values from the sequences
i1 = -1
i2 = -1
max1 = 0.
max2 = 0.
for i in range(len(weights)):
if weights[i] > max1:
max2 = max1
max1 = weights[i]
i2 = i1
i1 = i
elif weights[i] > max2:
max2 = weights[i]
i2 = i
return (i1, i2)
# Main loop
l_loop = True
loop_num = 0
best_gene = None
##test = gene()
##test.print_seq()
##print test.calc_seq()
# initialize
genes = []
for i in range(ns):
genes.append(gene(n_operations=sequence_length))
#print genes[-1].stringify(), genes[-1].val
f1 = gcurve(color=color.cyan)
while (l_loop and loop_num < max_itterations):
loop_num += 1
if (loop_num%10 == 0):
print "Loop: ", loop_num
# Calculate weights
weights = []
for i in range(ns):
weights.append(weight(target_val, genes[i].val))
# check for hit on target
if weights[-1] == "Bingo":
print "Bingo", genes[i].stringify(), genes[i].val
l_loop = False
best_gene = genes[i]
break
#print weights
if l_loop:
# indicate which was the best fit option (highest weight)
max_w = 0.0
max_i = -1
for i in range(len(weights)):
#print max_w, weights[i]
if weights[i] > max_w:
max_w = weights[i]
max_i = i
best_gene = genes[max_i]
## print "Best operation:", max_i, genes[max_i].stringify(), \
## genes[max_i].val, max_w
f1.plot(pos=(loop_num, best_gene.val))
# Pick parent gene sequences for next generation
# pick first of the genes using weigths for preference
## index = pick_value(weights)
## print "Picked operation: ", index, genes[index].stringify(), \
## genes[index].val, weights[index]
##
## # pick second gene
## index2 = index
## while index2 == index:
## index2 = pick_value(weights)
## print "Picked operation 2:", index2, genes[index2].stringify(), \
## genes[index2].val, weights[index2]
##
(index, index2) = pick_best(weights)
# Crossover: combine genes to get the new population
new_genes = []
for i in range(ns/2):
(a,b) = genes[index].crossover(genes[index2], crossover_rate)
new_genes.append(a)
new_genes.append(b)
# Mutate
for i in new_genes:
i.mutate(mutation_rate)
# update genes array
genes = []
for i in new_genes:
genes.append(i)
print
print "Best Gene:", best_gene.stringify(), best_gene.val
print "Number of iterations:", loop_num
##
When run, the code usually gets a valid answer, but does not always converge: The figure at the top of this post shows it finding a solution after 142 iterations (the solution it found was: +8.0 +8.0 *3.0 -6.0). The code is rough, but is all I have time for at the moment. However, it should be a reasonable starting point if I should decide to discuss these in class.
A quick program that animates scaling (dilation) of shapes by scaling the coordinates. You type in the dilation factor.
dilation.py
from visual import *
#axes
xmin = -10.
xmax = 10.
ymin = -10.
ymax = 10.
xaxis = curve(pos=[(xmin,0),(xmax,0)])
yaxis = curve(pos=[(0,ymin),(0,ymax)])
#tick marks
tic_dx = 1.0
tic_h = .5
for i in arange(xmin,xmax+tic_dx,tic_dx):
tic = curve(pos=[(i,-0.5*tic_h),(i,0.5*tic_h)])
for i in arange(ymin,ymax+tic_dx,tic_dx):
tic = curve(pos=[(-0.5*tic_h,i),(0.5*tic_h,i)])
#stop scene from zooming out too far when the curve is drawn
scene.autoscale = False
# define curve here
shape = curve(pos=[(-1,2), (5,3), (4,-1), (-1,-1)])
shape.append(pos=shape.pos[0])
shape.color = color.yellow
shape.radius = 0.1
shape.visible = True
#dilated shape
dshape = curve(color=color.green, radius=shape.radius*0.9)
for i in shape.pos:
dshape.append(pos=i)
#label
note = label(pos=(5,-8),text="Dilation: 1.0", box=False)
intext = label(pos=(5,-9),text="> x", box=False)
#scaling lines
l_scaling = False
slines = []
for i in range(len(shape.pos)):
slines.append(curve(radius=shape.radius*.5,color=color.red, pos=[shape.pos[i],shape.pos[i],shape.pos[i]]))
#animation parameters
animation_time = 1. #seconds
animation_smootheness = 30
animation_rate = animation_smootheness / animation_time
x = ""
while 1:
#x = raw_input("Enter Dilation: ")
if scene.kb.keys: # event waiting to be processed?
s = scene.kb.getkey() # get keyboard info
#print s
if s <> '\n':
x += s
intext.text = "> x "+x
else:
try:
xfloat = float(x)
note.text = "Dilation: " + x
endpoints = []
dp = []
for i in shape.pos:
endpoints.append(float(x) * i)
dp.append((endpoints[-1]-i)/animation_smootheness)
#print "endpoints: ", endpoints
#print "dp: ", dp
for i in range(animation_smootheness):
for j in range(len(dshape.pos)):
dshape.pos[j] = i*dp[j]+shape.pos[j]
rate(animation_smootheness)
if slines:
for i in range(len(shape.pos)):
slines[i].pos[1] = vector(0,0)
slines[i].pos[-1] = dshape.pos[i]
for i in range(len(shape.pos)):
dshape.pos[i] = endpoints[i]
slines[i].pos[-1] = dshape.pos[i]
for i in range(len(shape.pos)-1):
print shape.pos[i], "--->", dshape.pos[i]
except:
#print "FAIL"
failed = True
intext.text = "> x "
x = ""
This quick program is intended to introduce differentiation as a way of finding the slope of a line. Students know how to find the slope of a tangent line at least conceptually (by drawing). We pick a curve: in this case:
then enter values of x in the program to see how x, the function value and the differential compare to each other.
x
f(x)
f'(x)
0.5
0.25
1
1
1
2
2
2
4
3
9
6
Because it’s quick you have to change the function in the code, and enter the values for x in the python shell.
differentiation_intro_numeric.py
from visual import *
class tangent_line:
def __init__(self):
self.dx = 0.1
self.line = curve()
self.tangent_line = curve()
self.point = sphere(radius=.25,color=color.yellow)
self.point.visible = False
self.label = label(pos=(-5,-8))
'''CHANGE FUNCTION (y) HERE'''
# the original function
def f(self, x):
#y = sin(x)
y = x**2
return y
'''END CHANGE FUNCTION HERE'''
def find_slope(self, x):
sdx = .00001
m = (self.f(x+sdx)-self.f(x))/sdx
return round(m,3)
def draw(self):
for x in arange(xmin, xmax+self.dx, self.dx):
self.line.append(pos=(x, self.f(x)))
def draw_tangent(self, x):
m = self.find_slope(x)
y = self.f(x)
b = y - m * x
print "When x = ", x, " slope = ", m
self.label.text = "point: (%1.2f, %1.2f)\nSlope: %1.2f" % (x,y,m)
self.plot_point(x)
#draw tangent
self.tangent_line.visible = False
self.tangent_line = curve(pos=[(xmin,m*xmin+b),(xmax,m*xmax+b)], color=color.yellow)
def plot_point(self, x):
self.point.visible = True
self.point.pos = (x, self.f(x))
#axes
xmin = -10.
xmax = 10.
ymin = -10.
ymax = 10.
xaxis = curve(pos=[(xmin,0),(xmax,0)])
yaxis = curve(pos=[(0,ymin),(0,ymax)])
#tick marks
tic_dx = 1.0
tic_h = .5
for i in arange(xmin,xmax+tic_dx,tic_dx):
tic = curve(pos=[(i,-0.5*tic_h),(i,0.5*tic_h)])
for i in arange(ymin,ymax+tic_dx,tic_dx):
tic = curve(pos=[(-0.5*tic_h,i),(0.5*tic_h,i)])
#stop scene from zooming out too far when the curve is drawn
scene.autoscale = False
# draw curve
func = tangent_line()
func.draw()
# get input
while 1:
xin = raw_input("Enter x value: ")
func.draw_tangent(float(xin))
For some of my students with devices like Chromebooks, it has been a little challenging finding ways for them to do coding without a simple, built-in interpreter app. One interim option that I’ve found, and like quite a bit is the TutorialsPoint Coding Ground, which has online interfaces for quite a number of languages that are great for testing small programs, including Python.