Caleb Madrigal

Programming and Poetry


DNS Tunneling with Iodine

DNS Tunneling can be useful for getting out of a very restrictive corporate firewall (since almost nobody blocks DNS).

DNS Tunneling works by encoding IP protocol inside of DNS packets. Iodine is a cross-platform implementation of DNS Tunneling. This mini-guide will walk through the process of setting up a DNS Tunnel with Iodine.

Prerequisite: You will need a server with public IP and domain name

Setup DNS Tunnel

Setup DNS Records

Add 2 DNS records to your domain's DNS system:

* `A record`: tunnelhost -> your ip (maps to your server's ip)
* `NS record`: tunnel ->


DNS Setup

So now for me:

* `` now points to ``
* `` now points to ``
    - **This is the ...

Calling synchronous code from asyncio

I recently needed to call some synchronous code from asyncio. Thankfully, asyncio provides the run_in_executor function, which runs the specified function in a different thread. Here is an example of using it: download
import asyncio
import time
from urllib.request import urlopen

def count_to_10():
    for i in range(11):
        print("Counter: {}".format(i))
        yield from asyncio.sleep(.5)

def get_page_len(url):
    # This is the blocking sleep (not the async-friendly one)
    page = urlopen(url).read()
    return len(page)

def run_get_page_len():
   loop = asyncio.get_event_loop()

   future1 = loop.run_in_executor(None, get_page_len, '')

   #data1 = yield from future1
   return future1

def print_data_size():
   data = yield from run_get_page_len()
   print("Data size: {}".format(data))

loop = asyncio.get_event_loop()
tasks = [
    asyncio.async(print_data_size ...

Algorithms in Python

I recently decided to brush up on my algorithms and data structures by writing them in Python. Though these are not optimized, they could be helpful for reference. Here is the Github repo:

A some of the algorithms included:

Recursion with asyncio

Recursion is awesome, but has the downside of growing the stack, which can limit its usefulness. Some languages like Scheme, however, have Tail-call optimization, which lets programmers write Tail-recursive functions that don't grow the call stack. Python does not have Tail-call optimization (TCO), but with asyncio, we can have something like Tail-call optimization. Basically, this method uses the asyncio event loop like a trampoline function.

Example: download
import asyncio

# Tail-recursive factorial using asyncio event loop as a trampoline to
# keep the stack from growing.
def factorial(n, callback, acc=1):
    if n == 0:
        asyncio.async(factorial(n-1, callback, acc*n)) # async -> ensure_future in Python 3.4.4

def done_callback(result):
    print("Result: {}".format(result))
    loop = asyncio.get_event_loop()
    loop.stop ...

Falling alive

Falling alive
Undoing sleep
I awake
At the top of a peak
About to drop straight
Down the wooden track.

And all I can do
Is go with the flow
"You can't fight gravity,
So down you will go!"

And I guess it's not bad
Since falling is life.
We're all racing down
But it makes us alive.
  • Author: Caleb Madrigal
  • Date written: 3/20/2015

To answer the trees

I had to leave -
     [or was it "flee"?]

I felt entrapped by the drywall office halls
     [unable to breathe]
The sanitized artwork hanging in the pale florescent haze 
     reminded me what art should not be - 
          [what life should not be]

I hate the straight lines in man-made buildings
     that make everything feel so...concrete,
          [so unbending]
But I love the trees with their curvy lines that say,
     "reality can bend -
          come bend with us."

So that's why I am here - 
     [to answer the trees]
  • Author: Caleb Madrigal
  • Date written: 3/17/2015

This picture by Melissa Vincent was part of the inspiration for this poem: Melted Nights by Melissa Vincent

Simple D3 demos

I've been playing with d3 recently, and it awesome! I'm especially amazed by the ease of animating transitions with it.

There are plenty of amazing d3 demos out there, but I wanted to write some simpler ones to help me understand the basics of d3. You can find them here:

For example, here's how to draw a circle with d3:


  var width = 200;
  var height = 200;

  var x = width/2;
  var y = height/2;
  var ...

Spruce Needle Silhouette

Spruce Needle Silhouette

This is my 2nd painting (painted 2014-01-10). I like how it came out, but almost gave up while painting it because I was comparing it to the sky God painted:

Vanilla Sky Sunset Milwaukee WI

I took this photo of a beautiful "vanilla sky" outside of our home in Milwaukee, WI. It served as my model for the painting (along with other memories of looking at sunsets from beneath the spruce tree).

Dorian Gray Prelude

This year (2014) has been, for me, a year of art exploration. And to end it, I'd just like to share what I believe to be the greatest prelude ever written: the Prelude to The Picture of Dorian Gray by Oscar Wilde:

"The artist is the creator of beautiful things. To reveal art and conceal the artist is art's aim. The critic is he who can translate into another manner or a new material his impression of beautiful things.

The highest as the lowest form of criticism is a mode of autobiography. Those who find ugly meanings in beautiful things are corrupt without being charming. This is a fault.

Those who find beautiful meanings in beautiful things are the cultivated. For these there is hope. They ...


I recently finished reading Phantastes by George MacDonald for the 2nd time. The first time I read it, I really liked it. This second time, it became one of my all-time favorites. I've never experience any other work of art which feels so much like being in a dream.

To give you an idea of what it's about, here is the end of the Prelude:

"In a genuine fairy-story, everything must be miraculous, mysterious, and interrelated; everything must be alive, each in its own way. The whole of Nature must be wondrously blended with the whole world of the Spirit. In fairy-story the time of anarchy, lawlessness, freedom, the natural state of Nature makes itself felt in the world... The wolrd of the fairy-story is that ...

First painting

Yesterday (12/13/2014), my wife and I went to a painting studio to try our hands at painting. I loved the experience! Here is what I came out with:

My first painting

This was my first time painting since I was a kid doing watercolors in school. Definitely plan to try it again!

Why I changed my mind about Node.js


I first heard of Node.js about 4 years ago. I thought it was a ridiculous idea: using JavaScript as your server-side language! JavaScript always felt to me like an inferior language. Why would you use it unless you absolutely had to!? Now of course, you have to do JavaScript if you are writing code which runs in the browser, but on the server, you can use any language you choose. So why choose JavaScript?

I did choose to use JavaScript/Node.js on my current project. I have now been using it for the last several months, and I am quite impressed! Let me tell you what made me change my mind about Node.js.

Adoption by the Enterprise world

First, Node.js has been adopted ...

Sessions in Node.js and Express.js

In the process of trying to explain to a friend how sessions work, I decided to write a barebones example to demonstrate how to do sessions in Node.js with Express.js.

Here's a live demo:

And Here's what it looks like:

Before you enter your name:

After you enter your name:

And here is the code:

var express = require('express');
var cookieParser = require('cookie-parser');
var bodyParser = require('body-parser');
var session = require('cookie-session');

///////////////////////////////////////////////////////////////////////////////////////// MIDDLEWARE

var app = express();

// Needed to handle JSON posts

// Cookie parsing needed for sessions

// Session framework
app.use(session({secret: 'notsosecretkey123'}));

// Consider all URLs under /public/ as static files, and return them raw.
app.use(express.static(__dirname + '/public'));

/////////////////////////////////////////////////////////////////////////////////////////// HANDLERS

function ...

On Abstract Art

Too much detail in art is profanity - a vulgar attempt to sum up that which is beyond finite expression.

How to SSH Tunnel

When to use an SSH Tunnel?

An SSH Tunnel is perfect for the following scenarios:

  • You want to get to a website that a corporate network is blocking.
  • You want to encrypt the network traffic for a non-encrypted protocol.

What is an SSH Tunnel?

An SSH Tunnel is where you tell your local SSH Client to forward your local computer's traffic to a specified SSH Server (over the encrypted SSH protocol). The SSH Server will then act as a proxy for all requests you make on your local computer.

SSH Tunnel

How to set up an SSH Tunnel

There are 2 steps to set up an SSH Tunnel, and they both happen on your local computer (no configuration is needed on the SSH server which will be acting as ...

Art is a laser

The best art is more like a laser beam than a flood light - it doesn't hit everyone, but those it hits, it burns.

Fuel efficiency difference cost

I recently was car shopping, and I had the question about gas mileage: "How much does, say, the difference between 25 mpg and 30 mpg cost?"

To answer this question, I did a bit of analysis using IPython Notebook...


  • To find how much different fuel efficiencies cost in terms of dollars per year.
  • I want to estimate this value for the next 10 or so years.


  • I'm estimating the average price per gallon of gas (over the next 10 years) to be $4.50. It is around $3.60 now, I adjusted for inflation, and counter-adjusted for the time-value of money.
  • I'm estimating that we will drive about 14,000 miles per year (based on the last 3 years).


In [1]:
price_per_gallon = 4 ...

I cry to the night

I cry to the night
To the one in the sky
My echoes fly
With the birds on high.

My soul is one
With the trees and the sun
For we are all sung
From the same divine lung.

And He is near
And the whisper is clear
"My love, my dear
I am with you here."
  • Author: Caleb Madrigal
  • Date written: 5/18/2014
  • Inspiration: Sometimes, when I’m praying outside, I feel enfolded by nature - by God.

Together we will live forever

"We all must die."
"That's beautiful!", she cried.

But for now I will hold you in my embrace
Though fleeting be this time and place.

Fear not my love, there is more!
Just wait till we reach that other shore.

Beauty will become us - you and I,
And "together we will live forever" in the sky.
  • Author: Caleb Madrigal
  • Date written: 4/14/2014
  • Inspiration: Clint Mansell's "Together We Will Live Forever", Darren Aronofsky's "The Fountain", The Flaming Lips' "Do You Realize?". Every person you know will die. So let's not waste our life - let's live to the fullest, and hope for what is to come.

Ali Jardine ocean sky

Beautiful words

Over the last several years, I've been writing down words (or sets of words) which I consider to be particularly beautiful.

Here is my list:

  • Camp Lejeune
  • Long Boat Key
  • Devil May Cry
  • Effervescent
  • Cellar Door
  • Paramore
  • Nevermore
  • Evermore
  • Cognac
  • vis-à-vis
  • Commandeer

Art begets art

Art begets art
Hate begets hate
Love begets love
Grace begets grace.

Before we depart
We do place
A lasting mark
On others' fate.
  • Author: Caleb Madrigal
  • Date written: 3/18/2014
  • Inspiration: I've found that taking in good art makes me want to write.
  • Note: This isn't so much a poem as a truth put into a rhythmic format. It's a bit too "teachy" to be real poetry, in my opinion. Real poetry should not try to teach directly, but rather, should convey feelings (which may indirectly teach something).

Analogies are like functions

Analogies are to functions as facts are to variables - facts are data which the mind can reference, but analogies change the way the mind works.

Whenever possible, I try to explain things with analogies.


I feel gray
Cannot stay
In this daze.

Even pain - 
Let it rain!
but this haze.

I need light
Or even night.
These gray skies
Glaze my eyes.

Must awaken
To elation.
  • Author: Caleb Madrigal
  • Date written: 3/17/2014
  • Inspiration: I was in a gray mood today, and when I looked outside and saw the gray skies, I thought, “the weather matches my mood… or does my mood match the weather?” I didn’t feel like staying in this gray mood.

On the words of a song

Carried along 
On the words of a song,
We let ourselves go
In the waves as they roll.

This ebb and flow
Entraps our souls.
Enraptured so,
As our feelings grow.

And into this torrent,
Reason forfeit,
Our hearts implore us
To join the chorus.
  • Author: Caleb Madrigal
  • Date written: 3/10/2014
  • Inspiration: Music is universal. We enjoy it; we cannot help ourselves. And we want to join with it.

The deep ocean skies

Stay away! Stay away!
Keep away from the bay!
You know that at night,
Will in come the tide,
Sweeping out the mind
That is near the seaside.

And into this sky
You'll sink and you'll fly,
Fish floating by,
Enchanting your eyes,
No more to rise
From the deep ocean skies.
  • Author: Caleb Madrigal
  • Date written: 3/10/2014
  • Inspiration: Sometimes, when I need to get work done, I have to stay away from things which will get me into an artistic, or I'll be swept up in it for hours.

I think Ali Jardine was going for the same idea in this beautiful picture:

Ali Jardine ocean sky

The right side of my mind

Oft on cold and starry nights,
He raises arms, and the battle fights.
And storms the castle of my mind,
Removing from the throne my logical side.

And having the ship commandeered,
Into the stormy winds he steers,
And brings us into crashing waves -
Deep oceans are within his gaze.

And rejoicing on the captain's deck,
Tiring of appointments to respect,
He does not care if we should wreck
But only in beauty does he invest.

Careless and wild - this side of mine.
Roaring through life, not glancing behind
"Behold! the sails are filled with wind!
Onward towards the moon again!"
  • Author: Caleb Madrigal
  • Date written: 3/10/2014
  • Inspiration: What it feels like when my right-brain takes control. Often, starry nights and moonlight provoke me.

The stars as they shine

The stars as they shine
Will begin to divide
And in the sky
The Morning’s alive
Which never will die,
And then we ride
To eternal life
And to find our time
Was not lived a lie,
And to meet His smile
And to see His face
To bask in His grace
And to in love, embrace.
  • Author: Caleb Madrigal
  • Date written: 3/9/2014
  • Inspiration: Riding on a bus, looking at the beautiful sky, and thinking about meeting God.

That romantic flow

Life is beautiful
I feel it inside
But how can I express
The inner mind?
I suppose I can try
Let’s take a ride…

Heart overflowing
No need to hide
Love is a real thing
And there’s a spiritual side.

There is magic
Hidden from our eyes
Yet felt with the soul
That romantic flow.

And to this goal
I continue to stroll
Seeking that beacon
My spirit to strengthen.
  • Author: Caleb Madrigal
  • Date written: 3/9/2014
  • Inspiration: Looking at the sunrise in Chicago over the lake, after a night casually dressed and deep in conversation.

Passion not spent

What's my great fear?
I'll tell you; come near.
To lay down in death
with so much left.

Passion not spent -
Oh cowardly regret!
For fear of others?
The thousand deaths.

I'm afraid to die
With no twinkle in my eye
To pass meagerly by
Yet hidden inside.     

To walk through life
Not truly alive
And to leave in the night
With an unfelt "goodbye".
  • Author: Caleb Madrigal
  • Date written: 3/7/2014
  • Inspiration: An explanation of why living from my heart is so important to me.

A flip of her eyes

A flip of her eyes - eruption of mind
She sends me a smile - god I'm alive
Sensation of body - her's against mine
Can this not be forever? Oh curse of time!
  • Author: Caleb Madrigal
  • Date written: 3/5/2014
  • Inspiration: My wife :)

Experiencing fully the beautiful present

He views the world through an unfiltered mind -
No defenses built up, layers over time.
He sees it in all its shining brilliance
Experiencing fully the beautiful present.
  • Author: Caleb Madrigal
  • Date written: 3/4/2014
  • Inspiration: Rafe (my son) - when I look at his eyes, it looks like he is just driniking everything in.

Words are a song

Words are a song, and I drink them in
And drunken, fall from the parapet -
Into the heart and out of the mind,
And swim in the shimmering ocean inside.
  • Author: Caleb Madrigal
  • Date written: 3/3/2014
  • Inspiration: Molly Bloom's Soliloquy (James Joyce), Yayo (Lana Del Rey)

Poetic Mood

In the last week or so (starting Monday, March 3, 2014) I've been in a very poetic mood. It has been so interesting - it was light a light switch flipping on. I haven't been sitting down and trying to write poetry, they've just been coming to me. Often, as soon as one ends, another begins. It has been so pleasurable. I hope people enjoy them, but even if they don't, it has been so worth it to me!

NodeJS on Heroku

I've recently tried both Heroku and Node.js for the first time. Both are very simple, and work very nicely together. This is a quick guide to deploying a Node.js app to Heroku. To make it simple, I'll walk through deploying a Node.js sample project I created. It is a very simple message board.

Here are the steps to getting this simple app up and running on Heroku...

Get Heroku account

  • Go to Heroku, and click Sign Up
  • Download and install the Heroku Toolbeld for your operating system (which gives you the heroku command in your terminal)
  • Open your terminal and run heroku login, and enter your heroku credentials and the prompts

Deploy your app to heroku

In your terminal, do the ...

Display List as Table in IPython Notebook

IPython Notebook provides hook methods like _repr_html_ which can be defined by Python objects to allow richer representations. Here's an example:

In [1]:
class ListTable(list):
    """ Overridden list class which takes a 2-dimensional list of 
        the form [[1,2,3],[4,5,6]], and renders an HTML Table in 
        IPython Notebook. """
    def _repr_html_(self):
        html = ["<table>"]
        for row in self:
            for col in row:
        return ''.join(html)
In [23]:
import random
table = ListTable()
table.append(['x', 'y', 'x-y', '(x-y)^2'])
for i in xrange(7):
    x = random.uniform(0, 10)
    y = random.uniform(0, 10)
    table.append([x, y, x-y, (x-y)**2])
xyx-y(x-y ...

How To Draw Lines With Matplotlib

Simple example to show how to draw lines with Matplotlib (in IPython Notebook).

ipython notebook --pylab inline

In [5]:
import matplotlib.lines as lines
fig, ax = plt.subplots()

fig.set_size_inches(6,6)          # Make graph square
scatter([-0.1],[-0.1],s=0.01)     # Move graph window a little left and down

line1 = [(0,0), (1,0)]
line2 = [(0,0), (0,1)]

# Note that the Line2D takes a list of x values and a list of y values,
# not 2 points as one might expect.  So we have to convert our points
# an x-list and a y-list.
(line1_xs, line1_ys) = zip(*line1)
(line2_xs, line2_ys) = zip(*line2)

ax.add_line(Line2D(line1_xs, line1_ys, linewidth=2, color='blue'))
ax.add_line(Line2D(line2_xs, line2_ys, linewidth=2, color='red'))
Read On ↵

Opposite of zip function in Python

Is there a way to do the opposite of what the zip() function does? It turns out, there is - the zip() function with list unpacking...

Normal zip() usage

In [5]:
x = [1,2,3,4,5]
y = [10,20,30,40,50]
points = zip(x,y)
[(1, 10), (2, 20), (3, 30), (4, 40), (5, 50)]

Reverse of normal zip() with list unpacking

By using list unpacking (by using the asterisk before the list), zip can effectively act in reverse...

In [6]:
[(1, 2, 3, 4, 5), (10, 20, 30, 40, 50)]

Example of usefulness

In [19]:
import random
rand = lambda: random.gauss(mu=1, sigma=1)
points = [(rand(), rand()) for i in xrange(1000)]

# Make the graph square
fig = plt ...

Monte Carlo In Python

Monte Carlo Estimation of Area

Monte Carlo Estimation

Monte Carlo Estimation is a method of numerically estimating things which we don't (or can't) calculate numerically by randomly generating samples. In this IPython Notebook, I'm going to use Monte Carlo Estimation to estimate:

  1. The area under a curve
  2. The value of \(\pi\)

Monte Carlo Estimation of Area

Let's use Monte Carlo Estimation to estimate the area onder this curve (from 0 to 10):

\[y = 5 * \sin(6~x) + \sin(2~x) + 7\]

Here are the basic steps:

  1. Define a rectangle which encloses the part of the curve for which we want to find area.
  2. Randomly generate points within that region.
  3. Find which of the randomly generated points are under the curve by checking them against the equation of the function ...

Wordpress to Pelican Reasons

I recently migrated my blog from Wordpress to Pelican. Pelican is a static-site generating blog system which is written in Python and uses Jinja2 for templating. I'll probably do a post about the migration process later, but for now, I'll just give my reasons for moving to Pelican...

Data Longevity

I didn't want my blog data stored in a database; I vastly prefer it being stored in version-controlled Markdown format.


  • I wanted to write blog posts in Markdown (which is possible in Wordpress, but Wordpress isn't designed to use Markdown).
  • I also get to use vim to write my blog posts now, which is much nicer than the Wordpress editor.


  • I hate PHP, but love Python. It worried me that my blog ...

IPython Notebook on a VPS


This is a guide to set up IPython Notebook on a Server - specifically, on a DigitalOcean VPS. This will allow you to access your iPython Notebooks from anywhere.

Overview of Steps:

  • Set up a domain name
  • Get a VPS
  • Install IPython Notebook (and all dependencies)
  • Configure IPython Notebook to run in a server mode
  • Add SSL
  • Make IPython Notebook start automatically

Create a domain

Go to and click "Setup an account here" Go through the signup form Click on the activation link they send to your email This will bring you back to their site; Click the link you see there called "Add a subdomain" Here is how I filled out the form:

Create Domain


  • Leave the Destination alone for now, and leave the ...

Big graphs in IPython Notebook

I've been doing a good bit of graphing in IPython Notebook recently, and I often wanted to make the graphs larger. I also often wanted to label the graph axes. So I wrote this simple function and have been using it a lot.

# Graphing helper function
def setup_graph(title='', x_label='', y_label='', fig_size=None):
    fig = plt.figure()
    if fig_size != None:
        fig.set_size_inches(fig_size[0], fig_size[1])
    ax = fig.add_subplot(111)

Here's a demo of using it...

In [27]:
x = linspace(0, 2 * pi, 1000)
y = 5 * sin(1 * 2*pi*x) + 4 * sin(2 * 2*pi*x)

Without setup_graph()

In [24]:
_ = plot(x, y)
Read On ↵

Finding the Golden Ratio

I recently was exploring the Golden Ratio. I was really surprised what a simple (and recursive) relationship it comes from:


Starting with this this image, here is the derivation of the golden ratio...

From this relationship we see in this image, we can derive this equation: \[\frac{a}{b} = \frac{a+b}{a}\]

And if we then let b=1 (so we can find the ratio of a to 1), we get: \[a = \frac{a+1}{a}\]

Then, with this formula, we can define a function: \[f(a) = \frac{a+1}{a}\]

If we then find the fixed-point in this function (so that f(x) = x), then we will have found a solution to \[a = \frac{a+1}{a}\]

A simple way to think about the fixed-point of ...

How to graph with IPython Notebook

IPython Notebook / Matplotlib / Pylab / Numpy is great for graphing (among other things). Below is a simple demo of how to graph with it.

To run IPython Notebook, use this command: ipython notebook --pylab inline

Here's a screenshot:

IPython Notebook Example

Here's an embedded IPython Notebook showing a slightly easier way:

In [5]:
x = linspace(0, 2*pi, 42)
f1 = 5 * sin(x)
f2 = 2 * sin(2*x)
f3 = 1 * sin(3*x)
plot(x, f1)
plot(x, f2, 'ro')
plot(x, f3, 'g--')
Read On ↵

Running Ubuntu on VirtualBox in Windows

I'm mostly a Mac user and I also really like Linux (especially Ubuntu), but I hate Windows. However, I'm occasionally required to use Windows at work, so I've found a way to make using Windows more enjoyable: to use Ubuntu (installed on VirtualBox inside of Windows) instead of Windows for everything except for the few tasks which require Windows.

So I have made this little How to Install Ubuntu on VirtualBox in Windows setup guide:


  • Install VirtualBox
  • Download Ubuntu 12.04 ISO file
  • Create and configure a Virtual Machine
  • Tweak Virtual Machine hardware settings
  • Install Ubuntu in the Virtual Machine
  • Make Ubuntu nicer to use in VirtualBox
  • Using your new Virtual Ubuntu system

Install VirtualBox

  • Download installer from: ...

Pyknon Intro, Chords, and Intervals

I've continued on my (hopefully) short-term fascination with music+math (it's fun, but really bad for productivity). So I found this great library for generating music in Python called Pyknon. And it can be installed using pip: pip install pyknon.

I wrote a little python script just to help me understand some concepts in music theory like intervals and chords. It is meant to be read top to bottom (which is why I intersperse functions and variables throughout). It is NOT written in good modular form (in general, I don't recommend writing python like this). This code can also be used as an intro to the pyknon library.


from pyknon.genmidi import Midi
from import NoteSeq, Note, Rest

####### First we'll generate ...

Music Theory and Math Notes

For the last day, I've done some reading on music theory (trying to understand why things are the way they are in terms of math). Here are my raw unedited notes:

Important musical ratios:

  • Unison: 1:1 frequency
  • Octave: 2:1 frequency
    • 12 semitone increase
  • Fifth: 3:2 frequency (i.e. multiply by 1.5)
    • 7 semitone increase
  • Semitone: 2^(1./12) (12th root of 2) - about 1.059 - this is the "half-step" distance, so you can multiply the frequency of F by 1.059 to get the frequency for F#

Scales and keys

  • Chromatic scale: 12 notes (list of all semitones in an octave)
  • Diatomic scale: 7 notes
  • To be in a "key" (like C-major) means to select 7 notes from the 12 notes in the ...

Reduce with Python and Clojure

I was just playing around writing an annuity calculator function. Here is my first version:

def calculate_annuity(years, interest=0, addition_per_year=0, starting_amount=0):
    result = []
    running_total = starting_amount
    for year in range(years+1):
        running_total = (running_total + addition_per_year) * (1 + interest)
    return result

Here is a 2nd version, written using reduce instead of a loop:

def calculate_annuity2(years, interest=0, addition_per_year=0, starting_amount=0):
    return reduce(lambda result,addition: result + [(result[-1] + addition) * (1 + interest)],
                  [addition_per_year] * years, [starting_amount])

And here is a version in Clojure (basically a direct translation of the Python one:

(defn annuity [years interest addition_per_year init]
        #(conj %1 (* (+ (last %1) %2) (+ 1 interest)))
        (for [i (range years)] addition_per_year)))

Here are some example use cases:

; Find investment returns by year for a $1000 investment ...

Vim run current file

I just added a nice command to my .vimrc file to allow me to run the current file I am working on by typing :R:

command R !./%

On the VIM note, here is a link to my current vim config (.vimrc file and .vim directory) on github: Vim config.

Facial Detection with OpenCV and Python

I was able to get some basic facial detection working in OpenCV with Python. Here's what it looks like:

Facial recognition with OpenCV

And here is the 34-line python script to draw boxes around all detected faces in live video:

import cv

HAAR_CASCADE_PATH = "/opt/local/share/OpenCV/haarcascades/haarcascade_frontalface_default.xml"

def detect_faces(image):
    faces = []
    detected = cv.HaarDetectObjects(image, cascade, storage, 1.2, 2, cv.CV_HAAR_DO_CANNY_PRUNING, (100,100))
    if detected:
        for (x,y,w,h),n in detected:
    return faces

if __name__ == "__main__":
    cv.NamedWindow("Video", cv.CV_WINDOW_AUTOSIZE)

    capture = cv.CaptureFromCAM(CAMERA_INDEX)
    storage = cv.CreateMemStorage()
    cascade = cv.Load(HAAR_CASCADE_PATH)
    faces = []

    i = 0
    while True:
        image = cv.QueryFrame(capture)

        # Only run the Detection algorithm every 5 frames to improve performance
        if i%5==0:
            faces ...

Intro to OpenCV with Python

I have started my journey into the world of OpenCV using Python on my Mac. These are my first steps with it.

Installing OpenCV for Python on the Mac

# Install numpy as a prerequisite
sudo port install numpy

# Install OpenCV with the Python API
sudo port install opencv +python27

(The port command is part of Macports)

Capturing an images from a webcam

#!/usr/bin/env python2.7
import cv

cv.NamedWindow("w1", cv.CV_WINDOW_AUTOSIZE)

camera_index = 0
capture = cv.CaptureFromCAM(camera_index)

frame = cv.QueryFrame(capture)
cv.SaveImage("pic.jpg", frame)

(I tested this with the iSight camera on my Macbook Pro)

Displaying live video

#!/usr/bin/env python2 ...


I realized that I utilize the [NSArray head] and [NSArray tail] functions in various code snippets, but haven.t shared the actual code for that category. So here is my NSArray+FPcode.


#import <Foundation/Foundation.h>
typedef id(^MapBlock)(id);
@interface NSArray (FP)
- (NSArray *)map:(MapBlock)block;
- (id)head;
- (NSArray *)tail;


#import "NSArray+FP.h"
@implementation NSArray (FP)

- (NSArray *)map:(MapBlock)block {
    NSMutableArray *resultArray = [[NSMutableArray alloc] init];
    for (id object in self) {
        [resultArray addObject:block(object)];
    return resultArray;

- (id)head
    return [self objectAtIndex:0];

- (NSArray *)tail
    NSRange tailRange;
    tailRange.location = 1;
    tailRange.length = [self count] - 1;
    return [self subarrayWithRange:tailRange];


Eye movement on web pages

  • I've always preferred bulleted lists (and the like) over flat paragraph formats
  • I believe they can be visually parsed much faster than paragraphs
  • Now I have evidence: F-shaped reading pattern study :)

Here is an image which shows the eye movement of users on web pages (where red means more time was spent looking there): F Reading Pattern

Functional programming to deal with asynchronicity in Objective-C

In the last few weeks, I've used the [ALAssetsLibrary loadImages:callback:] method to load a list of images. For what I was doing, I wanted to wait till all the images were loaded before proceeding. I found that I could solve this problem very elegantly using a recursive solution:

- (void)loadImages:(NSArray *)imageUrls loadedImages:(NSArray *)loadedImages 
                                        callback:(void(^)(NSArray *))callback
    if (imageUrls == nil || [imageUrls count] == 0) {
    else {
        NSURL *head = [imageUrls head];
        __unsafe_unretained id unretained_self = self;        
        ALAssetsLibrary* library = [[ALAssetsLibrary alloc] init];
        [library assetForURL:head resultBlock:^(ALAsset *asset) {
            ALAssetRepresentation *assetRepresentation = asset.defaultRepresentation;

            UIImage *image = [UIImage imageWithCGImage:assetRepresentation.fullResolutionImage 

            [unretained_self loadImages:[imageUrls tail] loadedImages:[loadedImages 
                arrayByAddingObject:image] callback:callback];
        } failureBlock:^(NSError *error) {
            [unretained_self loadImages:[imageUrls tail] loadedImages:loadedImages callback:callback];

In ...

First look at Pylab/Matplotlib

Since I've been getting into Machine Learning/Artificial Intelligence recently, I've been looking at various computing environments recently. Some of the contenders are:

  • MATLAB - The traditional software stack for doing machine learning and statistical analysis
  • GNU Octave - An open-source MATLAB clone.
  • R - An open source clone of a statistical computing environment called S.
  • Julia - A language for doing statistical analysis. The goals are to compete with Matlab and R.
  • Matplotlib/Pylab/SciPy/NumPy - see below

Of these, I've tried Octave and Matplotlib. Matplotlib/Pylab is the software stack consisting of:

  • IPython - an interactive REPL for Python with things like tab completion
  • Matplotlib - a graphical plotting library
  • NumPy - a matrix library
  • SciPy - a collection of scientific and mathematical algorithms

I've only played with Matplotlib/Pylab ...

How to reference self in a block in Objective-C

When using ARC (automatic reference counting) in Objective-C, you need to be careful to avoid causing retain cycles. One place where a retain cycle can occur is where self is referenced in a block. To avoid a retain cycle, you can use the __unsafe_unretained modifier as such:

__unsafe_unretained id unretained_self = self;
reachability = [Reachability reachabilityWithHostname:@""];
reachability.reachableBlock = ^(Reachability *r) {
    dispatch_async(dispatch_get_main_queue(), ^{
        SurveyFormController *myself = unretained_self;
        self.mapView.reachable = YES;
        [myself configureView];

Linear Regression with Gradient Descent

I'm taking the Stanford Machine Learning class. The first algorithm we covered is Linear Regression using Gradient Descent. I implemented this algorithm is Python. Here's what it looks like:

import subprocess

def create_hypothesis(slope, y0):
    return lambda x: slope*x + y0

def linear_regression(data_points, learning_rate=0.01, variance=0.001):
    """ Takes a set of data points in the form: [(1,1), (2,2), ...] and outputs (slope, y0). """

    slope_guess = 1.
    y0_guess = 1.

    last_slope = 1000.
    last_y0 = 1000.
    num_points = len(data_points)

    while (abs(slope_guess-last_slope) > variance or abs(y0_guess - last_y0) > variance):
        last_slope = slope_guess
        last_y0 = y0_guess

        hypothesis = create_hypothesis(slope_guess, y0_guess)

        y0_guess = y0_guess - learning_rate * (1./num_points) * sum([hypothesis(point[0]) - point[1] for point in data_points])
        slope_guess = slope_guess - learning_rate * (1./num_points) * sum([ (hypothesis(point[0]) - point[1]) * point[0] for point ...

Embedding an XSLT in an XML document

Today, I had this XML document which I was wanting to make viewable in a browser. My plan was to run the XML document through an XSLT engine to transform it into an HTML document (which could then be opened in a browser).

Instead, I found that I could just embed the XSLT directly in the XML document. Then, when this file is opened in a browser, the XSLT executes and spits out HTML, which is then rendered into the browser window. This is great because I now have the original XML file (which we wanted to keep further analysis purposes) which can also be rendered nicely in a browser.

Here's an example:

<?xml version="1.0" encoding="ISO-8859-1"?>
<?xml-stylesheet type="text/xml" href="#stylesheet"?>
<!DOCTYPE catelog ...

Resampling Wheel Algorithm

I've been taking the Udacity CS373 (Robotic Car) class, and this week the topic was Particle filters. Particle filters are basically a localization algorithm that accounts for error in measurements and sensors.

Anyway, part of the Particle Filter algorithm requires the generation of a new set of these things called "particles" based on the particles' weights. So to accomplish this task, the Resample Wheel algorithm was presented in class. It is a particularly elegant method of generating a new set of particles by randomly drawing from an old set of particles (with replacement). The particle weight determines the likelihood of it being picked.

Here is the algorithm (with print statements - I used these to help me understand how the algorithm works):

import random

def generate_new_particles(old_particles, weights ...

JSONP for cross-origin AJAX

I was recently wanting to enable a webpage to utilize some server-side functionality which was on a different server than the one serving the original page. Traditionally, this has been problematic due to browsers' Same origin policy. JSONP is a solution to this problem. Here is how to do it with jQuery...

This webpage can be saved locally to any machine, opened in a browser, and can then load data from my web server:

<script src="" type="text/javascript"></script>
<script type="text/javascript">
            jsonpCallback: "jsonp_load",
    <button ...

Tail call optimization in Python

Since I've been getting into functional programming more recently, the fact that Python doesn't do tail-call optimization has been bothering me. So I did a bit of searching, and found this gem: Tail Call Optimization Decorator.

Here is a snippet from it:

import sys

class TailRecurseException:
  def __init__(self, args, kwargs):
    self.args = args
    self.kwargs = kwargs

def tail_call_optimized(g):
  This function decorates a function with tail call
  optimization. It does this by throwing an exception
  if it is it's own grandparent, and catching such
  exceptions to fake the tail call optimization.

  This function fails if the decorated
  function recurses in a non-tail context.
  def func(*args, **kwargs):
    f = sys._getframe()
    if f.f_back and f.f_back.f_back \
        and f.f_back.f_back.f_code == f.f_code ...

Standard Deviation in Python

I just wanted to go through the process of calculating standard deviation today, and this is how I did it in Python. Python makes such a nice calculator :)

>>> s = [2,4,4,4,5,5,7,9]
>>> def average(s): return sum(s) * 1.0 / len(s)
>>> avg = average(s)
>>> avg
>>> variance = map(lambda x: (x - avg)**2, s)
>>> variance
[9.0, 1.0, 1.0, 1.0, 0.0, 0.0, 4.0, 16.0]
>>> average(variance)
>>> import math
>>> standard_deviation = math.sqrt(average(variance))
>>> standard_deviation

Django on Hostmonster

I just finished going through the process of installing and configuring Django (with FastCGI) on Hostmonster's Shared hosting. It was more painful than I expected, so I decided write a post about how I got it working...

First, I wanted to create a subdomain which would host my django stuff. In order to do this, I created a subdomain using the cPanel (the item is called "Subdomains"). This created the directory and a basic .htaccess file (along with a few error pages).

I then went through the basic processes described on these pages:

When I finished these guides, I kept getting a 500 error. I eventually found that I can access the error log ...

String to Base64 String in Objective-C

Today, I needed to convert an ASCII-encoded NSString to a base64 NSString. I found a number of methods that convert from an NSString to a Base64 NSData, or from NSData to a Base64 string, but couldn't find one that did exactly what I needed. So here is a method that simply converts an NSString to a Base64 NSString:

+ (NSString *)base64String:(NSString *)str
    NSData *theData = [str dataUsingEncoding: NSASCIIStringEncoding];
    const uint8_t* input = (const uint8_t*)[theData bytes];
    NSInteger length = [theData length];

    static char table[] = "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/=";

    NSMutableData* data = [NSMutableData dataWithLength:((length + 2) / 3) * 4];
    uint8_t* output = (uint8_t*)data.mutableBytes;

    NSInteger i;
    for (i=0; i < length; i += 3) {
        NSInteger value = 0;
        NSInteger j;
        for (j = i; j < (i + 3); j++) {
            value <<= 8;

            if (j < length) {
                value |= (0xFF & input[j]);

        NSInteger theIndex ...

Function application and function composition in Haskell

In the process of learning haskell, I've been trying to become comfortable with the function applicator ($) and function composition (.). Here is some code to show how they relate (and how they relate to using parenthesis to enforce order of operations):

let dict = [(1,'a'),(2,'b'),(3,'c')]
-- The following 3 lines of code are equivalent...
snd . head . filter (\item@(k,v) -> k == 2) $ dict -- returns 'b'
snd $ head $ filter (\item@(k,v) -> k == 2) dict -- returns 'b'
snd (head (filter (\item@(k,v) -> k == 2) dict)) -- return 'b'

Map in Objective-C

Today, I was faced with an issue several times where I had an array of items of one type, and had to convert this into an array of items of a different type. This made me really want a map() function. So I added one to NSArray like so:


#import <Foundation/Foundation.h>
typedef id(^MapBlock)(id);
@interface NSArray (FP)
- (NSArray *)map:(MapBlock)block;


#import "NSArray+FP.h"
@implementation NSArray (FP)
- (NSArray *)map:(MapBlock)block {
    NSMutableArray *resultArray = [[NSMutableArray alloc] init];
    for (id object in self) {
        [resultArray addObject:block(object)];
    return resultArray;

So now, I can do this!

NSArray *strArray = [NSArray arrayWithObjects:@"1",@"1",@"2",@"3",@"5",@"8",nil];
NSArray *numArray = [strArray map:^id(id object) {
    NSString *str = object;
    return [NSNumber ...

How to place custom image in UITextField in iOS

Here is how to put an image in the right portion of a UITextField:

UITextField *textField = [[UITextField alloc] initWithFrame:CGRectMake(55, 10, 660, 30)];
textField.rightViewMode = UITextFieldViewModeWhileEditing;

UIImageView *imageView = [[UIImageView alloc] initWithFrame:CGRectMake(0, 0, 15, 15)];
imageView.image = [UIImage imageNamed:@"invalidFieldCircle"];
textField.rightView = imageView;

This also assumes that an image called "invalidFieldCircle.png" is in your project's "Resources" directory, which should be located just below the main project directory (you may have to create it).

Force weak reference in Objective-C

Today, I was trying to add an object to an NSDictionary and force it to be a weak reference. This can be done with the [NSValue valueWithNonretainedObject] call:

NSValue *weakRef = [NSValue valueWithNonretainedObject:loadDelegate];
NSDictionary *userInfo = [[NSDictionary alloc] initWithObjectsAndKeys:weakRef, @"loadDelegate", nil];

First steps in haskell

Just started looking into the Haskell programming language by going through the first few chapters of the awesome book: Learn You a Haskell which is free to read online. Here are some things I really like about what I have seen of Haskell so far:

  • Type inference allows you to have static typing but not all the verbosity of most other statically-typed languages.
  • Lazy evaluation is huge in Haskell, and it lets you do things like this: let factorial n = product (take n [1..]) So here, we have an infinite list ([1..] creates a infinite list from 1 to infinity!) which from which the first n items are taken and passed to the product function. Awesomeness...
  • Nice REPL

Consuming web service in Python with SUDS

import datetime
from suds.client import Client
url = "http://localhost:9080/dataplanws/DataPlanWebService/WEB-INF/wsdl/DataPlanWebService.wsdl"
client = Client(url)
print client.service.hello("Caleb")

Periods in Powershell

For running most commands, powershell and CMD work the same. One big difference I've found is that, when you use periods (.) in a command, they must be surrounded by quotation marks. For instance, this won't work:

mvn clean install -U -Docns.web.dynamic.content.version=2.6.0

But this will work:

mvn clean install -U "-Docns.web.dynamic.content.version=2.6.0"