Got a crazy project you need done? Looking to build an app? Or a robot? Or start to leverage machine learning in your business? My brother and I recently started offering our services at https://madconsulting.ai. Hit us up!
Truthy Graph
One time, I was looking at a graph, and the thought occurred to me, "this graph shows where the two sides of this equation are exactly equal, but what if a graph showed also where they were almost the same?", so I wrote a little graphing web app: https://truthygraph.github.io/.
I called it "Truthy Graph", since it shows where the 2 sides of the equation are nearly true. I think some interesting shapes show up. Here's a few screenshots:
Another cool thing, is that either side of the equation can use the 'y' or 'x' variables in any way (calling functions like sin
, can be squared, etc - the equations need not follow the 'y=' formula). Example:
I'm aware there are a few bugs (especially ...
First musical composition
I decided to try my hand at musical composition. This is my first piece. I call it "We Spin Together".
(This is just the melody)
And here's how it sounds:
Digital Radio Signal Generation
Recently, I was curious if I could generate a digital radio signal from scratch just using math, and use that signal to control this radio-controlled outlet that I have.
Here's the Github source: https://github.com/calebmadrigal/radio-hacking-scripts/blob/master/radio_signal_generation.ipynb
Generating radio signals
I want to be able to generate modulated digital signals such as this one...
This is a binary signal coded with On-off keying (also known as Amplitude-shift keying).
I'm taking the long pulses to mean "1" and the short pulses to mean "0", so this signal is transmitting the digital code, "0110100010000000".
# Imports and boilerplate to make graphs look better
%matplotlib inline
import matplotlib.pyplot as plt
import numpy as np
import scipy
import wave
from ...
Fourier Transform Notes
The following are my notes from a talk I gave at OSCON 2013. Here is the Github source of these Jupyter/IPython notebooks: https://github.com/calebmadrigal/FourierTalkOSCON
Sound Analysis with the Fourier Transform and Python
Link to repo: http://tinyurl.com/fourierpython
Intro
- Welcome
- Format of talk
- Everything is in iPython Notebook (on Github)
- You don't need to take notes
- Please save questions for the end
- Why this is interesting
- Sound processing is big - natural human-machine interfaces (e.g. Siri)
- Noise reduction, Compression, feature extraction (e.g. speech)
- Understanding our universe better (Superposition, Harmonics, Sound timbre)
Overview
- The Nature of Waves
- The Fourier Transform
- Fast Fourier Transform (FFT) in Python
- Audio analysis
%matplotlib inline
import matplotlib.pyplot as plt
import numpy as np
import ...
Hackers are the new lawyers
Hackers are the new Lawyers are the new Knights.
Back in the feudal days, Lords had Knights to protect them and their possessions from enemies (or to attack their neighbors and plunder their wealth).
In the business world today, lawyers are much like the knights of old. Companies can attack each other with the law, and lawyers are both the attacker and defenders.
In the increasingly computerized world, business are hacking each other to steal their Intellectual Property, understand their strategy, and sometimes, take them down all together (like what happened to HBGary, though that wasn't caused by another company). Today, these tactics are employed mostly by Chinese companies hacking competitors in other countries, but as multinational corporations continue to grow in power, this tactic might become ...
HackRF Replay Attack Jeep
I've recently been getting into Software-defined Radio (SDR), mostly using a HackRF - a radio tranceiver capable of operating from 1MHz to 6GHz (which is a huge range).
One of the most simple (and most interesting attacks) which can be done with SDR is what's called a Replay Attack. It works by simply recording a signal, and then rebroadcasting it. I was able to use this attack to lock and unlock my Jeep Patriot (2006) with my computer. Here's how...
First I recorded the "unlock" and "lock" signals from my keyless entry remote using this "flowgraph" in Gnu Radio Companion:
A couple things to note about this flow graph:
- The frequency is 315 MHz (the frequency at which pretty much all keyless entry)
- It records at ...
Editing radio signals with Audacity
You can capture radio signals with Software-defined Radios (SDR), such as the HackRF. Gnu Radio is the main software I use for receiving and transmitting radio signals, but I've found Audacity, a program meant primarily for editing sound files, to be a great program for viewing and editing radio signals.
So I had these remote-controlled outlets (which I bought at Home Depot):
And I wanted to see what their wireless communication looked like...
So first I recorded the signals of the different outlets with this Gnu Radio Companion flowgraph:
A few things to note about this flowgraph:
- It records at around 315 MHz, the frequency at which the remote transmits (which I found by looking up it's FCC ID code online)
- It records 2 million samples ...
HackRF Setup OSX 2016
These are some pretty raw notes I took while installing the software for using a HackRF on OSX. This is the software I installed:
- Gnu Radio (which comes with Gnu Radio Companion)
- Osmocom Gnu Radio Blocks
- gqrx
Gnu Radio Installation
Installed gnuradio via Homebrew
brew update
brew tap robotastic/homebrew-hackrf
brew install gnuradio --with-qt
brew install hackrf
brew install --HEAD rtlsdr
brew link --overwrite rtlsdr
This got GNU Radio and GNU Radio Companion installed, but then I needed to install the Osmocom Gnu Radio Blocks so that Gnu Radio could communicate with the HackRF.
Installing Osmocom Gnu Radio Blocks
This didn't work:
brew install --HEAD gr-osmosdr
Build with directions here: http://sdr.osmocom.org/trac/wiki/GrOsmoSDR
Had to add this to /Users/caleb/Documents/gnu_radio/repos ...
Raspberry Pi Home Security System
3 Years ago (2013), I built my own home security/automation system with a Raspberry Pi. Here's what it looks like:
What it does
- Allows controlling a couple outlets via a mobile web app
- Alarm system enabled/disabled via mobile web app
- If alarm enabled, magnetic door sensor would trigger siren and turn on outlets
Hardware used
- Raspberry Pi - $35
- Edimax EW-7811Un 150Mbps 11n Wi-Fi USB Adapter - $10
- Off-the-shelf remote controlled outlet kit - $20
- Off-the-shelf wireless magnetic door sensor/outlet - $25
- Car siren - $8
- Breadboard, transistors, photoresistor, resistors, wires, soldering iron
Total cost of parts: ~$115
Software used
I wrote the controller software in Python (with ZeroMQ, Flask, Rpi.GPIO, and jQuery Mobile): https://github.com/calebmadrigal/rpi-home-automation.
It's made of up 3 components:
DNS Tunneling with Iodine
DNS Tunneling can be useful for getting out of a very restrictive corporate firewall (since almost nobody blocks DNS).
DNS Tunneling works by encoding IP protocol inside of DNS packets. Iodine is a cross-platform implementation of DNS Tunneling. This mini-guide will walk through the process of setting up a DNS Tunnel with Iodine.
Prerequisite: You will need a server with public IP and domain name
Setup DNS Tunnel
Setup DNS Records
Add 2 DNS records to your domain's DNS system:
* `A record`: tunnelhost -> your ip (maps tunnelhost.yourdomain.com to your server's ip)
* `NS record`: tunnel -> tunnelhost.yourdomain.com
Example:
So now for me:
* `tunnelhost.calebmadrigal.com` now points to `104.236.122.169`
* `tunnel.calebmadrigal.com` now points to `tunnelhost.calebmadrigal.com`
- **This is the ...
Golden sky painting
Just another painting. I did this one earlier this year.
Calling synchronous code from asyncio
I recently needed to call some synchronous code from asyncio. Thankfully, asyncio provides the run_in_executor
function, which runs the specified function in a different thread. Here is an example of using it:
Algorithms in Python
I recently decided to brush up on my algorithms and data structures by writing them in Python. Though these are not optimized, they could be helpful for reference. Here is the Github repo: https://github.com/calebmadrigal/algorithms-in-python.
A some of the algorithms included:
Recursion with asyncio
Recursion is awesome, but has the downside of growing the stack, which can limit its usefulness. Some languages like Scheme, however, have Tail-call optimization, which lets programmers write Tail-recursive functions that don't grow the call stack. Python does not have Tail-call optimization (TCO), but with asyncio, we can have something like Tail-call optimization. Basically, this method uses the asyncio event loop like a trampoline function.
Example:
To answer the trees
I had to leave - [or was it "flee"?] I felt entrapped by the drywall office halls [unable to breathe] The sanitized artwork hanging in the pale florescent haze reminded me what art should not be - [what life should not be] I hate the straight lines in man-made buildings that make everything feel so...concrete, [so unbending] But I love the trees with their curvy lines that say, "reality can bend - come bend with us." So that's why I am here - [to answer the trees]
- Author: Caleb Madrigal
- Date written: 3/17/2015
This picture by Melissa Vincent was part of the inspiration for this poem:
Simple D3 demos
I've been playing with d3 recently, and it awesome! I'm especially amazed by the ease of animating transitions with it.
There are plenty of amazing d3 demos out there, but I wanted to write some simpler ones to help me understand the basics of d3. You can find them here:
http://calebmadrigal.github.io/d3-demos/
For example, here's how to draw a circle with d3:
Code:
var width = 200;
var height = 200;
var x = width/2;
var y = height/2;
var ...
Spruce Needle Silhouette
This is my 2nd painting (painted 2014-01-10). I like how it came out, but almost gave up while painting it because I was comparing it to the sky God painted:
I took this photo of a beautiful "vanilla sky" outside of our home in Milwaukee, WI. It served as my model for the painting (along with other memories of looking at sunsets from beneath the spruce tree).
Dorian Gray Prelude
This year (2014) has been, for me, a year of art exploration. And to end it, I'd just like to share what I believe to be the greatest prelude ever written: the Prelude to The Picture of Dorian Gray by Oscar Wilde:
"The artist is the creator of beautiful things. To reveal art and conceal the artist is art's aim. The critic is he who can translate into another manner or a new material his impression of beautiful things.
The highest as the lowest form of criticism is a mode of autobiography. Those who find ugly meanings in beautiful things are corrupt without being charming. This is a fault.
Those who find beautiful meanings in beautiful things are the cultivated. For these there is hope. They ...
Phantastes
I recently finished reading Phantastes by George MacDonald for the 2nd time. The first time I read it, I really liked it. This second time, it became one of my all-time favorites. I've never experience any other work of art which feels so much like being in a dream.
To give you an idea of what it's about, here is the end of the Prelude:
"In a genuine fairy-story, everything must be miraculous, mysterious, and interrelated; everything must be alive, each in its own way. The whole of Nature must be wondrously blended with the whole world of the Spirit. In fairy-story the time of anarchy, lawlessness, freedom, the natural state of Nature makes itself felt in the world... The wolrd of the fairy-story is that ...
First painting
Yesterday (12/13/2014), my wife and I went to a painting studio to try our hands at painting. I loved the experience! Here is what I came out with:
This was my first time painting since I was a kid doing watercolors in school. Definitely plan to try it again!
Why I changed my mind about Node.js
Intro
I first heard of Node.js about 4 years ago. I thought it was a ridiculous idea: using JavaScript as your server-side language! JavaScript always felt to me like an inferior language. Why would you use it unless you absolutely had to!? Now of course, you have to do JavaScript if you are writing code which runs in the browser, but on the server, you can use any language you choose. So why choose JavaScript?
I did choose to use JavaScript/Node.js on my current project. I have now been using it for the last several months, and I am quite impressed! Let me tell you what made me change my mind about Node.js.
Adoption by the Enterprise world
First, Node.js has been adopted ...
Sessions in Node.js and Express.js
In the process of trying to explain to a friend how sessions work, I decided to write a barebones example to demonstrate how to do sessions in Node.js with Express.js.
Here's a live demo: http://expressjs-session-example.herokuapp.com/
And Here's what it looks like:
Before you enter your name:
After you enter your name:
And here is the code:
var express = require('express');
var cookieParser = require('cookie-parser');
var bodyParser = require('body-parser');
var session = require('cookie-session');
///////////////////////////////////////////////////////////////////////////////////////// MIDDLEWARE
var app = express();
// Needed to handle JSON posts
app.use(bodyParser.json());
// Cookie parsing needed for sessions
app.use(cookieParser('notsosecretkey'));
// Session framework
app.use(session({secret: 'notsosecretkey123'}));
// Consider all URLs under /public/ as static files, and return them raw.
app.use(express.static(__dirname + '/public'));
/////////////////////////////////////////////////////////////////////////////////////////// HANDLERS
function ...
On Abstract Art
Too much detail in art is profanity - a vulgar attempt to sum up that which is beyond finite expression.
How to SSH Tunnel
When to use an SSH Tunnel?
An SSH Tunnel is perfect for the following scenarios:
- You want to get to a website that a corporate network is blocking.
- You want to encrypt the network traffic for a non-encrypted protocol.
What is an SSH Tunnel?
An SSH Tunnel is where you tell your local SSH Client to forward your local computer's traffic to a specified SSH Server (over the encrypted SSH protocol). The SSH Server will then act as a proxy for all requests you make on your local computer.
How to set up an SSH Tunnel
There are 2 steps to set up an SSH Tunnel, and they both happen on your local computer (no configuration is needed on the SSH server which will be acting as ...
Art is a laser
The best art is more like a laser beam than a flood light - it doesn't hit everyone, but those it hits, it burns.
Fuel efficiency difference cost
I recently was car shopping, and I had the question about gas mileage: "How much does, say, the difference between 25 mpg and 30 mpg cost?"
To answer this question, I did a bit of analysis using IPython Notebook...
Goal
- To find how much different fuel efficiencies cost in terms of dollars per year.
- I want to estimate this value for the next 10 or so years.
Assumptions
- I'm estimating the average price per gallon of gas (over the next 10 years) to be $4.50. It is around $3.60 now, I adjusted for inflation, and counter-adjusted for the time-value of money.
- I'm estimating that we will drive about 14,000 miles per year (based on the last 3 years).
Data
price_per_gallon = 4 ...
Beautiful words
Over the last several years, I've been writing down words (or sets of words) which I consider to be particularly beautiful.
Here is my list:
- Camp Lejeune
- Long Boat Key
- Devil May Cry
- Effervescent
- Cellar Door
- Paramore
- Nevermore
- Evermore
- Cognac
- vis-à-vis
- Commandeer
Art begets art
Art begets art Hate begets hate Love begets love Grace begets grace. Before we depart We do place A lasting mark On others' fate.
- Author: Caleb Madrigal
- Date written: 3/18/2014
- Inspiration: I've found that taking in good art makes me want to write.
- Note: This isn't so much a poem as a truth put into a rhythmic format. It's a bit too "teachy" to be real poetry, in my opinion. Real poetry should not try to teach directly, but rather, should convey feelings (which may indirectly teach something).
Analogies are like functions
Analogies are to functions as facts are to variables - facts are data which the mind can reference, but analogies change the way the mind works.
Whenever possible, I try to explain things with analogies.
Gray
Gray I feel gray Cannot stay In this daze. Even pain - Let it rain! Anything… but this haze. I need light Or even night. These gray skies Glaze my eyes. Stimulation! Exhumation! Must awaken To elation.
- Author: Caleb Madrigal
- Date written: 3/17/2014
- Inspiration: I was in a gray mood today, and when I looked outside and saw the gray skies, I thought, “the weather matches my mood… or does my mood match the weather?” I didn’t feel like staying in this gray mood.
On the words of a song
Carried along On the words of a song, We let ourselves go In the waves as they roll. This ebb and flow Entraps our souls. Enraptured so, As our feelings grow. And into this torrent, Reason forfeit, Our hearts implore us To join the chorus.
- Author: Caleb Madrigal
- Date written: 3/10/2014
- Inspiration: Music is universal. We enjoy it; we cannot help ourselves. And we want to join with it.
The deep ocean skies
Stay away! Stay away! Keep away from the bay! You know that at night, Will in come the tide, Sweeping out the mind That is near the seaside. And into this sky You'll sink and you'll fly, Fish floating by, Enchanting your eyes, No more to rise From the deep ocean skies.
- Author: Caleb Madrigal
- Date written: 3/10/2014
- Inspiration: Sometimes, when I need to get work done, I have to stay away from things which will get me into an artistic, or I'll be swept up in it for hours.
I think Ali Jardine was going for the same idea in this beautiful picture:
The right side of my mind
Oft on cold and starry nights, He raises arms, and the battle fights. And storms the castle of my mind, Removing from the throne my logical side. And having the ship commandeered, Into the stormy winds he steers, And brings us into crashing waves - Deep oceans are within his gaze. And rejoicing on the captain's deck, Tiring of appointments to respect, He does not care if we should wreck But only in beauty does he invest. Careless and wild - this side of mine. Roaring through life, not glancing behind "Behold! the sails are filled with wind! Onward towards the moon again!"
- Author: Caleb Madrigal
- Date written: 3/10/2014
- Inspiration: What it feels like when my right-brain takes control. Often, starry nights and moonlight provoke me.
The stars as they shine
The stars as they shine Will begin to divide And in the sky The Morning’s alive Which never will die, And then we ride To eternal life And to find our time Was not lived a lie, And to meet His smile And to see His face To bask in His grace And to in love, embrace.
- Author: Caleb Madrigal
- Date written: 3/9/2014
- Inspiration: Riding on a bus, looking at the beautiful sky, and thinking about meeting God.
That romantic flow
Life is beautiful I feel it inside But how can I express The inner mind? I suppose I can try Let’s take a ride… Heart overflowing No need to hide Love is a real thing And there’s a spiritual side. There is magic Hidden from our eyes Yet felt with the soul That romantic flow. And to this goal I continue to stroll Seeking that beacon My spirit to strengthen.
- Author: Caleb Madrigal
- Date written: 3/9/2014
- Inspiration: Looking at the sunrise in Chicago over the lake, after a night casually dressed and deep in conversation.
Passion not spent
What's my great fear? I'll tell you; come near. To lay down in death with so much left. Passion not spent - Oh cowardly regret! For fear of others? The thousand deaths. I'm afraid to die With no twinkle in my eye To pass meagerly by Yet hidden inside. To walk through life Not truly alive And to pass in the night With an unfelt "goodbye".
- Author: Caleb Madrigal
- Date written: 3/7/2014
- Inspiration: An explanation of why living from my heart is so important to me.
Experiencing fully the beautiful present
He views the world through an unfiltered mind - No defenses built up, layers over time. He sees it in all its shining brilliance Experiencing fully the beautiful present.
- Author: Caleb Madrigal
- Date written: 3/4/2014
- Inspiration: Rafe (my son) - when I look at his eyes, it looks like he is just driniking everything in.
Poetic Mood
In the last week or so (starting Monday, March 3, 2014) I've been in a very poetic mood. It has been so interesting - it was light a light switch flipping on. I haven't been sitting down and trying to write poetry, they've just been coming to me. Often, as soon as one ends, another begins. It has been so pleasurable. I hope people enjoy them, but even if they don't, it has been so worth it to me!
NodeJS on Heroku
I've recently tried both Heroku and Node.js for the first time. Both are very simple, and work very nicely together. This is a quick guide to deploying a Node.js app to Heroku. To make it simple, I'll walk through deploying a Node.js sample project I created. It is a very simple message board.
Here are the steps to getting this simple app up and running on Heroku...
Get Heroku account
- Go to Heroku, and click Sign Up
- Download and install the Heroku Toolbeld for your operating system (which gives you the
heroku
command in your terminal) - Open your terminal and run
heroku login
, and enter your heroku credentials and the prompts
Deploy your app to heroku
In your terminal, do the ...
Display List as Table in IPython Notebook
IPython Notebook provides hook methods like _repr_html_
which can be defined by Python objects to allow richer representations. Here's an example:
class ListTable(list):
""" Overridden list class which takes a 2-dimensional list of
the form [[1,2,3],[4,5,6]], and renders an HTML Table in
IPython Notebook. """
def _repr_html_(self):
html = ["<table>"]
for row in self:
html.append("<tr>")
for col in row:
html.append("<td>{0}</td>".format(col))
html.append("</tr>")
html.append("</table>")
return ''.join(html)
import random
table = ListTable()
table.append(['x', 'y', 'x-y', '(x-y)^2'])
for i in xrange(7):
x = random.uniform(0, 10)
y = random.uniform(0, 10)
table.append([x, y, x-y, (x-y)**2])
table
How To Draw Lines With Matplotlib
Simple example to show how to draw lines with Matplotlib (in IPython Notebook).
ipython notebook --pylab inline
import matplotlib.lines as lines
fig, ax = plt.subplots()
fig.set_size_inches(6,6) # Make graph square
scatter([-0.1],[-0.1],s=0.01) # Move graph window a little left and down
line1 = [(0,0), (1,0)]
line2 = [(0,0), (0,1)]
# Note that the Line2D takes a list of x values and a list of y values,
# not 2 points as one might expect. So we have to convert our points
# an x-list and a y-list.
(line1_xs, line1_ys) = zip(*line1)
(line2_xs, line2_ys) = zip(*line2)
ax.add_line(Line2D(line1_xs, line1_ys, linewidth=2, color='blue'))
ax.add_line(Line2D(line2_xs, line2_ys, linewidth=2, color='red'))
plot()
show()