After some analysis, it looks like** purchases greater than 50 QAR (~$14 USD) will trigger the alert, so the programmer must not have converted the raw sum before making the comparison that determines if an alert should happen. I’d be sympathetic, but I’ve been swallowing 3% “Foreign Currency Conversion Fee”s left and right… and what the heck for? :-) Fortunately, it failed in a way preferable to me and is more likely to alert. I’d hate to have to deal with this in a country where the raw number is less than 50 and the USD value is greater than 50.

I’m tempted to align a few more purchases in the > 50QAR and < 50USD range so I can get some more data points.

Everybody loves screenshots

* “Wells Fargo’s” reads pretty awkwardly, doesn’t it?
** Obligatory “correlation is not causation”

[And then there was the movie. Look at me, not doing spoilers or anything.]

There were Arabic subtitles through the whole thing. If I’d been a little bit further in “Alif Baa” I probably could’ve walked out able to write “Harry Potter” in Arabic, but life is tough. I did wander into Jarir Bookstore yesterday and got a copy of HP2 in Arabic… I had HP1 while I was learning Spanish, and it’s nice to have an end goal in sight that I can flip through. Interestingly, it had the American version of the cover art; I was almost certain it was going to be based on the British version.

Note Mark: That’s the devilish smile of a man who’s spent the last 15 minutes teasing his wife and daughter in the States via SMS about getting to see the movie before its release in the US.

Chessboard!

I went to Souq Waqif today with Stehlik and found a chessboard.

Mark did most of the haggling bits, but I offered in a Coke. And now I have the coolest chessboard ever.

I think I amused the shopkeeper… as we were leaving he gave me a bracelet. It’s pretty awesome, too.

Marquetry work is super amazing

Oh, and there's backgammon

Backgammon marquetry

The shopkeeper was cool

The Finch is a robot to teach introductory computer science. Information about it is available at finchrobot.com.

Getting the Initial Information

First plug the finch in and run dmesg to see what the system log made of it:

[merichar@dormouse ~]% dmesg | tail
[226206.828073] usb 5-1: new full speed USB device using uhci_hcd and address 11
[226207.008109] usb 5-1: New USB device found, idVendor=2354, idProduct=1111
[226207.008118] usb 5-1: New USB device strings: Mfr=1, Product=2, SerialNumber=0
[226207.008125] usb 5-1: Product: Finch
[226207.008130] usb 5-1: Manufacturer: BirdBrain Tools
[226207.146317] generic-usb 0003:2354:1111.0001: hiddev0,hidraw0: USB HID v1.11 Device [BirdBrain Tools Finch] on usb-0000:00:1d.0-1/input0
[226207.146386] usbcore: registered new interface driver usbhid
[226207.146388] usbhid: USB HID core driver

Since it was successfully presenting itself as a usb device, we could also use lsusb to obtain details about it (and the other usb devices associated):

[merichar@dormouse ~]% lsusb
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
...
Bus 003 Device 009: ID 0a5c:2110 Broadcom Corp. Bluetooth Controller
Bus 005 Device 013: ID 2354:1111

Even though lsusb didn’t provide the human-readable vendor and product information, the vendor and product IDs match those of the dmesg output above.
So now we know that the vendor ID for the finch is 2354 and the product id is 1111.

We can also look at the default permissions of the device, since we know from lsusb that it’s bus 5, device 13 (adjust those numbers to your reality):

[merichar@dormouse ~]% ls -l /dev/bus/usb/005/013
crw-rw-r-- 1 root root 189 ...

From dmesg we saw that it also identifies as a USB HID: hiddev0, so we can look at bus mapping-agnostic permissions:

[merichar@dormouse ~]% ls -l /dev/usb
total 0
crw------- 1 root root 180, 0 Jun 14 11:00 /dev/usb/hiddev0

What do we want?

The goal is to not have to run processing or java as root when using the finch. Therefore, we must adjust the permissions of the device such that we (and perhaps others) are able to communicate with it directly.

Getting what we want

udev rules are the best way to change the default permissions of the finch. udev is the linux device manager, so it’s queen of mapping nodes in /dev. In Debian, user created udev rules are placed within /etc/udev/rules.d as (usually) device-specific .rules files.

As root, we are going to create the file:

/etc/udev/rules.d/55-finch.rules

containing the following:

SUBSYSTEM=="usb", SYSFS{idVendor}=="2354", SYSFS{idProduct}=="1111", MODE="0660", GROUP="plugdev"

The expected behavior would be that the next time the finch, as identified by the 2-tuple of vendor id and product id, is introduced to the system, the device’s group changes from root to plugdev and the permissions become 0660 (i.e. readable and writable by both the user and group owners). The 55- prefix to the rules file is its priority. plugdev is a group whose members should be able to access hot-pluggable devices, more traditionally cameras and usb sticks; see Martin’s post on the subject: here.

Sanity Check

After the 55-finch.rules is in place, let’s try it out. If the finch is plugged in to a USB port unplug it. The mappings should be gone from within /dev. Now plug the finch back in. If you were to perform another dmesg or lsusb, the output should be almost identical. e.g.:

[merichar@dormouse ~]% lsusb
...
Bus 005 Device 014: ID 2354:1111

In my case, the device mapping incremented by one. The device mapping will not affect the implementation of our rules file, and as you can see the rules file does not reference a mapping at all. That is the joy of dynamic mapping with udev. The new mapping is important because we do want to confirm that the permissions and ownership we defined in the .rules file actually took to the new mapping and all subsequent mappings:

[merichar@dormouse ~]% ls -l /dev/bus/usb/005/014
crw-rw-r-- 1 root plugdev 189, 525 Jun 14 12:00 /dev/bus/usb/005/014

Therefore, anyone in group plugdev is able to communicate directly with the finch.
We do a quick check that the user we care about is actually in the group plugdev:

[merichar@dormouse ~]% groups merichar
merichar : merichar dialout cdrom floppy sudo audio video plugdev netdev powerdev svn

And that’s that. I startup processing or java normally and as the user merichar I’m able to control the finch.

A Trampoline Game for the Microsoft Kinect

This project is a game that incorporates a trampoline into the player’s interaction with a Kinect. The player controls a frog by jumping and using his or her arm to extend the frog’s tongue and catch flies. The goal is to catch as many flies as possible before reaching the water.

Trampolines offer a new degree of freedom while interacting with the Kinect, and I wanted to create a game that demonstrated how one could be used to enhance the Kinect experience. Unlike regular jumping, a trampoline creates a regular and more predictable rhythm of vertical motion and can be sustained for a longer period of time.

Starting with that concept I originally began to create a side-scrolling game similar to Super Mario Brothers that would have the user jump to hit boxes and jump over pipes and enemies. I abandoned this attempt when it was clear that
it was uncomfortable for the player to adjust their actions to one that affected an orthogonal vertex. Additionally, complex object intersection became arbitrary complexity for a project attempting to achieve a proof of concept of rhythm
detection, skeleton tracking while bouncing, and hop detection.

Frog Arms

In the next iteration, the player was a frog whose arms would reflect their relative vertical position. A spring force at the end of each hands directs upward motion slightly outward and creates the illusion that the player is a frog jumping forward at each hop.

Hop Detection

The most interesting phase of the project involved player hop detection. One full cycle of ascending and descending could be broken down into logical phases of a bounce. Using the middle of the shoulders as the most stable part of the body from which to determine the acts of ascending and descending, every update cycle would pull that vertical position and place it into a height cache. The height cache was an array of height values treated as a sliding window: Instead of comparing the current y position to the most recent y position it was compared to some defined number of former y positions. As the current y position would have to be larger than all the former readings within the window, the sliding window reduced the likelihood of a bad reading or jitter causing a false determination that the player has started a new phase of ascending or descending. After a period of ascension, the player is considered in stasis when a future y position fails to be larger than all members of the height cache.

This is reflected in the following logic:

bool asc = true;
bool desc = true;
for( int i=0; i < HEIGHT_CACHE_SIZE; i++) {
    float old = properties.heightCache[i];
    //height pulled from tracked user is relatively inversed
    if(old > y) {
        desc = false;
    }
    else if(old < y) {
        asc = false;
    }
}
 
// if we are either ascending or descending (not in stasis)
// update the previous state
if( asc || desc) {
    // on first time leaving stasis; preserve former asc/desc states
    if(!properties.inStasis) {
        properties.wasAscending = properties.isAscending;
        properties.wasDescending = properties.isDescending;
    }
    properties.isAscending = asc;
    properties.isDescending = desc;
    properties.inStasis = false;
}
else {
    if (!properties.inStasis){
        properties.wasAscending = properties.isAscending;
        properties.wasDescending = properties.isDescending;
        properties.isAscending = false;
        properties.isDescending = false;
        properties.inStasis = true;
    }
}

Finally, with this logic in place, a hop is accounted for by the following:

if( wasAscending() and isDescending() and 
        (abs(y - properties.average_weighted_y) > REST_Y_DELTA)) {
    properties.num_hops++;
}

NB: The REST_Y_DELTA is used to account for the trampoline’s affect on a player’s rest state.

Gameplay – Flies

With the foundation in place, gameplay became the next phase of development. Continuing with the frog theme, I introduced flies as a desired token within the game. As the arms were presently unused, they became the tool used for fly interaction. Once the fly came close enough (they would gradually become larger as they approached the player), the player could wave their hand above shoulder level to swipe the frog’s tongue across the screen and gather any flies that hadn’t escaped off screen.

Gameplay – Stats

In order to show a player’s progress and bring encouragement, their number of hops and the number of flies their frog had collected were displayed on clouds at the top of the screen.

Gameplay – Endgame

Finally, the end game became the grass descending into the sea. Each hop would keep the grass from descending some marginal amount increasing the length of the game and thus the time available to catch flies.

Conclusion

While gameplay left a lot to be desired, I believe playing to my strengths and focusing on logical determinations of the jumper’s state and the resulting generic ‘jumper’ class and header files, which can be dropped in to future games involving rhythmic vertical motion, will ultimately prove worthwhile. After the public exhibition, it became clear that smaller children were having trouble getting proper hop detection and corresponding calculations. Once I fix some of the code that relied on incorrect constant values, it should be ready for general release.

Icon & One Sentence Description

Fly Catching is a game using a Kinect and trampoline where the player controls a frog that can hop and catch flies.

Mix and match toys and games are a popular activity for children. It typically appears as an upright cylinder or rectangular prism with multiple standing human or animal figures independently painted or carved into the surface. The shape is divided into thirds that rotate independently, allowing the different heads, torsos, and legs of the painted or carved figures to align. They have existed in many forms of media and targeted different age groups over their two-hundred year history. Changeable Gentlemen, a card game published around 1810 targeting adults, was one of the earliest manufactured mix and match games. Adaptations in base material and figure subjects made it more suitable for a younger audience, and modern incarnations are generally as a classic toy for children.

I chose a mix and match toy as the interaction subject because it’s a ubiquitous toy and gives almost all the players a baseline familiarity. Also, while the media and figures are numerous over the last two hundred years, the method of interaction has always been hand manipulation. The Kinect enables the player to use their entire body to control the figure. The figures are from a card game “Mixies” published in 1956 by Ed-U-Cards. The player can rotate through different heads, torsos, and feet by swiping above, at, or below the player’s midsection, respectively. Swiping to the left will rotate to the next card to appear, and swiping to the right will rotate to the previous card. The figure will follow a player’s horizontal and vertical movement, bending, and if they turn around.

I used OpenFrameworks, ofxKinect, and OpenNI. Once the skeleton is detected, the images are overlayed at the position and angle of the corresponding section. Hands and arms do not directly affect card placement, so they are free to control the card rotation. Sweep/wave detection is simply a matter of the end of the left or right arm moving over a certain horizontal distance within a time threshold. If the user’s back is turned, the back of the card is displayed instead of the obverse.

Watch this video on YouTube.

The original incarnation of this project was inspired by the Good Morning! Twitter visualization created by Jer Thorp. A demonstration of CMU network traffic flows, it would show causal links for the available snapshot of the network traffic. All internal IP addresses had to be anonymized, making the internal traffic less meaningful. Focusing only on traffic with an endpoint outside of CMU was interesting, but distribution tended towards obeying the law of large numbers, albeit with a probability density function that favored Pittsburgh.

This forced me to consider what made network traffic interesting and valuable, and I settled on collecting my own HTTP traffic in real-time using tcpdump. I summarily rejected HTTPS traffic in order to be able to analyze the packet contents, from which I could extract the host, content type, and content length. Represented appropriately, those three items can provide an excellent picture of personal web traffic.

Implementation

The visualization has two major components: Collection and representation. Collection is performed by a bash script that calls tcpdump and passes the output to sed and awk for parsing. Parsed data is inserted into a mysql database. Representation is done by Processing and the mysql and jbox2d libraries for it.

Visualization Choices

Each bubble is a single burst of inbound traffic, e.g. html, css, javascript, or image file. The size of the bubble is a function of the content size, in order to demonstrate the relative amount of tube it takes up to other site elements. Visiting a low-bandwidth site multiple times will increase the number of bubbles and thus the overall area of its bubbles will approach and potentially overcome the area of bubbles representing fewer visits to a bandwidth-intensive site. The bubbles are labeled by host and colored by the top level domain of the host. In retrospect, a better coloring scheme would have been the content type of the traffic. Bubble proximity to the center is directly proportional to how recently the element was fetched; elements decay as they approach the outer edge.

The example above shows site visits to www.cs.cmu.edu, www.facebook.com (and by extension, static.ak.fbcdn.net), www.activitiesboard.org, and finally www.cmu.edu, in that order.

Network Bubbles in Action

Watch this video on YouTube.

Code Snippets

Drawing Circles

Create a circle in the middle of the canvas (offset by a little bit of jitter on the x&y axes) for a radius that’s a function of the content length.

Body new_body = physics.createCircle(width/2+i, height/2+j,sqrt(msql.getInt(5)/PI) );
new_body.setUserData(msql.getString(4));

Host Label

If the radius of the circle is sufficiently large, label it with the hostname.

if (radius>7.0f) {
    textFont(metaBold, 15); 
    text((String)body.getUserData(),wpoint.x-radius/2,wpoint.y);
  }

tcpdump Processing

Feeding tcpdump input into sed

#!/bin/bash
tcpdump -i wlan0 -An 'tcp port 80' | 
while read line
do
if [[ `echo $line |sed -n '/Host:/p'` ]]; then 
    activehost=`echo $line | awk '{print $2}' | strings`
...
fi

Source Code

Get the source here.

Processing.js

Processing Applet

Source Code:

void setup() {
 // define the window size & enable anti-aliasing
 size(550, 800);
 background(255);
 smooth();
 noFill(); // not solid squares
 noLoop(); // only draw() once
}
 
void draw() {
 // allow for some padding around the edge
 translate(30,10);
 
 int x = 0, y = 0, side = 30;
 int i = 0;    // increment just like the original
 float j1, j2; // random numbers for translation and rotation
 
 for (int k = 0; k < 24; k++ ) { // iterate through y-axis
   for (int j = 0; j < 16; j++ ) { // iterate through x-axis
     i++;
     translate(x, y); // re-define coordinate axis to square
 
     // pick increasingly variable random numbers for
     j1 = random(-15*i/244,15*i/244); // translation
     j2 = random(-15*i/244,15*i/244); // & rotation
 
     // rotation should only affect one square at a time,
     //   so push and then pop that alteration to the coordinate axis
     pushMatrix();
     rotate(radians(j2));  // rotate the axis
     rect(j1, j1, 30, 30); // plot the square
     popMatrix(); // return the axis to its original state
 
     x=30; // x-position advances for each square drawn in row
     y=0;  // y-position shouldn't change in the middle of a row
   }
   x=-450; // return x to beginning of row position
   y=30;   // adjust y to next row
 }
}

OpenFrameworks

Watch this video on YouTube.

Watch this video on YouTube.