Sunday, November 10, 2013

Kinect Pin Art (FAIL)

Back in the winter of 2011 to the spring of 2012, I worked on a project that did not go well. The idea was to take Kinect (a motion sensing camera from Microsoft) depth information and use the information to move a matrix of pins. The goal was to simulate a low resolution "pin art" toy in near real time.  The pixels did move based on depth information from the Kinect, but all of the 60 pixels were not finished.  It would have been nice to have a video of the operational pixels, but I did not get one at the time.

Figure 1: A pin art toy from E-bay

I recently saw this video from a group of 5 people from MIT:
The idea is similar to my Kinect Pin Art project. Although, they did a much, much better job of implementation.

Here is the hardware video I put together:

Since Hack-a-day has been asking for "failed" projects, this is my contribution.

There were two main difficulties:
1) Get the depth information from a Kinect sensor.
2) Build a 6x10 matrix of linear motion pixels.

The main difficulty was getting an array of pixels to move in a linear fashion over 1 to 2 inches. The least expensive way I could come up with linear motion in near real time was using a series of mini servos attached to the pixels through bicycle cables. The figure below shows only the bottom hobby servo tray installed on the plywood base.

Figure 2: The bottom servo tray installed early in the development

The "pixels" were the top half of Bic ballpoint pins.

Visual Basic Express 2010 was used to gather depth information from the Kinect sensor. In the figure below, the depth information is shown on the image near the middle. The user clicks and drags a rectangle over the middle image. The little green squares represent where the depth information would be gathered to be sent out to the servo controller.
Figure 3:  The desktop software in operation

The off the shelf servo controller could drive 32 servos.  There were two of them to drive the 60 servos needed for the display.

Why did the project fail?
Mostly, the hardware was my downfall. The bicycle cable and pixels tended to bind up at random times. PVC boards were used to hold the pixels. The drill holes through the PVC boards needed to be about 2.5 inches long. Drill bits tend to wander around some when going through PVC. Several different things were tried to prevent the pixels from binding up: aluminum tubing, and a bicycle cable alignment board. (see the images below)
The hardware as it stands today (there were only two servo trays attempted):

 Figure 4:  Two of the pixels sticking out of the pixel holders

Figure 5:  Aluminum alignment tubes for the bicycle cable

Figure 6:  All of the hardware

Looking back on the project, there are several things I could have done differently:
-As the Mythbusters say “When in doubt, lubricate!” (or some thing like that). Anyway, the bicycle cables could have used some generous lube.
-Instead of drilling such long holes, a couple of pre-drilled boards (like pegboard) would have helped out greatly for holding the pixels.
-Several parts of the hardware assembly screams out for either 3D printed parts or machined parts.

Anyway, if there are any other ideas on improvements to the hardware, I would like to hear them in the comments.


  1. Hey, Matt! So sorry for contacting you here but just saw your sump pump project and was wondering if you'd be interested in submitting your photo of it for a contest I'm helping run? It's the IEEE Standards Association "I Spy Ethernet" contest. No entry fee, you retain the photo copyright.

    The contest is being held at this Facebook page:, but additional information is also available in a press release here:

  2. The problem with cables is they aren't meant to be pushed. The reason they work great for bike brakes is that they are always being pulled, either by the handle or by the spring force that normally keeps the brake calipers open. If you got some firm steel wire and make it into lots of little arms (imagine your servos are an engine crankshaft and your pixels are pistons, and you need piston arms between them)

    The pegboard idea sounds good. If you want to pay or can get access to one, a laser cutter would produce a few pegboard-like frames you could use and with the great precision required.

  3. As stated by mattthegamer463, the bowden (bycicle) cables you used are supposed to pull only. When you try to push with them, although they look sturdy enough to push the pens, they may jam ocasionally, because you are not pushing them straight, but around the servo's center of rotation. My suggestion:
    change the horns you used on the servos with symmetrical ones, meaning with the ones that extend to two sides from the gear and connect one cable to one side of the horn to push the pen, and another to pull it back, to the other side of the horn. you may run in to some trouble to fit double the cables you intended to use though.

  4. use radio control model airplane flexible pushrods. They're made for exactly things like this

  5. In the aircraft parts business, we have a technique for drilling long holes in tough materials. We first drill, slightly undersized. Drills are designed to clear chips, and to do that they have to have large passages, which makes them flexible. Thus they don't make really straight holes. They also don't make really round holes! Typically holes will be slightly triangular.
    So then we go down the same holes with an end mill, which is much stiffer, to straighten them and make them round. On really critical holes, we then go in with a reamer to get accurate diameters and improve the surface finish.

  6. I'm not an engineer, but it seems like maybe you could make this work if you put springs behind the pixels.

  7. You guys are correct about the bowden cables. I should have not gone with that.
    Chuck, good tip about the drilling of long holes.
    Springs would have been a good idea.