CREU Project Blog

A Possible Refinement?
November 18, 2007, 9:53 pm
Filed under: Uncategorized

So while I was studying for my test yesterday, I suddenly came up with this idea.  The design we are currently considering involves tracking the IR light reflecting off the finger using a webcam.  This design has  a few downsides to it such as the issue that it can only operate in a dim environment and it cannot tell which blob is generated by which finger.  Also the spherical surface seems unnecessary especially when we no longer uses the surface for refracting the IR beams.   So I was just thinking that we can possibly do something like the following.  


You can download the pdf to the above image Here .  Logitech has recently released its new HD webcam with auto-focus and a high quality lens system.  We can have the user wear labels on their fingers (see picture above).  The pattern/lines on the label allow us to calculate the distance the user’s finger is from the camera and possibly identity the individual fingers.  I think that this is better than our current design for the following reasons.  This system can work under various lighting conditions and it will give us more information about the position of the user’s hand to play with.  This is an extension to the wiimote multi-touch that Johnny has demonstrated in the youtube video below.    

Picture of 3D Hand is courtesy of: 

November 16, 2007, 7:40 pm
Filed under: Uncategorized

Here is a quick recap of our idea inspired by Johnny’s youtube video   We are going to have an IR LED array just like the one in the video and behind the array will be a webcam.  We will then cover it with something transparent and spherical looking object as our multi-touch surface.  The whole purpose of this is that we are hoping to capture the IR light that bounces off the finger using the camera, and we can use this information to track finger movements.  As for the experiment, we weren’t able to get any IR Led from the ECE electronic shop because they don’t have any.  So we are currently ordering some from Ebay.  In the meanwhile I have a wii sensor bar lying around so we did some experiment with that but apparently the IR light emitting from that is not strong enough.  So we end up having to place our finger really close to the camera, and instead of tracking blobs, we only see a blur.  We will have more conclusive results when the IR LED comes in the mail next week.

More Brainstorming =____=
November 6, 2007, 7:46 pm
Filed under: Uncategorized

Yen Wen was sick today so she couldn’t come =(.  During the meeting with Professor Anind, we talked about potential problems that we might encounter if we proceed with the multi-touch sphere that uses a camera for finger tracking.  One of the biggest potential problem is getting IR rays to refract internally in a semi-spherical plastic surface.   It is very possible that the rays will penetrate the surface.  So we beside that it would be worthwhile to do a little experiment.  So this week we are going to try to get some IR LED and some sort of transparent plastic dome-shaped object and examine how much internal refraction we can get.  At the same time we would continue to investigate our previous design and hopefully it would somehow work out?  We somehow need  to find something useable that is semi-sphere, transparent that we can use for the experiment.  

Considering to use new Multitouch Technology by NYU Researcher
October 31, 2007, 2:38 pm
Filed under: Uncategorized

We met with professor Anind yesterday and we discussed about the problems with the LED sensing solution.  We are now considering a new possibility of construction a new device using a novel Multi-Touch technology developed by Jeff Han at NYU.  There seems to be a fair amount of documentations on how to constructed a multi-touch table online.   The theory of operation of the technology is as following.   IR light is shone into a plastic surface such that the IR light reflect internally inside the plastic.  When the reflection is disturbed by for example the touch of a finger the light will scatter and the camera placed underneath the plastic will sense blobs of IR light.  We can track the movement on the plastic by tracking the blobs of light sensed by the camera.  A picture of the setup described in the paper is shown below


However for the purpose of the project, we are going to make some slight change to this setup.  Our initial goal of the project is to build a 3D input device, that would allow easier manipulation of 3D models on the computer.  We are interested in tracking finger movements on a sphere using this technology.   We propose to construct a spherical device that is composed of two semi-spherical plastic surfaces.  We want to shine IR light into the two semi-spherical surface and track the finger movements on the sphere using two cameras facing back to back.

Reference:Jefferson Y. Han, Low-Cost Multi-Touch Sensing through Frustrated Total Internal Reflection 

Multitouch LED Display PART IV (NEW PROBLEM ARISES~!!)
October 21, 2007, 7:00 am
Filed under: LED Display, Multitouch

Now that the Bidirectional LED demo is working but we realized a few problems. 

The refresh rate is too slow.  It appears that it is taking too much time for the LED to discharge.  However this is quite minor, we fixed this by grounding the LED using a resistor. 

 When we try to extend this to the 8×8 LED Matrix we found a much bigger problem and we are quite clueless as to how to solve it.  When we work with the matrix, we are not dealing with a single LED.  All the LED are connected to each other by the matrix.  The problem is that when a user places a finger on an LED, all the matrix in the entire array is distrubed, so it is almost impossible to determine where the user is touching. 

We don’t really have a solution right now but we are thinking about switching to a multitouch surface instead. 

October 15, 2007, 6:44 am
Filed under: Uncategorized

After consulting with Johnny and debugging the bidirectional LED test circuit using an oscilloscope,  we figured out the problem.  Previously in the firmware code of the, we used the Arduino API to set the state of the pin (that is to set the digital pins to input/output state).  The problem occurs at the discharing stage.  When we set the digital pin to input mode using the pinMode function, the pull up resistor is also set.  This is definitely not a good sign, because setting the pull up resistor  prevents the LED from discharging.  What this means is that we cannot call the digital I/O functions in the Arduino API.   Since the Arduino API is actually built upon the AVR-GCC, a free C compiler for the Atmega series microcontroller, we simply have to manipulate the digital I/O using the functions provided in the AVR-GCC lib.  Here is a table on how to manipulate the digital I/O on the Atmega microcontroller.

Here is the modified code

int digi_fwd = 2;
int digi_rev = 3
unsigned int count;
void setup(){
DDRD |= _BV(digi_fwd); //set port D pin 2 as output
DDRD |= _BV(digi_rev);
void loop(){

PORTD |= _BV(digi_fwd); //set digi_fwd high
PORTD &= ~_BV(digi_rev); //set digi_rev low

//Reverse Bias
PORTD |= _BV(digi_rev); //set digi_rev high
PORTD &= ~_BV(digi_fwd); //set digi_fwd low

DDRD &= ~_BV(digi_rev); //set digi_rev input
PORTD &= ~_BV(digi_rev); //Disable pull up

count = 0;

Serial.println(count, DEC);

DDRD |= _BV(digi_rev); //set digi_rev input

October 8, 2007, 6:14 am
Filed under: download, LED Display

Although we have not yet been able to get any useful input from the LED.  We bought some LED Marix Display wired them up to the Arduino Stamp.  We wrote some firmware code in C and a demo app in Processing.  I was hoping to quickly draw a schematic of how the LED Matrix is wired, but I didn’t get to that yet.  Hopefully it will be up in the near future.  Below is a picture of the 8×8 LED Matrix setup.  Each row of the matrix is controlled by a digital I/O pin on the Arduino Stamp.  However there is not enough digital I/O on the Arduino Stamp to control the columns, we use a shift register to output data to control the columns on the matrix.


Each circle on the java applet corresponds to a single LED on the display.  

Below is also a youtube demonstration of the setup

 For those of you that are interested here is the Firmware Source Code in an Arduino Project and here is the Processing Source Code for the GUI as an Eclipse project.

**Note that the RxTxLib included in the GUI source code is strictly for MacOSX