This is a project that was done for my “man machine interface/ multimedia” course at university, with quite a few additions.
Without further ado , here is a small presentation video , more info , source code etc after the break.
Information and features:
The aim was to come up with a way to display 100+ photos in a photo-gallery, with original input methods being a bonus, it was written in Processing, with some additional Java elements.
Because of performance issues , i decided not to use openCV , but write my own blob detection/ tracking programming , using Gstreamer for the webcam feed acquisition, as well as a small gesture recognition system based on vector product.
Here are some of the features:
- each picture group is gathered from a different folder and is projected on a 3d sphere: (200+ photos in the first gallery) which allows an overview of a lot of pictures at the same time
- gesture recognition system ,based on the vector product between the last few (8 in this video) strokes drawn by the user and a set of reference gestures stored as arrays of vectors.
- i initially used a “brightest” element blob tracking system but later switched to a motion based one , as it is much more reliable under different lighting conditions
- during the development of the project i made big improvements in blob tracking speed ( with 30 pictures, runs at 30fps on a Asus 1000h)
- multiple galleries
- possibility to adjust tracking and gesture parameters from inside the program
Lessons learned and possible evolutions:
- I learned a great deal with this project, both in the field of image analysis , blobs tracking and gesture recognition, but on the practical side i must confess that spending half a day perfecting the calibration of gesture recognition make me doubt the usability of purely gesture based User Interfaces in a work environment (à la “Minority report”) , as it seems way too tiresome, but then again it could just be the problems of my specific implementation of such a system.
- On the other hand, i might one day adapt it to be usable with a system like the “Eye writer” (quite an amazing project : you can check it out here), which seems someone more natural , and less prone to require excessive movements (my neighbors by now most probably think i’m crazy to continually wave my hands around in front of the screen).
- I confess i love futuristic interfaces (from the Ghost in the Shell anime/manga and the Mass Effect 2 game specifically ), so that might show here 🙂
The program was tested on Windows Xp , Windows Seven and Ubuntu and should normally run even on relatively old machines.
If you want to try it out using your own pictures , just copy them into the data/testPics or data/testPics2 folder:
- You can download it here (edit some slight adjustment needs to be made before i re-upload it, should be done by tomorrow)
- The source code is here (same as above)
- The (small) manual is here