4.26.2009

Smoke Demo

I built touchlib and got the smoke demo running on our prototype today, as well as a simple touch app that prints simple touch events to stdout (finger down, finger up).

The way touchlib seems to work is definately as a library. Touch applications import touchlib and happen to read the same configuration file as far as background substraction and calibration settings and so forth, but drive the camera and do image processing themselves (hence no concept of multitasking touch apps).

I personally don't like that design choice, so I am glad that we are writing our own instead of adapting touchlib. It seems like our gesture system will work that way though, which perhaps makes more sense since we have decided gestures are more application specific.

I had to apply this patch to make touchlib not segfault:
--- ./src/CMakeLists.txt.orig 2008-04-22 19:47:46.000000000 +0000
+++ ./src/CMakeLists.txt 2008-04-22 19:47:56.000000000 +0000
@@ -9,3 +9,9 @@
# SET_TARGET_PROPERTIES(configapp
# PROPERTIES LINK_FLAGS ${OPENCV_LINKDIR} LINK_FLAGS ${OPENCV_LDFLAGS})
ENDIF(OPENGL_FOUND AND GLUT_FOUND)
+
+INCLUDE(UsePkgConfig)
+PKGCONFIG(gdk-2.0 GDK2_INCLUDE_DIR GDK2_LINK_DIR GDK2_LINK_FLAGS GDK2_CFLAGS)
+IF (GDK2_INCLUDE_DIR)
+ SET(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} ${GDK2_CFLAGS}")
+ENDIF (GDK2_INCLUDE_DIR)
--- ./src/configapp.cpp.orig 2008-04-22 19:43:36.000000000 +0000
+++ ./src/configapp.cpp 2008-04-22 19:47:56.000000000 +0000
@@ -835,6 +835,7 @@
}

if( keypressed == 13 || keypressed == 10) // enter = calibrate position

{

+ screen->unlockHighGUIMutex();

startGLApp(argc, argv);

}



--- ./src/CTouchScreen.cpp.orig 2008-04-22 19:46:11.000000000 +0000
+++ ./src/CTouchScreen.cpp 2008-04-22 19:55:09.000000000 +0000
@@ -19,6 +19,27 @@


using namespace touchlib;



+#ifdef linux

+#include

+/* A mutex lock to prevent multiple threads to access the HighGUI

+ * functions in the same time. For OpenCV <= 1.0.0 in Linux platform ,

+ * the cvWaitKey and cvShowImage function use different set of mutex lock,

+ * but in fact both of the functions would access the GDK critical

+ * session. Before the bug was being resolved in OpenCV, use this

+ * mutex lock to fix the problem.

+ * */

+#define highgui_mutex_init() { if (!g_thread_supported ()) { \

+ g_thread_init (NULL);\

+ gdk_threads_init();} \

+ }

+#define highgui_mutex_lock() gdk_threads_enter()

+#define highgui_mutex_unlock() gdk_threads_leave()

+#else

+#define highgui_mutex_init()

+#define highgui_mutex_lock()

+#define highgui_mutex_unlock()

+#endif

+

#ifdef WIN32

HANDLE CTouchScreen::hThread = 0;

HANDLE CTouchScreen::eventListMutex = 0;

@@ -31,6 +52,7 @@


CTouchScreen::CTouchScreen()

{

+ highgui_mutex_init();

frame = 0;



#ifdef WIN32

@@ -230,7 +252,9 @@
if(filterChain.size() == 0)

return false;

//printf("Process chain\n");

+ highgui_mutex_lock();

filterChain[0]->process(NULL);

+ highgui_mutex_unlock();

IplImage *output = filterChain.back()->getOutput();



if(output != NULL) {

@@ -803,6 +827,14 @@
}

}



+void CTouchScreen::lockHighGUIMutex(){

+ highgui_mutex_lock();

+}

+

+void CTouchScreen::unlockHighGUIMutex(){

+ highgui_mutex_unlock();

+}

+

// Code graveyard:

/*

// Transforms a camera space coordinate into a screen space coord

--- ./include/CTouchScreen.h.orig 2008-04-22 19:43:36.000000000 +0000
+++ ./include/CTouchScreen.h 2008-04-22 19:47:56.000000000 +0000
@@ -122,6 +122,8 @@
// returns -1 if none found..

int findTriangleWithin(vector2df pt);



+ void lockHighGUIMutex();

+ void unlockHighGUIMutex();





private:

--- ./include/ITouchScreen.h.orig 2008-04-22 19:43:36.000000000 +0000
+++ ./include/ITouchScreen.h 2008-04-22 19:56:39.000000000 +0000
@@ -83,6 +83,8 @@


// start the processing and video capturing

virtual void beginTracking() = 0;

+ virtual void lockHighGUIMutex() = 0;

+ virtual void unlockHighGUIMutex() = 0;



/**

* Sets the blob tracker.


The patch fixes a race condition in the way that OpenCV's HighGUI library interacts with one of the native X toolkits, GTK. I ported the patch to the latest version of touchlib in their cvs, r400. It ran without segfaulting, but it would stall out and hang very often (and come back as well). I assume that something else in their code was broken as well, so I went back to revision 393 which the patch is made for and it worked like a charm.

Sadly, Smoke does not seem as cool in person as in videos. My touches still seem pretty bad, me being me and all. Hopefully it is only a matter of me tweaking the touchlib calibration settings, since syncing LED pulsing against all the webcams sounds pretty tricky.

No comments:

Post a Comment