Archive for September, 2013

Charade dataset, collection of clean shots of Audrey Hepburn, Cary Grant, and Jacques Marin

Monday, September 30th, 2013

After remembering that the 1963 version of the film Charade is in the public domain, I thought it keep make a useful dataset for image processing and computer vision techniques. I watched the movie again and took screen grabs any time there was a clean shot of a main actor’s face.

charade still hepburn

charade still grant

charade still grant

charade still Marin

Download all 226 images

1920×1040 png (417 MB zip)

1920×1040 jpg (290 MB zip)

960×520 png (141 MB zip)

480×260 png (40 MB zip)

Stop X11 app from bouncing dock icon indefinitely

Monday, September 30th, 2013

Freeglut (unfortunately) uses X11 as a windowing system. This means ugly, non-native windowing on mac os x, but also seems to mean that it doesn’t play nicely with .app bundling. Bundled X11 apps create two dock icons. One for the individual app and then one for X11 (if it’s not already there). Upon appearing, the applications icon bounces indefinitely even though the X11 windows have opened and are working properly. It seems the only way to fix this is to trigger some event that’s caught by the Carbon or Cocoa API of Mac OS X.

So here’s an example of my hacky solution.

#include <OpenGL/OpenGL.h>
#include <GLUT/glut.h>

#include <CoreFoundation/CoreFoundation.h>
#include <objc/objc.h>
#include <objc/objc-runtime.h>
#include <iostream>
// Stop bouncing dock icon.
// 
static inline void cocoa_says_stop_bouncing()
{
  id app = NULL;
  id pool = objc_getClass("NSAutoreleasePool");
  if (!pool)
  {
      std::cerr << "Unable to get NSAutoreleasePool!\nAborting\n";
      return;
  }
  pool = objc_msgSend(pool, sel_registerName("alloc"));
  if (!pool)
  {
      std::cerr << "Unable to create NSAutoreleasePool...\nAborting...\n";
      return;
  }
  pool = objc_msgSend(pool, sel_registerName("init"));
  app = objc_msgSend(objc_getClass("NSApplication"),
    sel_registerName("sharedApplication"));
  id req = objc_msgSend(app,sel_getUid("requestUserAttention:"),0);
  objc_msgSend(app,sel_getUid("cancelUserAttentionRequest:"),req);
  objc_msgSend(pool, sel_registerName("release"));
}

void display()
{
  glClearColor(1,0.8,0.8,1);
  glClear(GL_COLOR_BUFFER_BIT);
  glutSwapBuffers();
  glutPostRedisplay();
}

int main(int argc, char** argv)
{
  glutInit(&argc,argv);
  glutCreateWindow("test");
  // Must come after glutCreateWindow
  cocoa_says_stop_bouncing();
  glutDisplayFunc(display);
  glutMainLoop();
  return 0;
}

Assuming you install freeglut with macports, then compile with:

g++ -framework Cocoa -o test test1.cpp -L/opt/local/lib -lglut -framework OpenGL

Then bundle into a test.app directory using

build_app test

and then double-click on your app and stand back, relieved that the bouncing has ceased.

Source, Other

Automatically bundle application into double-clickable .app

Monday, September 30th, 2013

Here’s a bash script to automatically bundle up your executable program into a “double-clickable” .app application directory. This works nicely for making GLUT applications look a bit more authentic. Save the following in a file called build_app:

#!/bin/bash

# based on https://raw.github.com/ixchow/Preshack/master/Basis/osx/build_app.pl
if [ "$#" -eq 0 ]
then
  USAGE="Usage:
  build_app.sh path/to/executable path/to/icon.icns path/to/new.app
or
  build_app.sh path/to/executable path/to/icon.icns
or
  build_app.sh path/to/executable
";
  echo "$USAGE";
  exit 1
fi

BIN="$1"

if [ "$#" -ge 2 ]
then
  ICNS="$2"
else
  ICNS=""
fi

if [ "$#" -ge 3 ]
then
  APP="$3"
else
  APP="$1.app"
fi


NAME=`basename $1`
SHORT=${NAME:0:4}

mkdir -p "$APP/Contents/Resources" || exit 1;
mkdir -p "$APP/Contents/MacOS" || exit 1;

PLIST="$APP/Contents/Info.plist";

echo "<?xml version=\"1.0\" encoding=\"UTF-8\"?>
<!DOCTYPE plist PUBLIC \"-//Apple Computer//DTD PLIST 1.0//EN\" \"http://www.apple.com/DTDs/PropertyList-1.0.dtd\">
<plist version=\"1.0\">
<dict>
    <key>CFBundleDevelopmentRegion</key>
    <string>English</string>
    <key>CFBundleExecutable</key>
    <string>$NAME</string>
    <key>CFBundleIconFile</key>
    <string>$NAME.icns</string>
    <key>CFBundleIdentifier</key>
    <string>com.tchow.$NAME</string>
    <key>CFBundleInfoDictionaryVersion</key>
    <string>6.0</string>
    <key>CFBundleNAME</key>
    <string>$NAME</string>
    <key>CFBundlePackageType</key>
    <string>APPL</string>
    <key>CFBundleSignature</key>
    <string>$SHORT</string>
    <key>CFBundleVersion</key>
    <string>1.0</string>
</dict>
</plist>" > $PLIST
cp $BIN $APP/Contents/MacOS/$NAME
if [ ! -z "$ICNS" ]
then
  cp $ICNS $APP/Contents/Resources/$NAME.icns
fi

Then you can run it on you executable like this:

build_app executable icon.icns output.app

Source

Compiling and running the nVidia dual depth peeling example on Mac OS X

Friday, September 27th, 2013

Achieving order-independent transparency is one of the biggest embarrassments to real-time graphics. We want something like:

glEnable(GL_SLOWER_BUT_CORRECT_TRANSPARENCY);

But instead we often resort to complicated, dirty hacks, expensive sorting with isn’t even correct, or giving up and living with ugly order-dependent alpha blending.

There are a few good ways to do it right and “Order Independent Transparency” and the follow-up “Order Independent Transparency with Dual Depth Peeling” are clever solutions.

Dual depth peeling demo running on mac

I finally managed to get there 2008(!) demo code running on my iMac (with AMD Radeon HD 6970M OpenGL Engine) running Mac OS X 10.7. This is quite a feat considering the OpenGL support on my machine is circa 1999.

Here are the steps to get this demo running. Download the source code.

Travel to the source directory

cd dual_depth_peeling/OpenGL/src/dual_depth_peeling

and create a Makefile with:

.PHONY: all

CXX=g++

all: obj dual_depth_peeling

CFLAGS += -O3 -Wall -DNDEBUG -Winvalid-pch -m64 -fopenmp -msse4.2

CPP_FILES=$(wildcard *.cpp)
OBJ_FILES=$(addprefix obj/,$(notdir $(CPP_FILES:.cpp=.o)))


NVIDIA_COMMON_INC=-I../../common/include/ -I../../common/nvModel/include

OPENGL_LIB=-framework OpenGL
GLUT_LIB=-framework GLUT

LIB+=$(OPENGL_LIB) $(GLUT_LIB)
INC+=$(NVIDIA_COMMON_INC)

dual_depth_peeling: $(OBJ_FILES)
    ${CXX} $(CFLAGS) -o $@ $(OBJ_FILES) $(LIB)

obj:
    mkdir -p obj

obj/%.o: %.cpp %.h
    ${CXX} $(CFLAGS) -o $@ -c $< $(INC) 

obj/%.o: %.cpp
    ${CXX} $(CFLAGS) -o $@ -c $< $(INC)

clean:
    rm -f $(OBJ_FILES)
    rm -f dual_depth_peeling

Before making we need to fix some bugs and compiler issues in the included math code. My edits are of the form:

// Alec
// Original code
New code

In nvMatrix.h:

// Alec
//for (int i = 0; i < 4; i++) v[i] = element(i,r);
for (int i = 0; i < 4; i++) v[i] = element(i,c);

In nvQuaternion.h

// Alec
//mult_vec(vec3(src_and_dst), src_and_dst);
mult_vec(vec3<T>(src_and_dst), src_and_dst);

and

// Alec
//vec3 axis;
vec3<T> axis;

and

// Alec
//return  &q[0];
return  &_array[0];

and

// Alec
//q0 = q[0];
//q1 = q[1];
//q2 = q[2];
//q3 = q[3];
q0 = _array[0];
q1 = _array[1];
q2 = _array[2];
q3 = _array[3];

and

  // Alec
  //vec3 v;
  vec3<T> v;

In nvShaderUtils.h and other files later replace

#include <GL/glew.h> 

with

// Alec
#ifdef __APPLE__
#include <OpenGL/GL.h>
#else   
#include <GL/glew.h>
#endif

In nvVector.h there are a bunch of lines that look like:

T::value_type ...

which produce errors like

../../common/include/nvVector.h:681:5: Error: error: need 'typename' before 'T:: value_type' because 'T' is a dependent scope

replace these lines with something like

typename T::value_type ...

also

// Alec
//return *this;
return lhs;

The Makefile we wrote is setup not to use the included nvModel.h. The original demo only included the header and a binary. Instead I got the header and cpp files: nvModel.h, nvModel.cc, nvModelObj.cc and nvModelQuery.cc from this random website I found by googling “nvModel.cpp”. Download those files into dual_depth_peeling/OpenGL/src/dual_depth_peeling and rename the *.cc files into *.cpp.

Go back to dual_depth_peeling/OpenGL/src/dual_depth_peeling and find and replace lines like

#include <GL/glew.h>
#include <GL/glut.h>

with

// Alec
#ifdef __APPLE__
#include <OpenGL/GL.h>
#include <GLUT/glut.h>
#else   
#include <GL/glew.h>
#include <GL/glut.h>
#endif  

and change the include to find our new nvModel.h

// Alec 
//#include <nvModel.h>
#include "nvModel.h"

In dual_depth_peeling.cpp change the following lines:

        // Alec
        //glTexImage2D(GL_TEXTURE_RECTANGLE_ARB, 0, GL_FLOAT_RG32_NV, g_imageWidth, g_imageHeight,
        glTexImage2D(GL_TEXTURE_RECTANGLE_ARB, 0, GL_RG32F, g_imageWidth, g_imageHeight,

and

        // Alec
        //glTexImage2D(GL_TEXTURE_RECTANGLE_ARB, 0, GL_DEPTH_COMPONENT32F_NV,
        glTexImage2D(GL_TEXTURE_RECTANGLE_ARB, 0, GL_DEPTH_COMPONENT,

and

// Alec
//glTexImage2D(GL_TEXTURE_RECTANGLE_ARB, 0, GL_FLOAT_R32_NV,
glTexImage2D(GL_TEXTURE_RECTANGLE_ARB, 0, GL_R32F,

and

                //Alec
                //g_opacity = max(g_opacity, 0.0);
                g_opacity = std::max(g_opacity, 0.0f);

and

                //Alec
                //g_opacity = min(g_opacity, 1.0);
                g_opacity = std::min(g_opacity, 1.0f);

and put and ifndef around the glew stuff

        //Alec
#ifndef __APPLE__
        if (glewInit() != GLEW_OK)
        {
                printf("glewInit failed. Exiting...\n");
                exit(1);
        }

    if (!glewIsSupported( "GL_VERSION_2_0 "
                          "GL_ARB_texture_rectangle "
                                                  "GL_ARB_texture_float "
                                                  "GL_NV_float_buffer "
                          "GL_NV_depth_buffer_float "))
    {
        printf("Unable to load the necessary extensions\n");
                printf("This sample requires:\n");
                printf("OpenGL version 2.0\n");
                printf("GL_ARB_texture_rectangle\n");
                printf("GL_ARB_texture_float\n");
                printf("GL_NV_float_buffer\n");
                printf("GL_NV_depth_buffer_float\n");
                printf("Exiting...\n");
        exit(1);
    }
#endif

We’re almost done. If you now issue:

make
./dual_depth_peeling

You’ll see that it compiles and runs but it fails to compile the shaders. So now we fix compatibility issues with the shaders. Comment any extension lines like this:

// Alec
//#extension ARB_draw_buffers : require

and change any sampleRECT and textureRECT lines like this

// Alec
//uniform samplerRECT TempTex;
uniform sampler2DRect TempTex;

and this

    // Alec
    //gl_FragColor = textureRect(TempTex, gl_FragCoord.xy);
    gl_FragColor = texture2DRect(TempTex, gl_FragCoord.xy);

Fix any issues with comparisons between floats and ints like this:

    // Alec
    //if (gl_FragColor.a == 0) discard;
    if (gl_FragColor.a == 0.0) discard;

Finally fix problems with operations not acting on vec3 and vec4 like this:

    // Alec
    //gl_FragColor.rgb = frontColor + backColor * alphaMultiplier;
    gl_FragColor.rgb = frontColor.rgb + backColor * alphaMultiplier;

also in shaders/shade_fragment.glsl swap fmod for mod like this:

    //Alec
    //color.rgb = (fmod(i, 2.0) == 0) ? vec3(.4,.85,.0) : vec3(1.0);
    color.rgb = (mod(i, 2.0) == 0.0) ? vec3(.4,.85,.0) : vec3(1.0);

That ought to do it. Looking through the code this probably won’t be the absolute easiest thing to integrate into my apps, but I’m happy to finally really see something that works on my own machine.

Here’s a table of the results. It compares this method to two naive methods for order independent alpha blending: averaging and summing. Those naive methods have obvious flaws.

Dual depth peeling compared to averaging and summing
Update: To skip all these steps and just trust me. Download my edited source.

OpenGL lighting not working, GL_POSITION is in homogenous coordinates

Monday, September 23rd, 2013

A while ago I somehow introduced a bug in my opengl app that occasionally caused my app to render a shaded object without lighting like this:

gl lighting not working

instead of what I wanted which was this:

gl lighting working

Now, there must be 101 ways that you can screw up the opengl lighting. What I was doing was setting the light position incorrectly. I was calling something like:

float light[4];
light[0] = x;
light[1] = y;
light[2] = z;
glLightfv(GL_LIGHT0,GL_POSITION,light);

I new enough to use a 4-dimensional (homogenous) vector for the light direction. But I wasn’t setting the w-component. I guess occasionally this was getting a crazy value like NaN and killing the lights.

float light[4];
light[0] = x;
light[1] = y;
light[2] = z;
light[2] = w;// e.g. w = 0 or 1
glLightfv(GL_LIGHT0,GL_POSITION,light);

Display all glut cursors

Monday, September 23rd, 2013

Here’s a little program to see all of the glut cursors available on your implementation.

#include <GLUT/glut.h>
#include <OpenGL/OpenGL.h>
#include <iostream>
#include <map>
#include <cstring>

std::map<int,const char *> m;
std::map<int,const char *>::const_iterator c;
const char * GLUT_CURSOR_STR[23]
{
  "GLUT_CURSOR_RIGHT_ARROW",        
  "GLUT_CURSOR_LEFT_ARROW",       
  "GLUT_CURSOR_INFO", 
  "GLUT_CURSOR_DESTROY",    
  "GLUT_CURSOR_HELP", 
  "GLUT_CURSOR_CYCLE",  
  "GLUT_CURSOR_SPRAY",  
  "GLUT_CURSOR_WAIT", 
  "GLUT_CURSOR_TEXT", 
  "GLUT_CURSOR_CROSSHAIR",      
  "GLUT_CURSOR_UP_DOWN",    
  "GLUT_CURSOR_LEFT_RIGHT",       
  "GLUT_CURSOR_TOP_SIDE",     
  "GLUT_CURSOR_BOTTOM_SIDE",        
  "GLUT_CURSOR_LEFT_SIDE",      
  "GLUT_CURSOR_RIGHT_SIDE",       
  "GLUT_CURSOR_TOP_LEFT_CORNER",            
  "GLUT_CURSOR_TOP_RIGHT_CORNER",             
  "GLUT_CURSOR_BOTTOM_RIGHT_CORNER",                
  "GLUT_CURSOR_BOTTOM_LEFT_CORNER",               
  "GLUT_CURSOR_FULL_CROSSHAIR",           
  "GLUT_CURSOR_NONE", 
  "GLUT_CURSOR_INHERIT"
};

int width,height;
void reshape(int width,int height)
{
  ::width = width;
  ::height = height;
  glViewport(0,0,width,height);
}

void display()
{
  using namespace std;
  glClearColor(1,1,1,0);
  glClear(GL_DEPTH_BUFFER_BIT | GL_COLOR_BUFFER_BIT);

  glMatrixMode( GL_PROJECTION ) ;
  glPushMatrix() ; // save
  glLoadIdentity();// and clear
  glMatrixMode( GL_MODELVIEW ) ;
  glPushMatrix() ;
  glLoadIdentity();
  gluOrtho2D(0,width,0,height);
  glDisable( GL_DEPTH_TEST ) ; // also disable the depth test so renders on top
  glTranslated(10,height/2,0);
  glScaled(.09,.09,.09);
  glLineWidth(2);

  //glRasterPos2f( 0,0 ) ; // center of screen. (-1,0) is center left.
  glColor4f(0.0f, 0.0f, 0.0f, 1.0f);
  char buf[300];
  sprintf( buf,"%s", c->second ) ;
  const char * p = buf ;
  do glutStrokeCharacter( GLUT_STROKE_ROMAN, *p ); while( *(++p) ) ;
  glutSwapBuffers();
  glutPostRedisplay();
}


void key(unsigned char k, int, int)
{
  if(k == ' ')
  {
    if(++c == m.end())
    {
      c = m.begin();
    }
    glutSetCursor(c->first);
  }
}

int main(int argc, char * argv[])
{
  using namespace std;
  m[GLUT_CURSOR_RIGHT_ARROW] = "GLUT_CURSOR_RIGHT_ARROW";        
  m[GLUT_CURSOR_LEFT_ARROW] = "GLUT_CURSOR_LEFT_ARROW";       
  m[GLUT_CURSOR_INFO] = "GLUT_CURSOR_INFO"; 
  m[GLUT_CURSOR_DESTROY] = "GLUT_CURSOR_DESTROY";    
  m[GLUT_CURSOR_HELP] = "GLUT_CURSOR_HELP"; 
  m[GLUT_CURSOR_CYCLE] = "GLUT_CURSOR_CYCLE";  
  m[GLUT_CURSOR_SPRAY] = "GLUT_CURSOR_SPRAY";  
  m[GLUT_CURSOR_WAIT] = "GLUT_CURSOR_WAIT"; 
  m[GLUT_CURSOR_TEXT] = "GLUT_CURSOR_TEXT"; 
  m[GLUT_CURSOR_CROSSHAIR] = "GLUT_CURSOR_CROSSHAIR";      
  m[GLUT_CURSOR_UP_DOWN] = "GLUT_CURSOR_UP_DOWN";    
  m[GLUT_CURSOR_LEFT_RIGHT] = "GLUT_CURSOR_LEFT_RIGHT";       
  m[GLUT_CURSOR_TOP_SIDE] = "GLUT_CURSOR_TOP_SIDE";     
  m[GLUT_CURSOR_BOTTOM_SIDE] = "GLUT_CURSOR_BOTTOM_SIDE";        
  m[GLUT_CURSOR_LEFT_SIDE] = "GLUT_CURSOR_LEFT_SIDE";      
  m[GLUT_CURSOR_RIGHT_SIDE] = "GLUT_CURSOR_RIGHT_SIDE";       
  m[GLUT_CURSOR_TOP_LEFT_CORNER] = "GLUT_CURSOR_TOP_LEFT_CORNER";            
  m[GLUT_CURSOR_TOP_RIGHT_CORNER] = "GLUT_CURSOR_TOP_RIGHT_CORNER";             
  m[GLUT_CURSOR_BOTTOM_RIGHT_CORNER] = "GLUT_CURSOR_BOTTOM_RIGHT_CORNER";                
  m[GLUT_CURSOR_BOTTOM_LEFT_CORNER] = "GLUT_CURSOR_BOTTOM_LEFT_CORNER";               
  m[GLUT_CURSOR_FULL_CROSSHAIR] = "GLUT_CURSOR_FULL_CROSSHAIR";           
  m[GLUT_CURSOR_NONE] = "GLUT_CURSOR_NONE"; 
  m[GLUT_CURSOR_INHERIT] = "GLUT_CURSOR_INHERIT";
  c = m.begin();

  glutInit(&argc,argv);
  glutInitDisplayString( "rgba depth double samples>=8 ");
  glutInitWindowSize(300,70);
  glutCreateWindow("cursor");
  glutKeyboardFunc(key);
  glutReshapeFunc(reshape);
  glutDisplayFunc(display);
  glutMainLoop();
  return 0;
}

On my Mac OS X machine, the built-in apple implementation looks like this:

glut cursors mac

whereas freeglut using X11 looks like

freeglut cursors mac

Notable freeglut doesn’t manage to make the cursor disappear with GLUT_CURSOR_NONE, but it does have a much better GLUT_CURSOR_DESTROY.

Sparse versus dense Cholesky factorization benchmark

Wednesday, September 18th, 2013

Here’s a very informal benchmark comparing the sparse to the dense Cholesky factorizations as implemented by MATLAB. The problem I’m considering is a Laplace/Poisson equation on an n by n regular grid (x-axis is N = n2). Here’s the code I used:

td = [];
ts = [];
N=[];
for n = 1:30
  [V,F] = create_regular_grid(n,n,0,0);
  L = cotmatrix(V,F);
  FL = full(L);
  s = @() chol(-L(2:end,2:end));
  d = @() chol(-FL(2:end,2:end));
  ts = [ts(:);timeit(s,3)];
  td = [td(:);timeit(d)];
  N=[N(:);size(V,1)];
  loglog(N,ts,N,td,'LineWidth',4);
  drawnow;
end

And here is the log-log plot.

sparse vs dense cholesky factorization in matlab benchmark

On my machine, an iMac 3.4 Intel Core i7 with 16GB ram, I see the cross-over point at about N = 200. Shortly after that sparse wins handily.

Templated Eigen SparseMatrix with triangularView

Saturday, September 14th, 2013

I was trying to use Eigen’s triangularView for a templated SparseMatrix like this:

template <typename T> void foo()
{
  Eigen::SparseMatrix<T> A;
  A.triangularView<Upper>();
}

But I kept getting errors like:

 In function 'void foo()':
22:34: error: expected primary-expression before ')' token
 In instantiation of 'void foo() [with T = double]':
81:15:   required from here
22:3: error: no match for 'operator<' in 'A.Eigen::SparseMatrixBase<Derived>::triangularView<Mode> < (Eigen::._64)2u'

Turns out this can be fixed by appending the word template before triangularView:

template <typename T> void foo()
{
  Eigen::SparseMatrix<T> A;
  A.template triangularView<Eigen::Upper>();
}

Sparse quadratic programming solver in libigl

Friday, September 13th, 2013

Active set method iterations, constant and linear inequality constraints

I’ve implemented an active set solver for solving larger sparse quadratic programming problems. It’s now part of our libigl library (Version ≥ 0.3.1). It’s implemented using Eigen and follows closely Section 2.2.3 of my phd thesis.

Calling the function looks like:

active_set(A,B,known,Y,Aeq,Beq,Aieq,Bieq,lx,ux,Z)

Which solves the optimization problem:


    argmin ZT * A * Z + ZT * B
      Z
    subject to:  Z_known = Y
                 Aeq * Z = Beq
                Aieq * Z ≤ Bieq
                       Z ≥ lx
                       Z ≤ ux

Note: Since Eigen does not have an LDLT solver for semi-positive-definite sparse matrices yet, I’m using SparseLU which is unnecessarily slow.

Note: I’m not checking for linearly dependent constraints, so the rows of [Aeq;Aieq] must be linearly independent.

Update: I now check for linearly dependent constraints. However, I can’t get a sparse Q matrix from Eigen’s Sparse QR decomposition so when I encounter linearly dependent constraints some sparse operations sadly seem to become dense.

Note: I have not attempted any performance optimizations (yet). Don’t expect this to beat MATLAB or Mosek’s quadprogs. But do enjoy that this is a free, easy to incorporate QP solver in C++.

Advanced Topics in Computer Graphics and Vision at ETHz

Thursday, September 12th, 2013

siggraph papers

I’ll be the coordinator from graphics for the ETH Zurich two-credit seminar course Advanced Topics in Computer Graphics and Vision.

We’ll be reading state-of-the-art research paper from top-tier conferences like SIGGRAPH, ICCV, CVPR.