# gluUnProject buggy for canonical views

## Alec Jacobson

## March 15, 2015

I just ran into a very frustrating bug in `gluUnProject`

. It seems that if the model view matrix is rotated to align the view axis with the y-axis then `gluUnProject`

is unreliable. Here's a little example:

Set up you model view, projection and viewport with:

```
MV = [1 0 0 0;0 0 -1 0;0 1 0 -4.3;0 0 0 1];
P = [1.81066 0 0 0;0 2.41421 0 0; 0 0 -1.00019 -0.0200019; 0 0 -1 0];
VP = [0 0 960 720];
```

Then given a point in the scene

```
p = [1.66667 0 0];
```

project it using `gluProject`

(this seems to work OK):

```
proj_p = [816.867 360 0.99777];
```

move it one pixel up (as if with the mouse):

```
proj_p = [816.867 361 0.99777];
```

then unproject with `gluUnProject`

:

```
wrong = [1.80114 -0.345327 -0.00534674];
```

Where did that movement in the y-direction come from? My guess is the implementation of gluUnProject is doing a bad job inverting the scene matrix in terms of floating point error. Luckily, libigl has an implementation of `igl::unproject`

which uses Eigen's robust `inverse()`

function. This solved my problem and gives the correct unprojection:

```
correct = [1.66667 0 -0.00494755];
```