Hi Juha.
Nice patch, thanks for that.

There is one thing I would like to clarify before committing it.
In KX_MouseFocusSensor.cpp you are changing the formula for the perspective projection as well.
I would expect that casting the point from Z -1.0 may lead to unexpected results.

For example, what happens if there is an object right behind the camera? I haven't tested it here, but I would expect it to be detected instead.
So I was wondering if we should keep 0.0 (or even cam->clipstart, although 0.0 should be fine) for perspective camera at least:
245 frompoint.setValue(.(2 * (m_x-x_lb) / width) - 1.0,
246 1.0 - (2 * (m_y_inv - y_lb) / height),
247 cam->GetCameraData()->m_perspective ? 0.0:-1.0,
248 1.0 );

What do you think? Maybe I understood this wrong and the normalized coords are always inside the camera frustrum?



245 . frompoint.setValue(.(2 * (m_x-x_lb) / width) - 1.0,
246 . . . . . . 1.0 - (2 * (m_y_inv - y_lb) / height),
247 . . . . . . cam->GetCameraData()->m_perspective ? 0.0:-1.0,
248 . . . . . . 1.0 );

Hi Dalai,

As I understand it the frustum is completely -1...1 in x,y,z when in normalized coords (post projection matrix + perspective division). Best way to make sure is to set a breakpoint at KX_MouseFocusSensor.cpp line 270 (in patched version) where frompoint and topoint have been multiplied with the inverted projection matrix only. Since the camera is pointing at negative Z-axis the values there should be:

frompoint[2] / frompoint[3] = negative clipstart
topoint[2] / topoint[3] = negative clipend.

Also you can test this with the mousefocus_camera.blend where there are two red pieces in front of the clipstart to both sides of the visible ones. Maybe not the best scenario since you first have to move the visible pieces from blocking the view of the first square on the gameboard; the remaining red pieces should become clickable in bge when you see them in normal 3d view camera (clipstart ~12.3). They're not visible in bge though because of backface culling unless you go to a lower clipstart.

From OpenGL coordinate transformations (had to cheat and look at the specs):
object coords -> ModelView -> eye coords -> Projection -> clip coords -> Perspective division -> normalized coords -> Viewport transformation -> window coords

Looking at it seems it's actually the final window coordinate z-value which ranges from 0 to 1 (or somewhere in between with glDepthRange set).

ok I will try to test this with the bp and get back to you/commit

Hi Juha,
in my test here (with mouse_focus_camera.blend alternating the camera mode to perspective/ortographic) I get:

The camera clip start is 12.5 and clipend is 17.5:

using -1.0
========
Perspective mode:
frompoint[2] / frompoint[3] = -17.5
topoint[2] / topoint[3] = -17.5

Ortographic
frompoint[2] / frompoint[3] = -12.5
topoint[2] / topoint[3] = -17.5

Using 0.0:
Perspective: same results (-17.5,-17.5)
Ortographic:
frompoint[2] / frompoint[3] = -15.0
topoint[2] / topoint[3] = -17.5


So for Ortographic you are spot on. But do you have any idea of do I always get the same value for Perspective regardless of the original frompoint[2] ?

Hi Dalai,

Hmm.. the -17.5, -17.5 for perspective is definitely not correct and I'm not seeing that here, any chance frompoint[2] is always set to 1.0? These are the patched lines for frompoint and topoint

frompoint.setValue((2 * (m_x-x_lb) / width) - 1.0,
1.0 - (2 * (m_y_inv - y_lb) / height),
-1.0,
1.0 );

topoint.setValue( (2 * (m_x-x_lb) / width) - 1.0,
1.0 - (2 * (m_y_inv-y_lb) / height),
1.0,
1.0 );

When frompoint first multiplied by the inverse projection matrix the values going in should always be X, X, -1.0, 1.0 and coming out they should have the "frompoint[2] / frompoint[3] = negative clipstart" to be correct. This is now in eyespace (or camera's localspace) so the clipstart and clipend should be at correct distances on the negative z-axis (have to divide by frompoint[3] to get the vector value when it's w component is 1.0, homogenous coordinates etc).

The next step in the code is multiplying with the camera's transform which will move them to worldspace and after that there's the division by frompoint[3] (it's a uniform division of all components in the vector (/matrix) so it can be done at any point in the calculation).

oh right. I was probably testing something wrong. I'm getting good results here. Great fix.

I will commit this as soon as the trunk is re-open (3 days from now). For 2.6 I think it would be a bit rushy since although the new result is correct, it may break old files (and it's too soon to test).

committed on rev. 41131
Thanks for the patch

This patch introduced -far +far when using 3dviewport camera (while fixed the same issue in the projection code itself).
I believe the right is to use +near +far all the time.

So for now, it was re-fixed in rev. 54733