I have removed the m_pHitObject, m_xray and m_testPropName and replace them by a temporary struct "RayCastData" which contains these datas and a collision mask. Finally i add a collision mask argument in the python function "rayCast" :
rayCast(to, from, dist, prop, face, xray, poly, mask)
It can be useful to hit only object which are on the right colision layer. for example if you have hitbox for a charater or vehicle you don't want to hit it with raycast.
test file :
left mouse click on two planes and see console messages.
Somewhat more elaborate test file by @Sybren A. Stüvel (sybren):
Look around and click on the cubes. One cube lamp responds, the other doesn't, based on their collision groups.