Hello, could you post the output of the assertion that fails? This would be helpful in trying to find the source of the problem

The assert is most likely unrelated to the bug, just some NaN.

The bug here would be in camera_world_to_ndc, or the data that goes into it.

I was able to trace down some things to direction_to_fisheye_equisolid but I am unsure of how that function was implemented. @Brecht Van Lommel (brecht) is there some documentation on how the equisolid fisheye lense was implemented?

There's no documentation for it that I know of. There's two functions here, which should do the inverse of the other. So unless the input to the functions is somehow wrong, it's a purely mathematical thing where the formulas are just not matching.

I tried verifying if the inputs are correct for the camera_world_to_ndc function and I am not sure they are quite correct. One thing I tried doing was changing the variables P and the kernel_data.cam.worldtocamera matrix such that the output moves into the correct position, but I did not get very far with that.

There's two functions here, which should do the inverse of the other

Could you please elaborate on that statement? Which two functions perform the inverse of each other? I will try verifying if they perform how they are supposed to once I know more about them.