Applying Lens Distortion to Panoramic Camera Model

[botanyldrm]

I am able to change camera model to Panoramic model using bpy package directly such as:
bpy.data.objects["Camera"].data.type = "PANO" bpy.data.objects["Camera"].data.cycles.panorama_type = "FISHEYE_EQUISOLID"

Now, I want to be sure whether camera distortion is applied correctly in blenderproc for this case. I mean, will following code result with correct distortion:

mapping_coords = bproc.camera.set_lens_distortion(k1, k2, k3, p1, p2)

[themasterlink]

@5trobl Could you help out here?

As far as I know, this only works for "Perspective" and not for "PANO".

But, @5trobl is the expert here.

Best,
Max

[5trobl]

I don't know how Cycle's 'PANO' type relates to 'PERSP', perhaps @cornerfarmer does. There might be some relation (shared/converted parameters) but 'PANO' applies some nonlinear distortion on top (e.g., it's impossible to perspectively project an AOV>=180° on a planar sensor). That distortion does not correspond to the Brown-Conrady model. Even in the case of smaller fish-eye lenses with ~150° AOV where the Brown-Conrady could still hold, that distortion model is inadequate because it physically does not match actual lenses and because mathematically it very easily blends back to the front direction such that distortion is not even defined for the surrounding regions. For that lenses the Kannala-Brandt model on angular distances instead of projected distances works much better with even fewer parameters but we have not implemented that one yet.

Since bproc.camera.set_lens_distortion() relies on both the intrinsics of the 'PERSP' camera type and the Brown-Conrady radial distortion model, it should not be used for 'PANO' cameras anyway. If you obtain some results, these will certainly not be geometrically accurate to any realistic camera. Perhaps we should add an assert statement not to allow this. Nice try though!

PS: I'm an expert in geom. calibration not in bpy.