🐛 Orientation have different values on front and back camera

[luicfrr]

What's happening?

I am not sure if it's supposed to work this way or if it's actually a bug, but I noticed an orientation inconsistency between Android and iOS when switching cameras.

Both devices were tested in portrait mode, with screen orientation locked, and only switching between the front and back cameras.

On iOS, for both cameras, frame.orientation returned UIImage.Orientation.right. The device didn't change its orientation, so the frame orientation stayed the same.

However, on Android, the front camera returns Orientation.LANDSCAPE_LEFT and the back camera returns Orientation.LANDSCAPE_RIGHT.

I believe one of the systems is returning the wrong orientation value for one of the cameras.

Reproduceable Code

const [
  facingFront,
  setFacingFront
] = useState<boolean>( true )
const cameraDevice = useCameraDevice( facingFront ? 'front' : 'back' )

...

<Camera
  style={ StyleSheet.absoluteFill }
  isActive={ true }
  device={ cameraDevice }
  outputOrientation={ 'device' }
/>
<Button
  onPress={ () => setFacingFront( ( current ) => !current ) }
  title={ 'Toggle Cam' }
/>

Relevant log output

IOS
both front and back camera: frame.isMirrored: false - frame.orientation: UIImageOrientation(rawValue: 2)

ANDROID:
front camera: frame.isMirrored: false - frame.orientation: LANDSCAPE_LEFT
back camera: frame.isMirrored: false - frame.orientation: LANDSCAPE_RIGHT

Camera Device

IOS: {
  "minFocusDistance": 12,
  "hardwareLevel": "full",
  "isMultiCam": false,
  "id": "com.apple.avfoundation.avcapturedevice.built-in_video:0",
  "maxZoom": 16,
  "hasTorch": true,
  "supportsFocus": true,
  "supportsRawCapture": false,
  "neutralZoom": 1,
  "physicalDevices": [
    "wide-angle-camera"
  ],
  "position": "back",
  "supportsLowLightBoost": false,
  "maxExposure": 8,
  "sensorOrientation": "portrait",
  "minExposure": -8,
  "name": "Back Camera",
  "hasFlash": true,
  "formats": [],
  "minZoom": 1
}

Android: {
  "hardwareLevel": "limited",
  "minExposure": -20,
  "neutralZoom": 1,
  "minZoom": 1,
  "supportsFocus": false,
  "formats": [],
  "supportsLowLightBoost": false,
  "hasTorch": false,
  "supportsRawCapture": false,
  "minFocusDistance": 0,
  "sensorOrientation": "landscape-right",
  "maxZoom": 4,
  "physicalDevices": [
    "wide-angle-camera"
  ],
  "isMultiCam": false,
  "position": "front",
  "id": "1",
  "hasFlash": false,
  "maxExposure": 20,
  "name": "1 (FRONT) androidx.camera.camera2"
}

Device

Iphone 11 (IOS 17.5.1) - Galaxy J5 (Android 8.0)

VisionCamera Version

4.4.1

Can you reproduce this issue in the VisionCamera Example app?

Yes, I can reproduce the same issue in the Example app here

Additional information

• I am using Expo
• I have enabled Frame Processors (react-native-worklets-core)
• I have read the Troubleshooting Guide
• I agree to follow this project's Code of Conduct
• I searched for similar issues in this repository and found none.

[maintenance-hans]

Guten Tag, Hans here.

Note

New features, bugfixes, updates and other improvements are all handled mostly by @mrousavy in his free time.
To support @mrousavy, please consider 💖 sponsoring him on GitHub 💖.
Sponsored issues will be prioritized.

[Marius-Adam]

+1 here.
I am using a frame processor and it impacts the text recognition (works as expected when phone in landscape but does not pick anything up when device held in portrait)

Also opened an issue here: #3051

[mrousavy]

I think this is intended.

frame.orientation tells you the orientation of the physical pixels to appear upright, targeting the current output orientation.

So if output orientation is portrait (aka you want portrait photos), frame.orientation tells you what orientation it's pixel buffer is relative to the output orientation (example; portrait).

[luicfrr]

@mrousavy I think this issue should be reopened because values are not consitent.

As I said before, IOS always return .left with device is held in portrait mode in both cameras but android return LANDSCAPE_LEFT on front camera and LANDSCAPE_RIGHT in back.

One is wrong:

• or Android should return LANDSCAPE_LEFT value in both cameras like IOS
• or IOS should return .left in front camera and .right on back camera.

This inconsitence is the real issue.

[mrousavy]

@luicfrr please see this PR #3077 and read the description.

This explains how this works, and why orientation is not the same value.