Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

🐛 useSkiaFrameProcessor renders black screen using front cameras #2951

Open
4 of 5 tasks
khlling opened this issue Jun 8, 2024 · 21 comments
Open
4 of 5 tasks

🐛 useSkiaFrameProcessor renders black screen using front cameras #2951

khlling opened this issue Jun 8, 2024 · 21 comments
Labels
🐛 bug Something isn't working

Comments

@khlling
Copy link

khlling commented Jun 8, 2024

What's happening?

When calling frame.render() within useSkiaFrameProcessor on an Android phone, the captured frames render as expected when the camera position is set to 'back'. However, setting the camera position to 'front' results in a black screen.

Omitting useSkiaFrameProcessor (i.e., not passing a frame processor as a prop to ReanimatedCamera) renders as expected
Recording and playing back the video output from the front camera works fine, indicating the frames are captured correctly but not rendered properly in real-time.

Reproduceable Code

let device = useCameraDevice('front');
  const format = useCameraFormat(device, Templates.FrameProcessing);
  const microphone = useMicrophonePermission();
  const location = useLocationPermission();

  const frameProcessor = useSkiaFrameProcessor((frame) => {
    'worklet';
    frame.render();
  }, []);

<ReanimatedCamera
                style={StyleSheet.absoluteFill}
                device={device}
                ref={camera}
                format={format}
                enableFpsGraph={true}
                photo={true}
                video={true}
                audio={microphone.hasPermission}
                enableLocation={location.hasPermission}
                frameProcessor={frameProcessor}
              />

Relevant log output

2024-06-09 00:28:56.362 10089-10089 VRI[MainActivity]       com.toggled.wuji                     D  Start draw after previous draw not visible
2024-06-09 00:28:56.389 10089-10089 CameraView              com.toggled.wuji                     I  Updating CameraSession...
2024-06-09 00:28:56.396 10089-10089 CameraSession           com.toggled.wuji                     I  configure { ... }: Waiting for lock...
2024-06-09 00:28:56.396 10089-10089 CameraSession           com.toggled.wuji                     I  configure { ... }: Updating CameraSession Configuration... Difference(deviceChanged=false, outputsChanged=false, sidePropsChanged=false, isActiveChanged=true, orientationChanged=false, locationChanged=true)
2024-06-09 00:28:56.397 10089-10089 CameraSession           com.toggled.wuji                     I  Camera Lifecycle changed to STARTED!
2024-06-09 00:28:56.397 10089-10183 UseCaseAttachState      com.toggled.wuji                     D  Active and attached use case: [] for camera: 1
2024-06-09 00:28:56.397 10089-10089 CapabilitiesByQuality   com.toggled.wuji                     D  profiles = ImmutableEncoderProfilesProxy{defaultDurationSeconds=60, recommendedFileFormat=2, audioProfiles=[AudioProfileProxy{codec=3, mediaType=audio/mp4a-latm, bitrate=96000, sampleRate=48000, channels=1, profile=1}], videoProfiles=[VideoProfileProxy{codec=2, mediaType=video/avc, bitrate=22000000, frameRate=30, width=1920, height=1080, profile=-1, bitDepth=8, chromaSubsampling=0, hdrFormat=0}]}
2024-06-09 00:28:56.397 10089-10089 CapabilitiesByQuality   com.toggled.wuji                     D  profiles = ImmutableEncoderProfilesProxy{defaultDurationSeconds=60, recommendedFileFormat=2, audioProfiles=[AudioProfileProxy{codec=3, mediaType=audio/mp4a-latm, bitrate=96000, sampleRate=48000, channels=1, profile=1}], videoProfiles=[VideoProfileProxy{codec=2, mediaType=video/avc, bitrate=12000000, frameRate=30, width=1280, height=720, profile=-1, bitDepth=8, chromaSubsampling=0, hdrFormat=0}]}
2024-06-09 00:28:56.397 10089-10089 CapabilitiesByQuality   com.toggled.wuji                     D  profiles = ImmutableEncoderProfilesProxy{defaultDurationSeconds=60, recommendedFileFormat=2, audioProfiles=[AudioProfileProxy{codec=3, mediaType=audio/mp4a-latm, bitrate=96000, sampleRate=48000, channels=1, profile=1}], videoProfiles=[VideoProfileProxy{codec=2, mediaType=video/avc, bitrate=6000000, frameRate=30, width=720, height=480, profile=-1, bitDepth=8, chromaSubsampling=0, hdrFormat=0}]}
2024-06-09 00:28:56.397 10089-10089 CapabilitiesByQuality   com.toggled.wuji                     D  Using supported quality of ConstantQuality{value=5, name=HD, typicalSizes=[1280x720]} for size 1280x720
2024-06-09 00:28:56.397 10089-10089 VideoConfigUtil         com.toggled.wuji                     D  MediaSpec video mime matches EncoderProfiles. Using EncoderProfiles to derive VIDEO settings [mime type: video/avc]
2024-06-09 00:28:56.397 10089-10089 VidEncVdPrflRslvr       com.toggled.wuji                     D  Resolved frame rate 30fps [Video profile frame rate: 30fps, Expected operating range: <UNSPECIFIED>]
2024-06-09 00:28:56.397 10089-10089 VidEncVdPrflRslvr       com.toggled.wuji                     D  Resolved VIDEO frame rate: 30fps
2024-06-09 00:28:56.397 10089-10089 VidEncVdPrflRslvr       com.toggled.wuji                     D  Using resolved VIDEO bitrate from EncoderProfiles
2024-06-09 00:28:56.397 10089-10089 VideoConfigUtil         com.toggled.wuji                     D  Base Bitrate(12000000bps) * Bit Depth Ratio (8 / 8) * Frame Rate Ratio(30 / 30) * Width Ratio(1280 / 1280) * Height Ratio(720 / 720) = 12000000
2024-06-09 00:28:56.397 10089-10089 VideoConfigUtil         com.toggled.wuji                     W  Unsupported mime type video/avc or profile level -1. Data space is unspecified.
2024-06-09 00:28:56.397 10089-10089 CameraOrientationUtil   com.toggled.wuji                     D  getRelativeImageRotation: destRotationDegrees=0, sourceRotationDegrees=270, isOppositeFacing=false, result=270
2024-06-09 00:28:56.397 10089-10089 VideoCapture            com.toggled.wuji                     D  Surface processing is enabled.
2024-06-09 00:28:56.400 10089-10206 NativeCrypto            com.toggled.wuji                     V  Read error: ssl=0xb400007a33915798: I/O error during system call, Software caused connection abort
2024-06-09 00:28:56.400 10089-10206 NativeCrypto            com.toggled.wuji                     V  SSL shutdown failed: ssl=0xb400007a33915798: I/O error during system call, Broken pipe
2024-06-09 00:28:56.406 10089-12027 OpenGlRenderer          com.toggled.wuji                     D  EGLContext created, client version 2
2024-06-09 00:28:56.407 10089-10089 VideoCapture            com.toggled.wuji                     D  camera timebase = REALTIME, processing timebase = REALTIME
2024-06-09 00:28:56.407 10089-10089 DeferrableSurface       com.toggled.wuji                     D  Surface created[total_surfaces=3, used_surfaces=0](androidx.camera.core.processing.SurfaceEdge$SettableSurface@4b889a5}
2024-06-09 00:28:56.407 10089-10089 DeferrableSurface       com.toggled.wuji                     D  Surface created[total_surfaces=4, used_surfaces=0](androidx.camera.core.processing.SurfaceEdge$SettableSurface@823a588}
2024-06-09 00:28:56.407 10089-10089 DeferrableSurface       com.toggled.wuji                     D  Surface created[total_surfaces=5, used_surfaces=0](androidx.camera.core.SurfaceRequest$2@9301107}
2024-06-09 00:28:56.407 10089-10089 DeferrableSurface       com.toggled.wuji                     D  New surface in use[total_surfaces=5, used_surfaces=1](androidx.camera.core.SurfaceRequest$2@9301107}
2024-06-09 00:28:56.407 10089-10089 DeferrableSurface       com.toggled.wuji                     D  use count+1, useCount=1 androidx.camera.core.SurfaceRequest$2@9301107
2024-06-09 00:28:56.407 10089-10089 DeferrableSurface       com.toggled.wuji                     D  Surface created[total_surfaces=6, used_surfaces=1](androidx.camera.core.SurfaceRequest$2@1a0f3cc}
2024-06-09 00:28:56.408 10089-10089 DeferrableSurface       com.toggled.wuji                     D  New surface in use[total_surfaces=6, used_surfaces=2](androidx.camera.core.SurfaceRequest$2@1a0f3cc}
2024-06-09 00:28:56.408 10089-10089 DeferrableSurface       com.toggled.wuji                     D  use count+1, useCount=1 androidx.camera.core.SurfaceRequest$2@1a0f3cc
2024-06-09 00:28:56.408 10089-10089 Recorder                com.toggled.wuji                     D  Surface is requested in state: CONFIGURING, Current surface: 0
2024-06-09 00:28:56.408 10089-10089 CameraOrientationUtil   com.toggled.wuji                     D  getRelativeImageRotation: destRotationDegrees=0, sourceRotationDegrees=270, isOppositeFacing=false, result=270
2024-06-09 00:28:56.408 10089-10386 CapabilitiesByQuality   com.toggled.wuji                     D  profiles = ImmutableEncoderProfilesProxy{defaultDurationSeconds=60, recommendedFileFormat=2, audioProfiles=[AudioProfileProxy{codec=3, mediaType=audio/mp4a-latm, bitrate=96000, sampleRate=48000, channels=1, profile=1}], videoProfiles=[VideoProfileProxy{codec=2, mediaType=video/avc, bitrate=22000000, frameRate=30, width=1920, height=1080, profile=-1, bitDepth=8, chromaSubsampling=0, hdrFormat=0}]}
2024-06-09 00:28:56.408 10089-10386 CapabilitiesByQuality   com.toggled.wuji                     D  profiles = ImmutableEncoderProfilesProxy{defaultDurationSeconds=60, recommendedFileFormat=2, audioProfiles=[AudioProfileProxy{codec=3, mediaType=audio/mp4a-latm, bitrate=96000, sampleRate=48000, channels=1, profile=1}], videoProfiles=[VideoProfileProxy{codec=2, mediaType=video/avc, bitrate=12000000, frameRate=30, width=1280, height=720, profile=-1, bitDepth=8, chromaSubsampling=0, hdrFormat=0}]}
2024-06-09 00:28:56.408 10089-10386 CapabilitiesByQuality   com.toggled.wuji                     D  profiles = ImmutableEncoderProfilesProxy{defaultDurationSeconds=60, recommendedFileFormat=2, audioProfiles=[AudioProfileProxy{codec=3, mediaType=audio/mp4a-latm, bitrate=96000, sampleRate=48000, channels=1, profile=1}], videoProfiles=[VideoProfileProxy{codec=2, mediaType=video/avc, bitrate=6000000, frameRate=30, width=720, height=480, profile=-1, bitDepth=8, chromaSubsampling=0, hdrFormat=0}]}
2024-06-09 00:28:56.408 10089-10386 Recorder                com.toggled.wuji                     D  Using supported quality of ConstantQuality{value=5, name=HD, typicalSizes=[1280x720]} for surface size 720x1280
2024-06-09 00:28:56.408 10089-10386 Recorder                com.toggled.wuji                     D  Try to safely release video encoder: null
2024-06-09 00:28:56.408 10089-10386 VideoEncoderSession     com.toggled.wuji                     D  closeInternal in RELEASED state, No-op
2024-06-09 00:28:56.408 10089-10386 VideoEncoderSession     com.toggled.wuji                     D  Create VideoEncoderSession: VideoEncoderSession@201851597 for androidx.camera.core.SurfaceRequest@73019ff
2024-06-09 00:28:56.408 10089-10386 VideoConfigUtil         com.toggled.wuji                     D  MediaSpec video mime matches EncoderProfiles. Using EncoderProfiles to derive VIDEO settings [mime type: video/avc]
2024-06-09 00:28:56.408 10089-10386 VidEncVdPrflRslvr       com.toggled.wuji                     D  Resolved frame rate 30fps [Video profile frame rate: 30fps, Expected operating range: <UNSPECIFIED>]
2024-06-09 00:28:56.408 10089-10386 VidEncVdPrflRslvr       com.toggled.wuji                     D  Resolved VIDEO frame rate: 30fps
2024-06-09 00:28:56.408 10089-10386 VidEncVdPrflRslvr       com.toggled.wuji                     D  Using resolved VIDEO bitrate from EncoderProfiles
2024-06-09 00:28:56.408 10089-10185 Camera2CameraImpl       com.toggled.wuji                     D  {Camera@3b810bd[id=1]} Use case androidx.camera.video.VideoCapture-e983a322-28ff-4f2c-a015-832f40f611a363247364 ACTIVE
2024-06-09 00:28:56.408 10089-10386 VideoConfigUtil         com.toggled.wuji                     D  Base Bitrate(12000000bps) * Bit Depth Ratio (8 / 8) * Frame Rate Ratio(30 / 30) * Width Ratio(720 / 1280) * Height Ratio(1280 / 720) = 12000000
2024-06-09 00:28:56.408 10089-10386 VideoConfigUtil         com.toggled.wuji                     W  Unsupported mime type video/avc or profile level -1. Data space is unspecified.
2024-06-09 00:28:56.408 10089-10185 UseCaseAttachState      com.toggled.wuji                     D  Active and attached use case: [] for camera: 1
2024-06-09 00:28:56.408 10089-10089 CameraSession           com.toggled.wuji                     I  Camera State: CLOSED (has error: false)
2024-06-09 00:28:56.408 10089-10089 CameraSession           com.toggled.wuji                     I  Camera Lifecycle changed to RESUMED!
2024-06-09 00:28:56.408 10089-10089 MetadataProvider        com.toggled.wuji                     I  Start updating location...
2024-06-09 00:28:56.408 10089-10185 Camera2CameraImpl       com.toggled.wuji                     D  {Camera@3b810bd[id=1]} Use cases [androidx.camera.core.ImageCapture-02b2839f-a563-40a9-ae06-867f752643cc24918039, androidx.camera.video.VideoCapture-e983a322-28ff-4f2c-a015-832f40f611a363247364, androidx.camera.core.ImageAnalysis-48884d1f-5484-404e-baa9-1fa69bbecc02102428141] now ATTACHED
2024-06-09 00:28:56.408 10089-10185 UseCaseAttachState      com.toggled.wuji                     D  All use case: [androidx.camera.video.VideoCapture-e983a322-28ff-4f2c-a015-832f40f611a363247364, androidx.camera.core.ImageCapture-02b2839f-a563-40a9-ae06-867f752643cc24918039, androidx.camera.core.ImageAnalysis-48884d1f-5484-404e-baa9-1fa69bbecc02102428141] for camera: 1
2024-06-09 00:28:56.409 10089-10185 Camera2CameraImpl       com.toggled.wuji                     D  mMeteringRepeating is ATTACHED, SessionConfig Surfaces: 3, CaptureConfig Surfaces: 1
2024-06-09 00:28:56.409 10089-10185 UseCaseAttachState      com.toggled.wuji                     D  Active and attached use case: [androidx.camera.video.VideoCapture-e983a322-28ff-4f2c-a015-832f40f611a363247364] for camera: 1
2024-06-09 00:28:56.409 10089-10185 Camera2CameraImpl       com.toggled.wuji                     D  {Camera@3b810bd[id=1]} Resetting Capture Session
2024-06-09 00:28:56.409 10089-10386 MediaCodecList          com.toggled.wuji                     D  codecHandlesFormat: no format, so no extra checks
2024-06-09 00:28:56.409 10089-10185 Camera2CameraImpl       com.toggled.wuji                     D  {Camera@3b810bd[id=1]} Releasing session in state INITIALIZED
2024-06-09 00:28:56.409 10089-10185 Camera2CameraImpl       com.toggled.wuji                     D  {Camera@3b810bd[id=1]} Attempting to force open the camera.
2024-06-09 00:28:56.409 10089-10185 CameraStateRegistry     com.toggled.wuji                     D  tryOpenCamera(Camera@3b810bd[id=1]) [Available Cameras: 1, Already Open: false (Previous state: CLOSED)] --> SUCCESS
2024-06-09 00:28:56.409 10089-10386 MediaCodecList          com.toggled.wuji                     D  codecHandlesFormat: no format, so no extra checks
2024-06-09 00:28:56.409 10089-10185 CameraStateRegistry     com.toggled.wuji                     D  Recalculating open cameras:
                                                                                                    Camera                                       State                 
                                                                                                    -------------------------------------------------------------------
                                                                                                    Camera@3b810bd[id=1]                         OPENING               
                                                                                                    Camera@4f1e65a[id=0]                         CLOSED                
                                                                                                    -------------------------------------------------------------------
                                                                                                    Open count: 1 (Max allowed: 1)
2024-06-09 00:28:56.409 10089-10185 Camera2CameraImpl       com.toggled.wuji                     D  {Camera@3b810bd[id=1]} Opening camera.
2024-06-09 00:28:56.409 10089-10185 Camera2CameraImpl       com.toggled.wuji                     D  {Camera@3b810bd[id=1]} Transitioning camera internal state: INITIALIZED --> OPENING
2024-06-09 00:28:56.409 10089-10185 CameraStateMachine      com.toggled.wuji                     D  New public camera state CameraState{type=OPENING, error=null} from OPENING and null
2024-06-09 00:28:56.409 10089-10185 CameraStateMachine      com.toggled.wuji                     D  Publishing new public camera state CameraState{type=OPENING, error=null}
2024-06-09 00:28:56.409 10089-10185 UseCaseAttachState      com.toggled.wuji                     D  All use case: [androidx.camera.video.VideoCapture-e983a322-28ff-4f2c-a015-832f40f611a363247364, androidx.camera.core.ImageCapture-02b2839f-a563-40a9-ae06-867f752643cc24918039, androidx.camera.core.ImageAnalysis-48884d1f-5484-404e-baa9-1fa69bbecc02102428141] for camera: 1
2024-06-09 00:28:56.410 10089-10089 CameraSession           com.toggled.wuji                     I  configure { ... }: Completed CameraSession Configuration! (State: RESUMED)
2024-06-09 00:28:56.412 10089-12032 CCodec                  com.toggled.wuji                     D  allocate(c2.exynos.h264.encoder)
2024-06-09 00:28:56.412 10089-10089 VideoCapture            com.toggled.wuji                     D  Stream info update: old: StreamInfo{id=0, streamState=INACTIVE, inProgressTransformationInfo=null} new: StreamInfo{id=0, streamState=INACTIVE, inProgressTransformationInfo=null}
2024-06-09 00:28:56.412 10089-10089 CameraSession           com.toggled.wuji                     I  Camera State: OPENING (has error: false)
2024-06-09 00:28:56.415 10089-12032 CCodec                  com.toggled.wuji                     I  setting up 'default' as default (vendor) store
2024-06-09 00:28:56.418 10089-12032 CCodec                  com.toggled.wuji                     I  Created component [c2.exynos.h264.encoder]
2024-06-09 00:28:56.418 10089-12032 CCodecConfig            com.toggled.wuji                     D  read media type: video/avc
2024-06-09 00:28:56.426 10089-12032 ReflectedParamUpdater   com.toggled.wuji                     D  extent() != 1 for single value type: output.subscribed-indices.values
2024-06-09 00:28:56.426 10089-12032 ReflectedParamUpdater   com.toggled.wuji                     D  extent() != 1 for single value type: input.buffers.allocator-ids.values
2024-06-09 00:28:56.427 10089-12032 ReflectedParamUpdater   com.toggled.wuji                     D  extent() != 1 for single value type: output.buffers.allocator-ids.values
2024-06-09 00:28:56.427 10089-12032 ReflectedParamUpdater   com.toggled.wuji                     D  extent() != 1 for single value type: output.buffers.pool-ids.values
2024-06-09 00:28:56.430 10089-12032 ReflectedParamUpdater   com.toggled.wuji                     D  ignored struct field coding.qp.values
2024-06-09 00:28:56.432 10089-12032 ReflectedParamUpdater   com.toggled.wuji                     D  ignored struct field coding.gop.values
2024-06-09 00:28:56.438 10089-12032 CCodecConfig            com.toggled.wuji                     D  ignoring local param raw.color (0xc2001809) as it is already supported
2024-06-09 00:28:56.438 10089-12032 CCodecConfig            com.toggled.wuji                     I  query failed after returning 21 values (BAD_INDEX)
2024-06-09 00:28:56.438 10089-12032 CCodecConfig            com.toggled.wuji                     D  c2 config diff is Dict {
                                                                                                      c2::u32 algo.bitrate-mode.value = 3
                                                                                                      c2::u32 algo.encoding-quality-level.value = 0
                                                                                                      c2::i32 algo.priority.value = 0
                                                                                                      c2::float algo.rate.value = 0
                                                                                                      c2::i32 coded.average-qp.value = 0
                                                                                                      c2::u32 coded.bitrate-mode.value = 1
                                                                                                      c2::u32 coded.bitrate.value = 64000
                                                                                                      c2::float coded.frame-rate.value = 15
                                                                                                      c2::u32 coded.hdr-format.value = 0
                                                                                                      c2::u32 coded.picture-type.value = 0
                                                                                                      c2::u32 coded.pl.level = 20496
                                                                                                      c2::u32 coded.pl.profile = 20484
                                                                                                      c2::u32 coding.intra-refresh.mode = 0
                                                                                                      c2::float coding.intra-refresh.period = 0
                                                                                                      c2::u32 coding.request-sync-frame.value = 0
                                                                                                      c2::i64 coding.sync-frame-interval.value = 1000000
                                                                                                      Buffer coding.temporal-layering = {
                                                                                                        00000000:  10 00 00 00 07 20 01 52  00 00 00 00 00 00 00 00  ..... .R........
                                                                                                      }
                                                                                                      c2::u32 input.delay.value = 0
                                                                                                      string input.media-type.value = "video/raw"
                                                                                                      c2::u32 output.buffers.prepend-header.value = 0
                                                                                                      string output.media-type.value = "video/avc"
                                                                                                      c2::u32 raw.color.matrix = 0
                                                                                                      c2::u32 raw.color.primaries = 0
                                                                                                     
2024-06-09 00:28:56.440 10089-10386 VideoEncoder            com.toggled.wuji                     D  mInputTimebase = REALTIME
2024-06-09 00:28:56.440 10089-10386 VideoEncoder            com.toggled.wuji                     D  mMediaFormat = {color-format=2130708361, i-frame-interval=1, mime=video/avc, width=720, bitrate=12000000, frame-rate=30, height=1280}
2024-06-09 00:28:56.440 10089-10171 CameraDevices           com.toggled.wuji                     I  Camera #1 is now unavailable.
2024-06-09 00:28:56.443 10089-10185 Camera2CameraImpl       com.toggled.wuji                     D  {Camera@3b810bd[id=1]} Use case androidx.camera.core.ImageCapture-02b2839f-a563-40a9-ae06-867f752643cc24918039 ACTIVE
2024-06-09 00:28:56.443 10089-10185 UseCaseAttachState      com.toggled.wuji                     D  Active and attached use case: [androidx.camera.video.VideoCapture-e983a322-28ff-4f2c-a015-832f40f611a363247364, androidx.camera.core.ImageCapture-02b2839f-a563-40a9-ae06-867f752643cc24918039] for camera: 1
2024-06-09 00:28:56.443 10089-10185 Camera2CameraImpl       com.toggled.wuji                     D  {Camera@3b810bd[id=1]} Use case androidx.camera.video.VideoCapture-e983a322-28ff-4f2c-a015-832f40f611a363247364 ACTIVE
2024-06-09 00:28:56.443 10089-10185 UseCaseAttachState      com.toggled.wuji                     D  Active and attached use case: [androidx.camera.video.VideoCapture-e983a322-28ff-4f2c-a015-832f40f611a363247364, androidx.camera.core.ImageCapture-02b2839f-a563-40a9-ae06-867f752643cc24918039] for camera: 1
2024-06-09 00:28:56.444 10089-10185 Camera2CameraImpl       com.toggled.wuji                     D  {Camera@3b810bd[id=1]} Use case androidx.camera.core.ImageAnalysis-48884d1f-5484-404e-baa9-1fa69bbecc02102428141 ACTIVE
2024-06-09 00:28:56.444 10089-10185 UseCaseAttachState      com.toggled.wuji                     D  Active and attached use case: [androidx.camera.video.VideoCapture-e983a322-28ff-4f2c-a015-832f40f611a363247364, androidx.camera.core.ImageCapture-02b2839f-a563-40a9-ae06-867f752643cc24918039, androidx.camera.core.ImageAnalysis-48884d1f-5484-404e-baa9-1fa69bbecc02102428141] for camera: 1
2024-06-09 00:28:56.445 10089-10185 Camera2CameraImpl       com.toggled.wuji                     D  {Camera@3b810bd[id=1]} CameraDevice.onOpened()
2024-06-09 00:28:56.445 10089-10185 Camera2CameraImpl       com.toggled.wuji                     D  {Camera@3b810bd[id=1]} Transitioning camera internal state: OPENING --> OPENED
2024-06-09 00:28:56.445 10089-10185 CameraStateRegistry     com.toggled.wuji                     D  Recalculating open cameras:
                                                                                                    Camera                                       State                 
                                                                                                    -------------------------------------------------------------------
                                                                                                    Camera@3b810bd[id=1]                         OPEN                  
                                                                                                    Camera@4f1e65a[id=0]                         CLOSED                
                                                                                                    -------------------------------------------------------------------
                                                                                                    Open count: 1 (Max allowed: 1)
2024-06-09 00:28:56.445 10089-10185 CameraStateMachine      com.toggled.wuji                     D  New public camera state CameraState{type=OPEN, error=null} from OPEN and null
2024-06-09 00:28:56.445 10089-10185 CameraStateMachine      com.toggled.wuji                     D  Publishing new public camera state CameraState{type=OPEN, error=null}
2024-06-09 00:28:56.445 10089-10185 UseCaseAttachState      com.toggled.wuji                     D  All use case: [androidx.camera.video.VideoCapture-e983a322-28ff-4f2c-a015-832f40f611a363247364, androidx.camera.core.ImageCapture-02b2839f-a563-40a9-ae06-867f752643cc24918039, androidx.camera.core.ImageAnalysis-48884d1f-5484-404e-baa9-1fa69bbecc02102428141] for camera: 1
2024-06-09 00:28:56.445 10089-10185 UseCaseAttachState      com.toggled.wuji                     D  Active and attached use case: [androidx.camera.video.VideoCapture-e983a322-28ff-4f2c-a015-832f40f611a363247364, androidx.camera.core.ImageCapture-02b2839f-a563-40a9-ae06-867f752643cc24918039, androidx.camera.core.ImageAnalysis-48884d1f-5484-404e-baa9-1fa69bbecc02102428141] for camera: 1
2024-06-09 00:28:56.446 10089-10089 CameraSession           com.toggled.wuji                     I  Camera State: OPEN (has error: false)
2024-06-09 00:28:56.446 10089-10185 SyncCaptureSessionBase  com.toggled.wuji                     D  [androidx.camera.camera2.internal.SynchronizedCaptureSessionImpl@bfa482c] getSurface done with results: [Surface(name=null)/@0x238df90, Surface(name=null)/@0xfb54089, Surface(name=android.graphics.SurfaceTexture@a0cd2a)/@0x9cf80b8]
2024-06-09 00:28:56.446 10089-10185 CaptureSession          com.toggled.wuji                     D  Opening capture session.
2024-06-09 00:28:56.447 10089-10185 SyncCaptureSessionImpl  com.toggled.wuji                     D  [androidx.camera.camera2.internal.SynchronizedCaptureSessionImpl@bfa482c] start openCaptureSession
2024-06-09 00:28:56.447 10089-10185 DeferrableSurface       com.toggled.wuji                     D  New surface in use[total_surfaces=6, used_surfaces=3](androidx.camera.core.impl.ImmediateSurface@f1ba57a}
2024-06-09 00:28:56.447 10089-10185 DeferrableSurface       com.toggled.wuji                     D  use count+1, useCount=1 androidx.camera.core.impl.ImmediateSurface@f1ba57a
2024-06-09 00:28:56.447 10089-10185 DeferrableSurface       com.toggled.wuji                     D  New surface in use[total_surfaces=6, used_surfaces=4](androidx.camera.core.impl.ImmediateSurface@fa7bf21}
2024-06-09 00:28:56.447 10089-10185 DeferrableSurface       com.toggled.wuji                     D  use count+1, useCount=1 androidx.camera.core.impl.ImmediateSurface@fa7bf21
2024-06-09 00:28:56.447 10089-10185 DeferrableSurface       com.toggled.wuji                     D  New surface in use[total_surfaces=6, used_surfaces=5](androidx.camera.core.processing.SurfaceEdge$SettableSurface@4b889a5}
2024-06-09 00:28:56.447 10089-10185 DeferrableSurface       com.toggled.wuji                     D  use count+1, useCount=1 androidx.camera.core.processing.SurfaceEdge$SettableSurface@4b889a5
2024-06-09 00:28:56.454 10089-12032 hw-BpHwBinder           com.toggled.wuji                     I  onLastStrongRef automatically unlinking death recipients
2024-06-09 00:28:56.456 10089-12032 CCodec                  com.toggled.wuji                     D  allocate(c2.exynos.h264.encoder)
2024-06-09 00:28:56.459 10089-12032 CCodec                  com.toggled.wuji                     I  setting up 'default' as default (vendor) store
2024-06-09 00:28:56.461 10089-12032 CCodec                  com.toggled.wuji                     I  Created component [c2.exynos.h264.encoder]
2024-06-09 00:28:56.461 10089-12032 CCodecConfig            com.toggled.wuji                     D  read media type: video/avc
2024-06-09 00:28:56.462 10089-12032 ReflectedParamUpdater   com.toggled.wuji                     D  extent() != 1 for single value type: output.subscribed-indices.values
2024-06-09 00:28:56.463 10089-12032 ReflectedParamUpdater   com.toggled.wuji                     D  extent() != 1 for single value type: input.buffers.allocator-ids.values
2024-06-09 00:28:56.463 10089-12032 ReflectedParamUpdater   com.toggled.wuji                     D  extent() != 1 for single value type: output.buffers.allocator-ids.values
2024-06-09 00:28:56.463 10089-12032 ReflectedParamUpdater   com.toggled.wuji                     D  extent() != 1 for single value type: output.buffers.pool-ids.values
2024-06-09 00:28:56.465 10089-12032 ReflectedParamUpdater   com.toggled.wuji                     D  ignored struct field coding.qp.values
2024-06-09 00:28:56.470 10089-12032 ReflectedParamUpdater   com.toggled.wuji                     D  ignored struct field coding.gop.values
2024-06-09 00:28:56.472 10089-12032 CCodecConfig            com.toggled.wuji                     D  ignoring local param raw.color (0xc2001809) as it is already supported
2024-06-09 00:28:56.472 10089-12032 CCodecConfig            com.toggled.wuji                     I  query failed after returning 21 values (BAD_INDEX)
2024-06-09 00:28:56.472 10089-12032 CCodecConfig            com.toggled.wuji                     D  c2 config diff is Dict {
                                                                                                      c2::u32 algo.bitrate-mode.value = 3
                                                                                                      c2::u32 algo.encoding-quality-level.value = 0
                                                                                                      c2::i32 algo.priority.value = 0
                                                                                                      c2::float algo.rate.value = 0
                                                                                                      c2::i32 coded.average-qp.value = 0
                                                                                                      c2::u32 coded.bitrate-mode.value = 1
                                                                                                      c2::u32 coded.bitrate.value = 64000
                                                                                                      c2::float coded.frame-rate.value = 15
                                                                                                      c2::u32 coded.hdr-format.value = 0
                                                                                                      c2::u32 coded.picture-type.value = 0
                                                                                                      c2::u32 coded.pl.level = 20496
                                                                                                      c2::u32 coded.pl.profile = 20484
                                                                                                      c2::u32 coding.intra-refresh.mode = 0
                                                                                                      c2::float coding.intra-refresh.period = 0
                                                                                                      c2::u32 coding.request-sync-frame.value = 0
                                                                                                      c2::i64 coding.sync-frame-interval.value = 1000000
                                                                                                      Buffer coding.temporal-layering = {
                                                                                                        00000000:  10 00 00 00 07 20 01 52  00 00 00 00 00 00 00 00  ..... .R........
                                                                                                      }
                                                                                                      c2::u32 input.delay.value = 0
                                                                                                      string input.media-type.value = "video/raw"
                                                                                                      c2::u32 output.buffers.prepend-header.value = 0
                                                                                                      string output.media-type.value = "video/avc"
                                                                                                      c2::u32 raw.color.matrix = 0
                                                                                                      c2::u32 raw.color.primaries = 0
                                                                                                     
2024-06-09 00:28:56.475 10089-12031 MediaCodec              com.toggled.wuji                     I  MediaCodec will operate in async mode
2024-06-09 00:28:56.475 10089-10386 VQApply                 com.toggled.wuji                     D  minquality: codec claims to implement minquality=1
2024-06-09 00:28:56.475 10089-10386 MediaCodec              com.toggled.wuji                     D  shapeMediaFormat: deltas(1): AMessage(what = 0x00000000) = {
                                                                                                        int32_t android._encoding-quality-level = 1
                                                                                                      }
2024-06-09 00:28:56.475 10089-12032 CCodec                  com.toggled.wuji                     D  [c2.exynos.h264.encoder] buffers are bound to CCodec for this session
2024-06-09 00:28:56.475 10089-12032 CCodecConfig            com.toggled.wuji                     D  no c2 equivalents for color-format
2024-06-09 00:28:56.475 10089-12032 CCodecConfig            com.toggled.wuji                     D  no c2 equivalents for flags
2024-06-09 00:28:56.475 10089-12032 CCodecConfig            com.toggled.wuji                     D  no c2 equivalents for encoder
2024-06-09 00:28:56.476 10089-12032 CCodecConfig            com.toggled.wuji                     D  c2 config diff is   c2::u32 algo.encoding-quality-level.value = 1
                                                                                                      c2::u32 coded.bitrate.value = 12000000
                                                                                                      c2::float coded.frame-rate.value = 30
                                                                                                      c2::u32 raw.size.height = 1280
                                                                                                      c2::u32 raw.size.width = 720
2024-06-09 00:28:56.477 10089-12032 Codec2Client            com.toggled.wuji                     W  query -- param skipped: index = 3254781982.
2024-06-09 00:28:56.477 10089-12032 CCodec                  com.toggled.wuji                     D  encoding statistics level = 0
2024-06-09 00:28:56.477 10089-12032 CCodec                  com.toggled.wuji                     D  setup formats input: AMessage(what = 0x00000000) = {
                                                                                                      int32_t android._color-format = 2130708361
                                                                                                      int32_t android._dataspace = 0
                                                                                                      int32_t bitrate-mode = 1
                                                                                                      int32_t frame-rate = 30
                                                                                                      int32_t height = 1280
                                                                                                      int32_t intra-refresh-period = 0
                                                                                                      string mime = "video/raw"
                                                                                                      int32_t prepend-sps-pps-to-idr-frames = 0
                                                                                                      int32_t priority = 0
                                                                                                      int32_t video-qp-average = 0
                                                                                                      int32_t width = 720
                                                                                                      Rect crop(0, 0, 719, 1279)
                                                                                                      int64_t android._C2MemoryUsage = 1125899906908160
                                                                                                      int32_t color-format = 2130708361
                                                                                                    }
2024-06-09 00:28:56.477 10089-12032 CCodec                  com.toggled.wuji                     D  setup formats output: AMessage(what = 0x00000000) = {
                                                                                                      int32_t bitrate = 12000000
                                                                                                      int32_t bitrate-mode = 1
                                                                                                      Rect crop(0, 0, 719, 1279)
                                                                                                      int32_t width = 720
                                                                                                      int32_t color-standard = 0
                                                                                                      int32_t color-range = 0
                                                                                                      int32_t color-transfer = 0
                                                                                                      int32_t frame-rate = 30
                                                                                                      int32_t height = 1280
                                                                                                      int32_t intra-refresh-period = 0
                                                                                                      int32_t level = 65536
                                                                                                      int32_t max-bitrate = 12000000
                                                                                                      string mime = "video/avc"
                                                                                                      int32_t prepend-sps-pps-to-idr-frames = 0
                                                                                                      int32_t priority = 0
                                                                                                      int32_t profile = 8
                                                                                                      int32_t video-qp-average = 0
                                                                                                    }
2024-06-09 00:28:56.477 10089-12032 CCodecConfig            com.toggled.wuji                     I  query failed after returning 22 values (BAD_INDEX)
2024-06-09 00:28:56.477 10089-12032 CCodec                  com.toggled.wuji                     D  input format changed to AMessage(what = 0x00000000) = {
                                                                                                      int32_t android._color-format = 2130708361
                                                                                                      int32_t android._dataspace = 260
                                                                                                      int32_t bitrate-mode = 1
                                                                                                      int32_t frame-rate = 30
                                                                                                      int32_t height = 1280
                                                                                                      int32_t intra-refresh-period = 0
                                                                                                      string mime = "video/raw"
                                                                                                      int32_t prepend-sps-pps-to-idr-frames = 0
                                                                                                      int32_t priority = 0
                                                                                                      int32_t video-qp-average = 0
                                                                                                      int32_t width = 720
                                                                                                      Rect crop(0, 0, 719, 1279)
                                                                                                      int64_t android._C2MemoryUsage = 1125899906908160
                                                                                                      int32_t color-format = 2130708361
                                                                                                      int32_t color-range = 2
                                                                                                      int32_t color-standard = 1
                                                                                                      int32_t color-transfer = 3
                                                                                                    }
2024-06-09 00:28:56.477 10089-12032 GraphicBufferSource     com.toggled.wuji                     D  setting dataspace: 0x104, acquired=0
2024-06-09 00:28:56.477 10089-12032 CCodec                  com.toggled.wuji                     D  ISConfig not changed
2024-06-09 00:28:56.479 10089-10386 VideoEncoder            com.toggled.wuji                     D  Transitioning encoder internal state: null --> CONFIGURED
2024-06-09 00:28:56.479 10089-10386 Recorder                com.toggled.wuji                     D  Video source has transitioned to state: ACTIVE_NON_STREAMING
2024-06-09 00:28:56.479 10089-10386 VideoEncoderSession     com.toggled.wuji                     D  provide surface: Surface(name=null)/@0x4462971
2024-06-09 00:28:56.480 10089-10386 Recorder                com.toggled.wuji                     D  VideoEncoder is created. androidx.camera.video.internal.encoder.EncoderImpl@5130fd7
2024-06-09 00:28:56.480 10089-10089 DeferrableSurface       com.toggled.wuji                     D  New surface in use[total_surfaces=6, used_surfaces=6](androidx.camera.core.processing.SurfaceEdge$SettableSurface@823a588}
2024-06-09 00:28:56.480 10089-10386 Recorder                com.toggled.wuji                     D  Transitioning streamId: 0 --> 71706993
2024-06-09 00:28:56.480 10089-10089 DeferrableSurface       com.toggled.wuji                     D  use count+1, useCount=1 androidx.camera.core.processing.SurfaceEdge$SettableSurface@823a588
2024-06-09 00:28:56.480 10089-10386 Recorder                com.toggled.wuji                     D  Transitioning Recorder internal state: CONFIGURING --> IDLING
2024-06-09 00:28:56.480 10089-10089 CameraOrientationUtil   com.toggled.wuji                     D  getRelativeImageRotation: destRotationDegrees=0, sourceRotationDegrees=270, isOppositeFacing=false, result=270
2024-06-09 00:28:56.481 10089-10089 VideoCapture            com.toggled.wuji                     D  Stream info update: old: StreamInfo{id=0, streamState=INACTIVE, inProgressTransformationInfo=null} new: StreamInfo{id=71706993, streamState=INACTIVE, inProgressTransformationInfo=null}
2024-06-09 00:28:56.496 10089-10185 SyncCaptureSessionImpl  com.toggled.wuji                     D  [androidx.camera.camera2.internal.SynchronizedCaptureSessionImpl@bfa482c] Session onConfigured()
2024-06-09 00:28:56.496 10089-10185 CaptureSession          com.toggled.wuji                     D  Attempting to send capture request onConfigured
2024-06-09 00:28:56.496 10089-10185 CaptureSession          com.toggled.wuji                     D  Issuing request for session.
2024-06-09 00:28:56.496 10089-10185 Camera2Cap...estBuilder com.toggled.wuji                     D  createCaptureRequest
2024-06-09 00:28:56.497 10089-10185 CaptureSession          com.toggled.wuji                     D  CameraCaptureSession.onConfigured() mState=OPENED
2024-06-09 00:28:56.497 10089-10185 CaptureSession          com.toggled.wuji                     D  CameraCaptureSession.onReady() OPENED
2024-06-09 00:28:56.499 10089-10136 TrafficStats            com.toggled.wuji                     D  tagSocket(89) with statsTag=0xffffffff, statsUid=-1
2024-06-09 00:28:56.504 10089-10206 TrafficStats            com.toggled.wuji                     D  tagSocket(123) with statsTag=0x4e864ee, statsUid=-1
2024-06-09 00:28:56.506 10089-10131 TrafficStats            com.toggled.wuji                     D  tagSocket(199) with statsTag=0xffffffff, statsUid=-1
2024-06-09 00:28:56.587 10089-10286 RNFBAppCheck            com.toggled.wuji                     D  Provider::getToken - delegating to native provider
2024-06-09 00:28:56.592 10089-10183 VideoCapture            com.toggled.wuji                     D  cameraCaptureResult timestampNs = 691665786150556, current system uptimeMs = 405991215, current system realtimeMs = 691665862
2024-06-09 00:28:56.595 10089-12123 NativeCrypto            com.toggled.wuji                     V  Read error: ssl=0xb400007a33966a98: I/O error during system call, Software caused connection abort
2024-06-09 00:28:56.595 10089-12123 NativeCrypto            com.toggled.wuji                     V  SSL shutdown failed: ssl=0xb400007a33966a98: I/O error during system call, Broken pipe
2024-06-09 00:28:56.598 10089-12123 TrafficStats            com.toggled.wuji                     D  tagSocket(107) with statsTag=0xffffffff, statsUid=-1
2024-06-09 00:28:56.768 10089-10136 GraphResponse           com.toggled.wuji                     E  {HttpStatus: 400, errorCode: 100, subErrorCode: 33, errorType: GraphMethodException, errorMessage: Unsupported get request. Object with ID 'changeme' does not exist, cannot be loaded due to missing permissions, or does not support this operation. Please read the Graph API documentation at https://developers.facebook.com/docs/graph-api}
2024-06-09 00:28:56.768 10089-10131 GraphResponse           com.toggled.wuji                     E  {HttpStatus: 400, errorCode: 100, subErrorCode: 33, errorType: GraphMethodException, errorMessage: Unsupported get request. Object with ID 'changeme' does not exist, cannot be loaded due to missing permissions, or does not support this operation. Please read the Graph API documentation at https://developers.facebook.com/docs/graph-api}
2024-06-09 00:28:56.809 10089-10131 GraphResponse           com.toggled.wuji                     E  {HttpStatus: 400, errorCode: 100, subErrorCode: 33, errorType: GraphMethodException, errorMessage: Unsupported get request. Object with ID 'changeme' does not exist, cannot be loaded due to missing permissions, or does not support this operation. Please read the Graph API documentation at https://developers.facebook.com/docs/graph-api}
2024-06-09 00:28:56.824 10089-10301 PlayCore                com.toggled.wuji                     I  UID: [10464]  PID: [10089] IntegrityService : requestIntegrityToken(IntegrityTokenRequest{nonce=AUeuxb0lUili18DwKwinWC92d41jZX-9mWDwPf-uRdMhe9iKllSzRFOHGL4DN8C5Z3TgPSwgejtTTH1_5O46_gLIe8w0wQ-x4w-Uh0PAPN2Ca-NQ53BBUqMjlOgOqUEdl_A1cYn5_VQHMUfw75hP5mrZNaTWviFmSw1_-WWG5nslBA==, cloudProjectNumber=103015055007, network=null})
2024-06-09 00:28:56.824 10089-10302 PlayCore                com.toggled.wuji                     I  UID: [10464]  PID: [10089] IntegrityService : Initiate binding to the service.
2024-06-09 00:28:56.832 10089-10089 PlayCore                com.toggled.wuji                     I  UID: [10464]  PID: [10089] IntegrityService : ServiceConnectionImpl.onServiceConnected(ComponentInfo{com.android.vending/com.google.android.finsky.integrityservice.IntegrityService})
2024-06-09 00:28:56.832 10089-10302 PlayCore                com.toggled.wuji                     I  UID: [10464]  PID: [10089] IntegrityService : linkToDeath
2024-06-09 00:28:56.846 10089-10089 ionCamera.video         com.toggled.wuji                     W  type=1400 audit(0.0:467655): avc:  denied  { getattr } for  name="/" dev="dmabuf" ino=1 scontext=u:r:untrusted_app:s0:c208,c257,c512,c768 tcontext=u:object_r:unlabeled:s0 tclass=filesystem permissive=0 app=com.toggled.wuji
2024-06-09 00:28:56.869 10089-10131 GraphResponse           com.toggled.wuji                     E  {HttpStatus: 400, errorCode: 100, subErrorCode: 33, errorType: GraphMethodException, errorMessage: Unsupported get request. Object with ID 'changeme' does not exist, cannot be loaded due to missing permissions, or does not support this operation. Please read the Graph API documentation at https://developers.facebook.com/docs/graph-api}
2024-06-09 00:28:56.874 10089-10089 ionCamera.video         com.toggled.wuji                     W  type=1400 audit(0.0:467656): avc:  denied  { getattr } for  name="/" dev="dmabuf" ino=1 scontext=u:r:untrusted_app:s0:c208,c257,c512,c768 tcontext=u:object_r:unlabeled:s0 tclass=filesystem permissive=0 app=com.toggled.wuji
2024-06-09 00:28:56.892 10089-10136 GraphResponse           com.toggled.wuji                     E  {HttpStatus: 400, errorCode: 100, subErrorCode: 33, errorType: GraphMethodException, errorMessage: Unsupported post request. Object with ID 'changeme' does not exist, cannot be loaded due to missing permissions, or does not support this operation. Please read the Graph API documentation at https://developers.facebook.com/docs/graph-api}
2024-06-09 00:28:56.906 10089-10089 ionCamera.video         com.toggled.wuji                     W  type=1400 audit(0.0:467657): avc:  denied  { getattr } for  name="/" dev="dmabuf" ino=1 scontext=u:r:untrusted_app:s0:c208,c257,c512,c768 tcontext=u:object_r:unlabeled:s0 tclass=filesystem permissive=0 app=com.toggled.wuji
2024-06-09 00:28:56.928 10089-10131 GraphResponse           com.toggled.wuji                     E  {HttpStatus: 400, errorCode: 100, subErrorCode: 33, errorType: GraphMethodException, errorMessage: Unsupported get request. Object with ID 'changeme' does not exist, cannot be loaded due to missing permissions, or does not support this operation. Please read the Graph API documentation at https://developers.facebook.com/docs/graph-api}
2024-06-09 00:28:56.950 10089-10089 ionCamera.video         com.toggled.wuji                     W  type=1400 audit(0.0:467658): avc:  denied  { getattr } for  name="/" dev="dmabuf" ino=1 scontext=u:r:untrusted_app:s0:c208,c257,c512,c768 tcontext=u:object_r:unlabeled:s0 tclass=filesystem permissive=0 app=com.toggled.wuji
2024-06-09 00:28:56.970 10089-10089 ionCamera.video         com.toggled.wuji                     W  type=1400 audit(0.0:467659): avc:  denied  { getattr } for  name="/" dev="dmabuf" ino=1 scontext=u:r:untrusted_app:s0:c208,c257,c512,c768 tcontext=u:object_r:unlabeled:s0 tclass=filesystem permissive=0 app=com.toggled.wuji
2024-06-09 00:28:56.984 10089-10089 om.toggled.wuji         com.toggled.wuji                     E  [SurfaceTexture-0-10089-12] updateAndRelease: EGLConsumer is not attached to an OpenGL ES context
2024-06-09 00:28:56.984 10089-10089 RNSkia                  com.toggled.wuji                     I  updateAndRelease() failed. The exception above can safely be ignored
2024-06-09 00:28:57.013 10089-10131 GraphResponse           com.toggled.wuji                     E  {HttpStatus: 400, errorCode: 100, subErrorCode: 33, errorType: GraphMethodException, errorMessage: Unsupported get request. Object with ID 'changeme' does not exist, cannot be loaded due to missing permissions, or does not support this operation. Please read the Graph API documentation at https://developers.facebook.com/docs/graph-api}
2024-06-09 00:28:57.064 10089-10131 GraphResponse           com.toggled.wuji                     E  {HttpStatus: 400, errorCode: 100, subErrorCode: 33, errorType: GraphMethodException, errorMessage: Unsupported get request. Object with ID 'changeme' does not exist, cannot be loaded due to missing permissions, or does not support this operation. Please read the Graph API documentation at https://developers.facebook.com/docs/graph-api}
2024-06-09 00:28:57.117 10089-10089 om.toggled.wuji         com.toggled.wuji                     E  [SurfaceTexture-0-10089-12] updateAndRelease: EGLConsumer is not attached to an OpenGL ES context
2024-06-09 00:28:57.117 10089-10089 RNSkia                  com.toggled.wuji                     I  updateAndRelease() failed. The exception above can safely be ignored
2024-06-09 00:28:57.189 10089-10089 CameraSession           com.toggled.wuji                     I  Output orientation changed! LANDSCAPE_RIGHT
2024-06-09 00:28:57.190 10089-10089 CameraSession           com.toggled.wuji                     I  Output orientation changed! LANDSCAPE_RIGHT
2024-06-09 00:28:57.190 10089-10089 CameraSession           com.toggled.wuji                     I  Output orientation changed! LANDSCAPE_RIGHT
2024-06-09 00:28:57.191 10089-10089 CapabilitiesByQuality   com.toggled.wuji                     D  profiles = ImmutableEncoderProfilesProxy{defaultDurationSeconds=60, recommendedFileFormat=2, audioProfiles=[AudioProfileProxy{codec=3, mediaType=audio/mp4a-latm, bitrate=96000, sampleRate=48000, channels=1, profile=1}], videoProfiles=[VideoProfileProxy{codec=2, mediaType=video/avc, bitrate=22000000, frameRate=30, width=1920, height=1080, profile=-1, bitDepth=8, chromaSubsampling=0, hdrFormat=0}]}
2024-06-09 00:28:57.191 10089-10089 CapabilitiesByQuality   com.toggled.wuji                     D  profiles = ImmutableEncoderProfilesProxy{defaultDurationSeconds=60, recommendedFileFormat=2, audioProfiles=[AudioProfileProxy{codec=3, mediaType=audio/mp4a-latm, bitrate=96000, sampleRate=48000, channels=1, profile=1}], videoProfiles=[VideoProfileProxy{codec=2, mediaType=video/avc, bitrate=12000000, frameRate=30, width=1280, height=720, profile=-1, bitDepth=8, chromaSubsampling=0, hdrFormat=0}]}
2024-06-09 00:28:57.191 10089-10089 CapabilitiesByQuality   com.toggled.wuji                     D  profiles = ImmutableEncoderProfilesProxy{defaultDurationSeconds=60, recommendedFileFormat=2, audioProfiles=[AudioProfileProxy{codec=3, mediaType=audio/mp4a-latm, bitrate=96000, sampleRate=48000, channels=1, profile=1}], videoProfiles=[VideoProfileProxy{codec=2, mediaType=video/avc, bitrate=6000000, frameRate=30, width=720, height=480, profile=-1, bitDepth=8, chromaSubsampling=0, hdrFormat=0}]}
2024-06-09 00:28:57.191 10089-10089 QualitySelector         com.toggled.wuji

Camera Device

{
  "formats": [],
  "sensorOrientation": "landscape-right",
  "hardwareLevel": "full",
  "maxZoom": 10,
  "minZoom": 1,
  "maxExposure": 24,
  "supportsLowLightBoost": true,
  "neutralZoom": 1,
  "physicalDevices": [
    "wide-angle-camera"
  ],
  "supportsFocus": true,
  "supportsRawCapture": false,
  "isMultiCam": false,
  "minFocusDistance": 0,
  "minExposure": -24,
  "name": "1 (FRONT) androidx.camera.camera2",
  "hasFlash": false,
  "hasTorch": false,
  "position": "front",
  "id": "1"
}

Device

Pixel 6 (Android 14)

VisionCamera Version

4.1.0

Can you reproduce this issue in the VisionCamera Example app?

Yes, I can reproduce the same issue in the Example app here

Additional information

@khlling khlling added the 🐛 bug Something isn't working label Jun 8, 2024
@DarkShtir
Copy link

DarkShtir commented Jun 10, 2024

Same issue.
But my version is "4.2.0"

@khlling
Copy link
Author

khlling commented Jun 10, 2024

I've updated my version to "4.2.0" also. By chance, I tipped my phone from portrait to landscape whilst using the front camera and it started working. As soon as I tip it back to portrait it goes black. Try that out @DarkShtir.

I assume therefore it's something to do with the new orientation changes in the recent release. Will have an investigate of the latest changes

@DarkShtir
Copy link

@khlling this works, but the same time is not easy and comfortable to use.
I Downgraded to "4.0.5" and use custom patch to transform, rotate and align canvas in skia fame processor
I'll try get time for additional investigation of this issue

@khlling
Copy link
Author

khlling commented Jun 11, 2024

@DarkShtir Agreed I'm sure it's not the intended behaviour.

What changes did you have to make in 4.0.5 to get it to work? Can you share your patch files?

@DarkShtir
Copy link

DarkShtir commented Jun 11, 2024

diff --git a/node_modules/react-native-vision-camera/src/skia/useSkiaFrameProcessor.ts b/node_modules/react-native-vision-camera/src/skia/useSkiaFrameProcessor.ts
index 30efc01..9468756 100644
--- a/node_modules/react-native-vision-camera/src/skia/useSkiaFrameProcessor.ts
+++ b/node_modules/react-native-vision-camera/src/skia/useSkiaFrameProcessor.ts
@@ -9,6 +9,7 @@ import type { SkCanvas, SkPaint, SkImage, SkSurface } from '@shopify/react-nativ
 import { SkiaProxy } from '../dependencies/SkiaProxy'
 import { withFrameRefCounting } from '../frame-processors/withFrameRefCounting'
 import { VisionCameraProxy } from '../frame-processors/VisionCameraProxy'
+import { Dimensions, Platform } from 'react-native'
 
 /**
  * Represents a Camera Frame that can be directly drawn to using Skia.
@@ -166,11 +167,48 @@ export function createSkiaFrameProcessor(
               'worklet'
               // rotate canvas to properly account for Frame orientation
               canvas.save()
-              const rotation = getRotationDegrees(frame.orientation)
-              canvas.rotate(rotation, frame.width / 2, frame.height / 2)
-              // render the Camera Frame to the Canvas
-              if (paint != null) canvas.drawImage(image, 0, 0, paint)
-              else canvas.drawImage(image, 0, 0)
+              // BUG in Andoid ⬇️ 
+              if(Platform.OS==='ios'){
+                const rotation = getRotationDegrees(frame.orientation)
+        
+                canvas.rotate(rotation, frame.width / 2, frame.height / 2)
+                // render the Camera Frame to the Canvas
+                if (paint != null) canvas.drawImage(image, 0, 0, paint)
+                else canvas.drawImage(image, 0, 0)
+              }
+              // BUG ⬆️
+
+              // FIX on Android⬇️
+              if(Platform.OS==='android'){
+                const rotationAngle = getRotationDegrees(frame.orientation)
+                
+                const { width: frameWidth, height: frameHeight } = getPortraitSize(frame)
+
+                // center of the canvas
+                const centerX = image.width() / 2
+                const centerY = image.height() / 2
+
+                const currentPaint = paint ?? Skia.Paint()
+
+                const alphaX = Math.abs((image.width()-frameWidth)/2);
+                const alphaY = Math.abs((image.height()-frameHeight)/2);
+
+
+                const srcRect = Skia.XYWHRect(0, 0, image.width(),image.height())
+                let dstRect = Skia.XYWHRect(0,0, image.width(),image.height())
+                // Define the destination rectangle on the canvas
+                if (frame.orientation==='landscape-right') {
+                   dstRect = Skia.XYWHRect(-alphaX,-alphaY, image.width(),image.height())
+                } else if (frame.orientation==='landscape-left') {
+                   dstRect = Skia.XYWHRect(alphaX,alphaY, image.width(),image.height())
+                }
+                // Draw the image on the canvas
+
+                canvas.rotate(rotationAngle, centerX, centerY)
+
+                canvas.drawImageRect(image, srcRect, dstRect, currentPaint)
+              }
+              // FIX ⬆️
 
               // restore transforms/rotations again
               canvas.restore()

No problem, is local fix for project, to use skiaFrameProcessor
But it isn't great, and has bugs, for example reverse rotations when using front camera. But now this fix is resolving our issues.
Hope this help you.

@tsnguyenducphuong
Copy link

I had the same blank screen issue on Android 14 (Samsung A53) and resolved the issue by changing minSdkVersion from 24 to 26 in android/build.gradle.

@tsnguyenducphuong
Copy link

Also add the following would help:
const pixelFormat = Platform.OS === 'ios' ? 'rgb' : 'yuv';

@mrousavy
Copy link
Owner

@tsnguyenducphuong wait I thought VisionCamera/Skia will throw an error if minSdkVersion is not 26 or higher? Need to double-check with @wcandillon, but we should definitely throw very concise errors in such cases instead of returning null and just logging the errors to the console..

@mrousavy
Copy link
Owner

Either way, the issue from OP (@khlling) is not related to minSdkversion as it renders fine on back camera, but not on front. I think this has to do with the orientation/matrix code - @khlling can you play around with this function to figure out where it goes wrong?

switch (frame.orientation) {
case 'portrait':
// do nothing
break
case 'landscape-left':
// rotate two flips on (0,0) origin and move X + Y into view again
canvas.translate(frame.height, frame.width)
canvas.translate(270, 0)
break
case 'portrait-upside-down':
// rotate three flips on (0,0) origin and move Y into view again
canvas.translate(frame.width, frame.height)
canvas.rotate(180, 0, 0)
break
case 'landscape-right':
// rotate one flip on (0,0) origin and move X into view again
canvas.translate(frame.height, 0)
canvas.rotate(90, 0, 0)
break
default:
throw new Error(`Invalid frame.orientation: ${frame.orientation}!`)
}

I unfortunately don't have the free time to look into this right now, but if you find a solution in there I'd appreciate a PR! Thanks

@mrousavy
Copy link
Owner

Maybe it'd also be helpful if you posted device.sensorOrientation as well as frame.orientation of both the back and the front camera here just so we know what values we're dealing with.

@khlling
Copy link
Author

khlling commented Jun 12, 2024

@tsnguyenducphuong wait I thought VisionCamera/Skia will throw an error if minSdkVersion is not 26 or higher? Need to double-check with @wcandillon, but we should definitely throw very concise errors in such cases instead of returning null and just logging the errors to the console..

I can confirm that VisionCamera/Skia does throw an error for this. I had to update the minSdkVersionon the back of the build error.

Post this update I'm getting a blank front camera preview.

@khlling
Copy link
Author

khlling commented Jun 12, 2024

Either way, the issue from OP (@khlling) is not related to minSdkversion as it renders fine on back camera, but not on front. I think this has to do with the orientation/matrix code - @khlling can you play around with this function to figure out where it goes wrong?

switch (frame.orientation) {
case 'portrait':
// do nothing
break
case 'landscape-left':
// rotate two flips on (0,0) origin and move X + Y into view again
canvas.translate(frame.height, frame.width)
canvas.translate(270, 0)
break
case 'portrait-upside-down':
// rotate three flips on (0,0) origin and move Y into view again
canvas.translate(frame.width, frame.height)
canvas.rotate(180, 0, 0)
break
case 'landscape-right':
// rotate one flip on (0,0) origin and move X into view again
canvas.translate(frame.height, 0)
canvas.rotate(90, 0, 0)
break
default:
throw new Error(`Invalid frame.orientation: ${frame.orientation}!`)
}

I unfortunately don't have the free time to look into this right now, but if you find a solution in there I'd appreciate a PR! Thanks

No worries @mrousavy let me do a bit of digging. Thanks for giving me a hint of where to start looking!

Will update this issue with logs and investigation.

@tsnguyenducphuong
Copy link

In my case, I had first the blank screen issue then the black screen and solved with the following steps: (I am using Vision Camera version 4.2.1)

  1. minSdkVersion = 26 (solved blank issue).

Solve black screen:

  1. Set pixelFormat
  2. Set outputOrientation = "device"
  3. Set photoQualityBalance = "speed"
@wcandillon
Copy link

@mrousavy it's interesting because in the video use-case the error is clean and concise, I assume it would be the same for the vision-camera integration? Could it be because of an older version of Skia (I did improve this recently I think)

@mrousavy
Copy link
Owner

True, maybe an old RN Skia version! I'll do more research thanks

@mfkrause
Copy link

mfkrause commented Jun 14, 2024

I'm experiencing the same bug, but on iOS. When a Skia frame processor is connected, the camera stays black as long as the iPhone is in portrait. Some observations:

When I rotate the iPhone to landscape (which our UI doesn't support), the camera appears, but is rotated by 90 degrees. On iPad, where we support both portrait and landscape mode for the UI, everything works as expected, though.

When I set outputOrientation to preview, the camera frame stays visible, but stays locked in its 90 degrees rotated position (no matter whether or not the device's screen orientation lock is enabled). Setting it to portrait makes the camera preview stay black permanently (in every device orientation).

I'm not sure if that's expected behavior for apps that don't support landscape mode, but I've also noticed onPreviewOrientationChanged never sends any events and stays locked on portrait.
onOutputOrientation fires events with an orientation that's rotated 90 degrees, too. For example, if the device is rotated to portrait mode, this fires an event with landscape-left. Rotating it to portrait upside down gives landscape-right. This issue might possibly be related to #2984.

From within the frame processor, logging frame.orientation leads to landscape-left when the device is actually in portrait; portrait when the device is actually in landscape left or portrait upside down; and portrait-upside-down when the device is in landscape right. When outputOrientation is set to preview, this will always log portrait-upside-down.

All in all, there seems to be something going wrong with detecting the device's orientation, leading to wrong transforms on the Skia frame.

I just rebuilt the app with the newest react-native-skia release (1.3.4) which didn't fix it, unfortunately.

Edit: My iPhone runs on iOS 17 while my iPad still runs on iOS 16. So, if this is related to #2972, this might be why it works fine on the iPad.

@mfkrause
Copy link

mfkrause commented Jun 28, 2024

After I digged a little into the different orientation values provided by the library, I realized most of my previous comment is actually the expected behavior and the problem must indeed lie in the frame rotation in useSkiaFrameProcessor.ts. I played around a little and came up with this:

switch (frame.orientation) {
  case 'portrait':
  case 'portrait-upside-down':
    canvas.scale(frame.height / frame.width, frame.height / frame.width)
    canvas.rotate(90,0,0)
    canvas.translate(0, -(frame.height * 2))
    break
  case 'landscape-left':
  case 'landscape-right':
    canvas.translate(frame.height, 0)
    canvas.rotate(90, 0, 0)
    break
  default:
    throw new Error(`Invalid frame.orientation: ${frame.orientation}!`)
}

This seems to work great, however, two caveats:

  1. There's a minor glitch when rotating the device from portrait to either landscape direction, where the frame visibly "jumps" by a few pixels. I wasn't able to reliably fix that unfortunately (it's 11pm and the matrix translation stuff isn't quite getting in my head right now). I suppose the scale() values aren't quite right yet, as the frame still seems to extend beyond the device borders when the device is in landscape.
  2. This doesn't work when the UI supports landscape mode and orientation lock is disabled. When rotating to landscape, the orientation is wrong again. The original / current rotation code seems to apply the correct rotation in most scenarios in this case, however 1. the image quality in landscape mode is incredibly bad and 2. when the device is portrait upside down, there's still a black screen.

So, this isn't even close to a full fix or PR ready yet, just some observations.

@WillyGV
Copy link

WillyGV commented Jul 15, 2024

Hello, I have the same problem, and you can solve it by changing the logic of the landscape-left case:

diff --git a/package/src/skia/useSkiaFrameProcessor.ts b/package/src/skia/useSkiaFrameProcessor.ts
index 10888109..16b667a2 100644
--- a/package/src/skia/useSkiaFrameProcessor.ts
+++ b/package/src/skia/useSkiaFrameProcessor.ts
@@ -95,8 +95,8 @@ function withRotatedFrame(frame: Frame, canvas: SkCanvas, previewOrientation: Or
         break
       case 'landscape-left':
         // rotate two flips on (0,0) origin and move X + Y into view again
-        canvas.translate(frame.height, frame.width)
-        canvas.translate(270, 0)
+        canvas.translate(0, frame.width)
+        canvas.rotate(270, 0)
         break
       case 'portrait-upside-down':
         // rotate three flips on (0,0) origin and move Y into view again

but I run into the problem that the camera creates a mirror effect, and I can't solve it.
I already tried to add the attribute to the

<camera
isMirrored={true}
/>

but the image cannot be reversed

@mrousavy
Copy link
Owner

but I run into the problem that the camera creates a mirror effect, and I can't solve it. I already tried to add the attribute to the

<camera
isMirrored={true}
/>

but the image cannot be reversed

You want to disable mirroring, but pass isMirrored={true}? If you want to disable it you need to set isMirroring={false}...

@WillyGV
Copy link

WillyGV commented Jul 17, 2024

I'm sorry @mrousavy.
What I wanted to say is that the mirror effect happens to me in the preview, I saw that in the code and in the documentation that isMirrored does not work for the preview, also I am only working on Android for now and in the Google documentation, the Preview class, Does not support SetMirrorMode for android versions prior to API 32,
I think the problem is in Skia's canvas.

I tried isMirrored={false} and isMirrored={true} in the camera component, I haven't found out if it can be changed in the preview

@mrousavy
Copy link
Owner

Ah, so the Preview is not mirrored with a Skia Frame Processor, but is properly mirrored without a Skia FP - is that what you're saying?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
🐛 bug Something isn't working
7 participants