fCal temporal calibration requires BRIGHTNESS but OpenCVVideo provides RGB_COLOR

[patigarciaa]

I am trying to perform temporal calibration using fCal with:

• Tracker: NDI Polaris Vega VT
• Ultrasound: Philips Lumify
• Video streamed from tablet using ScreenStream (MJPEG)

The video is received in PlusToolkit using OpenCVVideo.

The problem is that when I try temporal calibration, fCal reports:

expected image type is BRIGHTNESS, actual type is RGB_COLOR

I tried:

• setting ImageType="BRIGHTNESS"
• converting the stream to grayscale using ffmpeg
• replaying the video from file using OpenCVVideo

However fCal still detects the input as RGB_COLOR.

The video is grayscale visually but internally stored as 3-channel RGB.

Is there a way to force Plus/fCal to convert the input to BRIGHTNESS before temporal calibration?

PlusToolkit version: PlusApp-2.8.0.20250221-Win64

Operating system: Windows

Video source pipeline: Lumify → tablet → ScreenStream → MJPEG → OpenCVVideo → fCal

Configuration file: xml

option 1:

<DeviceSet Name="fCal_DIRECT"
           Description="da problema de brightness"/>

<!-- POLARIS -->
<Device
  Id="TrackerDevice"
  Type="PolarisTracker"
  NetworkHostname="169.254.6.28"
  NetworkPort="8765"
  ToolReferenceFrame="Tracker">
  <DataSources>
    <DataSource Type="Tool" Id="Reference" RomFile="C:/miau/8700339.rom"/>
  </DataSources>
  <OutputChannels>
    <OutputChannel Id="TrackerStream">
      <DataSource Id="Reference"/>
    </OutputChannel>
  </OutputChannels>
</Device>

<!-- MJPEG DIRECTO -->
<Device
  Id="VideoDevice"
  Type="OpenCVVideo"
  RequestedCaptureApi="CAP_FFMPEG"
  AcquisitionRate="25"
  VideoURL="http://192.168.0.242:8080/stream.mjpeg">
  <DataSources>
    <DataSource Type="Video" Id="Video" PortUsImageOrientation="MF" ImageType="BRIGHTNESS"/>
  </DataSources>
  <OutputChannels>
    <OutputChannel Id="VideoStream" VideoDataSourceId="Video"/>
  </OutputChannels>
</Device>

<!-- MIXER -->
<Device Id="TrackedVideoDevice" Type="VirtualMixer">
  <InputChannels>
    <InputChannel Id="TrackerStream"/>
    <InputChannel Id="VideoStream"/>
  </InputChannels>
  <OutputChannels>
    <OutputChannel Id="TrackedVideoStream"/>
  </OutputChannels>
</Device>

fcal_directo.xml

<DeviceSet
  Name="fCal_AVI_OpenCV"
  Description="AVI grayscale + Polaris tracker"/>

<!-- POLARIS -->
<Device
  Id="TrackerDevice"
  Type="PolarisTracker"
  NetworkHostname="169.254.6.28"
  NetworkPort="8765"
  ToolReferenceFrame="Tracker">
  <DataSources>
    <DataSource
      Type="Tool"
      Id="Reference"
      RomFile="C:/miau/8700339.rom"/>
  </DataSources>
  <OutputChannels>
    <OutputChannel Id="TrackerStream">
      <DataSource Id="Reference"/>
    </OutputChannel>
  </OutputChannels>
</Device>

<!-- VIDEO DESDE ARCHIVO AVI -->
<Device
  Id="VideoDevice"
  Type="OpenCVVideo"
  RequestedCaptureApi="CAP_FFMPEG"
  AcquisitionRate="25"
  VideoURL="C:/miau/lumify_gray_real.avi">
  <DataSources>
    <DataSource
      Type="Video"
      Id="Video"
      PortUsImageOrientation="MF"
      ImageType="BRIGHTNESS"/>
  </DataSources>
  <OutputChannels>
    <OutputChannel
      Id="VideoStream"
      VideoDataSourceId="Video"/>
  </OutputChannels>
</Device>

<!-- MIXER -->
<Device Id="TrackedVideoDevice" Type="VirtualMixer">
  <InputChannels>
    <InputChannel Id="TrackerStream"/>
    <InputChannel Id="VideoStream"/>
  </InputChannels>
  <OutputChannels>
    <OutputChannel Id="TrackedVideoStream"/>
  </OutputChannels>
</Device>

in both cases i get the error: |ERROR|010.151000| vtkPlusLineSegmentationAlgo video input data verification failed: video data orientation or type is not supported (MF orientation, BRIGHTNESS type is expected)| in E:\D\PSNP64b\PlusLib\src\PlusCalibration\vtkLineSegmentationAlgo\vtkPlusLineSegmentationAlgo.cxx(132)

Full error message:
|ERROR| vtkPlusTrackedFrameList::VerifyProperties failed: expected image type is BRIGHTNESS, actual type is RGB_COLOR
|ERROR| vtkPlusLineSegmentationAlgo video input data verification failed: video data orientation or type is not supported (MF orientation, BRIGHTNESS type is expected)