Integration & Pipeline

The dvrk_display bridges hardware performance with ROS interoperability. The core executable (main.cpp) constructs custom GStreamer topologies from its designated JSON schema sequentially.

Hardware Acceleration

Stereo hardware sources (left_stream, right_stream) bypass standard memory copying entirely whenever possible.

By passing variables like preserve_size and exact coordinate subsets directly down to standard elements (videocrop, videoscale), the pipeline resolves spatial padding locally before issuing payloads up to complex multi-sinks (e.g. glimagesink, OpenGL texture mappers, or generic appsink drop zones).

ROS Sub-Streaming Strategy

Unlike conventional nodes running single pipelines, the dvrk_display creates dynamic tee junctions mid-stream. This allows hardware-dependent low latency endpoints to pull directly from the video layer natively while simultaneously generating format-converted streams to publish strictly over traditional ROS topologies.

It is highly recommended to rely on native stream capabilities; the ROS image topics generated by this node are provided mostly for backward compatibility and ad-hoc legacy vision processing.

Depending on the targeted JSON array (ros_image_publishers), users can emit:

  • Distinct individual topics (left/image_raw and right/image_raw). Note that the viewer does not provide strict synchronization guarantees across these discrete branches.

  • Consolidated side-by-side matrices (stereo) for broad machine-vision operations.

Overlay HUD

The viewer renders a real-time heads-up display (HUD) directly onto the stereo video feed using Cairo. The overlay subscribes to the dVRK System Node ROS topics (via the dvrk_console_namespace) and reflects the current operational state. The overlay opacity is controlled by the overlay_alpha configuration field.

The following indicators are displayed:

Foot Pedals (bottom center)

Two small status circles appear at the bottom center of each eye. From left to right they represent:

  • Clutch: Filled when the clutch foot pedal is actively pressed.

  • Camera: Filled when the camera foot pedal is actively pressed.

PSM Teleop Indicators (bottom corners)

Numbered circles showing which PSMs are currently teleoperated:

  • Left column (bottom-left corner): PSMs controlled by the MTML (left master).

  • Right column (bottom-right corner): PSMs controlled by the MTMR (right master).

  • The circle displays the PSM number (e.g., 1, 2, 3).

  • Filled when the teleop following state is active (the surgeon is controlling the arm).

  • Outlined when selected but not currently following.

  • Red when the PSM’s measured_cp reports an invalid pose (zero timestamp).

  • The tool type label (e.g., “Large Needle Driver”) is displayed next to each PSM indicator.

Camera Teleop (top center)

A camera icon appears at the top center when camera (ECM) teleoperation is active:

  • Filled when the ECM following state is active.

  • Outlined when the ECM teleop is selected but not following.