第三方驅動程式、程式庫與範例#

本頁面提供有關搭配使用 Basler Stereo mini 相機的第三方程式庫與範例程式碼的相關資訊。

NumPy#

Python 範例中使用 NumPy 來處理 pypylon 所傳回的原始元件資料。

pip install numpy

OpenCV#

OpenCV 可用於處理從 Stereo mini 相機擷取的強度影像。強度元件提供 RGBA8 影像，該影像可轉換為適合 OpenCV 處理的格式。

pip install opencv-python

範例：將強度元件轉換為 OpenCV 影像

import cv2
import numpy as np
from pypylon import pylon

# ... camera setup (see Python Programmer's Guide) ...

data_container = grab_result.GetDataContainer()
for i in range(data_container.DataComponentCount):
    component = data_container.GetDataComponentByIndex(i)
    try:
        if component.ComponentType == pylon.ComponentType_Intensity:
            raw = component.GetData()
            h, w = component.Height, component.Width
            # Stereo mini intensity is RGBA8
            rgba = np.frombuffer(raw, dtype=np.uint8).reshape(h, w, 4)
            bgr = cv2.cvtColor(rgba, cv2.COLOR_RGBA2BGR)
            cv2.imshow("Intensity", bgr)
            cv2.waitKey(1)
    finally:
        component.Release()

如需詳細資訊，請參閱 OpenCV。

Open3D#

Open3D 用於 ShowPointCloud 範例中，以將來自 Stereo mini 相機的 3D 點雲視覺化。其中的範圍元件 Coord3D_ABC32f 格式提供直接的 XYZ 座標。

pip install open3d

範例：從範圍與強度資料建立 Open3D 點雲

import numpy as np
import open3d as o3d
from pypylon import pylon

# ... camera setup with ComponentSelector Range, PixelFormat Coord3D_ABC32f ...

data_container = grab_result.GetDataContainer()
xyz = None
colors = None

for i in range(data_container.DataComponentCount):
    component = data_container.GetDataComponentByIndex(i)
    try:
        if component.ComponentType == pylon.ComponentType_Range:
            h, w = component.Height, component.Width
            raw = component.GetData()
            pts = np.frombuffer(raw, dtype=np.float32).reshape(h * w, 3)
            valid = np.isfinite(pts).all(axis=1)
            xyz = pts[valid]
        elif component.ComponentType == pylon.ComponentType_Intensity:
            h, w = component.Height, component.Width
            raw = component.GetData()
            rgba = np.frombuffer(raw, dtype=np.uint8).reshape(h * w, 4)
            colors = rgba[..., :3].astype(np.float64) / 255.0
    finally:
        component.Release()

if xyz is not None and colors is not None:
    pcd = o3d.geometry.PointCloud()
    pcd.points = o3d.utility.Vector3dVector(xyz)
    pcd.colors = o3d.utility.Vector3dVector(colors[:len(xyz)])
    o3d.visualization.draw_geometries([pcd])

如需詳細資訊，請參閱 Open3D。

Point Cloud Library (PCL)#

PCL 可用於進階的 3D 點雲處理。來自 Stereo mini 相機且為 Coord3D_ABC32f 格式的範圍資料可饋入 PCL 資料結構中。

LinuxWindows

sudo apt-get install libpcl-dev

Download from PCL website.

如需詳細資訊，請參閱 C++ Programmer's Guide。

ROS 2#

Stereo mini 相機可透過使用 pypylon 作為橋接器並發布 sensor_msgs/Image 主題，來整合至 ROS 2 中。

必要的套件：

pip install pypylon numpy

sudo apt install ros-humble-rclpy ros-humble-sensor-msgs

在此方法中，影像資料是從 Stereo mini 資料元件 (Intensity, Range) 中讀取，並發布至 ROS 主題，例如：

/stereo_mini/intensity_rgba8 (編碼 rgba8)
/stereo_mini/range_c16_mm (編碼 mono16)

最小橋接器範例：

import numpy as np
import rclpy
from rclpy.node import Node
from sensor_msgs.msg import Image
from pypylon import pylon


class StereoMiniBridge(Node):
    def __init__(self):
        super().__init__("stereo_mini_bridge")
        self.pub_intensity = self.create_publisher(Image, "/stereo_mini/intensity_rgba8", 10)

        di = pylon.DeviceInfo()
        di.SetDeviceClass("BaslerGTC/Basler/TLModel_keep")
        self.camera = pylon.InstantCamera(pylon.TlFactory.GetInstance().CreateFirstDevice(di))
        self.camera.Open()

        self.camera.ComponentSelector.Value = "Intensity"
        self.camera.ComponentEnable.Value = True
        try:
            self.camera.PixelFormat.Value = "RGBA8"
        except Exception:
            self.camera.PixelFormat.Value = "RGBA8packed"

        self.camera.StartGrabbing()
        self.timer = self.create_timer(0.03, self.tick)

    def tick(self):
        grab = self.camera.RetrieveResult(3000, pylon.TimeoutHandling_ThrowException)
        try:
            if not grab.GrabSucceeded():
                return
            dc = grab.GetDataContainer()
            for i in range(dc.DataComponentCount):
                c = dc.GetDataComponentByIndex(i)
                try:
                    if c.ComponentType == pylon.ComponentType_Intensity:
                        h, w = c.Height, c.Width
                        rgba = np.frombuffer(c.GetData(), dtype=np.uint8).reshape(h, w, 4)
                        msg = Image()
                        msg.header.stamp = self.get_clock().now().to_msg()
                        msg.header.frame_id = "stereo_mini"
                        msg.height = h
                        msg.width = w
                        msg.encoding = "rgba8"
                        msg.is_bigendian = 0
                        msg.step = w * 4
                        msg.data = rgba.tobytes()
                        self.pub_intensity.publish(msg)
                finally:
                    c.Release()
        finally:
            grab.Release()


def main():
    rclpy.init()
    node = StereoMiniBridge()
    rclpy.spin(node)


if __name__ == "__main__":
    main()