open_source & open_packet_source¶
Note
SDK 1.0 introduces breaking changes to the data consumption APIs described here; see Migration from 0.16.2 to 1.0.0 when upgrading from 0.16.x.
The Ouster SDK provides two primary helper functions for data consumption, open_source and open_packet_source. The one you choose depends on your goal:
To work with complete frames (most common): Use
open_sourceto get a FrameSetSource that yields FrameSet objects.To work with raw packets (advanced): Use
open_packet_sourceto get a PacketSource that yields individual LidarPacket and ImuPacket objects.
Reading Frames from a Live Sensor¶
The FrameSetSource API uses the method name open_source which provides a uniform entry point for users to handle different source types
using the same API. Currently, the supported source types are live sensor, pcap file, or osf file. The Python API also supports rosbag files.
It detects the IO type from the string, calls the appropriate constructor under the hood and then applies extra options.
For example, to start consuming packets from a single live sensor, you can call open_source with the hostname of the sensor:
Imports
# Other imports
from ouster.sdk import open_source
#include "ouster/core/open_source.h"
using namespace ouster::sdk;
Instantiate open_source
source = open_source(hostname)
auto source = open_source(hostname);
Or to replay from a valid input file source file such as pcap or osf file, you can call open_source with the file path:
source = open_source(data_path)
auto source = open_source(data_path);
Load Metadata for Pcap Sources¶
PCAP (Packet Capture) files are the most common format for recording raw UDP packets from Ouster sensors. They preserve the original network packets exactly as received.
The source object returned by open_source provides access to LidarFrame objects,
regardless of whether the source data comes from a sensor, pcap, or osf file.
For pcap sources, if the metadata file is not in the same folder as the pcap and doesn’t have a shared name prefix the above method will fail.
To load and parse the metadata ourselves we only need to pass the metadata to the method through the optional meta parameter
and the method will take care of loading it and associating it with the
source object.
source = open_source(pcap_path,
meta=[sensor_info_path])
auto source = open_source(pcap_path, [&](FrameSetSourceOptions& opt) {
opt.meta = { sensor_info_path }; });
open_source Parameters¶
One can process data from single or multiple sensors by passing a hostname, file path, or a list of sources to the SDK. The output behavior is primarily controlled by the collate and sensor_idx parameters. The function signature is as follows:
def open_source(
source_url: Union[str, List[str]],
collate: bool = True,
sensor_idx: int = -1,
*args,
**kwargs) -> FrameSetSource: ...
core::AnyFrameSetSource open_source(
const std::string& source,
const std::function<void(FrameSetSourceOptions&)>& options = {},
bool collate = true,
int sensor_idx = -1
);
- collate (default: True):
True: Yields one synchronizedFrameSetper iteration with one entry per sensor — a “larger”FrameSetthat time-aligns multiple sensors’ frames together.False: Yields individualFrameSetobjects as soon as they’re available (only recommended for single-sensor use). On multi-sensor sources still yieldsFrameSetobjects, but they arrive one per sensor with no time alignment.
- sensor_idx (default: -1):
-1: Keeps every sensor stream in the source, allowing collation.>= 0: Returns only the sensor with the specified index; thecollateflag is ignored.
The output is always a FrameSet, which acts as a container. The parameters determine how many frames this set contains and how they are synchronized.
Parameter |
Type |
Default |
Purpose |
|---|---|---|---|
|
|
required |
Sensor hostname(s) or file path; all entries must be the same IO type. |
|
|
unset |
FrameSetSourceOptions available are listed at FrameSetSourceOptions Parameters. |
|
|
|
Returns synchronized |
|
|
|
|
Example for Single Sensor Source:
# Shortcut: select sensor 0 while opening. Note: sensor_idx >= 0 makes
# open_source ignore collate entirely, so its value here has no effect.
via_open_source = open_source(source_url, collate=False, sensor_idx=0)
// Shortcut: select sensor 0 while opening. Note: sensor_idx >= 0 makes
// open_source ignore collate entirely, so its value here has no effect.
auto via_open_source = open_source(source_url, {}, /*collate=*/false, /*sensor_idx=*/0);
Example for Multi Sensor Source:
# Replace with actual hostnames
# hostnames = ["os-xxxx.local", "os-yyyy.local"]
source = sdk.open_source(list(hosts), collate=True)
with source:
for count, frame_set in enumerate(source):
for idx, _ in enumerate(frame_set):
frame = frame_set[idx]
if frame:
print(f"frame:{count}" f"frame_id={frame.frame_id}")
if count + 1 >= limit:
break
// Replace with actual hostnames
// auto hostnames = {"os-xxxx.local", "os-yyyy.local"};
auto source = open_source(hostnames);
Note
The PCAP and OSF builders allows opening only one file at a time so open_source cannot be used for multiple file inputs such as the code shown below.
# paths = ["capture.osf", "capture2.osf"]
source = sdk.open_source(list(paths), collate=True)
// auto paths = {"capture.pcap", "capture2.pcap"};
auto source = open_source(paths);
std::size_t count = 0;
for (const auto& frame_set : source) {
for (std::size_t idx = 0; idx < frame_set.size();++idx) {
const auto& frame = frame_set[idx];
if (frame) {
std::cout << "frame: " << count
<< " id=" << frame->frame_id << "\n";
} else {
continue;
}
}
if (++count >= limit) {
break;
}
}
FrameSetSourceOptions Parameters¶
Option |
Type |
Default |
Applies to |
Purpose |
|---|---|---|---|---|
|
|
unset |
PCAP, OSF, live |
Load per-sensor extrinsics from JSON (matched by serial number). |
|
list of 4×4 matrices |
unset |
All |
Override file/default extrinsics with explicit transforms. |
|
|
unset |
All |
Restrict decoded channels ( |
|
|
|
All |
Accept packets/frames whose serial/init ID differs from metadata. |
|
|
|
File sources |
Force building an index before streaming. |
|
|
unset |
PCAP, OSF |
Provide explicit metadata files when auto-discovery fails. |
|
|
unset |
Live sensors |
Override UDP ports when connecting to hardware. |
|
|
|
Live sensors |
Leave existing sensor settings untouched on connect. |
|
|
|
Live sensors |
Prevent the SDK from rewriting the sensor’s UDP destination. |
|
|
|
All (iterators) |
Per-sensor wait time for frames when iterating. |
|
|
|
Live sensors |
HTTP/configuration timeout while staging sensor settings. |
|
|
|
Live sensors |
Cap buffered frames to avoid backlog growth. |
|
|
unset |
Live sensors |
Supply metadata without querying devices. |
|
|
|
All |
Include raw per-column headers or raw pixel words in each |
|
|
unset |
Live sensors |
Apply staged configuration before streaming. |
|
|
|
All |
Override SDK error-reporting callbacks. |
Reading raw packet data¶
While most applications should use a FrameSetSource to work with complete LidarFrame objects, the SDK also provides a lower-level API for working directly with raw packets.
The open_packet_source function is a high-level helper designed to automatically detect the source type and return the correct PacketSource implementation.
A PacketSource is an iterator that yields raw, individual LidarPacket and ImuPacket objects as they are received from a sensor or read from a file.
This is primarily useful for tasks if your application requires access to raw packet data before it’s batched into frames such as
recording data, network replay, or custom, low-level processing.
Valid inputs for open_packet_source include:
Live sensor hostnames/IPs
.pcap files
.bag files (Python only)
An example of using open_packet_source to read packets from a live sensor is shown below.:
Imports
from ouster.sdk import open_packet_source
from ouster.sdk import sensor
from contextlib import closing
#include "ouster/pcap/pcap_packet_source.h"
#include "ouster/sensor/sensor_frame_set_source.h"
using namespace ouster::sdk;
Read packet data
with closing(open_packet_source(hostname,
lidar_port=lidar_port,
imu_port=imu_port,
buffer_time_sec=1.0)) as packet_source:
auto packet_source = open_packet_source(source, [&](auto& options) {
options.lidar_port = lidar_port;
options.imu_port = imu_port;
options.buffer_time_sec = 1.0F; });
Frame Iteration¶
The FrameSetSource examines the timestamp of every frame from every sensor and returns a list of frames
that fit within the same time window as a single FrameSet. The number of slots in the FrameSet is fixed corresponding
to how many sensors are contained in the pcap or OSF file (one entry per sensor).
However, the collation could yield a null value if one or more of the sensors didn’t produce a LidarFrame
object that fits within the time frame of the current FrameSet or iteration.
Thus, depending on the operation at hand it is critical to check if we got a valid LidarFrame object
when examining the iteration output of a FrameSetSource.
To display the frame_id of LidarFrame objects that belong to the same FrameSet on the same line, the code needs to be updated to the following:
with source:
for count, frame_set in enumerate(source):
for idx, _ in enumerate(frame_set):
frame = frame_set[idx]
if frame:
print(f"frame:{count} frame_id={frame.frame_id}")
for (const auto& frame_set : source) {
for (std::size_t idx = 0; idx < frame_set.size();++idx) {
const auto& frame = frame_set[idx];
if (frame) {
std::cout << "frame: " << count
<< " id=" << frame->frame_id << "\n";
Selecting one sensor: sensor_idx vs single()¶
single(n) is the underlying operation that selects a single sensor; it returns a Singler that keeps only the
sensor at index n. single() must be called on an uncollated source, so open with collate=False first.
The sensor_idx parameter of open_source is simply a shortcut that opens the source uncollated and calls single(sensor_idx) for you, so these two are
equivalent:
# [doc-stag-single-opensource-nocollate]
# Shortcut: select sensor 0 while opening. Note: sensor_idx >= 0 makes
# open_source ignore collate entirely, so its value here has no effect.
via_open_source = open_source(source_url, collate=False, sensor_idx=0)
# [doc-etag-single-opensource-nocollate]
# Equivalent: open uncollated, then derive a single-sensor view.
via_single = open_source(source_url, collate=False).single(0)
// [doc-stag-single-opensource-nocollate]
// Shortcut: select sensor 0 while opening. Note: sensor_idx >= 0 makes
// open_source ignore collate entirely, so its value here has no effect.
auto via_open_source = open_source(source_url, {}, /*collate=*/false, /*sensor_idx=*/0);
// [doc-etag-single-opensource-nocollate]
// Equivalent: open uncollated, then derive a single-sensor view.
auto via_single = open_source(source_url, {}, /*collate=*/false).single(0);
For a source that contains only one sensor (such as a single live sensor), explicitly selecting sensor_idx=0 above
is equivalent to leaving sensor_idx at its default of -1: there is only one stream, so each iteration yields a
one-entry FrameSet either way. The difference only becomes observable with multi-sensor sources, where -1 keeps
all streams (collated) while sensor_idx >= 0 narrows the output to the single sensor at that index.
Either of these two forms yields an iterator over FrameSet objects, but since the source is now narrowed to one sensor, each
FrameSet holds only a single entry. Inside the loop, frame_set is that FrameSet: frame_set[0] grabs the
LidarFrame object, and printing frame_set[0].frame_id gives the frame number for that sensor.
# Both via_open_source and via_single hold live resources (sockets/file
# handles); python context manager will close both when the with block ends.
with via_open_source, via_single:
# Both are Singler-wrapped, single-sensor sources, so both narrow every
# FrameSet to one entry, accessible the same way: frame_set[0].
via_open_source_frame_set = next(iter(via_open_source))
via_single_frame_set = next(iter(via_single))
if len(via_open_source_frame_set) > 0 and via_open_source_frame_set[0]:
print(f"via sensor_idx: frame_id={via_open_source_frame_set[0].frame_id}")
if len(via_single_frame_set) > 0 and via_single_frame_set[0]:
print(f"via single(): frame_id={via_single_frame_set[0].frame_id}")
// Both are Singler-wrapped, single-sensor sources, so both narrow every
// FrameSet to one entry, accessible the same way: frame_set[0].
for (const auto& frame_set : via_open_source) {
if (frame_set.size() > 0 && frame_set[0]) {
std::cout << "via sensor_idx: frame_id=" << frame_set[0]->frame_id << "\n";
break;
}
}
for (const auto& frame_set : via_single) {
if (frame_set.size() > 0 && frame_set[0]) {
std::cout << "via single(): frame_id=" << frame_set[0]->frame_id << "\n";
break;
}
}
Calling single() on a collated source (the open_source default) raises an error because collation discards the
per-sensor stream boundaries — there is no way to recover them afterward. Selecting an index the source does not contain
(for example single(2) on a single-sensor source) also raises.
Closing a source¶
When you are finished with a source, you should close it to release its underlying resources. For a live sensor this frees the UDP sockets bound to the lidar and IMU ports; for pcap and osf replay it releases the open file handles. This behaves identically regardless of how you chain or single them.
Note
If you do not release a live source, the sensor’s UDP ports stay bound until the process exits. Attempting to reopen the same sensor before the port is released fails.
FrameSetSources in the SDK generally follow RAII: when they go out of scope or are otherwise destructed, they automatically
release their underlying resources. In C++ you can do this by just letting the object go out of scope when you are finished or
by doing something like storing the object in a smart pointer and resetting it. In Python you can do this by utilizing the source in
a context manager (with block). Both means will ensure the source is closed when you are done with it,
including if any exceptions are thrown.
In Python however, RAII based approaches do not always work so a close() method is also exposed. When this method is called the
resources from the source will immediately be released, making further interaction with the source prone to errors or failure.
It is recommended you follow up every close by setting the variable holding the source to None (or a new source) to ensure you do not interact with a closed source.
source = open_source(hostname, sensor_idx=0)
try:
frame_set = next(iter(source))
if len(frame_set) > 0 and frame_set[0]:
print(f"first frame_id: {frame_set[0].frame_id}")
finally:
# Release the underlying resources: the sensor's UDP sockets for a
# live source, or file handles for pcap/osf replay. This is the same
# regardless of how the source was opened.
source.close()
{
auto source = open_source(hostname, {}, /*collate=*/true, /*sensor_idx=*/0);
for (const auto& frame_set : source) {
if (frame_set.size() > 0 && frame_set[0]) {
std::cout << "first frame_id=" << frame_set[0]->frame_id << "\n";
break;
}
}
}
// Underlying resources (UDP sockets for live sources, file handles for
// replay) are released when source is destroyed at end of scope.