Using the API¶
We already have covered how Ouster SDK provides APIs to record data from live sensors and to read from data files in the previous section. In this section, we will demonstrate how to write the data that was read into various file formats using the API.
Recording Sensor Data to OSF¶
OSF (Open Sensor Format) files provide a more structured approach to data storage, including sensor info and frame-level organization. It is the recommended format used to store Ouster sensor data.
Instead of saving raw packets, one can batch packets from a live sensor into LidarFrame objects first, and then saves those LidarFrame objects to an .osf file. This is useful if you want to store processed, ready-to-use frames.
An API for writing to the OSF file format is also exposed. This is most often used for writing frames and sensor info, possibly with a reduced number of fields in order to save data.
Let’s walkthough an example of recording data from a live sensor and storing it into an OSF file.
Imports
# Other imports
from ouster.sdk import open_source
import ouster.sdk.osf as osf
#include "ouster/osf/osf_frame_set_source.h"
#include "ouster/osf/writer.h"
using namespace ouster::sdk;
Setup
source = open_source(hostname)
info = source.sensor_info[0]
auto source = open_source(hostname);
const auto& infos = source.sensor_info()[0];
Write to OSF file
A general scheme of writing frames to the OSF with Writer:
Create
osf.Writerwith the output file name, lidar metadata(s) (ouster.sdk.core.SensorInfo) and optionally the desired output frame fields.Use the writer’s
savefunctionwriter.save(index, frame)to encode theLidarFrameinto the underlying message buffer for lidarindexand finally push it to disk. If you have multiple lidar sensors you can save the frames simultaneously by providing them in an array towriter.save.
writer = osf.Writer(output_file, info)
written = 0
for frame_set in source:
for idx, frame in enumerate(frame_set):
if frame is None: continue # noqa
writer.save(idx, frame)
written += 1
if written >= n_frames: break # noqa
osf::Writer writer(output_file, *infos);
int written = 0;
for (const auto& frame_set : source) {
for (size_t idx = 0; idx < frame_set.size(); ++idx) {
const auto& frame = frame_set[idx];
if (!frame) { continue; }
writer.save(static_cast<int>(idx), *frame); }
if (++written >= n_frames) { break; }}
Modifying an OSF File¶
Next, we will look at an example where osf.Writer is used for saving the available OSF file into Lidar
Frames with a reduced fields. By reduce fields, we mean here that if LidarFrame has 7 channel
fields, we can keep only 3 and save the disk space and bandwidth during replay.
New field types should be a subset of fields in encoded LidarFrame so we just assume that RANGE, SIGNAL and REFLECTIVITY fields will be present in the input OSF file.
Create Writer with a subset of fields to save (i.e. slicing will happen automatically on write).
frames = osf.OsfFrameSetSource(osf_file)
fields_to_write = [
core.ChanField.RANGE,
core.ChanField.SIGNAL,
core.ChanField.REFLECTIVITY]
writer = osf.Writer(output_file,
frames.sensor_info,
fields_to_write)
# Read frames and write back
for frame_set in frames:
for idx, frame in enumerate(frame_set):
if frame is None:
continue
print(f"writing sliced frame with ts = \
{frame.get_first_valid_packet_timestamp()}")
writer.save(idx, frame)
writer.close()
osf::OsfFrameSetSource source(osf_file);
std::vector<std::string> fields_to_write = {
"RANGE",
"SIGNAL",
"REFLECTIVITY"};
osf::Writer writer(make_sliced_fname(osf_file),
dereference_infos(source.sensor_info()),
fields_to_write);
// Read frames and write back
for (const auto& frame_set : source) {
for (size_t idx = 0; idx < frame_set.size(); ++idx) {
const auto& frame = frame_set[idx];
if (!frame) { continue; }
std::cout << "writing sliced frame with ts = "
<< frame->get_first_valid_packet_timestamp() << std::endl;
writer.save(static_cast<int>(idx), *frame); } }
writer.close();
Creating an OSF File with custom LidarFrame¶
Below you can see an example which creates a frame and writes it to an OSF File using the Writer API:
Imports
import ouster.sdk.core as core
import ouster.sdk.osf as osf
#include "ouster/osf/writer.h"
using namespace ouster::sdk;
Create LidarFrame and save
The LidarFrame constructor has an overload taking — width (columns per frame), and height (pixels per column). The number of columns per packet is set to the default DEFAULT_COLUMNS_PER_PACKET which is 16.
# Create sensor info (single sensor) and writer
info = core.SensorInfo.from_default(core.LidarMode._512x10)
writer = osf.Writer(osf_file, info)
# Allocate an empty frame that matches the metadata
frame = core.LidarFrame(info)
# Manipulate the frame here if desired
# Write the frame to stream 0
writer.save(0, frame)
// Create default sensor info for a 512x10 sensor
auto info = core::SensorInfo::from_default(core::LidarMode::_512x10);
osf::Writer writer(osf_file, *info);
// Instantiate a lidar frame that matches the SensorInfo dimensions
auto frame = core::LidarFrame(info);
// Manipulate the frame as desired here
add_example_objects(frame);
// Write it to file on stream 0
writer.save(0, frame);
Playback¶
Both PCAP and OSF files can be played back using the open_source helper, which provides a FrameSetSource for iterating through FrameSet objects.
# paths = ["capture.osf", "capture2.osf"]
source = sdk.open_source(list(paths), collate=True)
// auto paths = {"capture.pcap", "capture2.pcap"};
auto source = open_source(paths);
std::size_t count = 0;
for (const auto& frame_set : source) {
for (std::size_t idx = 0; idx < frame_set.size();++idx) {
const auto& frame = frame_set[idx];
if (frame) {
std::cout << "frame: " << count
<< " id=" << frame->frame_id << "\n";
} else {
continue;
}
}
if (++count >= limit) {
break;
}
}
For a detailed guide on reading and iterating over LidarFrame objects from all source types, please see the previous section.
Converting PCAPs to Other Formats¶
Ouster SDK provides several examples demonstrating how to convert PCAP files to various formats such as CSV, LAS, and PCD. These examples can be found in the examples.