Using the API

New types and functions

The Ouster Perception API introduces the following types and functions:

  • Object — a generic object, represented by a bounding box, object_to_body and body_to_world poses, velocity, classification, a timestamp, and user-definable set of properties represented by strings.

  • DetectionConfig — base configuration for detection engines. Subclasses such as ClassicDetectionConfig provide settings for the classic cluster-based engine.

  • DetectionEngine — a base class that declares virtual methods for modifying (e.g. by adding objects to) LidarFrame and FrameSet.

  • ClassMap — A mapping from integers to strings, meant to map Object::class_id to a string description.

  • ClassMapSet — A mapping from strings to ClassMap instances, allowing users to specify different descriptions for Object instances derived from different DetectionEngine.

  • FrameSetSourceMetadataSet — A container for instances of ClassMapSet and other metadata a user may want to associate with a FrameSetSource. Currently, because OSF is the only format that supports LidarFrame and FrameSet, the OsfFrameSetSource is the only implementation that supports metadata.

Using DetectionEngine

To use the API, call DetectionEngine.create to instantiate the engine, passing the SensorInfo list (even for a single sensor) and an optional ClassicDetectionConfig (defaults to ClassicDetectionConfig{}). Then call update method, providing either a LidarFrame or FrameSet by reference. The following example describes how to instantiate the default DetectionEngine and use it to add objects representing lidar point clusters to every frame in the first sensor stream in the provided source.

View on GitHub 
# Open a frame source
source = open_source(source_url)

# Create a detection engine
engine = perception.DetectionEngine.create(source.sensor_info)

# Iterate over the frame sets in the frame source
frame_counter = 0
for frame_set in source:
    frame = frame_set[0]

    # Apply the engine to the frame (or the whole frame set.)
    engine.update(frame)

    # Inspect the objects added by the engine.
    print(f'Frame: {frame_counter}')
    frame_counter += 1
    for key, objects in frame.objects.items():
        print(f'Key {key} has {len(objects)} objects.')
View on GitHub 
// Open a frame set source
auto source = open_source(source_url);

// Create a detection engine
auto engine = perception::DetectionEngine::create(source.sensor_info());

// Iterate over the frame sets in the frame set source
int frame_counter{0};
for (auto frame_set : source) {
    auto& frame = *frame_set[0];

    // Apply the engine to the frame (or the whole frame set.)
    engine->update(frame);

    // Inspect the objects added by the engine.
    std::cerr << "Frame: " << frame_counter << '\n';
    frame_counter++;
    for (const auto& kv : frame.objects()) {
        std::cerr << "Key " << kv.first << " has " << kv.second.size() << " objects.\n";
    }
}

Adding Object to frames without a DetectionEngine

While the DetectionEngine interface is the most straightforward way to add objects to frames, users can also add objects directly to a frame without using a DetectionEngine. This may be useful for users who want to add objects derived from an external source, such as an annotated dataset, or who want to implement their own detection engine without inheriting from the base class.

To add an Object to a frame, simply create an instance of the Object class and add it to the frame’s objects vector.For example, the following code snippet creates an Object with poses, velocity, and class ID, and adds it to a frame:

View on GitHub 
from ouster.sdk import core
import numpy as np
objects = [core.Object(), core.Object()]
objects[0].id = 1
objects[0].creation_ts = 99
objects[0].timestamp = 199
objects[0].class_id = 1
objects[0].class_confidence = 0.9
objects[0].object_to_body.position = np.array([1, 2, 3])
objects[0].object_to_body.set_rotation(np.array([2, 2, 2]))
objects[0].body_to_world.position = np.array([10, 20, 30])
objects[0].body_to_world.set_rotation(np.array([0.1, 0.2, 0.3]))
objects[0].velocity = np.array([2, 3, 4])
objects[0].dimensions = np.array([1, 1, 1])
objects[0].properties["num_points"] = '[100]'
objects[0].properties["attributes"] = '["eats_icecream", "carries_bag"]'
objects[1].id = 2
objects[1].creation_ts = 100
objects[1].timestamp = 200
objects[1].class_id = 2
objects[1].class_confidence = 0.8
objects[1].object_to_body.position = np.array([3, 2, 1])
objects[1].object_to_body.set_rotation(np.array([1, 1, 1]))
objects[1].body_to_world.position = np.array([4, 5, 6])
objects[1].body_to_world.set_rotation(np.array([0.4, 0.5, 0.6]))
objects[1].velocity = np.array([4, 3, 2])
objects[1].dimensions = np.array([2, 2, 2])
objects[1].properties["num_points"] = '[50]'
objects[1].properties["attributes"] = '["parked_illegaly"]'
frame.objects["test_objects"] = objects
View on GitHub 
using namespace ouster::sdk;

std::vector<core::Object> objects(2);
objects[0].id = 1;
objects[0].creation_ts = 99;
objects[0].timestamp = 199;
objects[0].class_id = 1;
objects[0].class_confidence = 0.9;
objects[0].object_to_body.set_position(Eigen::Vector3d{1, 2, 3});
objects[0].object_to_body.set_rotation(Eigen::Vector3d{2, 2, 2});
objects[0].body_to_world.set_position(Eigen::Vector3d{10, 20, 30});
objects[0].body_to_world.set_rotation(Eigen::Vector3d{0.1, 0.2, 0.3});
objects[0].velocity = core::UnalignedVector3f{2, 3, 4};
objects[0].dimensions = core::UnalignedVector3f{1, 1, 1};
objects[0].properties["num_points"] = "[100]";
objects[0].properties["attributes"] = "\[\"eats_icecream\", \"carries_bag\"]";
objects[1].id = 2;
objects[1].creation_ts = 100;
objects[1].timestamp = 200;
objects[1].class_id = 2;
objects[1].class_confidence = 0.8;
objects[1].object_to_body.set_position(Eigen::Vector3d{3, 2, 1});
objects[1].object_to_body.set_rotation(Eigen::Vector3d{1, 1, 1});
objects[1].body_to_world.set_position(Eigen::Vector3d{4, 5, 6});
objects[1].body_to_world.set_rotation(Eigen::Vector3d{0.4, 0.5, 0.6});
objects[1].velocity = core::UnalignedVector3f{4, 3, 2};
objects[1].dimensions = core::UnalignedVector3f{2, 2, 2};
objects[1].properties["num_points"] = "\"[50]\"";
objects[1].properties["attributes"] = "[\"parked_illegaly\"]";
frame.objects()["test_objects"] = objects;

Using an approach like the above, FrameSet can also store Object instances. Since OSF supports saving LidarFrame and FrameSet, reading and writing datasets with objects can be done with OsfFrameSetSource and Writer or AsyncWriter.

Reading and writing ClassMap

In Ouster SDK 1.0, no DetectionEngine implementation produces Object s with classifications, yet. However, ClassMap, which creates an association between Object::class_id and a string, can still be useful for Object s imported from an external source, like an annotated lidar dataset.

ClassMaps are stored in a FrameSetSourceMetadataSet and associated with a frame source via metadata keys (for example, "class_maps").

Writing an OSF with ClassMaps is straightforward, since the Writer and AsyncWriter support saving FrameSetSourceMetadataSet like so:

View on GitHub 
writer = osf.Writer(str(output_path), [sensor_info])

class_map1 = core.ClassMap({
    1: 'dog',
    2: 'cat'
})

class_map2 = core.ClassMap({
    1: 'tree',
    2: 'bush'
})

class_maps = core.ClassMapSet({
    'four_legs': class_map1,
    'zero_legs': class_map2
})

metadata = core.FrameSetSourceMetadataSet()
metadata['class_maps'] = class_maps
metadata['additional_info'] = "Test additional info"


writer.save(metadata)
View on GitHub 
osf::Writer writer(output_osf_filename, {sensor_info});

core::ClassMap class_map1;
class_map1.class_map.emplace(1,
                             "dog");
class_map1.class_map.emplace(2,
                             "cat");
core::ClassMap class_map2;
class_map2.class_map.emplace(1,
                             "tree");
class_map2.class_map.emplace(2,
                             "bush");
core::ClassMapSet class_maps;
class_maps.class_maps.emplace("four_legs",
                              class_map1);
class_maps.class_maps.emplace("zero_legs",
                              class_map2);
core::FrameSetSourceMetadataSet frame_set_source_metadata_set;
frame_set_source_metadata_set.entries.emplace("class_maps", class_maps);
std::string additional_info = "Test additional info";
frame_set_source_metadata_set.entries.emplace("additional_info", additional_info);

writer.save(frame_set_source_metadata_set);

Reading a ClassMap from a FrameSetSource is as simple as accessing it using its string key from the frame set source metadata.

View on GitHub 
from ouster.sdk import open_source
src = open_source(str(osf_path))
# Unlike C++, no .is<>) check is needed: metadata() returns a
# ClassMapSet directly (the bindings unwrap the stored type for you).
class_map_set = src.metadata('class_maps')
dog_class = class_map_set['four_legs'][1]
View on GitHub 
using namespace ouster::sdk;
auto src = open_source(osf_path);
// In C++, FrameSetSource metadata uses type erasure to allow storing
// different types of metadata So the type must be checked and the value
// casted to the appropriate type.
if (!src.metadata("class_maps").is<core::ClassMapSet>()) {
    throw std::runtime_error("Unexpected metadata type!");
}
auto class_map_set = src.metadata("class_maps").as<core::ClassMapSet>();
auto dog_class = class_map_set.class_maps["four_legs"].class_map[1];