A method includes generating, by a first object detection model of a perception system of an autonomous vehicle and based on a first portion of sensor data describing a first portion of an environment of the autonomous vehicle, a first object detection output that comprises a first predicted class for an object; generating, by a second object detection model of the perception system and based on a second portion of sensor data describing a second portion of the environment, a second object detection output that comprises a predicted distribution over candidate classes for the object; adapting at least a portion of the first object detection output into an adapted representation of the first object detection output; providing the adapted representation in a shared prediction output space with the predicted distribution from the second object detection output; and generating a resolved object detection output based on the adapted representation and the predicted distribution.