Various implementations of the systems, methods, and electronic devices described herein estimate three-dimensional (3D) size and position of objects. An example process may include obtaining sensor data for a sequence of frames, determining a representation of an object, and determining 3D positions and size estimates. For each frame that includes image data and depth data, the 3D position is determined based on the device pose data and the depth data for each corresponding frame, and the size estimate is determined based on the image data and the depth data. For each frame of the sequence of frames that includes image data and excludes depth data, the size estimate is determined based on a prior size estimate from a prior frame, and the 3D position is determined based on the size estimate of the representation of the object and a portion of the image data.