Systems and methods are provided for implementing a multi-stage, ML model training process for autonomous or semi-autonomous driving. The multi-stage ML model training process comprises (1) 2D and 3D supervised losses during a synthetic data ML model training, (2) 2D supervised on real-world data, and (3) 3D self-supervised losses on real-world data. The improved ML training process may not rely on 3D object recognition with real-world 3D labeled data. Once the ML model is trained, in some examples, the trained ML model can implement an inference process to predict the 3D shape, size, and 6D pose of objects within a single image, operate at a category level, and eliminate the need for computer-aided design (CAD) models during inference.