In various examples, a visualization of an environment may be generated using a Panini projection that is optimized based on detected scene content. For example, image data of an environment may be perspective projected (e.g., using a rectilinear projection) to generate a reference projection image, which may be analyzed to detect the presence of vanishing points and/or horizontal lines (e.g., in a central region). The image data of the environment may be projected using a Panini projection that is optimized based on distances to detected objects, the absence of a detected vanishing point, and/or the presence of a detected horizontal line to generate a Panini projection image. In some embodiments, vertical compression is applied to the Panini projection image to correct for distortion of horizontal lines (e.g., based on the presence of a detected horizontal line).