Systems and methods are described for climbing of objects in an environment of a robot based on sensor data. A system can obtain sensor data of the environment. For example, the system can obtain sensor data from one or more sensors of robot. The system can identify the object based on the sensor data. Further, the system can determine that the object is climbable based on determining that the object corresponds to a particular training object. The system can determine that the object corresponds to the particular training object based on a particular characteristic of the object. The system can identify a climbing operation associated with the training object and instruct the robot to climb on the object based on the climbing operation.