Surgical systems and methods for removing material from a bone at a surgical site involve a robotic device having a robotic arm configured to support and move a cutting instrument along a tool path. A vision device is attached to the robotic device or the cutting instrument and is movable relative to the surgical site in response to movement of the cutting instrument along the tool path. A control system is coupled to the robotic device and the vision device. The control system acquires multiple vision data sets of the surgical site from different perspectives, determines a pose of a surface portion of the bone based on the vision data sets, and periodically updates the pose based on successive vision data sets acquired during continued movement of the cutting instrument along the tool path.