A system comprises an intraoral scanner and a computing device. The computing device is configured to: receive intraoral scan data of at least a first portion of a dental site; perform object recognition on the intraoral scan data to determine an identity of a tool in the intraoral scan data; use the identity of the tool to determine a known shape of the tool; generate a three-dimensional (3D) surface of the dental site based on the received intraoral scan data; and supplement the 3D surface of the dental site with additional surface data of a second portion of the dental site that is hidden from direct viewing by the intraoral scanner, wherein the additional surface data is based at least in part on the known shape of the tool.