Introduced here are approaches to generating programmatic definitions of activities that are performed in videos and are to be mimicked by users of a pose monitoring platform. While these approaches can be implemented by the pose monitoring platform, the videos could be obtained from another source (e.g., an online video sharing platform or a social media platform). These approaches help solve the problem of limited exercise therapy being available, allowing programmatic definitions to be readily created for new physical activities and improving adherence to exercise therapy programs that require consistent performance of physical activities.