Various embodiments of the present disclosure disclose machine-learning based evaluation techniques for detecting feature bias. An evaluation framework is provided that utilizes new evaluation data structures for comprehensibly evaluating feature bias in machine learning models. The evaluation framework includes receiving evaluation dataset for a machine learning model that includes one or more different feature classes of an evaluation feature. The evaluation framework includes generating, using an evaluation function, at least two performance metrics for the machine learning model and generating a unitless dissimilarity metric for the evaluation feature based at least in part on the first performance metric, the second performance metric, and an average between the first and second performance metrics. In this way, the unitless dissimilarity metric is based on a variability associated with the evaluation feature.