An embodiment of the present invention is directed to data quality processing of data in-flight within a data pipeline that includes reading source data from one or more input datasets into a dataframe. The system reads source data from an input dataset into a dataframe within a data pipeline and applies data quality rules to the dataframe while the source data is in-flight within the data pipeline. The system determines a pipeline action based on metadata associated with the one or more data quality rules. The pipeline action may be selected from a group of a drop action that discards failed records, a quarantine action that stores failed records in a separate table, a warn action that loads all records with quality flags, and a fail action that prevents data loading and sets the pipeline to a failed status.