A video coder classifies reference samples neighboring a reference block, of a current block, into at least: first reference samples in a first group, and second reference samples in a second group. A first filter is selected, for the first group, from first candidate filters based on first sample pairs including the first reference samples paired with corresponding first current samples neighboring the current block. A second filter is selected, for the second group, from second candidate filters based on second sample pairs including the second reference samples paired with corresponding second current samples neighboring the current block. The coder determines a prediction block from the reference block based on the first filter and the second filter. The coder codes the current block based on the prediction block.