This application provides a neural network model training method and a communication apparatus. The method includes: A transmit-end device performs, by using a first neural network model, forward inference on training data included in a first training process, to obtain a first forward inference result; and sends the first forward inference result to a receive-end device. Further, the transmit-end device receives a first feedback result for the first forward inference result from the receive-end device, and obtains a first intermediate result generated when forward inference is performed on training data included in the first training process. The transmit-end device calculates a gradient of the first training process based on the first feedback result, the first forward inference result, and the first intermediate result. Further, the transmit-end device updates a first model parameter to a second model parameter based on the gradient corresponding to the first training process.