A method for performing federated learning on progressively available data, which is suitable for implementation by resource-constrained devices, such as smartphones and edge devices is provided. A method includes obtaining, instructions to begin a FL round, determining a mode to use during the FL round, wherein the mode is one of a streaming mode involving sub-epoch training and a non-streaming mode involving epoch training, training, using the determined mode, a local ML model stored at the client device, the local ML model corresponding to the global ML model and transmitting a plurality of updated parameters of the local ML model when the training is complete.