A method and system for training a language model train a language model without a language degeneration phenomenon of the language model. The method comprises: specifying a first text sequence to be input into the language model to fine-tune the language model; processing the first text sequence as input to the language model; generating, by the language model, a second text sequence including a plurality of tokens based on the first text sequence; storing the second text sequence generated by the language model in a memory; performing masking on the second text sequence stored in the memory; and fine-tuning the language model using the second text sequence on which the masking is performed.