A method of fabricating a semiconductor memory device comprises preparing a semiconductor substrate and insulating and active layers; forming a back-gate electrode in the active layer; patterning the active layer to form first and second active patterns on the insulating layer in a cell array region and to form a third active pattern on the insulating layer in a peripheral circuit region; forming bit lines; bonding a second substrate to the bit lines; removing the semiconductor substrate and the insulating layer; forming a peripheral gate electrode; forming a first interlayer insulating pattern to cover second surfaces of the first and second active patterns in the cell array region; forming a first etch stop layer to cover the first interlayer insulating pattern and the peripheral gate electrode with a uniform thickness; and forming a second interlayer insulating pattern on the first etch stop layer in the peripheral circuit region.