The semiconductor memory device according to some embodiments include: a first gate stack structure; a first channel structure including a first pillar part and a second pillar part extending from the first pillar part; a first blocking insulating layer surrounding a sidewall of the first pillar part of the first channel structure; a first data storage layer; a first source select line overlapping with an end portion of the first blocking insulating layer and an end of the first data storage layer, the end portions of the first blocking insulating layer and the first data storage layer facing in an extending direction of the second pillar part of the first channel structure; a first tunnel insulating layer, wherein the first tunnel insulating layer is extending between the second pillar part of the first channel structure and the first source select line; and a peripheral circuit structure.