A method includes forming a fin structure with alternating first and second semiconductor layers stacked over a substrate, forming a dummy gate structure across the fin structure and depositing a spacer layer over the fin structure and the dummy gate structure. The method also includes implanting dopants into the spacer layer to form a first doped region over the dummy gate structure and a second doped region over the fin structure and removing the first and the second doped regions of the spacer layer to form gate spacers over sidewalls of the dummy gate structure and fin spacers over sidewalls of the fin structure. The fin structure protrudes from the fin spacers after removing the first doped region and the second doped region of the spacer layer. The method also includes etching the fin structure to form a source/drain recess and forming a source/drain structure in the source/drain recess.