A method for forming a semiconductor device is provided. The method includes forming a plurality of first channel nanostructures and a plurality of second channel nanostructures in an n-type device region and a p-type device region of a substrate, respectively, and sequentially depositing a gate dielectric layer, an n-type work function metal layer, and a cap layer surrounding each of the first and second channel nanostructures. The cap layer merges in first spaces between adjacent first channel nanostructures and merges in second spaces between adjacent second channel nanostructures. The method further includes selectively removing the cap layer and the n-type work function metal layer in the p-type device region, and depositing a p-type work function metal layer over the cap layer in the n-type device region and the gate dielectric layer in the p-type device region. The p-type work function metal layer merges in the second spaces.