A semiconductor device structure is provided. The semiconductor device structure includes a substrate having a base. The semiconductor device structure includes a first multilayer stack over the base. The first multilayer stack includes a first channel layer and a second channel layer over and spaced apart from the first channel layer. The semiconductor device structure includes a gate stack over the substrate. The gate stack wraps around the first multilayer stack. The semiconductor device structure includes an insulating feature over the base. A bottom part of the insulating feature extends into the base. The semiconductor device structure includes a first source/drain structure over the insulating feature and connected to the second channel layer. The insulating feature separates the first source/drain structure from the base.