A semiconductor device includes a substrate, a plurality of active structures, a trench, a lower epitaxy, an upper epitaxy and a bottom barrier portion. The active structures are formed on the substrate and arranged in a first direction. The trench passes through adjacent two of the active structures in a second direction and has a bottom recess. The lower epitaxy is formed on a lower portion of the trench. The upper epitaxy is formed on an upper portion of the trench and separated from the lower epitaxy. The bottom barrier portion is formed on the bottom recess and separates the substrate and the lower epitaxy.