In the current mainstream development of polymer semiconductors aiming for high charge carrier mobility, the degree of intermolecular orbital overlap has not been taken into deep consideration, since the molecular packing arrangement at the atomic scale cannot be predicted. Here, we demonstrate the utility of a unique π-conjugated building block for a p-type polymer semiconductor, namely, chryseno[2,1-b:8,7-b′]dithiophene (ChDT), in which the same phase of the highest occupied molecular orbital (HOMO) extends along the longitudinal molecular axis. Such characteristic orbital structure is expected to tolerate the misalignment of the backbone in the crystalline region, leading to efficient intermolecular carrier transport. By incorporating ChDT in the conjugated polymer with 2,1,3-benzothiadiazole (BT) as a donor-acceptor type composition (PChDTBT) and varying the branching position of the alkyl side chain, a short π-stacking distance with a transition in π-conjugated plane orientation from predominant face-on to edge-on packing was observed. In addition, a stark contrast in hole mobility (μ_h) was ascertained where polymers having edge-on orientation exhibit 2-3 orders of magnitude higher values compared to the one showing face-on orientation. Together with the theoretical calculation, it was suggested that predominant and efficient intermolecular carrier transport was realized in PChDTBT, reflecting the design concept for introducing ChDT as a building block for polymer semiconductors.