With the advance of next-generation sequencing techniques, phylogenetic analyses based on genomic data are becoming frequent in plants. However, the efficacy of different methods used to retrieve orthologs for phylogenetic studies based on transcriptome sequences is rarely compared and examined for closely related species with low genetic differentiation. In this study, we used both low-copy gene-based approaches and mapping-based approaches to construct a phylogeny of Betulaceae. We found that both approaches showed no distinct differences in phylogeny construction and associated bootstrap support, although mapping-based methods seemed to be superior to the former in all analyses particularly when the summed length of orthologs was not long enough. Phylogenetic relationships within the family, largely consistent with those previously based on chloroplast DNA and nuclear internal transcribed spacer sequence variations, received high support for all clades and subclades. However, we found that support values for the sister relationship between two polyploid genera were lower than those between diploid groups if only a few orthologs were sampled. This may result from ortholog misidentification due to genome duplications.