Perovskite solar cells have entered the research field of photovoltaics by storm, already reaching efficiencies close to highly optimized silicon solar cells. Coupling perovskite and silicon solar cells in a tandem configuration has the potential to considerably out-perform conventional solar cells. Under standard test conditions, perovskite/silicon tandem solar cells already outperform the silicon single-junction solar cell alone. Under realistic conditions, however, tandem solar cells made from current record cells are hardly more efficient than the silicon solar cell alone. We model the performance of realistic perovskite/silicon tandem solar cells under real-world climate conditions, by incorporating parasitic cell resistances, nonradiative recombination, and optical losses into the detailed-balance limit. We show quantitatively that, when optimizing these parameters in the perovskite top cell, perovskite/silicon tandem solar cells could reach efficiencies above 38% under realistic conditions, even while leaving the silicon cell untouched. Despite the rapid efficiency increase of perovskite solar cells, our results emphasize the need for a concerted effort in material development, careful device design, and light management strategies, all necessary to further increase the efficiency of perovskite cells, and develop highly efficient perovskite/silicon tandem solar cells.