Soy, a plant in the pea family, has been a staple of Asian diets for thousands of years. In recent decades, soy has been integrated into modern American diets. The potential health effects of soy are generally attributed to its bioactive byproducts such as Isoflavones. Soy isoflavones are structurally and functionally similar to natural human estrogen (17β-oestradiol), can weakly bind to estrogen receptors, and may exert either estrogenic or anti-estrogenic effects. As global consumption of soy products and its derivatives increases, it is important to further investigate its risks and benefits. This review aims to examine, consolidate and present current understanding of the effects of soy and its byproducts on the oocyte, ovary, sperm, testes and in vitro fertilization. A literature review was conducted which involved an electronic search of multiple databases. Results revealed several animal studies, which show adverse effects of phytoestrogens such as genestein and other soy derivatives such as equol. These adverse effects were mostly observed on the ovary and oocyte resulting in decreased oocyte maturation, multi-oocyte follicle formation and inhibition of steroidogenesis. These effects are also extended to male reproductivity by adversely affecting spermatogenesis and steroidogenesis. However, this review was limited by the paucity of published human studies, with inconclusive results. Additionally, many of the animal studies explore the effects of genistein or diadzein alone on limited aspect of the reproductive system using non-physiological doses, which makes human comparisons difficult. Thus we recommend randomized long-term studies in primates in order to validate the efficacy and safety of soy.