Testosterone and Estrogen levels Affect the Whole Body and the Brain

Research articles showing how Testosterone and Estrogen (Estradiol) cause different brain chemistry/function between men and women.

Clinical research has demonstrated a significant sex difference in the occurrence of depressive disorders. Beginning at pubertal onset, women report a higher incidence of depression than men. Women are also vulnerable to the development of depressive disorders such as premenstrual dysphoric disorder, postpartum depression, and perimenopausal depression. These disorders are associated with reproductive stages involving changes in gonadal hormone levels. Specifically, female depression and female affective behaviors are influenced by estradiol levels. 
Estrogenic mediati... [Prog Neuropsychopharmacol Biol Psychiatry. 2014] - PubMed - NCBI

The literature suggests that while there are many similarities in brain structure, function, and neurotransmission in healthy men and women, there are important differences that distinguish the male from the female brain. 
Evolving knowledge of sex differences in bra... [Biol Psychiatry. 2007] - PubMed - NCBI

Of particular interest are sex differences that have been identified in the brain. Although the brains of men and women are highly similar, they show consistent differences that have important implications for each sex. That is, brain sex differences uniquely affect biochemical processes, may contribute to the susceptibility to specific diseases, and may influence specific behaviors. Such biological differences should never be used to justify discrimination or sexism. However, we believe that a thorough understanding of these differences can inform researchers and clinicians so that they can better address important issues. Two examples include how genetic sex can lead to differences between the sexes in the etiology and the progression of disease and how differences in neural development may result in differences in cognition and behavior.
The Genetics of Sex Differences in Brain and Behavior

"...sociologist J. Richard Urdy has found that females' responsiveness to encouragement of femininity is inversely related to their mothers' testosterone levels during the second trimester of pregnancy.

"...female rhesus money injected with androgens during the prenatal critical perioid later display stereotypic male sexual and nonsexual behavior.  They engage in higher frequencies of threat behavior, play initiation, rough-and-tumble play, and chasing play...a consistent relationship between hormones and behavior emerges that is not easy to dismiss by invoking different ad hoc criticisms of the individual findings.
Book: Biology at Work: Rethinking Sexual Equality

The incidence of psychopathy in the general population is generally considered to be 0·6-4% with a higher proportion of males to females. Brain imaging studies of psychopaths suggest a smaller and less active amygdala and prefrontal cortex. There also appear to be physiological derangements in psychopathy, including alterations/dysregulation in neurotransmitter homeostasis (dopamine and serotonin), altered endocrine responses (testosterone and cortisol)
Psychopathy: clinical features, developmen... [J Clin Pharm Ther. 2014] - PubMed - NCBI

The recognition memory tasks, novel object and novel object location, have been beneficial to neuroendocrine research concerning the effects of gonadal and adrenal hormones on cognitive function...Experiments conducted across a number of laboratories show that gonadal hormones, both estradiol and testosterone, promote memory while the adrenal hormone, corticosterone, impairs memory.
Department of Psychology, Hunter College of CUNY.
Recognition memory tasks in neuroendocrine r... [Behav Brain Res. 2014] - PubMed - NCBI

The gonadal steroids play an important role in altering the survival or death of neurons within the brain as well regulating the development of neuronal processes (such as the number and length of dendrites and dendritic spines, or the degree of axonal branching). In this regard, we can say that steroid hormones have organizational effects that produce sex differences in the connections and functions of the brain. Gonadal steroids also have activational effects; these activational effects are transient and are thought to coordinate behavior with either an internal event, such as ovulation, or an external event, such as the presence of a sexual partner.
Psychology Course
http://www.umich.edu/~psycours/531/becker.chapter3.htm

The specific impact of sex hormones on brain development and acoustic communication is known from animal models. Sex steroid hormones secreted during early development play an essential role in hemispheric organization and the functional lateralization of the brain, e.g. language. In animals, these hormones are well-known regulators of vocal motor behaviour.
Sex hormone influence on human infants' sound char... [Biol Lett. 2014] - PubMed - NCBI

Boys with higher levels of prenatal testosterone who were also exposed to higher levels of nicotine and alcohol during pregnancy exhibited increased hyperactivity-impulsivity during early childhood, but girls did not exhibit this same pattern. Thus, high prenatal testosterone exposure seems to increase risk for DBD symptoms particularly in males by increasing susceptibility to prenatal environmental stressors.
Prenatal testosterone increases sensiti... [Neurotoxicol Teratol. 2014] - PubMed - NCBI

Genistein is a phytoestrogen, particularly abundant in soybeans, that is able to bind estrogen receptors exerting both estrogenic and antiestrogenic activities. Genistein is largely present in the human diet even during pregnancy. Embryos and fetuses are therefore, commonly exposed to genistein during the development and after birth. In the present study, we used a murine model as a test end-point to investigate the effects of early exposure to genistein on adult male behavior and related neural circuits. Exposure of dams to genistein (100μg/g of body weight) daily during late pregnancy and early lactation, produced in male offspring, when adults, significant changes in anxiety behavior and statistically significant variations in the number of neuronal nitric-oxide synthase positive cells in the amygdala. In conclusions, these data indicate that early exposure to phytoestrogens may induce life-long effects on the differentiation of brain structures and behaviors.
Perinatal exposure to genistein affects the no... [Physiol Behav. 2014] - PubMed - NCBI