Lawrence grew up an only living child, of Mary Elizabeth (Smith) Morgan, a schoolteacher, and the Reverend Sandy Alonzo Morgan, an Episcopal minister. They lived in Richmond, Virginia but traveled to New York City for Lawrence's birth, as their first child had died in the local segregated hospital. Lawrence said, "In childhood and through adolescence I said I wanted to become a doctor because of the death of my only sibling, a brother, at eleven months, and two years before I was born. Someone like me could have saved him." After Lawrence's birth, the family returned to Virginia, then moved to heavily segregated Vicksburg, Mississippi.