Set in the mid-nineteenth century, the novel's protagonist is Sarah Woodruff, the Woman of the title, also known unkindly as "Tragedy" and by the unfortunate nickname "The French Lieutenant's Whore". She lives in the coastal town of Lyme Regis, as a disgraced woman, supposedly abandoned by a French naval officer named Varguennes - married, unknown to her, to another woman - with whom she had supposedly had an affair and who had returned to France.

She spends her limited time off at the Cobb, a pier jutting out to sea, staring at the sea itself. One day, she is seen there by the gentleman Charles Smithson and his fiancée, Ernestina Freeman, the shallow-minded daughter of a wealthy tradesman whose origins are Scottish. Ernestina tells Charles something of Sarah's story, and he develops a strong curiosity about her. Eventually, he and she begin to meet clandestinely, during which times Sarah tells Charles her history, and asks for his support, mostly emotional. Despite trying to remain objective, Charles eventually sends Sarah to Exeter, where he, during a journey, cannot resist stopping in to visit and see her. At the time she has suffered an ankle injury; he visits her alone and after they have made love he realises that she had been, contrary to the rumours, a virgin. Simultaneously, he learns that his prospective inheritance from an elder uncle is in jeopardy; the uncle has become engaged to a woman young enough to bear him an heir.