Comments: 3
There's a fair bit about Deb Willet in Pepys's diaries; he is astonishingly explicit about his relations with her (and with other women -- besides his wife). Apparently he truly cared for Deb; he sought her out several times after his wife had forced her removal from their home (after having discovered them in flagrante delictu), and it was only as a result of an ultimatum by his wife that he ultimately stopped pursuing her.
This pretty much aligns with the book! Deb Willet is definitely a woman I would like to learn more about.
Yes, I would expect Pepys's diaries to have been one (if not even the chief) source for the novel.