My experience is the same. The plot is probably the least interesting thing in the book. I love the wide cast of characters, the fascination with architecture, the gentle piss-taking of English class stereotypes and the fairly real descriptions of how English police work is done - minus all the Falcon stuff of course.
It's also why I love reading the Nicci French Freida Klein series. Well, the ever-expanding cast and the travelogue. I'll be reading street names and neighborhoods and parks and such which are familiar, but 40 years gone, I don't recognize anything on Google maps.