I participated in an online discussion once that turned into an attempt to define "literature", which it was felt that Stephen King's work was not. (I love Stephen King's work. The Stand is one of my favorite novels, and Salem's Lot scares the stew out of me every time I read it.)
Here's basically what I said.
Sometimes a book is so rich that you feel that you know the characters and you're reluctant to leave that universe, even though it may be culturally very different from the one you live in. Middlemarch is like that for me. I always feel like I've returned from a long journey when I put it back on the shelf.
And sometimes a character is so richly drawn that by contemplating it, you actually get insight into real people and situations; I think that's significant.
Sometimes a book engenders so many thoughts that I feel that I have to discuss it with somebody. My husband was kind enough to read Ethan Frome a few years ago, after I'd read it for the first time, just so I could talk about it with him.
I'm not sure these things constitute literature (whatever that is). They raise a book in my estimation, though. And I do value a good story; in fact, a strong, original plot can cover a multitude of sins for me.