Are VWs Really Unreliable?

It seems that about once a month or so I'm talking to someone about a new car they're looking for. If I mention VW, about half the time they so "oh, they're too unreliable, I'd never buy one".

My first question about this is that a consumer is unlikely to have a broad view of the market and the failure rates for a particular brand. So, I assume this comes a) from the experience of friends or b) urban myth.

I also have the experience of my family owning five VWs (currently -- it is a big family). That isn't really a big enough data set to draw conclusions, but it is a bigger pool of data than most consumers would have. Two of the cars have had zero problems. Three have had one issue each, but two have over 100k miles on them. All the cars are 6 years old or less, and were purchased new. JD Power says the best new cars have about 1 problem every three years, so we're doing better than average (about one problem every six years).

So, why the reputation? Is it undeserved?