The difference can be quite obvious at the extremely end of the ISO range but not very much at the lowest end, Unless you start blowing them up to 16x20 or bigger. This difference is due largely to the sensor size. With the same mp count, the larger SLR sensor allows cleaner noiseless picture for a wider range of iso settings. This is because the larger area allows more space between sensor pixels which reduces signal interference between them, thus resulting in less noise generated.
Incidently, it is also this sensor size difference that require SLR lenses to be larger for the same magnification as prosumers.
Also keep in mind that with a bigger camera, you might opt to leave the camera at home on those days that you are not specfically going out to take pictures whereas, with a smaller prosumer camera, you might decide to bring it along just for the heck of it. On these days, which will be more than you think, you might get the oportunity to capture 'the picture of a lifetime'. Now comparing this picture with no picture at all, I'd say that the quality would be infinitely better than an SLR literaly.
Add to this, the nice little extras like being able to take movies as well, It'd really start you thinking wouldn't it?