It should be quite easy to ruin most modern cameras (and some older ones) using magnets. However, the magnets you're talking about are by far too wimpy to have any serious effect.
Yeah.. for that you'll probably need a huge powerful magnet to do that... :think: Then again, if the aperture actuation is by electromagnetic, then it might affect the exposure.