Given the fact that there should not be any fungus spores in your non-removable lenses, it should not have fungus growth.
But too much condensation may affect the circuitry of your camera though.
You may wish to put the camera in a ziploc bag when you move from a cold to hot place. You'll find the condensation occurs on the ziploc. Leave it for a while before opening the bag to use your camera. Tried this tips from some users of DP Review on my SLR and it works. Problem is, if you can wait a while before you shoot.