MPEG4 is a video format. As well as the codec they defined a file format. The file format is based on Quicktime apparently.
The audio in a video file I think you will find is still MPEG Layer 2 or MPEG Layer 3 audio. What you need is a program to 'demux' out the audio track.
All these so called 'files formats' are really just different 'containers' for holding video and audio data.
There are something like 27 video and audio formats supported by the Quicktime file format. The popular AVI file format probably has an even longer list, but the list is much the same list as Quicktime....