Depends on what you want...
First, the file format isn't really that important. It's just a container. What matters more is how the streams (video, audio, ...) are encoded. Of course, not all containers allow all kinds of streams.
GOOD container formats:
mp4 (MPEG-4): The new international standard, good format, not enough software support yet but there will be. The future.
mkv (Matroska): Nice open-source format, allows multiple video/audio/subtitle streams. Still some playback problems (aspect ratio etc.). The problem is, it's only popular among geeks, has virtually no hardware player support, and it doesn't seem to be developed any more.
MEDIOCRE but unavoidable:
avi: Old Microsoft format, not good from a technical viewpoint, but very widespread. Most movies are in this format. Lots of tools available.
flv (Flash): Best for online videos because most people have the Flash plugin in their browsers, and unlike other browser video plugins, it just works. The newest beta version of the Flash player supports mp4, so hopefully flv will no longer be needed.
mpg (MPEG-2): Very old, huge files, but plays everywhere. This is what DVDs use - you can copy a .VOB file from a DVD, rename it to .mpg and it plays.
BAD container formats:
VCD (MPEG-1),
SVCD (MPEG-2): Used to burn CDs that play on DVD players. No longer needed because most standalone players support avi with XviD/DivX + mp3.
ogm (OGG): An extension of the open-source OGG format, technical difficulties, small fan base, probably no future.
wmv (Windows),
mov (old Apple Quicktime),
rm (Real): Proprietary crap, hard to convert. Don't use them.
Best VIDEO CODECS are:
XviD, DivX: most movies have that video stream
AVC aka H.264, x264: half the size of DivX for same quality, but needs fast hardware to play.
XviD and x264 are free/open source implementations, at least as good as their commercial rivals.
Best AUDIO CODECS are:
mp3, old, most widespread
AAC, better, upcoming
AC3 (Dolby) is an old format for 5.1 surround sound. No longer needed as AAC does surround with a much better compression rate.
Google for "codec shootout" to learn more. For audio, go to
hydrogenaudio forum.
To play on a computer, the best format is mp4 (or mkv, or avi for compatibility) with x264 video and AAC audio. Use
StaxRip to rip or convert to mp4. If you're using a standalone player (DVD player) or an older computer, the best choice is probably avi with XviD/DivX video and mp3 audio. Most standalone players call this format "DivX". But not all avis play on all devices.
If you
really want to know which formats are the best, go to
doom9.