Guys and gals, I am not sure if anyone else is watching H264 format video on his/her 5800, but I'd just like to share the settings that work fine for me.

Finally, after months of torturing uncle Google and testing, I have found the optimal settings for H.264 video playback on the 5800..

* using Mediacoder 0.6.2.x *

video encoder - mencoder
video format - H264
video container - MP4
video bit rate - 512kbps
video resolution - 352x288 or 352x240 (I use the latter since it's nearer to the 16:9 ratio)
video frame rate - 25fps (30fps works too)

audio encoder - nero
audio format - LC AAC
audio bit rate - 128kbps
audio channels - stereo
audio sampling rate - 44KHz

Sorry, I don't recall seeing any H.264 conversion guides on the first page. If I am not mistaken, H264 (MPEG4-AVC) is better than xvid or divx (MPEG4-ASP).