is there any converter/encoder that can convert my video file to 10bit? i got a video file and i want to try to convert to 10bit to shrink the file size without losing much quality

You can give MediaCoder a try, it's open source and support many conversion options.

can that software able to ultilized dual core/quad core? i got a core i5 and i see some conversion software did not fully utilize it, it only use maximum of 50% of the processor load

MeGUI?
And what is the source? if mpeg2 dvd or bluray is fine,
But if the source video you want to re-encode is mkv(yeah, i know its h264 blabla) fansub,
dont waste your time to reencode them again, cause reencoding = quality reduction, even if the encoder is x264 with Hi10P support.
Some more x264 Hi10P encoding speed is much slower than normal 8bit x264.

You should read this: Generation Loss http://en.wikipedia.org/wiki/Generation_loss
Unless you dont mind about quality or the fansub is too large.

it is a mp4/avc video format, about 1 hour 40 mins, full HD cost 7GB+

Oh my, good luck with the transcoding lol
CPU transcoding still slow as snail anyway. Hi10P aka 10bit is even slower.
You should try MeGUI as i said, but replace the x264 executable with the one with Hi10P support.