For the record, no it will NOT harm your PSP when you are playing with the AC. However, if you have the battery inside your PSP when playing the AC, your battery life could reduce eventhough it is a long-term effect.

Infact, there are some games on the PSP which actually suggest you play with the AC as well(Monster Hunter for example). I have played with my PSP for several hours on the AC before many times, no problems.