First, I copied all the plugin files (SEPLUGINS directory, you'll only need these) into my memstick. Don't forget to ENABLE it.

And then I tried to install the driver. Failed, since I have a corrupted key (firmware key I suppose).

So, I ran KeyCleaner 1.3 and fixed the corrupted keys (and I'll say, most were corrupted). But there are still 2 corrupted keys which KeyCleaner 1.3 couldn't fix. If I remembered it correctly, it was key 41 (holds the version of the PSP, mine is PSP-1000) and 45 (holds the USB description of the PSP) (it says these are just a copy of 5 and 7, was it 5 and 7... can't remember but it did state the problem was a copy of some other key).

To fix this, I downloaded idrepair, idstoragemanager and idstoragechanger. And then I ran idstoragemanager (if this homebrew can't replace keys, try with the other two) to replace the keys 41 and 45 with a good one. After that, I ran KeyCleaner 1.3 again to check if its good. 

YES! I received a Congratulations message. Now I can install the LibUSB driver. After installing this, run Start-USBHostFS_PC.bat, Start-PSPSH.bat and Start-RemoteJoy.bat (without FPS) or Start-RemoteJoy-FPS.bat (with FPS).

I tested it out with:
Virtua Tennis 3
Killzone: Liberation

One problem I encountered was the home key is not useable. It won't let me get back to the XMB It just hangs at Please wait...
So, I had to took out the battery
The problem lies in the plugin, when I disabled the plugin, its useable again

Warning:
DO NOT UNPATCH key 5 while in KeyCleaner 1.3 as it might BRICK your PSP. And then you have to get yourself a Pandora Battery, etc.