First, you have to type at least 3 letters for PS to start the search process.
Second, how large is you search index? If you go to the smart search app, look into the index, you should see how many items are in there.
Also, predictive search may not be a good thing for a large index with a lot of items. Because you are essentially doing a search without getting to the search result page. Try to create a test page with the search result with dialog box web part, use the same index. See what the performance of that is. If that is also slow, then it maybe the overall search performance on your site, not just PS.