Custom search fitler is a right way to go.
Smart search searches all field configured as searchable the object (custom table in your case), so it will cover Keyword search. Than you can use filters to narrow down the results by particular field(s).
So you need to:
Hope this helps