When this is the case a small buoyant bait presented at the correct depth in the water column will often induce a bite. Zig Rigs work better when the hook and bait are touching so make sure the hair is as short as possible.
eBay