On close inspection of the box leaves, you may find densely woven webbing, and the caterpillar will probably be protecting itself within the webbing. Ensure you spray into the box plant and not just onto the outside surface.
eBay