Smart casual has gotten iteself a bad wrap. The phrase seems to throw even the most skilled dressers into a state of utter perplexion and panic. Odd, because once understood, it's simple really, and the two words are not in fact directly opposed. Look at The Duke and Duchess, how at ease they are with the whole thing. That's sort of the point of smart casual, your touchstone for polo style – it should convey a certain sense of put-together easiness.
In the words of Coco Chanel, "luxury must be comfortable, otherwise it is not luxury."