If you make it to Skye, way on the north western edge there's Castle MacLeod. It has a very nice garden.
Probably not worth driving all the way there just for that. But we stayed with a very nice lady who lived at the Northwest point of Skye, and that was an attraction that was relatively close.