While a golem might accidentally damage the environment, generally speaking they won't do so directly. Otherwise, they would take the most direct path to anywhere and that would cause them to attempt to dig, pound, bash their way through walls.
As a feature of their environmental navigation logic, therefore they won't try to destroy walls to get at their target.
Since the golem is non-reasoning, it will attempt to use an alternate known route, or simply wait until the target they sense moves back into attack range.
Unlike even an animal, the golem can't get enraged or frustrated. It isn't going to go berserk on the door. it will assess that it cannot attempt a hit against the target and wait until the target moves into attack range or leaves sensory range or attempt an alternate route.
