Boorman's general skill isn't the problem. His suitability for a project like this is. I mean, look at his other projects from this era.
I did. I mentioned twice already how right after this he made a movie that was nominated for Academy awards, Golden Globes, and BAFTA awards. Wouldn't a movie he made in the mid 80s be considered the same era as the mid 80s when Gary approached him and he agreed to do it?
Also you somewhat slanderous if amusing scuttlebutt list of "wasted on set" people misses the point - Welles was giving a lot of phoned in and terrible performances and had no physicality by that point. He'd have been bad in anything but a cameo.
He was slotted to do narration. Which he did pretty well in History of the World part I and the voice of Unicron in Transformers the Movie (and he died right after doing that, before the film actually was released)