An interesting note - Hit points matter, generally, when you run out of them. As a PC fights just as effectively when they have 1 or 160, the characters generally suffer only when they drop below zero. When you think about this in terms of the barbarian and the benefits of the class, you may come to the conclusion I did: Defense is a huge waste of resources.Cloak of Displacement, it grants disadvantage to all attacks against you until your are hit, and then its effect comes back online at the start of your turn.
This would cancel the granted advantage when using Reckless Attack and make them even more hard to bring down. Particularly useful against missile attacks early on in the fight, allowing them to just charge ahead unscathed.
One of the benefits of a barbarian is that they have a lot of hit points and damage resistance. It allows them to take a huge amount of damage and keep on ticking. If your barbarian is not being knocked down often, then expending resources to increase their defenses may not be a great use of resources. You'd likely be better off devoting those resources to offense or utility capabilities (or non-hp based defenses like saving throws).