My instinctive response was "yes, you have to roll," but when I went to look up the rules, I found it was not as clear-cut as I originally thought. Here are the concentration rules:
And the temporary hit points rules:
Note the phrasing of the final sentence: "If you have 5 temporary hit points and take 7 damage... then
take 2 damage." The way this is phrased suggests that the temporary hit points are intervening before the damage actually hits: "Take 7 damage" is replaced with "lose the temporary hit points and take 2 damage." So "take 4 damage" would be replaced with "lose 4 temporary hit points and take no damage," which would mean no Concentration save.
Purely based on the text of the temporary hit point rules, this is the logical reading. However, this has somewhat far-reaching implications. It means that any time a PC with temporary hit points gets hit by something with a rider effect (e.g., poison), you have to consider whether preventing the damage should prevent the rider effect as well.
It also makes temporary hit points far more valuable to spellcasters.
Armor of Agathys, for example, would be a must-have spell for bladelocks. That isn't necessarily a bad thing--bladelocks kinda need the help--but it's something to be aware of.
Ultimately, I think I would go with my original response, but that is my personal ruling, based less on the RAW than on a sense of "do not open a can of worms if you don't have to."
(If you put stock in Sage Advice,
Jeremy Crawford says you have to make the save. However, he does not justify this with any citations from the rules.)