To add to my previous comments, it occurs to me as important that neither grapple/shove nor Bladesinger cantrips replace a weapon attack. Rather they are alternative forms of attacks that are compatible with the Attack action (and in Bladesinger’s case, compatible with the Extra Attack feature in limited cases).
The text describing the Attack action (whether or not Extra Attacks is involved) does not specify it is used for weapon attacks. It is defined as something allowing you to perform attacks. This can include weapon attacks (melee or ranged), but can also include grappling and shoving, and can even include spell attacks (for example, the Sun Soul monk gains a special spell attack that is specifically compatible with the Attack action). These are all specific types of attacks, subsets of the broader category, which are allowed with the Attack action. This is the general rule.
So when a specific rule (such as haste) says that its Attack action is limited to a single weapon attack, it is becoming more and more apparent to me that there is no question as to the rule. It restricts you to a specific subset of attacks allowed by the Attack action, in this case weapon attacks. It excludes all other attacks that might otherwise be used with the Attack action, whether the generally available grapple and shove, or specifically available subclass features such as those granted to the Sun Soul monk or the Bladesinger wizard.