Ability checks aren’t something the player “uses” at their leisure, and players cannot “use social skill mechanics” to force an NPC to act however they want. At the core of D&D 5th edition is the basic pattern of play wherein the DM describes the environment, the players describe what their characters do, and the DM describes the results. If necessary, the DM may call for an attack roll, ability check, or saving throw to determine the results if they are uncertain what the results will be. So no, a player can’t force an NPC to do whatever they want, because it is always within the DM’s power to determine how the NPC reacts and describe the results without calling for a roll. And even if the DM is uncertain, they set the terms for what the results of success and failure on the check (or attack or save) are.