Your warlock will probably be a bit more versatile, and the broader spell list may make them more powerful in a day where they can take many short rests between social/exploration encounters (I tend to limit short rests to no more than 3 until you take a long rest). If you have a wizard in the part it may make them feel that one of their class aspects (spell versatility) is being slightly eroded in terms of niche protection.
That said, I did the exact same thing for the fiend pact warlock in the game I DM, and the DM in the game I play in did the same for my genie pact warlock (Sha'ir). Its played fine and has added to everyone's fun. The classes with few spells known tends to force optimization in order to get max use out of your list, and this allows for some more fun and fluffy choices. To me, that's always a good thing!