Well, if you're making a normal melee attack with the pact weapon I'd say yes, because RAW thirsting blade doesn't give you the Extra Attack feature (don't know if this is RAI or not), which is required for making a second off-weapon attack.
However, if you're using Eldritch Scythe, at level 5...