Or the Warlock's patron can BE the familiar.
This was my first thought, too. It answers the question 5E poses: What value do familiars actually grant? Answer: Everything.
If you want to make things really creepy, the familiar has found a way to increase his personal power by working with a Warlock. The complication is that the power sits almost entirely within the Warlock until he dies, at which point the familiar consumes/traps/whatever the Warlock's soul to take its power. The familiar views the Warlock as a sort of magical nursery. It wants the Warlock to grow as powerful as possible without being able to break the original deal. There's always the tension between the two about whether the Warlock has finally decided to sever the tie or the familiar is ready to collect the debt. This may or may not be the first time the familiar has made this sort of deal, but it has to "pocket" its other, accumulated power for the duration of the pact.
That gives the whole thing a classic "deal with the devil" feel. I'd be very comfortable with this, IMC, and love the idea.
For a fey patron, the term might be eternal servitude in the courts of the fey, instead of the Warlock's soul. The familiar is a servant of a higher lord, rather than the lord itself. It doesn't want the Warlock to die, because he has to be bodily taken to the Feywild. Otherwise, the whole thing plays out pretty similarly.
Not totally sure how it'd work for a GOO patron, but I'd probably start by having the familiar appear pretty standard to everyone besides the Warlock. To the Warlock, it regularly talks and might even be slightly anthropomorphic -- taking notes, making tea, playing rock, scissors, paper with the Warlock, etc. The pact isn't so much about
increasing the patron's power as it is about widening an opening for its power. The Warlock's power comes directly from said opening, which is actually a mental rift. At a certain point, the Warlock will crack and be of no further use to the GOO. Up to that point, he is an increasingly useful tool. Ideally, the familiar would
only be visible to the Warlock, but I think that might be hard -- definitely have the familiar be around and visible to the Warlock even when in its pocket; maybe reskin the manifested familiar to be very coincidental to everyone else.