My computer is a load of garbage, and did have some pretty big performance issues in the first two acts, but nothing so major it significantly reduced my enjoyment of the game. Travelling to the Last Light Inn and, too a lesser extent, the Druid Grove did lower my FPS, but I just had to avoid the areas with a lot of NPCs, and the game would be pretty smooth.
I started Act 3 today, and so far it has been a nightmare. I'm not even in the city proper yet (I'm still in Rivington), and the game has crashed twice. Every other conversation/cutscene takes 5 minutes to load, and the game overall has been running much slower. I'm currently debating whether Act 3 is even playable on my PC. There's just way too many people in the same map. From what I've heard, the performance issues in Act 3 are pretty major for people with far better computers than mine.