So trying to decipher what appears to me to be somewhat vague comments, I think your calling to independent test by two different players not a group test but hold them accountable as a group.....? In my mind the when you let multiple people role for the same test in the same location and create "setbacks" that could make group participation accountable and impact party members who are present... it is a group test, however it seems like your trying to counting second attempts by another party member as a completely separate test while applying group consequences.
Your example restated with my translation: (Maybe wrong but I am honestly trying to read you in text and this is what I got)
Player 1 does investigation check, succeeds but announces a roll like a 15, Player 2 thinks a higher success might have revealed something more so wants to try again with a second investigation check, rolls a 18 and finds nothing new but does not cause problems.
Player 1 does investigation check, fails and finds nothing with a role of 5, Player 2 thinks he can succeed where player one failed rolls a 15 and succeeds, but you want to add a penalty because you don't like check spam test because Player 2 doesn't know why player 1 found nothing unless player 2 is metagaming and knows the role is low. My solution to dealing with that is that I would make it a group check that they can both do if they agree to join into the search when it is announced, if not, then they already stated that they had not interest in searching and any attempt to search after nothing is found would be meta gaming and not allowed. Alternately, you said, " Another player declares he's also searching." to me this reads as a player joining into a group test which is different from a help action as they use their own modifiers instead of giving the player advantage. Where the down side is that if a third person helps and two of them fail the entire test is a failure no matter how good the successful test was.... But you said your not doing that. Which means your letting multiple people check separately on the same test in the same room as individuals even though that amounts to a group searching the room and is the reason for group tests as I understand it. If that is the case its your lack of requiring a group check as a restraint to endless individual tests that is creating the problem, and my confusion on your prior posts. The one case where having a group test is not as good an option is when you have a group of two and their is no chance of less than a 50% success, its only a failure if both fail. In that case you can consider it the aid could be counter productive or on positive. If it could be counter productive, give the primary the help action, have the other role the advantage dice with the primary players stats and narrate the result. If it can only help, like multiple people pushing a bolder, then allow two man group test where its just more likely (but not guaranteed) that they succeed duo to having more weight behind it. Your searching example, it reasonable that more eyes and ideas would only get them too look in more places with more perspectives and a better chance of finding something but if you disagree and you think the second person could be in the way "destroying evidence" then make it a help action.
Player 1 and Player 2 fail, bad things happen failure is its own punishment but if your adding "setbacks" to that just because of the redundant checks your punishing team work on top of that.
My overall point is that I would make them work together if they are together doing the same thing. If I got your intent right this time its because I would consider individuals in a group doing checks as individuals in the same location as group activities and make them roll accordingly because that is what I understand a group test to be for, unless their is some competition between them and they have no intention of sharing and working together with what they find. At that point I would treat them as separately but I would run one and then the other with an initiative role and the success of first might allow the hinderance of the other if they so chose, however that's going to be very rare.
...Any closer?
