Sounds to me like the problem is one of meta-knowledge, and the best way to resolve that is to push for non-meta justification, and non-meta consequences.
So: they saw you as a player
roll a save. What did the character
see? Was there some kind of tell, and if so, what was it, and how
did they see it?
Maybe they saw your eye twitch at a bitter flavor. How did they see that? Since you consumed the fruit with no visible issues (having passed your save), the signs must have been relatively minor. That sounds, to me, like some kind of Perception check, or possibly an Insight check against the host to determine their true intent.
Beyond that, let's just say for the sake of argument that the DM gives them this meta-knowledge, some kind of reason or even just gut feeling that the fruit is unsafe. You mentioned hospitality is a big deal in this culture, which usually means that breaking
the laws of hospitality is a HUGE no-no. The guest and the host must equally refrain from injuring one another, so long as they are bound by the guest right; in cultures that observe this sort of thing IRL, a host who violates their sacred duties has committed a horrible
offense and will likely be ostracized by the community as a result (and, at least in myth, should expect a special delivery atmospheric electrical discharge courtesy of everyone's favorite divine philanderer.) This then means that an accusation
of violating these sacred duties is an incredibly serious charge, one that should not be made lightly.
Hence, I would expect some kind of save or social-skill roll to do that without giving massive offense. Just because they believe
the fruit may be unsafe, does not mean they can simply refuse outright. They've been caught in a trap of social expectations, and have to find their way out.
There may be other ways. Perhaps eagle-eyed or medically-trained characters could spot something awry about the fruit itself, or deceptive characters could try to bluff the host into thinking
they've enjoyed the fruit when they haven't, or some other way of wriggling out of this trap without actually setting it off. But they can't just blanket refuse, without in-character
reasons, and expect that to be the end of it.
Edit: That said, yes, the DM could also have altered their approach so as to prevent some of this, by asking how each player responded to the feast before
detailing what it was like. At that point, more diverse responses open up, e.g. the players have taken
fruit but get a chance to try to spot
that eager-fruit-eater @iserith
had an odd response to it, and thus a chance to change their behavior just in the nick of time...or not.