(Emphasis added.) This appears to be another philosophical difference between us.
For me, if someone reads the text differently than I thought it could be, I now have evidence that it can be read more ways than I thought. Since the definition of "ambiguous" is: "that can be understood in more than one way; having different meanings" (Oxford American) the text must, by definition, be more ambiguous than I originally thought.
From the bolded section of your post, however, it sounds like, if someone reads the text differently than you thought it could be, you interpret that as evidence that they must be basing their interpretation on something other than the plain text. Is that correct?