So, under the asssumption that the Kaiju is made of NewMatter in a NewConfiguration (like the various ways we find better structures out of carbon, spider silk, titanium, etc), the Kaiju must be designed to survive high pressure under deep water AND be able to survive "normal" pressure on the surface. A layman would asssume the latter is irrelevant if it's sturdy enough to survive the former, but let's assume we at least don't have a jelly fish problem (where outside of water, the thing can't support itself).
Within that framework, why do we assume the kaiju is indestructible? Being able to withstand large pressures and strains spread across the surface or joint is not the same as being immune to penetration attacks.
Elephants are big, bulky and have thick skin. A .50 cal still drops them because high mass, high velocity rounds are still VERY small in comparison to the surface they are striking. A surface that was designed to resist a broad force and stress, rather than a high force, small surface attack (like .22s penetrating bullet proof vests).
In any event, I'd like to see sciency refutation or support of the point that we are more likely to build a better bullet to kill a kaiju than a battle mech to kill a Kaiju. the kaiju's system is more likely to withstand rough and tumble brawling (as it can with its own kin and original environment) than faster, massier bullets fired at it. Especially given that if a kaiju has special design/materials that enables it to exist, we're less likely to master that in time to build robots capable of the same, than we are to build a better gun.