I don't see why you're linking me a "how google works" article. I understand how search engines work. Perhaps you can state the point you're trying to make?
Because you didn't seem to quite get why the 5e wikidot was the first result. If you know how search engines work, then it shouldn't be much of a surprise.
Is it that there is no problem with search engines facilitating piracy? Or no point in trying to stop them? Or are they unable to meaningfully address it?
That's a bit like saying brick-and-mortar stores facilitate shoplifting.
I'm not asking for us to come up with a legal system here. I'm asking about the ethics. Here, let's make the argument precise:
1) Google directs people to pirated content very effectively
2) Google knows this and is able to modify search to do so less effectively
3) They choose not to, beause people like pirated content and like that they can find it using Google. The legalities are grey enough that google can get away with it. As a result they can sell more ads.
If (1)-(3) are true, is google acting ethically? Or do you take issue with any of the propositions?
1) Stores have plenty of blind spots where cameras and employees can't see effectively, thus enabling shoplifting.
2) Stores know this and are able to modify their layout to make shoplifting less effective.
3) They choose not to, because people like to shoplift and like they can do so in the store.
OK, you have to realize that what you're saying makes no sense. In reality, it's this:
1) Google (also Bing; I checked) leads people to pirate sites because those sites know how to use the search algorithms to their advantage. Also, word of mouth led to those sites becoming popular, which causes them to get shown higher in a search result.
2) It's not cost-effective for Google to hire people to go to every single website in existence to determine whether or not the site contains pirated material.
2a) Nor is it Google's job to do so, any more than it's the job of whoever put out the Yellow Pages to make sure that every company that put out an ad in them was completely on the up-and-up.
2b) Instead, it's the job of WotC's legal team to go around, find sites, and take them down.
2bi) WotC has apparently chosen to pick their battles in this case, probably because the 5e wikidot and similar sites aren't going out of their way to broadcast their ill-gotten gain.
This also has nothing to do with using AI instead of paying actual human writers and artists. Please stop with the whataboutisms.
I suspect we'll end up in a similar place with LLMs. Libgen is out, web crawls are ok, and the legalities will be grey enough they can get away with it. As you state for search, I'm not sure LLMs can exist in the same way without web based data.
You can entirely train them on material you own and/or that is in the public domain.
However, then you're still using AI instead of paying actual writers and artists.
If 5 years from now the legal system decides that is ok, do we then conclude the LLMs are ethical because they are legal? I don't think that tracks.
Lots of things that are or were legal are unethical, and lots of things that are ethical, or at least not unethical, are or were illegal. There's no point conflating the two.