A) Published art and text
It is a problem if someone publishes/distributes/sells art, text or imagery, that is grossly derivative or plagiarizing, to the extent that it violates intellectual property (and here the different legal framework across the world is a complicating factor). It shouldn't matter whether the production involved reproduction by human hands, "classic" software and/or AI*. If you rip off someone's work - you're doing a bad thing.
Or even that good, old, photocopy. Also, it should only matter if it's used to redistribute, because you can't rip off by just consuming for yourself. I know it might be controversial for some, but the people who just, say, download an image on his computer to print on his character sheet is doing no harm, since he's using the artwork privately for his own purpose. The result of the "mixing of points" could result in an objection to a technology in general, or an objection to a particular source for models, depriving the possibility to use a technology in a non-infringing way (the "non copy-protected outcome" chosen in the US) or ony private use since you can't infringe on copyright through private use.
B) Software and services
It is a problem if software (desktop application or web-based service) contains copy-righted material. It is also a problem if it is capable of producing imagery which violates the intellectual property of others. You can't sell an application which reproduces such works. This is a bit more fuzzy than it seems at first. At one extreme you have "give me the full text of the book Shadow of Abracadara" at the other extreme you have a word processor, which you can use to type in the full text of a book and then store it locally on your machine. Word processors are obviously not a problem - but the point at which a tool is too good at following instructions isn't quite as easily definable as one might imagine.
Indeed, and I'd defende the word processor analogy. We don't regulate word processors, even if they can be used to retype the whole Shadow of Abracabra. We make it illegal to distribute your (home-typed) shadows of abracadabra books. It should be the same with AI. If it somehow was able to actively infringe, as in "write me a derivative work of an existing work, like the true ending of the Game of Thrones series", then it's the outcome that should be monitored, not the tool that can type out the book.
C) Training data
High-quality training data for neural networks is an extremely valuable commodity. It is not hard to crawl the internet for everything and train an AI. It is very hard and very expensive to curate a collection of material (be it art, text or something else). Some websites, like say Deviant Art or certain places for people to share fan fiction - even discussion forums like this one - are very good sources of high quality training data. Companies which make use of such data must obtain the consent of the owners of the material.
It's a contentious point, for some at least. For example, some have claimed that Adobe "strongarmed" them into relinquishing their rights because they didn't knew/understand what they relinquished and others say that they couldn't understand AI back when they entered into the contact. So you'll find people to refer to ethics even after giving consent contractually. At the other hand of the spectrum, you have the datamining exception validated in the EU, where a non-profit (in many case, public) research institution can basically do ignore copyright as long as they don't benefit financially from the outcome. Some will find it acceptable (the EU, obviously, who considered the overall benefit of having AI companies flourishing and paying taxes for the greater good), other might find that it's unfair to right holders who put their work on the Internet.
D) Selling training data is not selling intellectual property
This one I think is probably the most overlooked. An artist who sells the right to use work in training data, is not selling the right to reproduce said work. Those things must be kept apart. Why would a company want training data if they are not allowed to reproduce it? Well, that's because AI tooling can, when used by humans (and in the future perhaps even autonomously), produce (or contribute to the production of) creative works. It might not be art. It might be ugly. It might be derivative. But it can be something that if done by a human would be considered legal. And therefore training data has a value in making the AI tooling better at creating such things (better might not mean quality, it could also mean other things, so consider it broadly). Artists need to retain their intellectual property rights regardless of whether they consent to their work being used for training.
Indeed. I think however that nobody ever suggested (apart maybe a few who equate training to theft) that they were losing any right over their own work by allowing it to be used in training. It would extreme if the data in the training data had to be transfered and not only licensed for this use. Also, it would prevent using public domain works (because you can't appropriate them). I could see some case where it would be the case that the trainer owns exclusive rights over the material used in training: a private company that wouldn't disclose anything about their database, or a country-wide effort (if China or India tasks its art teachers to draw each a few specific pieces and caption them well, much better than the LAION-5B crap, they wouldn't need as many work to create an effective training model) and they might want to host the next "digital holywood without pesky actors and writers..." Those are fringe case, though. And TBH if a sovereign entity wanted to have a model like that, it would most probably create an exception for public training database much like they created an exception for public libraries to lend book.
Right now the focus is very much on the interplay between training data and the capacity/tendency to reproduce in a way which violates intellectual property - and defining those lines. That's an important discussion. But it's not the only discussion. And it's really important for the future livelihood of artists that this is not reduced to a question of training data and what workflows are acceptable for artists who use AI tooling.
TBH I am rather pessimistic over the livelyhood of many, many people, including many highly-paid jobs. But wealth-sharing method in a society is political so I won't say much more on this.
To the precise point of reproducing art in the way that it doesn't infringe copyright, it is easily within reach (in satisfactory or not satisfactory manners for the artists) especiall since it's a transient problem: at some point, there will be enough well-captioned, different pictures in the public domain or under free license that developping a model with them will be hasslefree and not the sole perview of Adobe.
I am pretty sure that worrying regulations, or the fear thereof, will only spur companies toward developping this database earlier than later. Also, it is a problem that can be lessened by explaining better how it works. There are people who honestly think that models are hundred of GB large because they must contain all the images in a compressed format (despite having functional model that are less than GB files). I guess many people think that "in the style of greg rutkowski" is a prompt that produces art that have the style of greg rutkowski in them, while it's not the case. It just happened that the autocaptioning of fantasy images displayed on artstation where labelled as "art by greg rutkowski", irrespective of whomever draw the picture. So it is a keyword that orients the generation toward generic fantasy art, with no link to greg rutkowski. If the artstation art had been labelled as art by George Washington, then you could ask the AI to draw dragons and pretty elves in the style of the late president's famous paintings... errr or not. Dispelling these worries would help allaying the fear of plagiarism (everyone can check by interrogating CLIP over their own artworks, it will propose styles of several artists none of which are you, generally). I have CLIP-ed a photography of a colleague I had just taken and she was said to be "an artwork by X, Y and Z" where those three were 19th century painters. I suppose that my Android phone has some dead painters's soul spliced to it ;-)
