F'rex: If you were building a generative AI to, say, ONLY produce legal forms, it would need a much, much smaller training set.
Sure, but legal forms are utterly trivial to automate in much more straightforward ways than using generative AI. This is why none of the various AI-based legal-sector-oriented document automation has really caught on yet*. There's established tech in this field (that I work with every day), and AI is just another way of attacking the same problem, and a more expensive and less reliable way that also requires a lot more data to reliably do its job. I mean, with conventional document automation, you need one or a small group of lawyers to go through a document, and note how to automate it, and you can do it on documents where your "data set" is pretty small or specialized (theoretically it can be zero, even, for a novel document). We've looked at AI-based document automation since I believe 2018, and it just requires too much data and the outputs are still too janky to really save time/money. I notice one company we looked at has in fact basically given up on a proper bespoke document automation (which was their pitch when I first came across them), and is now just doing generic stuff and no longer looking at working with law firms but rather than non-legal clients, particularly SMEs. I guess at least they didn't go bust! There's that!
I'm aware there are situations where a smaller data set can work - I've heard about non-LLM and more specialized generative AIs being proposed to be used for various tasks, and potentially more of a genuinely transformative tech, though I'm unsure how many, if any of those have gone ahead.
* = AI has been used for years in the legal field in complex/large-scale discovery, but that's not generative AI, that's AI where you train it to recognise certain things. And indeed recognition-oriented AI can be used in other ways and has been for a long time (you can use it to pull certain kinds data from gigantic packs of discovery docs pretty well for example). But for actual document automation, of which automated legal forms are a very simple kind, AI just hasn't been cutting it compared to conventional methods - and part of that is that, at least with we're aware of, you need an awful lot of data to train it.
"Brute force" isn't about the size of the data set, but about the character/nature of how that data is used. And, with all due respect - there's a whole lot of cleverness and nuance in that. Training a generative AI is not a computational sledgehammer.
I kind of agree, I certainly don't want to be too annoying about it, and I know I can be so sorry about that! It's not a sledgehammer, but for me "brute force" isn't very emotive (colourful, yes), and isn't as specifically literal as that - for me it's means anything functions in large part because it has a huge amount of power and data. And whilst "generative AI" can potentially work with a smaller set of data, that's not the stuff that's making headline.
The last even moderate "leap" technologically and conceptually with generative AI, as I understand it (and I am totally prepared to be corrected on this) was in, like 2017, and the core concepts go back deep into the 20th century, but couldn't really be implemented more because the processing power wasn't there rather than the ideas.
Without mass-scraping of truly insane amounts of data, none of the LLMs would exist, none of the AI art generators would exist. The only reason ChatGPT can put out programming stuff, for example, is that it ingested the whole of GitHub! Maybe "brute force" it the wrong phrase for that, but it's something in that direction, imho.
