Upgrade your account to a Community Supporter account and remove most of the site ads.

Reply to thread

Message: <blockquote data-quote="RareBreed" data-source="post: 9090711" data-attributes="member: 6945590">Depends on the model architecture.  Reinforcement learning  may update weights based on past experience.  There's also bayesian weights that have conditional probabilities and some other architectures that can have stochastic weights/biases.Also, I'm not a data scientist, I'm actually an SDET on the data engineering side <img src="https://cdn.jsdelivr.net/joypixels/assets/8.0/png/unicode/64/1f642.png" class="smilie smilie--emoji" loading="lazy" width="64" height="64" alt=":)" title="Smile    :)"  data-smilie="1"data-shortname=":)" />  While I've done some simple models (mostly RNNs), the bulk of my knowledge has been reading.  While I have a very high level picture of the Transformer architecture and Generative Adversial Networks, I have no experience with them.  I've been meaning to play around with  <img class="smilie smilie--emoji" alt="🤗" src="https://cdn.jsdelivr.net/joypixels/assets/8.0/png/unicode/64/1f917.png" title="Hugging face    :hugging:" data-shortname=":hugging:" loading="lazy" width="64" height="64" /> transformers at work, but other priorities keep knocking it down.  I haven't come up with a use case for GAN's, though some of my colleagues in other departments have.I believe the problem here is that adding layers isn't linear.  One of the courses I took showed this very interestingly.  The instructor is a big advocate of experimentation, so some of the exercises were to tweak various things, like weights, training epochs, learning rates or even the number of hidden layers.  He's also a big advocate of visualizing results (made easy with matplotlib or plotly on jupyter notebooks).  What was interesting was that in some cases, you would see the cost functions plateau, and just by adding a single layer, you would see a dramatic increase.But i think the biggest factor is simply cost.  Even if the first couple of iterations are fast because there are so few layers, N-1, N-2, ... where N is millions of layers is just way too expensive.Ah I gotcha now.  I would agree with that.Yes, interesting in theory, but I'm not sure there's a way they can detect it.  I've only read some articles on some of Chalmer's work, but one of the more interesting takes on consciousness/mind/intelligence is still from Hofstadter's excellent 40+ year old book, Godel, Escher Bach: An Eternal Golden Braid.  The idea that self-recursion can cause consciousness is pretty interesting. I'll try to remember that I'm attempting to create consciousness the next time I create a recursive function that stack overflows <img src="https://cdn.jsdelivr.net/joypixels/assets/8.0/png/unicode/64/1f609.png" class="smilie smilie--emoji" loading="lazy" width="64" height="64" alt=";)" title="Wink    ;)"  data-smilie="2"data-shortname=";)" /></blockquote>

Verification