All of m_j_r's Comments + Replies
Radical Energy Abundance
I like this perspective in the reality of atoms, but I'd like to offer a cofactor that also compounds with this.
Electrical power is never freed from thermodynamic constraints. In simple terms, it would be obtuse to maximize the transmission of electrical power, because that inherently maximizes waste heat & associated destructive risk. Fundamentally, our environment favors transmission of signal upon local consumption of power, usually digital bits above a minimum SNR. This is not a single paradigm, because our economy has been growing considerab... (read more)
AI is still economically-bound computation (e.g. https://twitter.com/clementdelangue/status/1711732659443863978) The smaller model(s) + deterministic glue + more expressive HITL paradigm is just going keep percolating for a while, clearly we haven't hit the frontier with those resource constraints and data technique is the most competitive element respective to those constraints (e.g. https://huggingface.co/HuggingFaceH4/zephyr-7b-alpha ).
On the other hand, a lot of markets maintain a equilibrium due to their capture, so any hypothetical gain in competitiv... (read more)