I really want to believe, but if I had a penny for every time that analog, optical, ternary, clockless, or other radically non-standard computing paradigms were supposed to revolutionize the industry... I'd have a nice pile of pennies.
Electronic signaling is just so marvelously easy to scale that the right path was clear pretty much from day one. We don't have that path for other operating principles right now. As for synchronous operation, binary signaling, and so forth, they're once again just scaling tools that let us crank out designs with billions of transistors without hand-crafting every piece or making the abstractions more leaky than they already are.
A difference now is that moore’s law per power density has been dead a few years and physics says it can’t get much better.
The other difference is that computers are now powerful enough to do the 10^20 calculations required to design efficient optical metamaterials for optical inference.
Progress doesn't happen out of thin air. Someone has to go in and do the work, and find out the limits or feasibility of such and such tech. More interest in this is good in the long run, even if the first few iterations don't prove revolutionary.
You can have as many iterations as it takes, you need only one to work. Thousands or millions is fine. If you keep track of what was tried and how hard everything is a gain.
Your perspective only makes sense in the context of machine learning and general purpose computation.
I'm not really seeing any ASICs that are built for running optimizers. Meanwhile the "AOC" is really good at solving unconstrained/equality-constrained quadratic programs.
Analog will ultimately leave binary AI/LLMs in the dustpile. Meaning is not accessible through counting, it will require syntax (parallel differences) and analoga in optic flow. We live in a century of toy experiments, not reasoned task variability/scale invariance.
I think that all real progress in this space will come from materials science and optics, not computer science. The optical-electronic-optical path just doesn’t scale. You can use analog electronics all you like, but what silicon area and energy do they consume, and how does that compare with digital equivalents? How does the parts count scale with the network size and channels? When you put it all together, usually you’re way down on digital. I don’t think this is any different.
Show me a way to keep the network all in optics, all the way through the network - including native positive and negative weights, weights that can actually be larger than 1, and activation functions - without any digital conversion and re-emission.
This architecture appears to overlap with the, then, Bell Labs LambdaXtreme telephone switch. It was sold to Alcatel as it appears to be an optimum design for a swich that covers the E.U. Switching was likely in the control plane, and perhaps an optical fabric was used for administrative changes in both the control and dats pkane.
the AI inference workloads shown in the paper are extremely far from what is implied when one says "... computer for AI inference". No discussion of issues around the memory hierarchy and how the presented architecture solves those. No mention of transformers, except for a vague reference to energy-based models
I really want to believe, but if I had a penny for every time that analog, optical, ternary, clockless, or other radically non-standard computing paradigms were supposed to revolutionize the industry... I'd have a nice pile of pennies.
Electronic signaling is just so marvelously easy to scale that the right path was clear pretty much from day one. We don't have that path for other operating principles right now. As for synchronous operation, binary signaling, and so forth, they're once again just scaling tools that let us crank out designs with billions of transistors without hand-crafting every piece or making the abstractions more leaky than they already are.
A difference now is that moore’s law per power density has been dead a few years and physics says it can’t get much better.
The other difference is that computers are now powerful enough to do the 10^20 calculations required to design efficient optical metamaterials for optical inference.
> [...] and physics says it can’t get much better.
What part of physics do you have in mind?
Probably the lower bound on transistor size
https://en.m.wikipedia.org/wiki/Quantum_tunnelling
kT generated per electron based gate activation.
Progress doesn't happen out of thin air. Someone has to go in and do the work, and find out the limits or feasibility of such and such tech. More interest in this is good in the long run, even if the first few iterations don't prove revolutionary.
You can have as many iterations as it takes, you need only one to work. Thousands or millions is fine. If you keep track of what was tried and how hard everything is a gain.
Your perspective only makes sense in the context of machine learning and general purpose computation.
I'm not really seeing any ASICs that are built for running optimizers. Meanwhile the "AOC" is really good at solving unconstrained/equality-constrained quadratic programs.
naive question: isn't fiber optic cables for communication a counter-example to your thesis?
Fiber optic is a communication medium, not a computation device.
You are right.
> The current hardware includes 16 microLEDs and 16 photodetectors, supporting a 16-variable state vector…
Analog will ultimately leave binary AI/LLMs in the dustpile. Meaning is not accessible through counting, it will require syntax (parallel differences) and analoga in optic flow. We live in a century of toy experiments, not reasoned task variability/scale invariance.
I think that all real progress in this space will come from materials science and optics, not computer science. The optical-electronic-optical path just doesn’t scale. You can use analog electronics all you like, but what silicon area and energy do they consume, and how does that compare with digital equivalents? How does the parts count scale with the network size and channels? When you put it all together, usually you’re way down on digital. I don’t think this is any different.
Show me a way to keep the network all in optics, all the way through the network - including native positive and negative weights, weights that can actually be larger than 1, and activation functions - without any digital conversion and re-emission.
I love it! Just take the computer that was implemented by God Himself (physics) and use them it to calculate your problem almost indefinitely faster.
This architecture appears to overlap with the, then, Bell Labs LambdaXtreme telephone switch. It was sold to Alcatel as it appears to be an optimum design for a swich that covers the E.U. Switching was likely in the control plane, and perhaps an optical fabric was used for administrative changes in both the control and dats pkane.
the AI inference workloads shown in the paper are extremely far from what is implied when one says "... computer for AI inference". No discussion of issues around the memory hierarchy and how the presented architecture solves those. No mention of transformers, except for a vague reference to energy-based models
Spatial light modulators are many orders slower than a cpu clock cycle. How many of these in parallel would be required to compete with an H100?
In performance per watt? One ;)
This is important work. I'd wondered whether optics could do maths, this looks to show it can.
See also: https://en.wikipedia.org/wiki/TWINKLE