This was used in his shape aware language FiSh, for dealing with multidimensional arrays. Shape compatibilities were statically type checked, if I recall correctly. Shapes were also used to optimize the loops.
I wonder… I know Eigen has some tricks it can do when the size of a matrix is known beforehand. The obvious example, 4x4 matrix inverse gets special treatment. I assume they also be smart about loop unrolling, that sort of stuff.
Anything similar in here?
If not—actually, optimizing compilers are pretty okay nowadays anyway. I wonder if you’ve tried just seeing what Rust will do automatically with different optimization levels?
Glowstick just provides the shape types and associated traits as a layer you can put on top of another tensor implementation. Since it's just verifying shapes and forwarding the operations to the underlying tensor (e.g. from candle/burn), I don't think there's any great way to get performance benefits from these integrations. It's mainly about the developer experience- getting errors at compile time vs runtime, checking shapes, etc.
That being said, it seems reasonable that you could make some optimizations like this if you had deeper integration of these types with a framework or similar. It's not something I've explored personally, sounds interesting though.
Very interesting work! Starred the project. Would love to see such features integrated into the compiler itself
Anyway, fully agree with you on the complementarity of ML and Rust
Could you take a look at Barry Jay's shape theory.
https://web.archive.org/web/20111015133833/http://www-staff....
This was used in his shape aware language FiSh, for dealing with multidimensional arrays. Shape compatibilities were statically type checked, if I recall correctly. Shapes were also used to optimize the loops.
[Programming in FISh] https://link.springer.com/article/10.1007/s100090050037
[Towards Dynamic Shaping] https://www.researchgate.net/publication/265975794_Towards_D...
It's cool to see that others have explored some of these concepts more generally, thanks for the links!
The glowstick shape is also a type-level list of integers, and I could definitely see how other shapes might be useful in different situations.
Doing this sort of thing in Rust is a bit of a stretch for sure, but it makes the outcome easy to apply which is nice.
I wonder… I know Eigen has some tricks it can do when the size of a matrix is known beforehand. The obvious example, 4x4 matrix inverse gets special treatment. I assume they also be smart about loop unrolling, that sort of stuff.
Anything similar in here?
If not—actually, optimizing compilers are pretty okay nowadays anyway. I wonder if you’ve tried just seeing what Rust will do automatically with different optimization levels?
Glowstick just provides the shape types and associated traits as a layer you can put on top of another tensor implementation. Since it's just verifying shapes and forwarding the operations to the underlying tensor (e.g. from candle/burn), I don't think there's any great way to get performance benefits from these integrations. It's mainly about the developer experience- getting errors at compile time vs runtime, checking shapes, etc.
That being said, it seems reasonable that you could make some optimizations like this if you had deeper integration of these types with a framework or similar. It's not something I've explored personally, sounds interesting though.
Very interesting work! Starred the project. Would love to see such features integrated into the compiler itself Anyway, fully agree with you on the complementarity of ML and Rust