1# SpinQuant 2 3This is an implementation of the [Fast Hadamard 4Transform](https://en.wikipedia.org/wiki/Fast_Walsh–Hadamard_transform) 5as used in [SpinQuant](https://arxiv.org/abs/2405.16406) (for the R3 6and R4 matrices), [QuaRot](https://arxiv.org/abs/2404.00456), and 7[Quip#](https://arxiv.org/pdf/2402.04396). We follow those papers' 8method (as implemented in 9https://github.com/Dao-AILab/fast-hadamard-transform/) for extending 10the transform to non-power-of-two input sizes. CUDA is not considered 11because https://github.com/Dao-AILab/fast-hadamard-transform/ is 12already available. 13 14The intended long-term destination for this code is pytorch/ao; it is 15in ExecuTorch temporarily until we get C++ dependency from ExecuTorch 16on torchao figured out. 17