• Home
  • Line#
  • Scopes#
  • Navigate#
  • Raw
  • Download
1gemmlowp: a small self-contained low-precision GEMM library
2===========================================================
3
4This is not a full linear algebra library, only a GEMM library: it only does
5general matrix multiplication ("GEMM").
6
7The meaning of "low precision" is detailed in this document:
8  doc/low-precision.txt
9
10Some of the general design is explained in
11  doc/design.txt
12
13
14Disclaimer
15==========
16
17This is not an official Google product (experimental or otherwise), it is just
18code that happens to be owned by Google.
19
20
21Mailing list
22============
23
24gemmlowp-related discussion, about either development or usage, is welcome
25on this Google Group (mailing list / forum):
26
27  https://groups.google.com/forum/#!forum/gemmlowp
28
29
30Portability, target platforms/architectures
31===========================================
32
33Should be portable to any platform with some C++11 and POSIX support,
34while we have optional optimized code paths for specific architectures.
35
36Required:
37  C++11 (a small conservative subset of it)
38
39Required for some features:
40  * Some POSIX interfaces:
41    * pthreads (for multi-threaded operation and for profiling).
42    * sysconf (for multi-threaded operation to detect number of cores;
43               may be bypassed).
44
45Optional:
46  Architecture-specific code paths use intrinsics or inline assembly.
47  See "Architecture-specific optimized code paths" below.
48
49Architecture-specific optimized code paths
50==========================================
51
52We have some optimized code paths for specific instruction sets.
53Some are written in inline assembly, some are written in C++ using
54intrinsics. Both GCC and Clang are supported.
55
56At the moment, we have a full set of optimized code paths (kernels,
57packing and unpacking paths) only for ARM NEON, supporting both
58ARMv7 (32bit) and ARMv8 (64bit).
59
60We also have a partial set of optimized code paths (only kernels
61at the moment) for Intel SSE. It supports both x86 and x86-64 but
62only targets SSE4. The lack of packing/unpacking code paths means
63that performance isn't optimal yet.
64
65Details of what it takes to make an efficient port of gemmlowp, namely
66writing a suitable GEMM kernel and accompanying packing code, are
67explained in this file:
68  doc/kernels.txt
69
70
71Public interfaces
72=================
73
741. gemmlowp public interface
75----------------------------
76
77  gemmlowp's main public interface is in the public/ subdirectory. The
78  header to include is
79      public/gemmlowp.h.
80  This is a headers-only library, so there is nothing to link to.
81
822. EightBitIntGemm standard interface
83-------------------------------------
84
85  Additionally, the eight_bit_int_gemm/ subdirectory provides an
86  implementation of the standard EightBitIntGemm interface. The header
87  to include is
88    eight_bit_int_gemm/eight_bit_int_gemm.h
89  This is *NOT* a headers-only library, users need to link to
90    eight_bit_int_gemm/eight_bit_int_gemm.cc.
91  The API is similar to the standard BLAS GEMM interface, and implements
92  C = A * B. If the transpose flags for a matrix argument are false, its memory
93  order is treated as column major, and row major if its true.
94
95
96Building
97========
98
99Building by manually invoking your compiler
100-------------------------------------------
101
102Because gemmlowp is so simple, working with it involves only
103single-command-line compiler invokations. Therefore we expect that
104most people working with gemmlowp will either manually invoke their
105compiler, or write their own rules for their own preferred build
106system.
107
108Keep in mind (previous section) that gemmlowp itself is a pure-headers-only
109library so there is nothing to build, and the eight_bit_int_gemm library
110consists of a single eight_bit_int_gemm.cc file to build.
111
112For a Android gemmlowp development workflow, the scripts/ directory
113contains a script to build and run a program on an Android device:
114  scripts/test-android.sh
115
116Building using Bazel
117--------------------
118
119That being said, we also maintain a Bazel BUILD system as part of
120gemmlowp. Its usage is not mandatory at all and is only one
121possible way that gemmlowp libraries and tests may be built. If
122you are interested, Bazel's home page is
123  http://bazel.io/
124And you can get started with using Bazel to build gemmlowp targets
125by first creating an empty WORKSPACE file in a parent directory,
126for instance:
127
128$ cd gemmlowp/..  # change to parent directory containing gemmlowp/
129$ touch WORKSPACE # declare that to be our workspace root
130$ bazel build gemmlowp:all
131
132
133Testing
134=======
135
136Testing by manually building and running tests
137----------------------------------------------
138
139The test/ directory contains unit tests. The primary unit test is
140  test/test.cc
141Since it covers also the EightBitIntGemm interface, it needs to be
142linked against
143  eight_bit_int_gemm/eight_bit_int_gemm.cc
144It also uses realistic data captured from a neural network run in
145  test/test_data.cc
146
147Thus you'll want to pass the following list of source files to your
148compiler/linker:
149  test/test.cc
150  eight_bit_int_gemm/eight_bit_int_gemm.cc
151  test/test_data.cc
152
153The scripts/ directory contains a script to build and run a program
154on an Android device:
155  scripts/test-android.sh
156
157It expects the CXX environment variable to point to an Android toolchain's
158C++ compiler, and expects source files (and optionally, cflags) as
159command-line parameters. To build and run the above-mentioned main unit test,
160first set CXX e.g.:
161
162$ export CXX=/some/toolchains/arm-linux-androideabi-4.8/bin/arm-linux-androideabi-g++
163
164Then run:
165
166$ ./scripts/test-android.sh \
167test/test.cc \
168eight_bit_int_gemm/eight_bit_int_gemm.cc \
169test/test_data.cc
170
171
172Testing using Bazel
173-------------------
174
175Alternatively, you can use Bazel to build and run tests. See the Bazel
176instruction in the above section on building. Once your Bazel workspace
177is set up, you can for instance do:
178
179$ bazel test gemmlowp:all
180
181
182Troubleshooting Compilation
183===========================
184
185If you're having trouble finding the compiler, follow these instructions to
186build a standalone toolchain:
187https://developer.android.com/ndk/guides/standalone_toolchain.html
188
189Here's an example of setting up Clang 3.5:
190
191$ export INSTALL_DIR=~/toolchains/clang-21-stl-gnu
192$ $NDK/build/tools/make-standalone-toolchain.sh \
193--toolchain=arm-linux-androideabi-clang3.5 --platform=android-21 \
194--install-dir=$INSTALL_DIR
195$ export CXX="$INSTALL_DIR/bin/arm-linux-androideabi-g++ \
196--sysroot=$INSTALL_DIR/sysroot"
197
198Some compilers (e.g. the default clang++ in the same bin directory) don't
199support NEON assembly. The benchmark build process will issue a warning if
200support isn't detected, and you should make sure you're using a compiler like
201arm-linux-androideabi-g++ that does include NEON.
202
203
204Benchmarking
205============
206
207The main benchmark is
208  benchmark.cc
209It doesn't need to be linked to any
210other source file. We recommend building with assertions disabled (-DNDEBUG).
211
212For example, the benchmark can be built and run on an Android device by doing:
213
214$ ./scripts/test-android.sh test/benchmark.cc -DNDEBUG
215
216If GEMMLOWP_TEST_PROFILE is defined then the benchmark will be built with
217profiling instrumentation (which makes it slower) and will dump profiles.
218See next section on profiling.
219
220
221Profiling
222=========
223
224The profiling/ subdirectory offers a very simple non-interrupting sampling
225profiler that only requires pthreads (no signals).
226
227It relies on source code being instrumented with pseudo-stack labels.
228See profiling/instrumentation.h.
229A full example of using this profiler is given in profiling/profiler.h.
230
231
232Contributing
233============
234
235Contribution-related discussion is always welcome on the gemmlowp
236mailing list (see above).
237
238We try to keep a current list of TODO items in the todo/ directory.
239Prospective contributors are welcome to pick one to work on, and
240communicate about it on the gemmlowp mailing list.
241
242Details of the contributing process, including legalese, are in CONTRIBUTING.
243
244Performance goals
245=================
246
247Our performance goals differ from typical GEMM performance goals in the
248following ways:
249
2501. We care not only about speed, but also about minimizing power usage.
251   We specifically care about charge usage in mobile/embedded devices.
252   This implies that we care doubly about minimizing memory bandwidth usage:
253   we care about it, like any GEMM, because of the impact on speed, and we
254   also care about it because it is a key factor of power usage.
255
2562. Most GEMMs are optimized primarily for large dense matrix sizes (>= 1000).
257   We do care about large sizes, but we also care specifically about the
258   typically smaller matrix sizes encountered in various mobile applications.
259   This means that we have to optimize for all sizes, not just for large enough
260   sizes.
261