• Home
Name Date Size #Lines LOC

..--

functions/04-Jul-2025-1,528989

utils/04-Jul-2025-380280

FunctionsManual.cppD04-Jul-2025246.9 KiB7,1894,957

FunctionsManual.hD04-Jul-202531.8 KiB1,1161,079

InferenceMode.hD04-Jul-2025156 116

README.mdD04-Jul-20251.8 KiB3428

TraceTypeManual.cppD04-Jul-202510.6 KiB299256

VariableTypeManual.cppD04-Jul-202517.5 KiB563459

VariableTypeUtils.hD04-Jul-202514.2 KiB441364

anomaly_mode.cppD04-Jul-20252.2 KiB7861

anomaly_mode.hD04-Jul-20251.7 KiB7238

autograd.cppD04-Jul-20256.4 KiB218193

autograd.hD04-Jul-20255.2 KiB10521

autograd_meta.cppD04-Jul-202511.5 KiB318171

autograd_not_implemented_fallback.cppD04-Jul-202525.1 KiB633481

autograd_not_implemented_fallback.hD04-Jul-20251.1 KiB3314

cpp_hook.cppD04-Jul-20251.9 KiB6857

cpp_hook.hD04-Jul-2025865 3022

custom_function.cppD04-Jul-202522.2 KiB586435

custom_function.hD04-Jul-202517.6 KiB486315

edge.hD04-Jul-20251.6 KiB5734

engine.cppD04-Jul-202561.6 KiB1,6741,047

engine.hD04-Jul-202510.5 KiB289167

forward_grad.cppD04-Jul-20252.5 KiB7960

forward_grad.hD04-Jul-20258.7 KiB21175

function.cppD04-Jul-20253.3 KiB11366

function.hD04-Jul-202529 KiB764385

function_hook.hD04-Jul-20252 KiB6549

grad_mode.hD04-Jul-2025210 127

graph_task.hD04-Jul-20259 KiB227105

init.cppD04-Jul-202547.5 KiB1,4161,257

input_buffer.cppD04-Jul-20258.9 KiB249165

input_buffer.hD04-Jul-20251.4 KiB4626

input_metadata.cppD04-Jul-20256.3 KiB205159

input_metadata.hD04-Jul-20252.9 KiB11479

jit_decomp_interface.cppD04-Jul-2025307 1812

jit_decomp_interface.hD04-Jul-20251.8 KiB5121

profiler.hD04-Jul-2025112 53

profiler_kineto.cppD04-Jul-202536.5 KiB1,084903

profiler_kineto.hD04-Jul-20257.8 KiB219128

profiler_legacy.cppD04-Jul-202522.9 KiB679510

profiler_legacy.hD04-Jul-202510.4 KiB402299

profiler_python.cppD04-Jul-202538.7 KiB1,145885

profiler_python.hD04-Jul-202584 84

python_anomaly_mode.cppD04-Jul-20253.8 KiB128104

python_anomaly_mode.hD04-Jul-20251.1 KiB4333

python_autograd.hD04-Jul-2025375 1811

python_cpp_function.cppD04-Jul-202511.8 KiB400336

python_cpp_function.hD04-Jul-20254.7 KiB10992

python_engine.cppD04-Jul-202517.6 KiB520439

python_engine.hD04-Jul-20251.2 KiB4535

python_enum_tag.hD04-Jul-2025120 85

python_fft_functions.hD04-Jul-202587 84

python_function.cppD04-Jul-202561.2 KiB1,8421,522

python_function.hD04-Jul-20255.4 KiB16184

python_hook.cppD04-Jul-202510.7 KiB353270

python_hook.hD04-Jul-20252 KiB5643

python_legacy_variable.cppD04-Jul-20254.9 KiB166137

python_legacy_variable.hD04-Jul-2025257 135

python_linalg_functions.hD04-Jul-202590 84

python_nested_functions.hD04-Jul-2025164 105

python_nested_functions_manual.cppD04-Jul-20251.3 KiB4334

python_nn_functions.hD04-Jul-202586 84

python_saved_variable_hooks.cppD04-Jul-20253 KiB9171

python_saved_variable_hooks.hD04-Jul-2025933 3426

python_sparse_functions.hD04-Jul-202590 84

python_special_functions.hD04-Jul-202591 84

python_torch_functions.hD04-Jul-2025650 2618

python_torch_functions_manual.cppD04-Jul-202530.3 KiB803704

python_variable.cppD04-Jul-202584.8 KiB2,3821,786

python_variable.hD04-Jul-20253.4 KiB11583

python_variable_indexing.cppD04-Jul-202518.5 KiB549453

python_variable_indexing.hD04-Jul-20252.7 KiB10083

record_function_ops.cppD04-Jul-20256.2 KiB167122

record_function_ops.hD04-Jul-2025935 2818

saved_variable.cppD04-Jul-202510.5 KiB288195

saved_variable.hD04-Jul-20254.5 KiB12157

saved_variable_hooks.hD04-Jul-2025296 149

symbolic.hD04-Jul-2025300 1711

variable.cppD04-Jul-202533.4 KiB910696

variable.hD04-Jul-202539.2 KiB949416

variable_info.cppD04-Jul-2025844 3325

variable_info.hD04-Jul-2025481 2215

README.md

1## Autograd
2
3Autograd is a hotspot for PyTorch performance, so most of the heavy lifting is
4implemented in C++. This implies that we have to do some shuffling between
5Python and C++; and in general, we want data to be in a form that is convenient
6to manipulate from C++.
7
8Our general model is that for any key data type that autograd manipulates,
9there are two implementations: a C++ type and a Python object type.  For
10example, consider variables in autograd: we have both `Variable` in `variable.h`
11(the C++ type) and `THPVariable` in `python_variable.h` (the Python type.)
12(By the way, THP stands for TorcH Python, not to be confused with THPP, TorcH
13C++).  `Variable` contains the payload of a variable, while `THPVariable` just
14contains a `shared_ptr` reference to `Variable`, as well as references to other
15Python objects which the Python runtime needs to know about.  A lot of
16data accessor implementations in `python_variable.cpp` simply reach through
17to the underlying `Variable` and return the appropriate value.
18
19The most complicated application of this principle is Function, which also
20supports users implementing custom behavior in Python.  We have the following
21classes:
22
23* `Node` in `function.h`, the C++ type.
24* `THPFunction` in `python_function.h`, the Python object type.  In
25  `python_function.cpp`, you can see the boilerplate that tells the Python
26  interpreter about this object.
27* `PyNode` in `python_function.h`, a subclass of `Node` which forwards
28  `apply` to a Python `THPFunction`. (NOT a Python object, despite its name!)
29
30Outside of `PyNode`, the C++ objects largely avoid referencing Python
31objects (there are a few exceptions, like `pyobj` in `Variable`, and
32`PyNode`, whose whole point is to let C++ call into Python). And `pyobj`
33in `Node` to ensure uniqueness of the associated python wrapper (if it exists).
34