1========================== 2Remote Processor Framework 3========================== 4 5Introduction 6============ 7 8Modern SoCs typically have heterogeneous remote processor devices in asymmetric 9multiprocessing (AMP) configurations, which may be running different instances 10of operating system, whether it's Linux or any other flavor of real-time OS. 11 12OMAP4, for example, has dual Cortex-A9, dual Cortex-M3 and a C64x+ DSP. 13In a typical configuration, the dual cortex-A9 is running Linux in a SMP 14configuration, and each of the other three cores (two M3 cores and a DSP) 15is running its own instance of RTOS in an AMP configuration. 16 17The remoteproc framework allows different platforms/architectures to 18control (power on, load firmware, power off) those remote processors while 19abstracting the hardware differences, so the entire driver doesn't need to be 20duplicated. In addition, this framework also adds rpmsg virtio devices 21for remote processors that supports this kind of communication. This way, 22platform-specific remoteproc drivers only need to provide a few low-level 23handlers, and then all rpmsg drivers will then just work 24(for more information about the virtio-based rpmsg bus and its drivers, 25please read Documentation/staging/rpmsg.rst). 26Registration of other types of virtio devices is now also possible. Firmwares 27just need to publish what kind of virtio devices do they support, and then 28remoteproc will add those devices. This makes it possible to reuse the 29existing virtio drivers with remote processor backends at a minimal development 30cost. 31 32User API 33======== 34 35:: 36 37 int rproc_boot(struct rproc *rproc) 38 39Boot a remote processor (i.e. load its firmware, power it on, ...). 40 41If the remote processor is already powered on, this function immediately 42returns (successfully). 43 44Returns 0 on success, and an appropriate error value otherwise. 45Note: to use this function you should already have a valid rproc 46handle. There are several ways to achieve that cleanly (devres, pdata, 47the way remoteproc_rpmsg.c does this, or, if this becomes prevalent, we 48might also consider using dev_archdata for this). 49 50:: 51 52 void rproc_shutdown(struct rproc *rproc) 53 54Power off a remote processor (previously booted with rproc_boot()). 55In case @rproc is still being used by an additional user(s), then 56this function will just decrement the power refcount and exit, 57without really powering off the device. 58 59Every call to rproc_boot() must (eventually) be accompanied by a call 60to rproc_shutdown(). Calling rproc_shutdown() redundantly is a bug. 61 62.. note:: 63 64 we're not decrementing the rproc's refcount, only the power refcount. 65 which means that the @rproc handle stays valid even after 66 rproc_shutdown() returns, and users can still use it with a subsequent 67 rproc_boot(), if needed. 68 69:: 70 71 struct rproc *rproc_get_by_phandle(phandle phandle) 72 73Find an rproc handle using a device tree phandle. Returns the rproc 74handle on success, and NULL on failure. This function increments 75the remote processor's refcount, so always use rproc_put() to 76decrement it back once rproc isn't needed anymore. 77 78Typical usage 79============= 80 81:: 82 83 #include <linux/remoteproc.h> 84 85 /* in case we were given a valid 'rproc' handle */ 86 int dummy_rproc_example(struct rproc *my_rproc) 87 { 88 int ret; 89 90 /* let's power on and boot our remote processor */ 91 ret = rproc_boot(my_rproc); 92 if (ret) { 93 /* 94 * something went wrong. handle it and leave. 95 */ 96 } 97 98 /* 99 * our remote processor is now powered on... give it some work 100 */ 101 102 /* let's shut it down now */ 103 rproc_shutdown(my_rproc); 104 } 105 106API for implementors 107==================== 108 109:: 110 111 struct rproc *rproc_alloc(struct device *dev, const char *name, 112 const struct rproc_ops *ops, 113 const char *firmware, int len) 114 115Allocate a new remote processor handle, but don't register 116it yet. Required parameters are the underlying device, the 117name of this remote processor, platform-specific ops handlers, 118the name of the firmware to boot this rproc with, and the 119length of private data needed by the allocating rproc driver (in bytes). 120 121This function should be used by rproc implementations during 122initialization of the remote processor. 123 124After creating an rproc handle using this function, and when ready, 125implementations should then call rproc_add() to complete 126the registration of the remote processor. 127 128On success, the new rproc is returned, and on failure, NULL. 129 130.. note:: 131 132 **never** directly deallocate @rproc, even if it was not registered 133 yet. Instead, when you need to unroll rproc_alloc(), use rproc_free(). 134 135:: 136 137 void rproc_free(struct rproc *rproc) 138 139Free an rproc handle that was allocated by rproc_alloc. 140 141This function essentially unrolls rproc_alloc(), by decrementing the 142rproc's refcount. It doesn't directly free rproc; that would happen 143only if there are no other references to rproc and its refcount now 144dropped to zero. 145 146:: 147 148 int rproc_add(struct rproc *rproc) 149 150Register @rproc with the remoteproc framework, after it has been 151allocated with rproc_alloc(). 152 153This is called by the platform-specific rproc implementation, whenever 154a new remote processor device is probed. 155 156Returns 0 on success and an appropriate error code otherwise. 157Note: this function initiates an asynchronous firmware loading 158context, which will look for virtio devices supported by the rproc's 159firmware. 160 161If found, those virtio devices will be created and added, so as a result 162of registering this remote processor, additional virtio drivers might get 163probed. 164 165:: 166 167 int rproc_del(struct rproc *rproc) 168 169Unroll rproc_add(). 170 171This function should be called when the platform specific rproc 172implementation decides to remove the rproc device. it should 173_only_ be called if a previous invocation of rproc_add() 174has completed successfully. 175 176After rproc_del() returns, @rproc is still valid, and its 177last refcount should be decremented by calling rproc_free(). 178 179Returns 0 on success and -EINVAL if @rproc isn't valid. 180 181:: 182 183 void rproc_report_crash(struct rproc *rproc, enum rproc_crash_type type) 184 185Report a crash in a remoteproc 186 187This function must be called every time a crash is detected by the 188platform specific rproc implementation. This should not be called from a 189non-remoteproc driver. This function can be called from atomic/interrupt 190context. 191 192Implementation callbacks 193======================== 194 195These callbacks should be provided by platform-specific remoteproc 196drivers:: 197 198 /** 199 * struct rproc_ops - platform-specific device handlers 200 * @start: power on the device and boot it 201 * @stop: power off the device 202 * @kick: kick a virtqueue (virtqueue id given as a parameter) 203 */ 204 struct rproc_ops { 205 int (*start)(struct rproc *rproc); 206 int (*stop)(struct rproc *rproc); 207 void (*kick)(struct rproc *rproc, int vqid); 208 }; 209 210Every remoteproc implementation should at least provide the ->start and ->stop 211handlers. If rpmsg/virtio functionality is also desired, then the ->kick handler 212should be provided as well. 213 214The ->start() handler takes an rproc handle and should then power on the 215device and boot it (use rproc->priv to access platform-specific private data). 216The boot address, in case needed, can be found in rproc->bootaddr (remoteproc 217core puts there the ELF entry point). 218On success, 0 should be returned, and on failure, an appropriate error code. 219 220The ->stop() handler takes an rproc handle and powers the device down. 221On success, 0 is returned, and on failure, an appropriate error code. 222 223The ->kick() handler takes an rproc handle, and an index of a virtqueue 224where new message was placed in. Implementations should interrupt the remote 225processor and let it know it has pending messages. Notifying remote processors 226the exact virtqueue index to look in is optional: it is easy (and not 227too expensive) to go through the existing virtqueues and look for new buffers 228in the used rings. 229 230Binary Firmware Structure 231========================= 232 233At this point remoteproc supports ELF32 and ELF64 firmware binaries. However, 234it is quite expected that other platforms/devices which we'd want to 235support with this framework will be based on different binary formats. 236 237When those use cases show up, we will have to decouple the binary format 238from the framework core, so we can support several binary formats without 239duplicating common code. 240 241When the firmware is parsed, its various segments are loaded to memory 242according to the specified device address (might be a physical address 243if the remote processor is accessing memory directly). 244 245In addition to the standard ELF segments, most remote processors would 246also include a special section which we call "the resource table". 247 248The resource table contains system resources that the remote processor 249requires before it should be powered on, such as allocation of physically 250contiguous memory, or iommu mapping of certain on-chip peripherals. 251Remotecore will only power up the device after all the resource table's 252requirement are met. 253 254In addition to system resources, the resource table may also contain 255resource entries that publish the existence of supported features 256or configurations by the remote processor, such as trace buffers and 257supported virtio devices (and their configurations). 258 259The resource table begins with this header:: 260 261 /** 262 * struct resource_table - firmware resource table header 263 * @ver: version number 264 * @num: number of resource entries 265 * @reserved: reserved (must be zero) 266 * @offset: array of offsets pointing at the various resource entries 267 * 268 * The header of the resource table, as expressed by this structure, 269 * contains a version number (should we need to change this format in the 270 * future), the number of available resource entries, and their offsets 271 * in the table. 272 */ 273 struct resource_table { 274 u32 ver; 275 u32 num; 276 u32 reserved[2]; 277 u32 offset[0]; 278 } __packed; 279 280Immediately following this header are the resource entries themselves, 281each of which begins with the following resource entry header:: 282 283 /** 284 * struct fw_rsc_hdr - firmware resource entry header 285 * @type: resource type 286 * @data: resource data 287 * 288 * Every resource entry begins with a 'struct fw_rsc_hdr' header providing 289 * its @type. The content of the entry itself will immediately follow 290 * this header, and it should be parsed according to the resource type. 291 */ 292 struct fw_rsc_hdr { 293 u32 type; 294 u8 data[0]; 295 } __packed; 296 297Some resources entries are mere announcements, where the host is informed 298of specific remoteproc configuration. Other entries require the host to 299do something (e.g. allocate a system resource). Sometimes a negotiation 300is expected, where the firmware requests a resource, and once allocated, 301the host should provide back its details (e.g. address of an allocated 302memory region). 303 304Here are the various resource types that are currently supported:: 305 306 /** 307 * enum fw_resource_type - types of resource entries 308 * 309 * @RSC_CARVEOUT: request for allocation of a physically contiguous 310 * memory region. 311 * @RSC_DEVMEM: request to iommu_map a memory-based peripheral. 312 * @RSC_TRACE: announces the availability of a trace buffer into which 313 * the remote processor will be writing logs. 314 * @RSC_VDEV: declare support for a virtio device, and serve as its 315 * virtio header. 316 * @RSC_LAST: just keep this one at the end 317 * @RSC_VENDOR_START: start of the vendor specific resource types range 318 * @RSC_VENDOR_END: end of the vendor specific resource types range 319 * 320 * Please note that these values are used as indices to the rproc_handle_rsc 321 * lookup table, so please keep them sane. Moreover, @RSC_LAST is used to 322 * check the validity of an index before the lookup table is accessed, so 323 * please update it as needed. 324 */ 325 enum fw_resource_type { 326 RSC_CARVEOUT = 0, 327 RSC_DEVMEM = 1, 328 RSC_TRACE = 2, 329 RSC_VDEV = 3, 330 RSC_LAST = 4, 331 RSC_VENDOR_START = 128, 332 RSC_VENDOR_END = 512, 333 }; 334 335For more details regarding a specific resource type, please see its 336dedicated structure in include/linux/remoteproc.h. 337 338We also expect that platform-specific resource entries will show up 339at some point. When that happens, we could easily add a new RSC_PLATFORM 340type, and hand those resources to the platform-specific rproc driver to handle. 341 342Virtio and remoteproc 343===================== 344 345The firmware should provide remoteproc information about virtio devices 346that it supports, and their configurations: a RSC_VDEV resource entry 347should specify the virtio device id (as in virtio_ids.h), virtio features, 348virtio config space, vrings information, etc. 349 350When a new remote processor is registered, the remoteproc framework 351will look for its resource table and will register the virtio devices 352it supports. A firmware may support any number of virtio devices, and 353of any type (a single remote processor can also easily support several 354rpmsg virtio devices this way, if desired). 355 356Of course, RSC_VDEV resource entries are only good enough for static 357allocation of virtio devices. Dynamic allocations will also be made possible 358using the rpmsg bus (similar to how we already do dynamic allocations of 359rpmsg channels; read more about it in rpmsg.txt). 360