1Remote Processor Framework 2 31. Introduction 4 5Modern SoCs typically have heterogeneous remote processor devices in asymmetric 6multiprocessing (AMP) configurations, which may be running different instances 7of operating system, whether it's Linux or any other flavor of real-time OS. 8 9OMAP4, for example, has dual Cortex-A9, dual Cortex-M3 and a C64x+ DSP. 10In a typical configuration, the dual cortex-A9 is running Linux in a SMP 11configuration, and each of the other three cores (two M3 cores and a DSP) 12is running its own instance of RTOS in an AMP configuration. 13 14The remoteproc framework allows different platforms/architectures to 15control (power on, load firmware, power off) those remote processors while 16abstracting the hardware differences, so the entire driver doesn't need to be 17duplicated. In addition, this framework also adds rpmsg virtio devices 18for remote processors that supports this kind of communication. This way, 19platform-specific remoteproc drivers only need to provide a few low-level 20handlers, and then all rpmsg drivers will then just work 21(for more information about the virtio-based rpmsg bus and its drivers, 22please read Documentation/rpmsg.txt). 23Registration of other types of virtio devices is now also possible. Firmwares 24just need to publish what kind of virtio devices do they support, and then 25remoteproc will add those devices. This makes it possible to reuse the 26existing virtio drivers with remote processor backends at a minimal development 27cost. 28 292. User API 30 31 int rproc_boot(struct rproc *rproc) 32 - Boot a remote processor (i.e. load its firmware, power it on, ...). 33 If the remote processor is already powered on, this function immediately 34 returns (successfully). 35 Returns 0 on success, and an appropriate error value otherwise. 36 Note: to use this function you should already have a valid rproc 37 handle. There are several ways to achieve that cleanly (devres, pdata, 38 the way remoteproc_rpmsg.c does this, or, if this becomes prevalent, we 39 might also consider using dev_archdata for this). See also 40 rproc_get_by_name() below. 41 42 void rproc_shutdown(struct rproc *rproc) 43 - Power off a remote processor (previously booted with rproc_boot()). 44 In case @rproc is still being used by an additional user(s), then 45 this function will just decrement the power refcount and exit, 46 without really powering off the device. 47 Every call to rproc_boot() must (eventually) be accompanied by a call 48 to rproc_shutdown(). Calling rproc_shutdown() redundantly is a bug. 49 Notes: 50 - we're not decrementing the rproc's refcount, only the power refcount. 51 which means that the @rproc handle stays valid even after 52 rproc_shutdown() returns, and users can still use it with a subsequent 53 rproc_boot(), if needed. 54 - don't call rproc_shutdown() to unroll rproc_get_by_name(), exactly 55 because rproc_shutdown() _does not_ decrement the refcount of @rproc. 56 To decrement the refcount of @rproc, use rproc_put() (but _only_ if 57 you acquired @rproc using rproc_get_by_name()). 58 59 struct rproc *rproc_get_by_name(const char *name) 60 - Find an rproc handle using the remote processor's name, and then 61 boot it. If it's already powered on, then just immediately return 62 (successfully). Returns the rproc handle on success, and NULL on failure. 63 This function increments the remote processor's refcount, so always 64 use rproc_put() to decrement it back once rproc isn't needed anymore. 65 Note: currently rproc_get_by_name() and rproc_put() are not used anymore 66 by the rpmsg bus and its drivers. We need to scrutinize the use cases 67 that still need them, and see if we can migrate them to use the non 68 name-based boot/shutdown interface. 69 70 void rproc_put(struct rproc *rproc) 71 - Decrement @rproc's power refcount and shut it down if it reaches zero 72 (essentially by just calling rproc_shutdown), and then decrement @rproc's 73 validity refcount too. 74 After this function returns, @rproc may _not_ be used anymore, and its 75 handle should be considered invalid. 76 This function should be called _iff_ the @rproc handle was grabbed by 77 calling rproc_get_by_name(). 78 793. Typical usage 80 81#include <linux/remoteproc.h> 82 83/* in case we were given a valid 'rproc' handle */ 84int dummy_rproc_example(struct rproc *my_rproc) 85{ 86 int ret; 87 88 /* let's power on and boot our remote processor */ 89 ret = rproc_boot(my_rproc); 90 if (ret) { 91 /* 92 * something went wrong. handle it and leave. 93 */ 94 } 95 96 /* 97 * our remote processor is now powered on... give it some work 98 */ 99 100 /* let's shut it down now */ 101 rproc_shutdown(my_rproc); 102} 103 1044. API for implementors 105 106 struct rproc *rproc_alloc(struct device *dev, const char *name, 107 const struct rproc_ops *ops, 108 const char *firmware, int len) 109 - Allocate a new remote processor handle, but don't register 110 it yet. Required parameters are the underlying device, the 111 name of this remote processor, platform-specific ops handlers, 112 the name of the firmware to boot this rproc with, and the 113 length of private data needed by the allocating rproc driver (in bytes). 114 115 This function should be used by rproc implementations during 116 initialization of the remote processor. 117 After creating an rproc handle using this function, and when ready, 118 implementations should then call rproc_register() to complete 119 the registration of the remote processor. 120 On success, the new rproc is returned, and on failure, NULL. 121 122 Note: _never_ directly deallocate @rproc, even if it was not registered 123 yet. Instead, if you just need to unroll rproc_alloc(), use rproc_free(). 124 125 void rproc_free(struct rproc *rproc) 126 - Free an rproc handle that was allocated by rproc_alloc. 127 This function should _only_ be used if @rproc was only allocated, 128 but not registered yet. 129 If @rproc was already successfully registered (by calling 130 rproc_register()), then use rproc_unregister() instead. 131 132 int rproc_register(struct rproc *rproc) 133 - Register @rproc with the remoteproc framework, after it has been 134 allocated with rproc_alloc(). 135 This is called by the platform-specific rproc implementation, whenever 136 a new remote processor device is probed. 137 Returns 0 on success and an appropriate error code otherwise. 138 Note: this function initiates an asynchronous firmware loading 139 context, which will look for virtio devices supported by the rproc's 140 firmware. 141 If found, those virtio devices will be created and added, so as a result 142 of registering this remote processor, additional virtio drivers might get 143 probed. 144 145 int rproc_unregister(struct rproc *rproc) 146 - Unregister a remote processor, and decrement its refcount. 147 If its refcount drops to zero, then @rproc will be freed. If not, 148 it will be freed later once the last reference is dropped. 149 150 This function should be called when the platform specific rproc 151 implementation decides to remove the rproc device. it should 152 _only_ be called if a previous invocation of rproc_register() 153 has completed successfully. 154 155 After rproc_unregister() returns, @rproc is _not_ valid anymore and 156 it shouldn't be used. More specifically, don't call rproc_free() 157 or try to directly free @rproc after rproc_unregister() returns; 158 none of these are needed, and calling them is a bug. 159 160 Returns 0 on success and -EINVAL if @rproc isn't valid. 161 1625. Implementation callbacks 163 164These callbacks should be provided by platform-specific remoteproc 165drivers: 166 167/** 168 * struct rproc_ops - platform-specific device handlers 169 * @start: power on the device and boot it 170 * @stop: power off the device 171 * @kick: kick a virtqueue (virtqueue id given as a parameter) 172 */ 173struct rproc_ops { 174 int (*start)(struct rproc *rproc); 175 int (*stop)(struct rproc *rproc); 176 void (*kick)(struct rproc *rproc, int vqid); 177}; 178 179Every remoteproc implementation should at least provide the ->start and ->stop 180handlers. If rpmsg/virtio functionality is also desired, then the ->kick handler 181should be provided as well. 182 183The ->start() handler takes an rproc handle and should then power on the 184device and boot it (use rproc->priv to access platform-specific private data). 185The boot address, in case needed, can be found in rproc->bootaddr (remoteproc 186core puts there the ELF entry point). 187On success, 0 should be returned, and on failure, an appropriate error code. 188 189The ->stop() handler takes an rproc handle and powers the device down. 190On success, 0 is returned, and on failure, an appropriate error code. 191 192The ->kick() handler takes an rproc handle, and an index of a virtqueue 193where new message was placed in. Implementations should interrupt the remote 194processor and let it know it has pending messages. Notifying remote processors 195the exact virtqueue index to look in is optional: it is easy (and not 196too expensive) to go through the existing virtqueues and look for new buffers 197in the used rings. 198 1996. Binary Firmware Structure 200 201At this point remoteproc only supports ELF32 firmware binaries. However, 202it is quite expected that other platforms/devices which we'd want to 203support with this framework will be based on different binary formats. 204 205When those use cases show up, we will have to decouple the binary format 206from the framework core, so we can support several binary formats without 207duplicating common code. 208 209When the firmware is parsed, its various segments are loaded to memory 210according to the specified device address (might be a physical address 211if the remote processor is accessing memory directly). 212 213In addition to the standard ELF segments, most remote processors would 214also include a special section which we call "the resource table". 215 216The resource table contains system resources that the remote processor 217requires before it should be powered on, such as allocation of physically 218contiguous memory, or iommu mapping of certain on-chip peripherals. 219Remotecore will only power up the device after all the resource table's 220requirement are met. 221 222In addition to system resources, the resource table may also contain 223resource entries that publish the existence of supported features 224or configurations by the remote processor, such as trace buffers and 225supported virtio devices (and their configurations). 226 227The resource table begins with this header: 228 229/** 230 * struct resource_table - firmware resource table header 231 * @ver: version number 232 * @num: number of resource entries 233 * @reserved: reserved (must be zero) 234 * @offset: array of offsets pointing at the various resource entries 235 * 236 * The header of the resource table, as expressed by this structure, 237 * contains a version number (should we need to change this format in the 238 * future), the number of available resource entries, and their offsets 239 * in the table. 240 */ 241struct resource_table { 242 u32 ver; 243 u32 num; 244 u32 reserved[2]; 245 u32 offset[0]; 246} __packed; 247 248Immediately following this header are the resource entries themselves, 249each of which begins with the following resource entry header: 250 251/** 252 * struct fw_rsc_hdr - firmware resource entry header 253 * @type: resource type 254 * @data: resource data 255 * 256 * Every resource entry begins with a 'struct fw_rsc_hdr' header providing 257 * its @type. The content of the entry itself will immediately follow 258 * this header, and it should be parsed according to the resource type. 259 */ 260struct fw_rsc_hdr { 261 u32 type; 262 u8 data[0]; 263} __packed; 264 265Some resources entries are mere announcements, where the host is informed 266of specific remoteproc configuration. Other entries require the host to 267do something (e.g. allocate a system resource). Sometimes a negotiation 268is expected, where the firmware requests a resource, and once allocated, 269the host should provide back its details (e.g. address of an allocated 270memory region). 271 272Here are the various resource types that are currently supported: 273 274/** 275 * enum fw_resource_type - types of resource entries 276 * 277 * @RSC_CARVEOUT: request for allocation of a physically contiguous 278 * memory region. 279 * @RSC_DEVMEM: request to iommu_map a memory-based peripheral. 280 * @RSC_TRACE: announces the availability of a trace buffer into which 281 * the remote processor will be writing logs. 282 * @RSC_VDEV: declare support for a virtio device, and serve as its 283 * virtio header. 284 * @RSC_LAST: just keep this one at the end 285 * 286 * Please note that these values are used as indices to the rproc_handle_rsc 287 * lookup table, so please keep them sane. Moreover, @RSC_LAST is used to 288 * check the validity of an index before the lookup table is accessed, so 289 * please update it as needed. 290 */ 291enum fw_resource_type { 292 RSC_CARVEOUT = 0, 293 RSC_DEVMEM = 1, 294 RSC_TRACE = 2, 295 RSC_VDEV = 3, 296 RSC_LAST = 4, 297}; 298 299For more details regarding a specific resource type, please see its 300dedicated structure in include/linux/remoteproc.h. 301 302We also expect that platform-specific resource entries will show up 303at some point. When that happens, we could easily add a new RSC_PLATFORM 304type, and hand those resources to the platform-specific rproc driver to handle. 305 3067. Virtio and remoteproc 307 308The firmware should provide remoteproc information about virtio devices 309that it supports, and their configurations: a RSC_VDEV resource entry 310should specify the virtio device id (as in virtio_ids.h), virtio features, 311virtio config space, vrings information, etc. 312 313When a new remote processor is registered, the remoteproc framework 314will look for its resource table and will register the virtio devices 315it supports. A firmware may support any number of virtio devices, and 316of any type (a single remote processor can also easily support several 317rpmsg virtio devices this way, if desired). 318 319Of course, RSC_VDEV resource entries are only good enough for static 320allocation of virtio devices. Dynamic allocations will also be made possible 321using the rpmsg bus (similar to how we already do dynamic allocations of 322rpmsg channels; read more about it in rpmsg.txt). 323