1###################################### 2Dynamic Linking and Loading with glibc 3###################################### 4 5.. contents:: 6 :local: 7 :backlinks: none 8 :depth: 2 9 10Introduction 11============ 12 13.. Note:: 14 :class: caution 15 16 Portable Native Client currently only supports static linking, and the 17 only C library available for it is newlib. This page is only valid for 18 Native Client, though PNaCl will eventually support some form of 19 dynamic linking. 20 21This document describes how to create and deploy dynamically linked and loaded 22applications with the glibc library in the Native Client SDK. Before reading 23this document, we recommend reading :doc:`Building Native Client Modules 24<building>` 25 26.. _c_libraries: 27 28C standard libraries: glibc and newlib 29-------------------------------------- 30 31The Native Client SDK comes with two C standard libraries --- glibc and 32newlib. These libraries are described in the table below. 33 34+--------+----------+-------------+--------------------------------------------+ 35| Library| Linking | License | Description | 36+========+==========+=============+============================================+ 37| glibc | dynamic | GNU Lesser | glibc is the GNU implementation of the | 38| | or static| General | POSIX_ standard runtime library for the C | 39| | | Public | programming language. Designed for | 40| | | License | portability and performance, glibc is one | 41| | | (LGPL) | of the most popular implementations of the | 42| | | | C library. It is comprised of a set of | 43| | | | interdependent libraries including libc, | 44| | | | libpthreads, libdl, and others. For | 45| | | | documentation, FAQs, and additional | 46| | | | information about glibc, see GLIBC_ | 47+--------+----------+-------------+--------------------------------------------+ 48| newlib | static | Berkeley | newlib is a C library intended for use in | 49| | | Software | embedded systems. Like glibc, newlib is a | 50| | | Distribution| conglomeration of several library parts. | 51| | | (BSD) type | It is available for use under BSD-type free| 52| | | free | software licenses, which generally makes it| 53| | | software | more suitable to link statically in | 54| | | licenses | commercial, closed-source applications. For| 55| | | | documentation, FAQs, and additional | 56| | | | information about newlib, see the newlib_ | 57| | | | documentation. | 58+--------+----------+-------------+--------------------------------------------+ 59 60For proprietary (closed-source) applications, your options are to either 61statically link to newlib, or dynamically link to glibc. We recommend 62dynamically linking to glibc, for a couple of reasons: 63 64* The glibc library is widely distributed (it's included in Linux 65 distributions), and as such it's mature, hardened, and feature-rich. Your 66 code is more likely to compile out-of-the-box with glibc. 67 68* Dynamic loading can provide a big performance benefit for your application if 69 you can structure the application to defer loading of code that's not needed 70 for initial interaction with the user. It takes some work to put such code in 71 shared libraries and to load the libraries at runtime, but the payoff is 72 usually worth it. In future releases, Chrome may also support caching of 73 common dynamically linked libraries such as libc.so between applications. 74 This could significantly reduce download size and provide a further potential 75 performance benefit (for example, the hello_world example would only require 76 downloading a .nexe file that's on the order of 30KB, rather than a .nexe 77 file and several libraries, which are on the order of 1.5MB). 78 79Native Client support for dynamic linking and loading is based on glibc. Thus, 80**if your Native Client application must dynamically link and load code (e.g., 81due to licensing considerations), we recommend that you use the glibc 82library.** 83 84.. Note:: 85 :class: note 86 87 **Notes:** 88 89 * **None of the above constitutes legal advice, or a description of the legal 90 obligations you need to fulfill in order to be compliant with the LGPL or 91 newlib licenses. The above description is only a technical explanation of 92 the differences between newlib and glibc, and the choice you must make 93 between the two libraries.** 94 95 * Static linking with glibc is rarely used. Use this feature with caution. 96 97 * The standard C++ runtime in Native Client is provided by libstdc++; this 98 library is independent from and layered on top of glibc. Because of 99 licensing restrictions, libstdc++ must be statically linked for commercial 100 uses, even if the rest of an application is dynamically linked. 101 102SDK toolchains 103-------------- 104 105The Native Client SDK contains multiple toolchains, which are differentiated by 106:ref:`target architecture <target_architectures>` and C library: 107 108=================== ========= =============================== 109Target architecture C library Toolchain directory 110=================== ========= =============================== 111x86 newlib toolchain/<platform>_x86_newlib 112x86 glibc toolchain/<platform>_x86_glibc 113ARM newlib toolchain/<platform>_arm_newlib 114PNaCl newlib toolchain/<platform>_pnacl 115=================== ========= =============================== 116 117In the directories listed above, <platform> is the platform of your development 118machine (i.e., win, mac, or linux). For example, in the Windows SDK, the x86 119toolchain that uses glibc is in ``toolchain/win_x86_glibc``. 120 121.. Note:: 122 :class: note 123 124 **Note:** The ARM and PNaCl toolchains are currently restricted to newlib. 125 126To use the glibc library and dynamic linking in your application, you **must** 127use a glibc toolchain. (Currently the only glibc toolchain is 128``<platform>_x86_glibc``.) Note that you must build all code in your application 129with one toolchain. Code from multiple toolchains cannot be mixed. 130 131Specifying and delivering shared libraries 132------------------------------------------ 133 134One significant difference between newlib and glibc applications is that glibc 135applications must explicitly list and deploy the shared libraries that they 136use. 137 138In a desktop environment, when the user launches a dynamically linked 139application, the operating system's program loader determines the set of 140libraries the application requires by reading explicit inter-module 141dependencies from executable file headers, and loads the required libraries 142into the address space of the application process. Typically the required 143libraries will have been installed on the system as a part of the application's 144installation process. Often the desktop application developer doesn't know or 145think about the libraries that are required by an application, as those details 146are taken care of by the user's operating system. 147 148In the Native Client sandbox, dynamic linking can't rely in the same way on the 149operating system or the local file system. Instead, the application developer 150must identify the set of libraries that are required by an application, list 151those libraries in a Native Client :ref:`manifest file <manifest_file>`, and 152deploy the libraries along with the application. Instructions for how to build 153a dynamically linked Native Client application, generate a Native Client 154manifest (.nmf) file, and deploy an application are provided below. 155 156Building a dynamically linked application 157========================================= 158 159Applications built with the glibc toolchain will by dynamically linked by 160default. Application that load shared libraries at runtime using ``dlopen()`` 161must link with the libdl library (``-ldl``). 162 163Like other gcc-based toolchains building a dynamic library for NaCl is normally 164done by linking with the ``-shared`` flag and compiling with the ``-fPIC`` flag. 165The SDK build system will do this automatically when the ``SO_RULE`` Makefile 166rule is used. 167 168The Native Client SDK includes an example that demonstrates how to build a 169shared library, and how to use the ``dlopen()`` interface to load that library 170at runtime (after the application is already running). Many applications load 171and link shared libraries at launch rather than at runtime, and hence do not 172use the ``dlopen()`` interface. The SDK example is nevertheless instructive, as 173it demonstrates how to build Native Client modules (.nexe files) and shared 174libraries (.so files) with the x86 glibc toolchain, and how to generate a 175Native Client manifest file for glibc applications. 176 177The SDK example, located in ``examples/tutorial/dlopen``, includes three C++ 178files: 179 180eightball.cc 181 This file implements the function ``Magic8Ball()``, which is used to provide 182 whimsical answers to user questions. This file is compiled into a shared 183 library called ``libeightball.so``. This library gets included in the 184 .nmf file and is therefore directly loadable with ``dlopen()``. 185 186reverse.cc 187 This file implements the function ``Reverse()``, which returns reversed 188 copies of strings that are passed to it. This file is compiled into a shared 189 library called ``libreverse.so``. This library is **not** included in the 190 .nmf file and is loaded via an http mount using the :ref:`nacl_io library 191 <nacl_io>`. 192 193dlopen.cc 194 This file implements the Native Client module, which loads the two shared 195 libraries and handles communcation with with JavaScript. The file is compiled 196 into a Native Client executable (.nexe). 197 198Run ``make`` in the dlopen directory to see the commands the Makefile executes 199to build x86 32-bit and 64-bit .nexe and .so files, and to generate a .nmf 200file. These commands are described below. 201 202.. Note:: 203 :class: note 204 205 **Note:** The Makefiles for most of the examples in the SDK build the 206 examples using multiple toolchains (x86 newlib, x86 glibc, ARM, and PNaCl). 207 With a few exceptions (listed in the :ref:`Release Notes 208 <sdk-release-notes>`), running "make" in each example's directory builds 209 multiple versions of the example using the SDK toolchains. The dlopen example 210 is one of those exceptions – it is only built with the x86 glibc toolchain, 211 as that is currently the only toolchain that supports glibc and thus dynamic 212 linking and loading. Take a look at the example Makefiles and the generated 213 .nmf files for details on how to build dynamically linked applications. 214 215.. _dynamic_loading_manifest: 216 217Generating a Native Client manifest file for a dynamically linked application 218============================================================================= 219 220The Native Client manifest file specifies the name of the executable to run 221and must also specify any shared libraries that the application directly 222depends on. For indirect dependencies (such as libraries opened via 223``dlopen()``) it is also convenient to list libraries in the manifest file. 224However it is possile to load arbitrary shared libraries at runtime that 225are not mentioned in the manifest by using the `nacl_io library <nacl_io>`_ 226to mount a filesystem that contains the shared libraries which will then 227allow ``dlopen()`` to access them. 228 229In this example we demonstrate both loading directly from via the manifest 230file (``libeightball.so``) and loading indirectly via a http mount 231(``libreverse.so``). 232 233Take a look at the manifest file in the dlopen example to see how 234a glibc-style manifest file is structured. (Run ``make`` in the dlopen directory to 235generate the manifest file if you haven't done so already.) Here is an excerpt 236from ``dlopen.nmf``:: 237 238 { 239 "files": { 240 "libeightball.so": { 241 "x86-64": { 242 "url": "lib64/libeightball.so" 243 }, 244 "x86-32": { 245 "url": "lib32/libeightball.so" 246 } 247 }, 248 "libstdc++.so.6": { 249 "x86-64": { 250 "url": "lib64/libstdc++.so.6" 251 }, 252 "x86-32": { 253 "url": "lib32/libstdc++.so.6" 254 } 255 }, 256 "libppapi_cpp.so": { 257 "x86-64": { 258 "url": "lib64/libppapi_cpp.so" 259 }, 260 "x86-32": { 261 "url": "lib32/libppapi_cpp.so" 262 } 263 }, 264 ... etc. 265 266In most cases, you can use the ``create_nmf.py`` script in the SDK to generate 267a manifest file for your application. The script is located in the tools 268directory (e.g. ``pepper_28/tools``). 269 270The Makefile in the dlopen example generates the manifest automatically using 271the ``NMF_RULE`` provided by the SDK build system. Running ``make V=1`` will 272show the full command line which is used to generate the nmf:: 273 274 create_nmf.py -o dlopen.nmf glibc/Release/dlopen_x86_32.nexe \ 275 glibc/Release/dlopen_x86_64.nexe glibc/Release/libeightball_x86_32.so \ 276 glibc/Release/libeightball_x86_64.so -s ./glibc/Release \ 277 -n libeightball_x86_32.so,libeightball.so \ 278 -n libeightball_x86_64.so,libeightball.so 279 280Run python ``create_nmf.py --help`` to see a full description of the command-line 281flags. A few of the important flags are described below. 282 283``-s`` *directory* 284 use *directory* to stage libraries (libraries are added to ``lib32`` and 285 ``lib64`` subfolders) 286 287``-L`` *directory* 288 add *directory* to the library search path. The default search path 289 already includes the toolchain and SDK libraries directories. 290 291.. Note:: 292 :class: note 293 294 **Note:** The ``create_nmf`` script can only automatically detect explicit 295 shared library dependencies (for example, dependencies specified with the -l 296 flag for the compiler/linker). If you want to include libraries that you 297 intend to dlopen() at runtime you must explcitly list them in your call to 298 ``create_nmf``. 299 300As an alternative to using ``create_nmf``, it is possible to manually calculate 301the list of shared library dependencies using tools such as ``objdump_``. 302 303Deploying a dynamically linked application 304========================================== 305 306As described above, an application's manifest file must explicitly list all the 307executable code modules that the application directly depends on, including 308modules from the application itself (.nexe and .so files), modules from the 309Native Client SDK (e.g., libppapi_cpp.so), and perhaps also modules from 310`naclport <naclports_>`_ or from `middleware systems <../../community/middleware>`_ that 311the application uses. You must provide all of those modules as part of the 312application deployment process. 313 314As explained in :doc:`Distributing Your Application 315<../distributing>`, there are two basic ways to deploy an application: 316 317* **hosted application:** all modules are hosted together on a web server of 318 your choice 319 320* **packaged application:** all modules are packaged into one file, hosted in 321 the Chrome Web Store, and downloaded to the user's machine 322 323You must deploy all the modules listed in your application's manifest file for 324either the hosted application or the packaged application case. For hosted 325applications, you must upload the modules to your web server. For packaged 326applications, you must include the modules in the application's Chrome Web 327Store .crx file. Modules should use URLs/names that are consistent with those 328in the Native Client manifest file, and be named relative to the location of 329the manifest file. Remember that some of the libraries named in the manifest 330file may be located in directories you specified with the -L option to 331``create_nmf.py``. You are free to rename/rearrange files and directories 332referenced by the Native Client manifest file, so long as the modules are 333available in the locations indicated by the manifest file. If you move or 334rename modules, it may be easier to re-run ``create_nmf.py`` to generate a new 335manifest file rather than edit the original manifest file. For hosted 336applications, you can check for name mismatches during testing by watching the 337request log of the web server hosting your test deployment. 338 339Opening a shared library at runtime 340=================================== 341 342Native Client supports a version of the POSIX standard ``dlopen()`` interface 343for opening libraries explicitly, after an application is already running. 344Calling ``dlopen()`` may cause a library download to occur, and automatically 345loads all libraries that are required by the named library. 346 347.. Note:: 348 :class: note 349 350 **Caution:** Since ``dlopen()`` can potentially block, you must initially 351 call ``dlopen()`` off your application's main thread. Initial calls to 352 ``dlopen()`` from the main thread will always fail in the current 353 implementation of Native Client. 354 355The best practice for opening libraries with ``dlopen()`` is to use a worker 356thread to pre-load libraries asynchronously during initialization of your 357application, so that the libraries are available when they're needed. You can 358call ``dlopen()`` a second time when you need to use a library -- per the 359specification, subsequent calls to ``dlopen()`` return a handle to the 360previously loaded library. Note that you should only call ``dlclose()`` to 361close a library when you no longer need the library; otherwise, subsequent 362calls to ``dlopen()`` could cause the library to be fetched again. 363 364The dlopen example in the SDK demonstrates how to open a shared libraries 365at runtime. To reiterate, the example includes three C++ files: 366 367* ``eightball.cc``: this is the shared library that implements the function 368 ``Magic8Ball()`` (this file is compiled into libeightball.so) 369* ``reverse.cc``: this is the shared library that implements the function 370 ``Reverse()`` (this file is compiled into libreverse.so) 371* ``dlopen.cc``: this is the Native Client module that loads the shared libraries 372 and makes calls to ``Magic8Ball()`` and ``Reverse()`` in response to requests 373 from JavaScript. 374 375When the Native Client module starts, it kicks off a worker thread that calls 376``dlopen()`` to load the two shared libraries. Once the module has a handle to 377the library, it fetches the addresses of the ``Magic8Ball()`` and ``Reverse()`` 378functions using ``dlsym()``. When a user types in a query and clicks the 'ASK!' 379button, the module calls ``Magic8Ball()`` to generate an answer, and returns 380the result to the user. Likewise when the user clicks the 'Reverse' button 381it calls the ``Reverse()`` function to reverse the string. 382 383Troubleshooting 384=============== 385 386If your .nexe isn't loading, the best place to look for information that can 387help you troubleshoot the JavaScript console and standard output from Chrome. 388See :ref:`Debugging <devcycle-debugging>` for more information. 389 390Here are a few common error messages and explanations of what they mean: 391 392**/main.nexe: error while loading shared libraries: /main.nexe: failed to allocate code and data space for executable** 393 The .nexe may not have been compiled correctly (e.g., the .nexe may be 394 statically linked). Try cleaning and recompiling with the glibc toolchain. 395 396**/main.nexe: error while loading shared libraries: libpthread.so.xxxx: cannot open shared object file: Permission denied** 397 (xxxx is a version number, for example, 5055067a.) This error can result from 398 having the wrong path in the .nmf file. Double-check that the path in the 399 .nmf file is correct. 400 401**/main.nexe: error while loading shared libraries: /main.nexe: cannot open shared object file: No such file or directory** 402 If there are no obvious problems with your main.nexe entry in the .nmf file, 403 check where main.nexe is being requested from. Use Chrome's Developer Tools: 404 Click the menu icon |menu-icon|, select Tools > Developer Tools, click the 405 Network tab, and look at the path in the Name column. 406 407**NaCl module load failed: ELF executable text/rodata segment has wrong starting address** 408 This error happens when using a newlib-style .nmf file instead of a 409 glibc-style .nmf file. Make sure you build your application with the glic 410 toolchain, and use the create_nmf.py script to generate your .nmf file. 411 412**NativeClient: NaCl module load failed: Nexe crashed during startup** 413 This error message indicates that a module crashed while being loaded. You 414 can determine which module crashed by looking at the Network tab in Chrome's 415 Developer Tools (see above). The module that crashed will be the last one 416 that was loaded. 417 418**/lib/main.nexe: error while loading shared libraries: /lib/main.nexe: only ET_DYN and ET_EXEC can be loaded** 419 This error message indicates that there is an error with the .so files listed 420 in the .nmf file -- either the files are the wrong type or kind, or an 421 expected library is missing. 422 423**undefined reference to 'dlopen' collect2: ld returned 1 exit status** 424 This is a linker ordering problem that usually results from improper ordering 425 of command line flags when linking. Reconfigure your command line string to 426 list libraries after the -o flag. 427 428.. |menu-icon| image:: /images/menu-icon.png 429.. _objdump: http://en.wikipedia.org/wiki/Objdump 430.. _GLIBC: http://www.gnu.org/software/libc/index.html 431.. _POSIX: http://en.wikipedia.org/wiki/POSIX 432.. _newlib: http://sourceware.org/newlib/ 433
