1# ArkVM Development guide 2 3ArkVM and ArkTS and related projects are hosted at [Gitee](https://gitee.com) and [GitCode](https://gitcode.com). 4 5## Table of content 6 7Fast introduce: 8 91. [Quick setup](#quick-setup) 101. [Launch "Hello world!" on ArkVM](#launch-hello-world-on-arkvm) 11 12Detailed info: 13 141. [How to use es2panda, ark, ark_aot, verifier](#how-to-use-es2panda-ark-ark_aot-verifier) 151. [Build modes](#build-modes) 161. [Toolchains](#toolchains) 171. [Enable CPU features](#enable-cpu-features) 181. [Plugins system](#plugins-system) 191. [Building with GN (still not all targets are supported)](#building-with-gn) 201. [Building with LLVM Backend](#building-with-llvm-backend) 211. [Launch cross-compiled ARM64/ARM32 tests with QEMU](#launch-cross-compiled-arm64arm32-tests-with-qemu) 221. [Install third party](#install-third-party) 231. [Testing your changes](#testing-your-changes) 24 25## Quick setup 26 27### Download project 28 29```bash 30mkdir arkcompiler 31cd arkcompiler 32 33BASELINE_BRANCH="OpenHarmony_feature_20241108" 34 35git clone --branch ${BASELINE_BRANCH} https://gitee.com/openharmony/arkcompiler_runtime_core.git runtime_core 36 37git clone --branch ${BASELINE_BRANCH} https://gitee.com/openharmony/arkcompiler_ets_frontend ets_frontend 38 39# Set specific link for ets_frontend 40ln -s ../../../ets_frontend/ets2panda runtime_core/static_core/tools/es2panda 41 42# (optional) Install ecmascript plugin 43git clone --branch ${BASELINE_BRANCH} https://gitcode.com/openharmony-sig/arkcompiler_ets_runtime.git runtime_core/static_core/plugins/ecmascript 44``` 45 46### Install dependent programs 47 48Ubuntu 22.04 is actual base system for development. 49 50```bash 51cd runtime_core/static_core/ 52sudo ./scripts/install-deps-ubuntu -i=dev -i=test -i=arm-all -i=llvm-prebuilts 53# It is minimal pack of tools for start develop on Linux x86-64, you can see more options by pass "--help" to script 54``` 55 56### Build project 57 58Let's configure a common build in `Debug` mode using `Clang-14` (more information about the build configuration in [Build modes](#build-modes) and [Toolchains](#toolchains)): 59 60```bash 61mkdir build 62cd build 63 64cmake <PATH_TO_ARKCOMPILER>/runtime_core/static_core -GNinja -DCMAKE_BUILD_TYPE=Debug -DCMAKE_TOOLCHAIN_FILE=cmake/toolchain/host_clang_14.cmake 65ninja ark ark_aot es2panda 66``` 67 68**Congratulation, project and primary programs are built!** 69 70### Launch "Hello world!" on ArkVM 71 72```bash 73echo 'console.log("Hello world!");' > hello.ets 74 75BUILD=<PATH_TO_BUILD> 76# Generate .abc files from source code 77${BUILD}/bin/es2panda --extension=ets --output hello.abc hello.ets 78 79# Launch Virtual Machine 80${BUILD}/bin/ark --load-runtimes=ets --boot-panda-files=${BUILD}/plugins/ets/etsstdlib.abc hello.abc hello.ETSGLOBAL::main 81``` 82 83**Congratulation, you launch the ArkTS code on ArkVM!** 84 85## Detailed description 86 87## How to use es2panda, ark, ark_aot, verifier 88 89The most frequent used options for each program are described here. 90 91### es2panda 92 93`es2panda` is used to compile source code to .abc file. 94 95| Option name | Type of option | Default value | Description | 96| --- | --- | --- | --- | 97| --extension | string | "ets" | Parse the input as the given extension | 98|--opt-level| number | 2 | Compiler optimization level | 99|--output| string | filename with extension .abc | Path to output file | 100 101Compile `ets` file: 102 103```bash 104./bin/es2panda --exttension=ets <file>.ets 105``` 106 107You can see more options by pass `--help`. 108 109--- 110 111### ark 112 113`ark` is virtual machine, is used to run .abc files. 114 115| Option name | Type of option | Default value | Description | 116| --- | --- | --- | --- | 117|--load-runtimes|string|"core"|Load specified class and intrinsic spaces and define runtime type| 118|--boot-panda-files|string[]|[]|Boot panda files (.abc) separated by colon| 119|--interpreter-type|string|irtoc|Interpreter implementation type| 120|--compiler-enable-jit|bool|true|Enables/disables JIT compiler| 121|--no-async-jit|bool|false|Perform compilation in the main thread or in parallel worker| 122|--compiler-hotness-threshold|uint32_t|3000|Threshold for "hotness" counter of the method after that it will be compiled| 123|--aot-files|string[]|[]|List of aot files (.an) to be loaded separated by colon| 124|--enable-an|bool|false|Try to load ARK .an file base on abc file location| 125|--enable-an:force|bool|false|Crash if there is no .an file for location based on .abc file| 126 127Launch `ets` abc file: 128 129```bash 130./bin/ark --load-runtimes=ets --boot-panda-files=plugins/ets/etsstdlib.abc <file>.abc <file>.ETSGLOBAL::main 131``` 132 133You can see more options by pass `--help`. 134 135--- 136 137### ark_aot 138 139`ark_aot` tool aims to compile input panda files into the single AOT file, that can be consumed by `ark`. 140 141| Option name | Type of option | Default value | Description | 142| --- | --- | --- | --- | 143|--load-runtimes|string|"core"|Load specified class and intrinsic spaces and define runtime type| 144|--paoc-panda-files|string[]|[]|List of input panda files to compiler separated by colon| 145|--boot-panda-files|string[]|[]|Boot panda files (.abc) separated by colon| 146|--paoc-output|string|out.an|Path to output file| 147 148Compile an `ets` abc file to AOT file: 149 150```bash 151./bin/ark_aot --boot-panda-files=plugins/ets/etsstdlib.abc --load-runtimes=ets --paoc-panda-files=<file>.abc --paoc-output=<file>.an 152``` 153 154> *NOTE*: `boot-panda-files` should be the same as it will be during launch `ark`, otherwise will be throw error during execution about mismach class hierarchy. 155 156You can see more options by launch `ark_aot` without options. 157 158--- 159 160### verifier 161 162`verifier` tool aims to verify specified Panda files. 163 164| Option name | Type of option | Default value | Description | 165| --- | --- | --- | --- | 166|--load-runtimes|string|"core"|Load specified class and intrinsic spaces and define runtime type| 167|--boot-panda-files|string[]|[]|Boot panda files (.abc) separated by colon| 168 169```bash 170./bin/verifier --load-runtimes=ets --boot-panda-files=plugins/ets/etsstdlib.abc <file>.abc 171``` 172 173> *NOTE*: `boot-panda-files` should be the same as it will be during launch `ark` or `ark_aot`. 174 175You can see more options by pass `--help`. 176 177## Build modes 178 179Recommended way to build Panda is to set `CMAKE_BUILD_TYPE` variable explicitly during configurations. Supported values are: 180 181| Mode | Assertions | Optimizations | Debug info | 182| ---- | ---------- | ------------- | ---------- | 183| `Debug` | Yes | None (CMake default) | `-g` (CMake default) | 184| `Release` | No | `-O3` (CMake default) | None (CMake default) | 185| `FastVerify` | Yes | `-O2` | `-ggdb3` | 186 187*Notes*: 188 189* Other common modes (`RelWithDebInfo`, `MinSizeRel`, `DebugDetailed`) should work but they are not tested in CI. 190* Unlike `RelWithDebInfo`, `FastVerify` preserves assertions (and provides more verbose debug information). 191 Use this build type for running heavy test suites when you want both fast-running code and debuggability. 192* `DebugDetailed` gives more debug information than `Debug`, it can be useful for debugging unit tests for example. 193 194## Toolchains 195 196All types of toolchains are located by path: `runtime_core/static_core/cmake/toolchain`. Main toolchains which are used in development: 197 198* `cmake/toolchain/host_clang_14.cmake` - build by clang-14 on Linux 199* `cmake/toolchain/host_gcc_11.cmake` - build by gcc-11 on Linux 200* `cmake/toolchain/cross-clang-14-qemu-aarch64.cmake` - cross build by clang-14 for arm64 with launch on Linux x86-64 201* `cmake/toolchain/cross-clang-14-qemu-arm-linux-gnueabi.cmake` - cross build for arm32 (gnueabi) with launch on Linux x86-64 by clang-14 202* `cmake/toolchain/cross-clang-14-x86_64-w64-mingw32-static.cmake` - build by mingw clang-14 on Windows x86-64 203 204You can already see how the toolchain name is built and what it is used for. By analogy, so for all the others. 205 206For more details, see the [build system](cmake/README.md). 207 208## Enable CPU features 209 210To specify what CPU features will be supported by the target CPU, the `PANDA_TARGET_CPU_FEATURES` variable may be set. 211This list of the features will be passed into numerous custom tools during the building of the VM itself, e.g., in 212[irtoc](irtoc/README.md) and influence what instructions will be selected by these tools: 213 214``` 215$ cmake -DPANDA_TARGET_CPU_FEATURES=jscvt,atomics 216``` 217 218## Plugins system 219 220ArkVM has a multilanguage virtual machine, each language attach as a plugin. All plugins located as a separate folder by path `runtime_core/static_core/plugins/`. By default, extension is plugged in configure if folder is exist. 221 222> *Example*: there is folder `runtime_core/static_core/plugins/ets/`, so language `ets` will implicitly be added to configure for building. 223 224You can explicitly enable/disable language by cmake option `-DPANDA_WITH_<LANG>`. 225 226> *Example*: option for `ets` will be `--PANDA_WITH_ETS=ON/OFF`. 227 228## Building with GN 229 230### Using bootstrap 231 232Build `arkts_bin`, `ark_aot`, `es2panda`, `verifier_bin` and `ets_interop_js_napi` targets.Launch cross-compiled ARM64/ARM32 tests with QEMU 233 234```bash 235cd runtime_core/static_core 236./scripts/build-panda-with-gn 237``` 238 239### Manually 240 2411. Getting GN binary 242 243```bash 244$ git clone https://gn.googlesource.com/gn 245$ cd gn 246$ python build/gen.py 247$ ninja -C out 248``` 249 2502. Build panda using gn (`arkts_asm`, `arkts_disasm`, `ark_aot`, `ark_aotdump`, `arkts_bin`, `es2panda`, and `verifier_bin` targets are supported) 251 252```bash 253$ cd /path/to/panda/repository 254$ /path/to/gn/repository/out/gn gen out 255$ ninja -C out arkts_bin 256``` 257 258When standard system, use 259 260```bash 261$ cd /path/to/panda/repository 262$ /path/to/gn/repository/out/gn --args=is_standard_system=true gen out 263$ ninja -C out <target name> 264``` 265 266By default, gn build uses LLVM Backend, so one must provide `llvm_target_dir="/path/to/llvm-15-{type}-{arch}"` if it is not in default location. To build without llvm add the following arguments: 267 268```bash 269$ /path/to/gn/repository/out/gn out is_llvmbackend=false ... 270``` 271 272Option `is_llvmbackend=true` in gn is the same scenarios as `-DPANDA_LLVM_BACKEND=true` option in cmake builds 273 274## Building with LLVM Backend 275 276If you want to build Ark with LLVM Backend you need to special modified LLVM 15 binaries, there are 2 way to get them: 277 2781. Download prebuilds *(preferred way)* 279 280 The prebuilds will be installed in the folder `/opt/`: 281 282 ```bash 283 cd runtime_core/static_core/ 284 sudo ./scripts/install-deps-ubuntu -i=llvm-prebuilts 285 ``` 286 2872. Build binaries locally 288 289 * Clone repository from [gitee.com](https://gitee.com/openharmony/third_party_llvm-project) 290 * Check [README](scripts/llvm/README.md) file about build process 291 * Use build [script](scripts/llvm/build_llvm.sh) 292 293If modified LLVM available in `/opt`, the following two options are necessary 294to build Ark with LLVM Backend functions. 295 296```cmake 297cmake -DPANDA_LLVM_BACKEND=true -DLLVM_TARGET_PATH=/opt/llvm-15-{type}-{arch} ... 298``` 299 300The `PANDA_LLVM_BACKEND` enables: 301 3021. LLVM Irtoc Interpreter. Use `-DPANDA_LLVM_INTERPRETER=OFF` to disable. 3032. LLVM Fastpaths compilation. Use `-DPANDA_LLVM_FASTPATH=OFF` to disable. 3043. LLVM Interpreter inlining. Use `-DPANDA_LLVM_INTERPRETER_INLINING=OFF` to disable. 3054. LLVM AOT compiler. Use `-DPANDA_LLVM_AOT=OFF` to disable. 306 307`PANDA_LLVM_INTERPRETER`, `PANDA_LLVM_FASTPATH`, and `PANDA_LLVM_AOT` are `ON` if `PANDA_LLVM_BACKEND` is turned on. 308 309It is recommended to choose `clang` compiler using toolchain files: `-DCMAKE_TOOLCHAIN_FILE=cmake/toolchain/host_clang_14.cmake`. 310By default GNU compiler `c++` is used, but some features are not available in such `gcc` builds. 311 312### Building with LLVM Backend for cross ARM64 build 313 314For cross-build, when e.g. `-DCMAKE_TOOLCHAIN_FILE=cmake/toolchain/cross-clang-14-qemu-aarch64.cmake` is used, 315two LLVM-paths options are required when all LLVM Backend functions are enabled. 316 317* First one is target LLVM, like `-DLLVM_TARGET_PATH=/opt/llvm-15-debug-aarch64` 318 * It is required for AOT, so, alternatively, you can use `-DPANDA_LLVM_AOT=OFF`. 319* Second one is host LLVM, like `-DLLVM_HOST_PATH=/opt/llvm-15-debug-x86_64` 320 * It is required for Irtoc compilation, so, alternatively, you can disable appropriate Interpreter and FastPath options (see above). 321 322## Launch cross-compiled ARM64/ARM32 tests with QEMU 323 324Recommended QEMU version for running tests is 6.2.0 (but 5.1+ should be ok, too). By default, it is downloaded and installed during environment bootstrap. Any system-installed package is left intact. If recommended QEMU version is not accessible via $PATH it can be specified during configuration time: 325 326```bash 327# If QEMU is available as /opt/qemu-6.2.0/bin/qemu-aarch64 or download manually 328$ cmake -DQEMU_PREFIX=/opt/qemu-6.2.0 ... 329``` 330 331## Install third party 332 333Panda uses third party libraries. During firs configure of `cmake` third-party will be installed automatically. To manually download libraries run: 334 335```bash 336cd runtime_core/static_core 337./scripts/install-third-party 338``` 339 340Useful options: 341 342* `--force-clone` - force download libraries, usually used when one of part third-party have updated 343* `--repo-name=repo` - install specific repo 344* `--node` - install node 345 346You can see more options by pass `--help` to script 347 348## Testing your changes 349 350For testing, the following umbrella targets that guarantee building prior to running may be used: 351 352* `ninja tests`, for running all testing suites. 353* `ninja tests_full`, for running all testing suites and various code linters. 354 355`clang-format` and `clang-tidy` checks are integrated into build system and can be called by target of build system: 356 357* `ninja code-style-check` - clang-format 358* `ninja clang-tidy-check` - clang-tidy 359 360> *NOTE* `ninja` is launched from the build directory 361