1<?xml version="1.0"?> 2<!DOCTYPE book PUBLIC "-//OASIS//DTD DocBook XML V4.3//EN" 3 "http://www.oasis-open.org/docbook/xml/4.3/docbookx.dtd" [ 4 <!ENTITY % local.common.attrib "xmlns:xi CDATA #FIXED 'http://www.w3.org/2003/XInclude'"> 5 <!ENTITY version SYSTEM "version.xml"> 6]> 7<chapter id="what-is-harfbuzz"> 8 <title>What is HarfBuzz?</title> 9 <para> 10 HarfBuzz is a <emphasis>text-shaping engine</emphasis>. If you 11 give HarfBuzz a font and a string containing a sequence of Unicode 12 codepoints, HarfBuzz selects and positions the corresponding 13 glyphs from the font, applying all of the necessary layout rules 14 and font features. HarfBuzz then returns the string to you in the 15 form that is correctly arranged for the language and writing 16 system. 17 </para> 18 <para> 19 HarfBuzz can properly shape all of the world's major writing 20 systems. It runs on all major operating systems and software 21 platforms and it supports the modern font formats in use 22 today. 23 </para> 24 <section id="what-is-text-shaping"> 25 <title>What is text shaping?</title> 26 <para> 27 Text shaping is the process of translating a string of character 28 codes (such as Unicode codepoints) into a properly arranged 29 sequence of glyphs that can be rendered onto a screen or into 30 final output form for inclusion in a document. 31 </para> 32 <para> 33 The shaping process is dependent on the input string, the active 34 font, the script (or writing system) that the string is in, and 35 the language that the string is in. 36 </para> 37 <para> 38 Modern software systems generally only deal with strings in the 39 Unicode encoding scheme (although legacy systems and documents may 40 involve other encodings). 41 </para> 42 <para> 43 There are several font formats that a program might 44 encounter, each of which has a set of standard text-shaping 45 rules. 46 </para> 47 <para>The dominant format is <ulink 48 url="http://www.microsoft.com/typography/otspec/">OpenType</ulink>. The 49 OpenType specification defines a series of <ulink url="https://github.com/n8willis/opentype-shaping-documents">shaping models</ulink> for 50 various scripts from around the world. These shaping models depend on 51 the font including certain features in its <literal>GSUB</literal> 52 and <literal>GPOS</literal> tables. 53 </para> 54 <para> 55 Alternatively, OpenType fonts can include shaping features for 56 the <ulink url="https://graphite.sil.org/">Graphite</ulink> shaping model. 57 </para> 58 <para> 59 TrueType fonts can also include OpenType shaping 60 features. Alternatively, TrueType fonts can also include <ulink url="https://developer.apple.com/fonts/TrueType-Reference-Manual/RM09/AppendixF.html">Apple 61 Advanced Typography</ulink> (AAT) tables to implement shaping 62 support. AAT fonts are generally only found on macOS and iOS systems. 63 </para> 64 <para> 65 Text strings will usually be tagged with a script and language 66 tag that provide the context needed to perform text shaping 67 correctly. The necessary <ulink 68 url="https://docs.microsoft.com/en-us/typography/opentype/spec/scripttags">Script</ulink> 69 and <ulink 70 url="https://docs.microsoft.com/en-us/typography/opentype/spec/languagetags">language</ulink> 71 tags are defined by OpenType. 72 </para> 73 </section> 74 75 <section id="why-do-i-need-a-shaping-engine"> 76 <title>Why do I need a shaping engine?</title> 77 <para> 78 Text shaping is an integral part of preparing text for 79 display. Before a Unicode sequence can be rendered, the 80 codepoints in the sequence must be mapped to the corresponding 81 glyphs provided in the font, and those glyphs must be positioned 82 correctly relative to each other. For many of the scripts 83 supported in Unicode, these steps involve script-specific layout 84 rules, including complex joining, reordering, and positioning 85 behavior. Implementing these rules is the job of the shaping engine. 86 </para> 87 <para> 88 Text shaping is a fairly low-level operation. HarfBuzz is 89 used directly by text-handling libraries like <ulink 90 url="https://www.pango.org/">Pango</ulink>, as well as by the layout 91 engines in Firefox, LibreOffice, and Chromium. Unless you are 92 <emphasis>writing</emphasis> one of these layout engines 93 yourself, you will probably not need to use HarfBuzz: normally, 94 a layout engine, toolkit, or other library will turn text into 95 glyphs for you. 96 </para> 97 <para> 98 However, if you <emphasis>are</emphasis> writing a layout engine 99 or graphics library yourself, then you will need to perform text 100 shaping, and this is where HarfBuzz can help you. 101 </para> 102 <para> 103 Here are some specific scenarios where a text-shaping engine 104 like HarfBuzz helps you: 105 </para> 106 <itemizedlist> 107 <listitem> 108 <para> 109 OpenType fonts contain a set of glyphs (that is, shapes 110 to represent the letters, numbers, punctuation marks, and 111 all other symbols), which are indexed by a <literal>glyph ID</literal>. 112 </para> 113 <para> 114 A particular glyph ID within the font does not necessarily 115 correlate to a predictable Unicode codepoint. For instance, 116 some fonts have the letter "a" as glyph ID 1, but 117 many others do not. In order to retrieve the right glyph 118 from the font to display "a", you need to consult 119 the table inside the font (the <literal>cmap</literal> 120 table) that maps Unicode codepoints to glyph IDs. In other 121 words, <emphasis>text shaping turns codepoints into glyph 122 IDs</emphasis>. 123 </para> 124 </listitem> 125 <listitem> 126 <para> 127 Many OpenType fonts contain ligatures: combinations of 128 characters that are rendered as a single unit. For instance, 129 it is common for the <literal>fi</literal> letter 130 combination to appear in print as the single ligature glyph 131 "fi". 132 </para> 133 <para> 134 Whether you should render an "f, i" sequence 135 as <literal>fi</literal> or as "fi" does not 136 depend on the input text. Instead, it depends on the whether 137 or not the font includes an "fi" glyph and on the 138 level of ligature application you wish to perform. The font 139 and the amount of ligature application used are under your 140 control. In other words, <emphasis>text shaping involves 141 querying the font's ligature tables and determining what 142 substitutions should be made</emphasis>. 143 </para> 144 </listitem> 145 <listitem> 146 <para> 147 While ligatures like "fi" are optional typographic 148 refinements, some languages <emphasis>require</emphasis> certain 149 substitutions to be made in order to display text correctly. 150 </para> 151 <para> 152 For example, in Tamil, when the letter "TTA" (ட) 153 letter is followed by "U" (உ), the pair 154 must be replaced by the single glyph "டு". The 155 sequence of Unicode characters "டஉ" needs to be 156 substituted with a single "டு" glyph from the 157 font. 158 </para> 159 <para> 160 But "டு" does not have a Unicode codepoint. To 161 find this glyph, you need to consult the table inside 162 the font (the <literal>GSUB</literal> table) that contains 163 substitution information. In other words, <emphasis>text shaping 164 chooses the correct glyph for a sequence of characters 165 provided</emphasis>. 166 </para> 167 </listitem> 168 <listitem> 169 <para> 170 Similarly, each Arabic character has four different variants 171 corresponding to the different positions it might appear in 172 within a sequence. Inside a font, there will be separate 173 glyphs for the initial, medial, final, and isolated forms of 174 each letter, each at a different glyph ID. 175 </para> 176 <para> 177 Unicode only assigns one codepoint per character, so a 178 Unicode string will not tell you which glyph variant to use 179 for each character. To decide, you need to analyze the whole 180 string and determine the appropriate glyph for each character 181 based on its position. In other words, <emphasis>text 182 shaping chooses the correct form of the letter by its 183 position and returns the correct glyph from the font</emphasis>. 184 </para> 185 </listitem> 186 <listitem> 187 <para> 188 Other languages involve marks and accents that need to be 189 rendered in specific positions relative a base character. For 190 instance, the Moldovan language includes the Cyrillic letter 191 "zhe" (ж) with a breve accent, like so: "ӂ". 192 </para> 193 <para> 194 Some fonts will provide this character as a single 195 zhe-with-breve glyph, but other fonts will not and, instead, 196 will expect the rendering engine to form the character by 197 superimposing the separate "ж" and "˘" 198 glyphs. 199 </para> 200 <para> 201 But exactly where you should draw the breve depends on the 202 height and width of the preceding zhe glyph. To find the 203 right position, you need to consult the table inside 204 the font (the <literal>GPOS</literal> table) that contains 205 positioning information. 206 In other words, <emphasis>text shaping tells you whether you 207 have a precomposed glyph within your font or if you need to 208 compose a glyph yourself out of combining marks—and, 209 if so, where to position those marks.</emphasis> 210 </para> 211 </listitem> 212 </itemizedlist> 213 <para> 214 If tasks like these are something that you need to do, then you 215 need a text shaping engine. You could use Uniscribe if you are 216 writing Windows software; you could use CoreText on macOS; or 217 you could use HarfBuzz. 218 </para> 219 <note> 220 <para> 221 In the rest of this manual, the text will assume that the reader 222 is that implementor of a text-layout engine. 223 </para> 224 </note> 225 </section> 226 227 228 <section> 229 <title>What does HarfBuzz do?</title> 230 <para> 231 HarfBuzz provides text shaping through a cross-platform 232 C API that accepts sequences of Unicode codepoints as input. Currently, 233 the following OpenType shaping models are supported: 234 </para> 235 <itemizedlist> 236 <listitem> 237 <para> 238 Indic (covering Devanagari, Bengali, Gujarati, 239 Gurmukhi, Kannada, Malayalam, Oriya, Tamil, Telugu, and 240 Sinhala) 241 </para> 242 </listitem> 243 <listitem> 244 <para> 245 Arabic (covering Arabic, N'Ko, Syriac, and Mongolian) 246 </para> 247 </listitem> 248 <listitem> 249 <para> 250 Thai and Lao 251 </para> 252 </listitem> 253 <listitem> 254 <para> 255 Khmer 256 </para> 257 </listitem> 258 <listitem> 259 <para> 260 Myanmar 261 </para> 262 </listitem> 263 264 <listitem> 265 <para> 266 Tibetan 267 </para> 268 </listitem> 269 270 <listitem> 271 <para> 272 Hangul 273 </para> 274 </listitem> 275 276 <listitem> 277 <para> 278 Hebrew 279 </para> 280 </listitem> 281 <listitem> 282 <para> 283 The Universal Shaping Engine or <emphasis>USE</emphasis> 284 (covering complex scripts not covered by the above shaping 285 models) 286 </para> 287 </listitem> 288 <listitem> 289 <para> 290 A default shaping model for non-complex scripts 291 (covering Latin, Cyrillic, Greek, Armenian, Georgian, Tifinagh, 292 and many others) 293 </para> 294 </listitem> 295 <listitem> 296 <para> 297 Emoji (including emoji modifier sequences, flag sequences, 298 and ZWJ sequences) 299 </para> 300 </listitem> 301 </itemizedlist> 302 303 <para> 304 In addition to OpenType shaping, HarfBuzz supports the latest 305 version of Graphite shaping (the "Graphite 2" model) and AAT 306 shaping. 307 </para> 308 309 <para> 310 HarfBuzz can read and understand TrueType fonts (.ttf), TrueType 311 collections (.ttc), and OpenType fonts (.otf, including those 312 fonts that contain TrueType-style outlines and those that 313 contain PostScript CFF or CFF2 outlines). 314 </para> 315 316 <para> 317 HarfBuzz is designed and tested to run on top of the FreeType 318 font renderer. It can run on Linux, Android, Windows, macOS, and 319 iOS systems. 320 </para> 321 322 <para> 323 In addition to its core shaping functionality, HarfBuzz provides 324 functions for accessing other font features, including optional 325 GSUB and GPOS OpenType features, as well as 326 all color-font formats (<literal>CBDT</literal>, 327 <literal>sbix</literal>, <literal>COLR/CPAL</literal>, and 328 <literal>SVG-OT</literal>) and OpenType variable fonts. HarfBuzz 329 also includes a font-subsetting feature. HarfBuzz can perform 330 some low-level math-shaping operations, although it does not 331 currently perform full shaping for mathematical typesetting. 332 </para> 333 334 <para> 335 A suite of command-line utilities is also provided in the 336 source-code tree, designed to help users test and debug 337 HarfBuzz's features on real-world fonts and input. 338 </para> 339 </section> 340 341 <section id="what-harfbuzz-doesnt-do"> 342 <title>What HarfBuzz doesn't do</title> 343 <para> 344 HarfBuzz will take a Unicode string, shape it, and give you the 345 information required to lay it out correctly on a single 346 horizontal (or vertical) line using the font provided. That is the 347 extent of HarfBuzz's responsibility. 348 </para> 349 <para> 350 It is important to note that if you are implementing a complete 351 text-layout engine you may have other responsibilities that 352 HarfBuzz will <emphasis>not</emphasis> help you with. For example: 353 </para> 354 <itemizedlist> 355 <listitem> 356 <para> 357 HarfBuzz won't help you with bidirectionality. If you want to 358 lay out text that includes a mix of Hebrew and English, you 359 will need to ensure that each buffer provided to HarfBuzz 360 has all of its characters in the same order and that the 361 directionality of the buffer is set correctly. This may mean 362 segmenting the text before it is placed into HarfBuzz buffers. In 363 other words, the user will hit the keys in the following 364 sequence: 365 </para> 366 <programlisting> 367 A B C [space] ג ב א [space] D E F 368 </programlisting> 369 <para> 370 but will expect to see in the output: 371 </para> 372 <programlisting> 373 ABC אבג DEF 374 </programlisting> 375 <para> 376 This reordering is called <emphasis>bidi processing</emphasis> 377 ("bidi" is short for bidirectional), and there's an 378 algorithm as an annex to the Unicode Standard which tells you how 379 to process a string of mixed directionality. 380 Before sending your string to HarfBuzz, you may need to apply the 381 bidi algorithm to it. Libraries such as <ulink 382 url="http://icu-project.org/">ICU</ulink> and <ulink 383 url="http://fribidi.org/">fribidi</ulink> can do this for you. 384 </para> 385 </listitem> 386 <listitem> 387 <para> 388 HarfBuzz won't help you with text that contains different font 389 properties. For instance, if you have the string "a 390 <emphasis>huge</emphasis> breakfast", and you expect 391 "huge" to be italic, then you will need to send three 392 strings to HarfBuzz: <literal>a</literal>, in your Roman font; 393 <literal>huge</literal> using your italic font; and 394 <literal>breakfast</literal> using your Roman font again. 395 </para> 396 <para> 397 Similarly, if you change the font, font size, script, 398 language, or direction within your string, then you will 399 need to shape each run independently and output them 400 independently. HarfBuzz expects to shape a run of characters 401 that all share the same properties. 402 </para> 403 </listitem> 404 <listitem> 405 <para> 406 HarfBuzz won't help you with line breaking, hyphenation, or 407 justification. As mentioned above, HarfBuzz lays out the string 408 along a <emphasis>single line</emphasis> of, notionally, 409 infinite length. If you want to find out where the potential 410 word, sentence and line break points are in your text, you 411 could use the ICU library's break iterator functions. 412 </para> 413 <para> 414 HarfBuzz can tell you how wide a shaped piece of text is, which is 415 useful input to a justification algorithm, but it knows nothing 416 about paragraphs, lines or line lengths. Nor will it adjust the 417 space between words to fit them proportionally into a line. 418 </para> 419 </listitem> 420 </itemizedlist> 421 <para> 422 As a layout-engine implementor, HarfBuzz will help you with the 423 interface between your text and your font, and that's something 424 that you'll need—what you then do with the glyphs that your font 425 returns is up to you. 426 </para> 427 </section> 428 429 <section id="why-is-it-called-harfbuzz"> 430 <title>Why is it called HarfBuzz?</title> 431 <para> 432 HarfBuzz began its life as text-shaping code within the FreeType 433 project (and you will see references to the FreeType authors 434 within the source code copyright declarations), but was then 435 extracted out to its own project. This project is maintained by 436 Behdad Esfahbod, who named it HarfBuzz. Originally, it was a 437 shaping engine for OpenType fonts—"HarfBuzz" is 438 the Persian for "open type". 439 </para> 440 </section> 441</chapter> 442