CTAN
	(I don't like the name)
	the files with patterns as authors should submit it
source
	sources of pattern files, like programs to generate them or word lists
	please contribute!
TL
	files that should go to TeX Live?

*NOTES ABOUT NORWEGIAN*

The Norwegian language has two written forms, called bokmål and nynorsk, whose
history dates back to the 19th century during the Danish occupation.  The most
widely used one of them is bokmål, but nynorsk is used as well, and they are
both official languages of Norway.  The ISO 639-1 code are ‘nb’ for bokmål and
‘nn’ for nynorsk.  There is also a ISO 639-1 code for “Norwegian”, ‘no’.

  Traditionally, LaTeX has been supporting Norwegian under the common name
“norsk”, with “norwegian” as an alias in language.dat.  The captions in Babel's
norsk.ldf are in bokmål.  Since a few months, there are two additional pattern
files available on CTAN labelled as bokmål and nynorsk, which correct
hyphenations for a small number of words – two words for each language,
actually: in bokmål, attende and betre are hyphenated at-ten-de and be-tre,
whereas in nynorsk it's att-en-de and bet-re.  With the current patterns for
“Norwegian” (nohyphbx.tex), though, they are hyphenated atten-de and betre,
which is correct in neither language.  The new pattern files provide therefore
provide tiny improvements.

  Since the Babel's current support is suitable for bokmål, and the bokmål
patterns are only slightly different from the current ones, we can use them as
the successor to the norsk patterns, and ship the nynorsk patterns in addition
to this.  We thus can use the following definitions in language.no.dat:

	norsk		loadhyph-nb.tex
	=norwegian
	=bokmal
	nynorsk		loadhyph-nn.tex

  Of course, the nynorsk patterns still can't be used directly as long as there
is no Babel support, but we hope that support will be added on the future, and
the patterns will already be there.

		Arthur, 2008-06-11

[
  {
    "Name": "TEX hyphenation patterns",
    "License": "MIT;GPL;GPL-2;LGPL-2.1;LGPL-3;LPPL-1;LPPL-1.2;LPPL-1.3;MPL-1.1",
    "License File": "LICENSE-MIT;LICENSE-GPL;LICENSE-GPL2;LICENSE-LGPL2.1;LICENSE-LGPL3;LICENSE-LPPL;LICENSE-LPPL1.2;LICENSE-LPPL1.3;LICENSE-MPL1.1",
    "Version Number": "CTAN-2021.03.21",
    "Owner": "wangxiaokai@huawei.com",
    "Upstream URL": "https://github.com/hyphenation/tex-hyphen",
    "Description": "This is the central repository for all hyphenation patterns"
  }
]

[![Build Status](https://travis-ci.com/hyphenation/tex-hyphen.svg?branch=master)](https://travis-ci.com/hyphenation/tex-hyphen)

This is the central repository for all hyphenation patterns (that we know of).
These patterns are encoded in UTF-8, but can also be used with 8-bit TeX
engines such as pdfTeX with the help of mechanisms provided with the package.
It is meant to be a low-level package and is integrated with all major TeX
distributions (TeX Live, MiKTeX, W32TeX); most TeX users should thus not
concern themselves with this package, unless of course they’re working on the
hyphenation patterns themselves.

We upload the package to CTAN regularly and use git tags to identify versions;
a CTAN upload has a version date in the form `yyyy-mm-dd`, corresponding to tag
`CTAN-yyyy.mm.dd` in this git repository. Because the package contains
contributions from many different sources that are not updated at the same
time, we felt this was this best choice for version identifiers.

If you are an author of hyphenation patterns and want to add or update them,
please contact the maintainers through the mailing list: tex-hyphen@tug.org
You can also visit the [TeX hyphenation page](http://www.hyphenation.org/tex) for more
information and technical details.

# Running
`bundle` (or `bundle install`)
`bundle exec rake` runs the default task in the Rakefile.

# tex-hyphen
## Introduction
tex-hyphen is a hyphenation pattern library for the TeX system. It can correctly hyphenate words in multiple languages to improve typesetting quality.

Source: tex-hyphen
URL: https://github.com/hyphenation/tex-hyphen
Version: CTAN-2021.03.21
License: Various combinations

## Background
In multilingual document processing and typesetting, correct hyphenation is crucial. tex-hyphen provides a comprehensive set of hyphenation patterns that support multiple languages, ensuring high-quality typesetting. Introducing tex-hyphen into OpenHarmony can significantly enhance the typesetting quality of multilingual documents.

## Language Classification
The tex directory contains multiple hyphenation patterns from TeX hyphenation patterns, each using different open-source licenses. The classification is as follows:
* MIT License
* GPL, GPL 2
* LGPL 1, LGPL 2.1
* LPPL 1, LPPL 1.2, LPPL 1.3
* MPL 1.1
* BSD 3

Languages used in OHOS,the following languages all use a user-friendly open-source license.:
* be - Belarusian
* cs - Czech
* cy - Welsh
* da - Danish
* de-1901 - German (1901 orthography)
* de-ch-1901 - Swiss German (1901 orthography)
* el-monoton - Modern Greek (monotonic)
* el-polyton - Modern Greek (polytonic)
* en-gb - British English
* en-us - American English
* es - Spanish
* et - Estonian
* fr - French
* ga - Irish
* gl - Galician
* hr - Croatian
* hu - Hungarian
* hy - Armenian
* id - Indonesian
* is - Icelandic
* it - Italian
* ka - Georgian
* lt - Lithuanian
* lv - Latvian
* mk - Macedonian
* mn-cyrl - Mongolian (Cyrillic script)
* nl - Dutch
* pt - Portuguese
* ru - Russian
* sh-cyrl - Serbo-Croatian (Cyrillic script)
* sh-latn - Serbo-Croatian (Latin script)
* sk - Slovak
* sl - Slovenian
* sr-cyrl - Serbian (Cyrillic script)
* sv - Swedish
* th - Thai
* tk - Turkmen
* tr - Turkish
* uk - Ukrainian
* zh-latn-pinyin - Chinese (Pinyin)

## Directory Structure
```
third_party_tex-hyphen
├── collaboration
│   ├── original
│   ├── repository
│   └── source
├── data/language-codes
├── docs
│   └── languages
├── encoding
│   └── data
├── hyph-utf8
│   ├── doc
│   ├── source
│   └── tex
├── misc
├── ohos
│   ├── src
│   └── hpb-binary
├── old
├── source
├── tests
├── TL
├── tools
└── webpage
```
collaboration/       JavaScript dependencies and XML configuration files required by the tex-hyphen official website
ohos/                OpenHarmony compilation files and hpb binary files
data/                Language library
docs/                Documentation related to hyphenation
encoding/            Contains files related to character set encodings, handling different character sets.
hyph-utf8/           Hyphenation pattern package for TeX, providing hyphenation patterns encoded in UTF-8
misc/                An example of a hyphenation file for the en-gb language.
old/                 Contains older hyphenation pattern files that may have been updated or replaced.
source/              Contains source code files used to generate and process hyphenation patterns.
TL/                  tlpsrc resource files, which are package source files in the TeX Live system, used to describe metadata of TeX Live packages
tools/               Contains utility scripts to assist in processing hyphenation pattern files.
webpage/             tex-hyphen official homepage, providing detailed information and resources about the hyph-utf8 package


## Value Brought to OpenHarmony
**1. Improved Typesetting Quality:** By introducing tex-hyphen, OpenHarmony can achieve more accurate hyphenation, improving the readability and aesthetics of documents.
**2. Enhanced Small Screen Experience:** Using hyphenation patterns on small screen devices can display more content in the same area, enhancing the reading experience.

## How to Use tex-hyphen in OpenHarmony
### 1. Compile the HPB Binary
#### Compilation Steps
Open the terminal (or command prompt), navigate to the directory containing the [hyphen_pattern_processor.cpp](ohos%2Fsrc%2Fhyphen-build%2Fhyphen_pattern_processor.cpp) file, and run the following command to compile the code:

```
cd ohos/src/hyphen-build/
g++ -g -Wall hyphen_pattern_processor.cpp -o transform
```

Explanation of the command:
- g++: Invoke the GCC compiler.
- -g: Add debugging information.
- -Wall: Enable all warnings.
- hyphen_pattern_processor.cpp: Source code file.
- -o transform: Specify the output executable file name as transform.

#### Execution Steps
After compilation, you can run the generated executable file and process the specified .tex file using the following command:

```
./transform hyph-en-us.tex ./out/
```

Explanation of the command:
- ./transform: Run the generated transform executable file.
- hyph-en-us.tex: Input file (the .tex file to be processed).
- ./out/: Output directory (the processed files will be stored in this directory).

After successful execution, the processed files will be stored in the ./out/ directory.

#### Batch Compilation
- Dependencies:
```
jq:JSON file parsing tool
```
- Configure the files to be compiled using the JSON configuration file [build-tex.json](ohos%2Fbuild%2Fbuild-tex.json):
```
[
    {
        "filename": "example1.tex"
    },
    {
        "filename": "example2.tex"
    }
]
```
filename: Specifies the name of the TeX file to be compiled. The file must be located in the [tex](hyph-utf8%2Ftex%2Fgeneric%2Fhyph-utf8%2Fpatterns%2Ftex) directory.

The build-tex.json file defines all supported languages, and the script will compile all of them by default. Developers can control the addition or removal of languages by modifying build-tex.json.
For example:
To remove the example2 language, modify the file as follows:
```
[
    {
        "filename": "example1.tex"
    }
]
```
To add the example3 language, modify the file as follows:
```
[
    {
        "filename": "example1.tex"
    },
    {
        "filename": "example2.tex"
    },
    {
        "filename": "example3.tex"
    }
]
```

- Open a terminal (or command prompt), navigate to the directory containing the [build.sh](ohos%2Fbuild%2Fbuild.sh) file, and run the following commands to compile the code:
```
chmod +x build.sh
./build.sh
```
After successful compilation, the compiled output will be placed in the ./out_hpb directory.
### 2. Parse Word Hyphenation Positions Using HPB
#### compilation Steps
Open a terminal (or command prompt), navigate to the directory containing the [hyphen_pattern_reader.cpp](ohos%2Fsrc%2Fhyphen-build%2Fhyphen_pattern_reader.cpp) file, and run the following command to compile the code:

```
cd ohos/src/hyphen-build/
g++ -g -Wall hyphen_pattern_reader.cpp -o reader
```
Explanation of the command:
- g++: Calls the GCC compiler.
- -g: Adds debugging information.
- -Wall: Enables all warnings.
- hyphen_pattern_reader.cpp: The source code file.
- -o reader: Specifies the output executable file name as reader.

#### Running Steps
After compilation, you can parse the hyphenation positions of words in the specified language using the following command:

```
./reader hyph-en-us.hpb helloworld
```
Explanation of the command:
- ./reader: Runs the generated reader executable.
- hyph-en-us.hpb: The input file (the binary file to be parsed).
- helloworld: The word to be parsed.

After successful execution, the log will output the hyphenation information of the parsed word.

### 3. Batch Verification
You can use the [generate_report.py](ohos%2Ftest%2Fgenerate_report.py) Python script to read the [report_config.json](ohos%2Ftest%2Freport_config.json) configuration file and perform batch verification to check the validity of the generated binary files.
#### Preparation
- Python 3.x
- transform and reader executables, placed in the same directory as the script.
- report_config.json configuration file

#### Usage
1. Prepare the Configuration File First, create a JSON configuration file named report_config.json with the following content:
```
{
    "file_path": "path/to/tex/files",
    "tex_files": [
        {
            "filename": "example.tex",
            "words": ["word1", "word2", "word3", "word4", "word5", "word6", "word7", "word8", "word9", "word10"]
        },
        ...
    ]
}
```
2. Run the Script Run the following command in the terminal:
```
python generate_report.py report_config.json
```
3. Log Files The script will generate a timestamped subdirectory under the report directory, containing the following log files:
```
match.log: Records successful matches.
unmatch.log: Records unsuccessful matches.
```

# 三方开源软件tex-hyphen

## tex-hyphen简介
tex-hyphen是一个用于TeX系统的断字模式库，它能够在多种语言中正确地将单词断行，以改善排版效果。

来源：tex-hyphen
URL：https://github.com/hyphenation/tex-hyphen
版本：CTAN-2021.03.21
License：多种组合

## 引入背景陈述
在多语言文档处理和排版中，正确的断字处理是至关重要的。tex-hyphen 提供了一套通用的断字模式，支持多种语言，为高质量的排版提供了基础保障。在OpenHarmony中，引入tex-hyphen可以显著提升多语言文档的排版质量。

## 语种归类
tex目录下包含了多个来自TeX hyphenations patterns的连字符规则，不同语种使用的开源许可证各不相同，整理归类：
* MIT License
* GPL,GPL 2
* LGPL 1,LGPL 2.1
* LPPL 1,LPPL 1.2,LPPL 1.3
* MPL 1.1
* BSD 3

OHOS中使用以下语种资源，以下语种均使用友好型开源协议：
* be - 白俄罗斯语（Belarusian）
* cs - 捷克语（Czech）
* cy - 威尔士语（Welsh）
* da - 丹麦语（Danish）
* de-1901 - 德语（German,1901orthography）
* de-ch-1901 - 瑞士德语（SwissGerman,1901orthography）
* el-monoton - 现代希腊语（ModernGreek,monotonic）
* el-polyton - 现代希腊语（ModernGreek,polytonic）
* en-gb - 英式英语（BritishEnglish）
* en-us - 美式英语（AmericanEnglish）
* es - 西班牙语（Spanish）
* et - 爱沙尼亚语（Estonian）
* fr - 法语（French）
* ga - 爱尔兰语（Irish）
* gl - 加利西亚语（Galician）
* hr - 克罗地亚语（Croatian）
* hu - 匈牙利语（Hungarian）
* hy - 亚美尼亚语（Armenian）
* id - 印度尼西亚语（Indonesian）
* is - 冰岛语（Icelandic）
* it - 意大利语（Italian）
* ka - 格鲁吉亚语（Georgian）
* lt - 立陶宛语（Lithuanian）
* lv - 拉脱维亚语（Latvian）
* mk - 马其顿语（Macedonian）
* mn-cyrl - 蒙古语（Mongolian,Cyrillicscript）
* nl - 荷兰语（Dutch）
* pt - 葡萄牙语（Portuguese）
* ru - 俄语（Russian）
* sh-cyrl - 塞尔维亚-克罗地亚语（Serbo-Croatian,Cyrillicscript）
* sh-latn - 塞尔维亚-克罗地亚语（Serbo-Croatian,Latinscript）
* sk - 斯洛伐克语（Slovak）
* sl - 斯洛文尼亚语（Slovenian）
* sr-cyrl - 塞尔维亚语（Serbian,Cyrillicscript）
* sv - 瑞典语（Swedish）
* th - 泰语（Thai）
* tk - 土库曼语（Turkmen）
* tr - 土耳其语（Turkish）
* uk - 乌克兰语（Ukrainian）
* zh-latn-pinyin - 汉语拼音（Chinese,Pinyin）

## 目录结构

```
third_party_tex-hyphen
├── collaboration
│   ├── original
│   ├── repository
│   └── source
├── data/language-codes
├── docs
│   └── languages
├── encoding
│   └── data
├── hyph-utf8
│   ├── doc
│   ├── source
│   └── tex
├── misc
├── ohos
│   ├── src
│   └── hpb-binary
├── old
├── source
├── tests
├── TL
├── tools
└── webpage

collaboration/       tex-hyphen官网依赖的js脚本、xml配置文件
ohos/                OpenHarmony编译文件和hpb二进制文件
data/                语种库
docs/                项目的文档资料
encoding/            包含编码相关的文件，用于处理不同字符集的编码问题。
hyph-utf8/           TeX 的断字模式包，提供了以 UTF-8 编码的断字模式
misc/                en-gb语种断词文件案例
old/                 包含一些旧的断词模式文件，这些文件可能已经被更新或替换。
source/              包含源代码文件，用于生成和处理断词模式。
TL/                  tlpsrc资源文件，tlpsrc文件是TeX Live系统中的一个包源文件，用于描述TeX Live包的元数据
tools/               包含一些工具脚本，用于辅助处理断词模式文件。
webpage/             tex-hyphen官网主页，提供了关于 hyph-utf8 包的详细信息和资源
```

## 为 OpenHarmony 带来的价值

**1.提高排版质量：** 通过引入 tex-hyphen，OpenHarmony 可以实现更为精准的断字处理，提高文档的可读性和美观度。
**2.提升小屏设备体验：** 在小屏设备中使用断词模式，能够在相同区域内显示更多内容，提升阅读体验。

## OpenHarmony中如何使用tex-hyphen

### 1、编译hpb二进制
#### 编译步骤
打开终端（或命令提示符），导航到包含 [hyphen_pattern_processor.cpp](ohos%2Fsrc%2Fhyphen-build%2Fhyphen_pattern_processor.cpp) 文件的目录，并运行以下命令来编译代码：

```
cd ohos/src/hyphen-build/
g++ -g -Wall hyphen_pattern_processor.cpp -o transform
```
上述命令说明：
- g++: 调用 GCC 编译器。
- -g: 添加调试信息。
- -Wall: 启用所有警告。
- hyphen_pattern_processor.cpp: 源代码文件。
- -o transform: 指定输出的可执行文件名为 transform。

#### 运行步骤
编译完成后，可以使用以下命令来运行生成的可执行文件，并处理指定的 .tex 文件：
```
./transform hyph-en-us.tex ./out/
```
上述命令说明：
- ./transform: 运行生成的 transform 可执行文件。
- hyph-en-us.tex: 输入文件（待处理的 .tex 文件）。
- ./out/: 输出目录（处理后的文件将存储在此目录中）。

运行成功后，处理后的.hpb二进制文件将存储在 ./out/ 目录中。

#### 批量编译
- 依赖：
```
jq：json文件解析工具
```

- 通过json配置文件[build-tex.json](ohos%2Fbuild%2Fbuild-tex.json)，配置需要编译的文件。
```
[
    {
        "filename": "example1.tex"
    },
    {
        "filename": "example2.tex"
    }
]
```
**filename** ：指定需要编译的tex文件名，文件须在 [tex](hyph-utf8%2Ftex%2Fgeneric%2Fhyph-utf8%2Fpatterns%2Ftex) 目录下

build-tex.json文件中定义了全量支持语种，脚本会默认全量编译。开发者可通过修改build-tex.json来控制新增或者删除语种。
例如：
需要移除example2语种，修改结果如下：
```
[
    {
        "filename": "example1.tex"
    }
]
```
需要新增example3语种，修改结果如下：
```
[
    {
        "filename": "example1.tex"
    },
    {
        "filename": "example2.tex"
    },
    {
        "filename": "example3.tex"
    }
]
```

- 打开终端（或命令提示符），导航到包含 [build.sh](ohos%2Fbuild%2Fbuild.sh) 文件的目录，并运行以下命令来编译代码
```
chmod +x build.sh
./build.sh
```
编译成功后，编译产物将会放置在./out_hpb目录下

### 2、通过hpb解析单词断词位置
#### 编译步骤
打开终端（或命令提示符），导航到包含 [hyphen_pattern_reader.cpp](ohos%2Fsrc%2Fhyphen-build%2Fhyphen_pattern_reader.cpp) 文件的目录，并运行以下命令来编译代码：

```
cd ohos/src/hyphen-build/
g++ -g -Wall hyphen_pattern_reader.cpp -o reader
```
上述命令说明：
- g++: 调用 GCC 编译器。
- -g: 添加调试信息。
- -Wall: 启用所有警告。
- hyphen_pattern_reader.cpp: 源代码文件。
- -o reader: 指定输出的可执行文件名为 reader。

#### 运行步骤
编译完成后，可以使用以下命令来解析对应语种的单词：
```
./reader hyph-en-us.hpb helloworld
```
上述命令说明：
- ./reader: 运行生成的 reader 可执行文件。
- hyph-en-us.hpb: 输入文件（待解析的二进制文件）。
- helloworld: 待解析的单词。

运行成功后，日志中将会输出本次解析的单词断词信息。

### 3、自动化验证
通过[generate_report.py](ohos%2Ftest%2Fgenerate_report.py) Python脚本读取[report_config.json](ohos%2Ftest%2Freport_config.json)配置文件，可实现批量校验生成的二进制文件是否有效
#### 准备
- Python 3.x
- transform和reader可执行文件，并将可执行文件放在脚本同一级目录
- report_config.json配置文件
#### 使用方法
1. 准备配置文件
首先，创建一个JSON格式的配置文件report_config.json，包含以下内容：
```
   {
       "file_path": "path/to/tex/files",
       "tex_files": [
           {
           "filename": "example.tex",
           "words": ["word1", "word2", "word3", "word4", "word5", "word6", "word7", "word8", "word9", "word10"]
           },
       ...
       ]
   }
```
- file_path：TeX文件所在的目录路径。
- tex_files：包含多个TeX文件及其对应的单词列表。
2. 运行脚本
在终端中运行以下命令：
```
python generate_report.py report_config.json
```
3. 日志文件
脚本会在 report 目录下生成一个带有时间戳的子目录，包含以下日志文件：
```
match.log：记录匹配成功的结果。
unmatch.log：记录匹配失败的结果。
```