README.md
1# Content
2
3<!-- TOC -->
4
5- [Overview](#overview)
6- [Model Architecture](#model-architecture)
7- [Dataset](#dataset)
8- [Environment Requirements](#environment-requirements)
9- [Quick Start](#quick-start)
10- [Script Detailed Description](#script-detailed-description)
11
12<!-- /TOC -->
13
14# Overview
15
16This folder holds code for Training-on-Device of a LeNet model. Part of the code runs on a server using MindSpore infrastructure, another part uses MindSpore Lite conversion utility, and the last part is the actual training of the model on some android-based device.
17
18# Model Architecture
19
20LeNet is a very simple network which is composed of only 5 layers, 2 of which are convolutional layers and the remaining 3 are fully connected layers. Such a small network can be fully trained (from scratch) on a device in a short time. Therefore, it is a good example.
21
22# Dataset
23
24In this example we use the MNIST dataset of handwritten digits as published in [THE MNIST DATABASE](http://yann.lecun.com/exdb/mnist/)
25
26- Dataset size：52.4M，60,000 28*28 in 10 classes
27 - Test：10,000 images
28 - Train：60,000 images
29- Data format：binary files
30 - Note：Data will be processed in dataset.cc
31
32- The dataset directory structure is as follows:
33
34```text
35mnist/
36├── test
37│ ├── t10k-images-idx3-ubyte
38│ └── t10k-labels-idx1-ubyte
39└── train
40 ├── train-images-idx3-ubyte
41 └── train-labels-idx1-ubyte
42```
43
44# Environment Requirements
45
46- Server side
47 - [MindSpore Framework](https://www.mindspore.cn/install/en): it is recommended to install a docker image
48 - MindSpore ToD Framework
49 - [Downloads](https://www.mindspore.cn/lite/docs/en/r1.5/use/downloads.html)
50 - [Build](https://www.mindspore.cn/lite/docs/en/r1.5/use/build.html)
51 - [Android NDK r20b](https://dl.google.com/android/repository/android-ndk-r20b-linux-x86_64.zip)
52 - [Android SDK](https://developer.android.com/studio?hl=zh-cn#cmdline-tools)
53- A connected Android device
54
55# Quick Start
56
57After installing all the above mentioned, the script in the home directory could be run with the following arguments:
58
59```bash
60sh ./prepare_and_run.sh -D DATASET_PATH [-d MINDSPORE_DOCKER] [-r RELEASE.tar.gz] [-t arm64|x86]
61```
62
63where:
64
65- DATASET_PATH is the path to the [dataset](#dataset),
66- MINDSPORE_DOCKER is the image name of the docker that runs [MindSpore](#environment-requirements). If not provided MindSpore will be run locally
67- RELEASE.tar.gz is a pointer to the MindSpore ToD release tar ball. If not provided, the script will attempt to find MindSpore ToD compilation output
68- target is defaulted to arm64, i.e., on-device. If x86 is provided, the demo will be run locally. Note that infrastructure is not optimized for running on x86. Also, note that user needs to call "make clean" when switching between targets.
69
70# Script Detailed Description
71
72The provided `prepare_and_run.sh` script is performing the following:
73
74- Prepare the trainable lenet model in a `.ms` format
75- Prepare the folder that should be pushed into the device
76- Copy this folder into the device and run the scripts on the device
77
78See how to run the script and parameters definitions in the [Quick Start Section](#quick-start)
79
80## Preparing the model
81
82Within the model folder a `prepare_model.sh` script uses MindSpore infrastructure to export the model into a `.mindir` file. The user can specify a docker image on which MindSpore is installed. Otherwise, the python script will be run locally.
83The script then converts the `.mindir` to a `.ms` format using the MindSpore ToD converter.
84The script accepts a tar ball where the converter resides. Otherwise, the script will attempt to find the converter in the MindSpore ToD build output directory.
85
86## Preparing the Folder
87
88The `lenet_tod.ms` model file is then copied into the `package` folder as well as scripts, the MindSpore ToD library and the MNIST dataset.
89Finally, the code (in src) is compiled for arm64 and the binary is copied into the `package` folder.
90
91### Running the code on the device
92
93To run the code on the device the script first uses `adb` tool to push the `package` folder into the device. It then runs training (which takes some time) and finally runs evaluation of the trained model using the test data.
94
95# Folder Directory tree
96
97``` python
98train_lenet/
99├── Makefile # Makefile of src code
100├── model
101│ ├── lenet_export.py # Python script that exports the LeNet model to .mindir
102│ ├── prepare_model.sh # script that export model (using docker) then converts it
103│ └── train_utils.py # utility function used during the export
104├── prepare_and_run.sh # main script that creates model, compiles it and send to device for running
105├── README.md # English manual
106├── README_CN.md # Chinese manual
107├── scripts
108│ ├── eval.sh # on-device script that load the train model and evaluates its accuracy
109│ └── train.sh # on-device script that load the initial model and train it
110├── src
111│ ├── net_runner.cc # program that runs training/evaluation of models
112│ ├── net_runner.h # net_runner header
113│ └── utils.h # general utilities
114```
115
116When the `prepare_and_run.sh` script is run, the following folder is prepared. It is pushed to the device and then training runs
117
118``` python
119├── package
120│ ├── bin
121│ │ └── net_runner # the executable that performs the training/evaluation
122│ ├── dataset
123│ │ ├── test
124│ │ │ ├── t10k-images-idx3-ubyte # test images
125│ │ │ └── t10k-labels-idx1-ubyte # test labels
126│ │ └── train
127│ │ ├── train-images-idx3-ubyte # train images
128│ │ └── train-labels-idx1-ubyte # train labels
129│ ├── eval.sh # on-device script that load the train model and evaluates its accuracy
130│ ├── lib
131│ │ ├── libjpeg.so.62
132│ │ ├── libminddata-lite.a
133│ │ ├── libminddata-lite.so
134│ │ ├── libmindspore-lite.a
135│ │ ├── libmindspore-lite-jni.so
136│ │ ├── libmindspore-lite.so
137│ │ ├── libmindspore-lite-train.a
138│ │ ├── libmindspore-lite-train-jni.so
139│ │ ├── libmindspore-lite-train.so
140│ │ ├── libturbojpeg.so.0
141│ │ └── mindspore-lite-java.jar
142│ ├── model
143│ │ └── lenet_tod.ms # model to train
144│ └── train.sh # on-device script that load the initial model and train it
145```
146