README.md
1# hacksaw
2
3**HACK** in a **S**peedy **A**ccess **W**orkspace
4
5## What is Hacksaw?
6
7If you have a large multi-gigabyte codebase spread out through multiple git projects it can take a long time branch off a clean workspace. Hacksaw is a tool that
8lets you split off a clean workspace in seconds. It does so by only copying git projects that you
9explicitly select to be edited. All other projects are read-only bind mounts. This lets you build without cloning the full codebase to a new location!
10
11## How much faster is it, really?
12
13Lets look at some performance numbers for creating a hacksaw workspace using as a codebase the AOSP master branch as of 2020-8-4. The machine used was a c2-standard-60 Google Cloud Platform VM with 60 vCPUs and 240 GiB of RAM. Each action was performed at least 10 times then averaged out.
14
15* Create a new Hacksaw workspace
16 + Time: 0.4 sec
17 + Disk usage: 7.9 MiB
18
19* Remove a Hacksaw workspace with no edits or build artifacts.
20 + Time: 0.6 sec
21
22* Create a new Hacksaw workspace and edit build/make project.
23 + Time: 0.6 sec
24 + Disk usage: 18 MiB
25
26* Create a new Hacksaw workspace and edit frameworks/base project.
27 + Time: 7.5 sec
28 + Disk usage: 1.3 GiB
29
30As you can see, the time it takes to set up a new hacksaw workspace is proportional to
31the git projects checked out for editing. Contrast that with how long it takes
32to create a workspace using a full repo sync with a local
33mirror.
34
35* Create a new full repo workspace [using a fresh local mirror](https://source.android.com/setup/build/downloading#using-a-local-mirror)
36 + Time: 12 min 32 sec
37 + Disk usage: 88 GiB
38
39* Remove a full repo workspace with no build artifacts
40 + Time: 28 seconds
41
42## Can you give me an example?
43
44```
45$ mkdir ~/aosp
46$ cd ~/aosp
47$ repo init -u https://android.googlesource.com/platform/manifest
48...
49$ repo sync --quiet --current-branch --no-tags --no-clone-bundle --jobs=$(nproc)
50...
51$ hacksaw codebase add aosp ~/aosp
52Added codebase aosp
53$ hacksaw codebase default aosp
54Default codebase set to aosp
55$ hacksaw workspace new big-feature
56Composing.................................................................
57..........................................................................
58..........................................................................
59..........................................................................
60..........................................................................
61..........................................................................
62..........................................................................
63..........................................................................
64..........................................................................
65..........................................................................
66..........................................................................
67...........................................
68Workspace composed
69Created big-feature at ~/hacksaw/big-feature
70$ hacksaw edit ~/hacksaw/big-feature/tools/treble
71Created branch big-feature on project ~/hacksaw/big-feature/tools/treble
72$ hacksaw workspace new quick-fix
73Composing.................................................................
74..........................................................................
75..........................................................................
76..........................................................................
77..........................................................................
78..........................................................................
79..........................................................................
80..........................................................................
81..........................................................................
82..........................................................................
83..........................................................................
84...........................................
85Workspace composed
86Created big-feature at ~/hacksaw/quick-fix
87$ hacksaw edit ~/hacksaw/quick-fix/tools/treble
88Created branch quick-fix on project ~/hacksaw/quick-fix/tools/treble
89```
90
91## How do I install it?
92
93Building hacksaw requires [golang to be installed](https://golang.org/doc/install).
94To install the hacksaw client run the following:
95
96```
97go get android.googlesource.com/platform/tools/treble.git/hacksaw/cmd/hacksaw
98```
99
100This will install hacksaw to ~/go/bin/hacksaw. You may choose to copy that
101to a location in your path. For example:
102
103```
104sudo cp ~/go/bin/hacksaw /usr/local/bin
105sudo chmod 755 /usr/local/bin/hacksaw
106```
107
108## How do I make sure that creating a hacksaw workspace is fast?
109
110Hacksaw creates bind mounts for all git projects in a codebase. It then
111copies **everything** else. Make sure you remove all build artifacts from a
112codebase before create a workspace, otherwise it may spend a long time copying
113them.
114
115## How do I run it with sudo?
116
117Commands that mount and unmount will require sudo. That includes commands like
118
119* `hacksaw workspace new`
120* `hacksaw edit`
121* `hacksaw workspace remove`
122
123Other commmands like `hacksaw workspace list` or `hacksaw add codebase` do not
124mount or unmount so do not require sudo.
125
126If you would like to avoid using sudo you may install hacksawd as explained below.
127
128## How do I run it without sudo?
129
130Hacksawd is a privileged system daemon whose only job is to manage bind mounts.
131The provided install script will install to your system
132
133```
134go get android.googlesource.com/platform/tools/treble.git/hacksaw/cmd/hacksawd
135sudo cp ~/go/bin/hacksawd /usr/local/bin
136sudo chmod 755 /usr/local/bin/hacksawd
137sudo ~/go/src/android.googlesource.com/platform/tools/treble.git/hacksaw/scripts/install-service.sh
138```
139
140The installation scripts creates a new "hacksaw" group and adds you to it. You
141will need to log out and log back in for the group changes to take effect. After that you should be able to run any hacksaw command without sudo.
142
143If you wish to uninstall the service then run:
144
145```
146sudo ~/go/src/android.googlesource.com/platform/tools/treble.git/hacksaw/scripts/uninstall-service.sh
147sudo rm /usr/local/bin/hacksawd
148```
149## How do I sync?
150
151You sync your codebases using `repo sync`. All updates will be propagated to workspaces.
152Except for projects that you are currently editing. Those will require you to `git pull`
153manually in the workspace project.
154
155## How does hacksaw work?
156
157Hacksaw uses read-only bind mounts to create project references from
158a workspace to a codebase. When you mark a project for editing then
159its read-only bind mount gets replaced by a writable Git worktree.
160
161
162
163
164## What are the known issues?
165
166* Some repo commands don't work yet. Namely: `repo start` and `repo upload`.
167 So at the moment you can only upload to Gerrit [using git
168 push](https://gerrit-review.googlesource.com/Documentation/user-upload.html#_git_push).
169* Failing to create a workspace is not rolled back.
170* Editing nested projects is not supported yet. So if you have a git project
171 that contains other git projects you will get some unexpected behaviour.
172* Git submodules are not supported yet, but the tool is designed with
173 future git submodule support in mind.
174* Syncing a codebase does update the existing projects in all attached
175 workspaces but it does not remove or add new projects. Perhaps there
176 should be a new "workspace sync" command for that?
177
178## Where can I get more help?
179
180You can ask hacksaw-users@googlegroups.com by [joining the group](https://groups.google.com/forum/#!forum/hacksaw-users).
