Android 11 and Debian 10 base image #31
Conversation
ypid
changed the title
|
Hello, Thanks for pinging me and the the interest in https://github.com/mobilesec/reproducible-builds-aosp . It's a project from the Institute of Networks and Security at the Johannes Kepler University Linz, with me being the main developer currently. There are indeed quite a few similarities between our project and this one. I did stumble upon this repository when researching the requirements for our project, but in the end settled to implement simple shell scripts based on the instruction from https://source.android.com myself. The primary goal of our project is to perform the official build instructions, only deviating when there's no other way (e.g. additional APT dependencies that are not properly documented), and analyzing the output w.r.t. to reference builds. Inspecting the scripts of this repository leads me to believe that there are certain changes that deviate from the reference instructions (e.g. With regards to the other projects you mentioned (GrapheneOS, LineageOS, and similar): A similar argument to the one from above applies to these. Many of them are focused on augmenting the standard AOSP builds with additional features or security measures, which is interesting in its own right, not quite what we need. If you know of any other project that aims to build AOSP in a completely unchanged way, please let me know (minus robotnix, whose author @danielfullmer we are already in touch with). However, you are fully correct that the lack of a locking mechanism of the input dependencies (i.e. apt and pip packages) is a weakness of our project as of now and something worth looking into for the future. |
|
Thanks for your answer.
This is configurable from the config file ;-) So to clarify, this project https://github.com/hashbang/aosp-build is then exactly what you are looking for. A build system for unmodified or with only necessary changes to build it AOSP. The stuff I was talking about is just my use case that is developed in another repo (https://github.com/hashbang/os) which just uses the Docker image and bundled default scripts aosp-build provides. I don’t know other projects except the once we already mentioned. Except https://ci.android.com which would be interesting to look at. But even if we had access to the scripts, I guess their use case and threat model is different. Lets do it like this, I can ping you when the Android 11 and Debian 10 work has been merged into master, and I have possibly tested building an unmodified AOSP with it, then you can take a look and discuss this in your department. |
It could be discussed if this exception might not be useful for linting purposes of the config.yml, but for development it is not useful to have the script fail just because remove_groups and remove_paths both match.
https://android.googlesource.com/kernel/manifest now contains prebuilts that are already hash locked.
It gets in the way.
This is expected to improve security because the AOSP build system uses nsjail so this sandbox can be way more confined than what we can create with Docker here. Further work on build isolation improvements should be done with nsjail so that it can benefit all AOSP users.
If you want the steps to run together, you can still call make with both targets. I just found that during development, I always want to run those steps separately and then inspect the `config.yml` before running the next step. Also, manifest takes much longer so it makes even more sense to review in between. config is pretty fast.
Unable to find latest build tag for factory build RP1A.201005.004 for device taimen Unable to find latest build tag for factory build RP1A.201005.004 for device walleye
Needed for using a different manifest source for example GrapheneOS.
A git repo per device is needed for the repo command. Before, it was confusing because there were to git repos created, on top repo `config/manifests` and then additionally one per device by the fetch script.
Before if an error happened you could not tell in which repo the issue occurred.
Now `-p` helps with this.
```
-e, --abort-on-errors
Abort if a command exits unsuccessfully
-p Show project headers before output
-v, --verbose Show command error messages
```
It seems the repo command runs this single only one job by default. So depending on your built system, using --jobs gives you a real performance gain. I have tested this an can confirm it.
@mpoel |
|
Thanks for keeping me updated. I tested your work on this PR (as of commit a550c23) in a clean installation of an Ubuntu 20.04 VM, attempting build a standard AOSP build for the crosshatch device. As instructed in the
Attached you find a full patch that encompasses the points mentioned above and enabled me run a full build. I hope this helps and is interesting to you. Obviously, feel free to ask for more details. I like your project and it's great to offer multiple backends (SOAP effectively has 3 as well, native, docker and Jenkins), but I don't want to promise you too much. We've already got our infrastructure set up and running well with our Jenkins integration for some basic monitoring. You probably know the saying with the running systems and such. But it would definitely be interesting to perform builds with aosp-build in addition to our SOAP builds and generate SOAP difference reports for them. Patch: |
Commit in updater after which the new metadata format is required: commit b51bf7a3c16b34539cce8b2792fbc8b89c05a89d Author: Daniel Micay <[email protected]> Date: 2020-06-02 13:47:14 -0400 sanity check channel name
|
That is very valuable feedback. Thank you! It also means that I did not break Android 10 support which is good to know :)
I did not bother to update manifests for devices other than sargo to be honest. Did that now.
I have been thinking about this as well. I will need to switch to https://snapshot.debian.org/.
Got it :) I will go over your other points in a few weeks. |
Done that. |
ypid
changed the title
|
I should probably merge that. Otherwise people would need to submit patches/base their work on my branch/this PR. |
I decided to push my draft WIP state now because I just found https://github.com/mobilesec/reproducible-builds-aosp which has many similarities with this project and I would like sync up with @mpoel.
Closes: #31
Hi @mpoel
It seems you have started a project on 2019-10-16 that is similar to this one. It might make sense to join forces? I would also be interested if you considered any of the known projects (refs: 1, 2) in the AOSP-based hardening community? I checked your code right now. You have slightly different goals than this project does, but still pretty similar direction. I am also very interested in reproducible builds. What your project does not do is locking all it’s build inputs. I will probably read over your Bachelor’s thesis the next days. Until then, I just want to ping you as you also worked on your project today, same as me.
This PR is a draft, but it can give you an idea of the current state of the project. Note that I have not tested building AOSP with this PR. I just built GrapheneOS with it for now.