Download AOSP Source Code (repo init & repo sync) | Android Internals From Scratch Ep 4
Summary
This video explains the process of downloading and synchronizing the Android 14 source code using the 'repo' tool. It covers the necessity of repo for managing over 800 Git repositories, the steps for initialization with the correct manifest and branch, and the importance of a local manifest for specific hardware like the Raspberry Pi. The guide emphasizes storage requirements, command breakdowns (repo init, repo sync), and verifying the successful download of all source files.
Key Insights
Repo tool simplifies managing multi-repository Android builds.
Repo is a tool developed by Google specifically for managing Android builds that involve numerous repositories. It utilizes a manifest file (XML) to list all repositories, their remote URLs, branches, and local placement, facilitating parallel fetching and synchronized management of changes across these repositories.
Specifying a precise branch/tag ensures reproducible builds.
Using a specific release tag, like 'R67' for Android 14, is crucial because it provides a reproducible build. This ensures that every sync fetches the exact same source code, including all security patches and updates applied up to that specific release.
Local manifests integrate hardware support by adding necessary repositories.
For Raspberry Pi support, a local manifest adds repositories like the device tree, specific kernel source, and HAL implementations. Without this, a generic AOSP build would not be compatible with Raspberry Pi hardware.
Repo sync can be resumed if interrupted without losing progress.
If the synchronization process is interrupted (e.g., due to connection drops or accidental terminal closure), running 'repo sync' again will resume the download from where it left off, as the tool tracks already downloaded content.
Sections
Introduction to Android Source Code Management
Android source code is split across over 800 Git repositories, not a single large one.
The Android source code is organized into more than 800 individual Git repositories. This includes separate repositories for the kernel, framework, each system service, hardware interfaces, and build tools. Managing these manually would be impractical.
Repo tool simplifies managing multi-repository Android builds.
Repo is a tool developed by Google specifically for managing Android builds that involve numerous repositories. It utilizes a manifest file (XML) to list all repositories, their remote URLs, branches, and local placement, facilitating parallel fetching and synchronized management of changes across these repositories.
Two primary commands: repo init and repo sync.
The core commands for managing the Android source code are 'repo init,' which reads the manifest and configures the local workspace, and 'repo sync,' which fetches the actual source code from all listed repositories.
Setting Up the Workspace
Create a dedicated, explicitly named directory for the source code.
It is recommended to create a specific directory for the Android source code, such as 'AOSP-Android 14', to clearly identify the version and avoid confusion when working with multiple versions.
Ensure sufficient storage space (350-400 GB) before synchronization.
A minimum of 350 to 400 GB of free storage is required on the drive before proceeding with the synchronization process. This is due to the large size of the aggregated source code from hundreds of repositories.
Initializing the Repository
Use ‘repo init’ with URL and specific branch for desired Android version.
The 'repo init' command initializes the workspace. The '-u' flag specifies the URL of the manifest repository (e.g., Google's official AOSP manifest on googlesource.com), and the '-b' flag specifies the branch or tag (e.g., 'Android 14.0.0 R67').
Specifying a precise branch/tag ensures reproducible builds.
Using a specific release tag, like 'R67' for Android 14, is crucial because it provides a reproducible build. This ensures that every sync fetches the exact same source code, including all security patches and updates applied up to that specific release.
.repo directory stores workspace configuration and manifest files.
After 'repo init' completes, a hidden '.repo' directory is created. This directory acts as the control center for the workspace, containing the active 'manifest.xml' file, a directory with all manifest files, and the 'repo' tool itself.
Manifest files define all repositories for the source tree.
The 'manifest.xml' file (often referencing 'default.xml') lists every single repository involved in the build. Each project entry specifies its name, path, and sometimes a specific version, dictating where each piece of Android source code is placed locally.
Adding Device-Specific Manifests (Raspberry Pi Example)
Local manifests are separate XML files for device-specific additions.
Local manifests are additional XML files placed in the '.repo/local_manifests' directory. They are used to include repositories not present in the main AOSP manifest, such as hardware support for specific devices like the Raspberry Pi.
Local manifests integrate hardware support by adding necessary repositories.
For Raspberry Pi support, a local manifest adds repositories like the device tree, specific kernel source, and HAL implementations. Without this, a generic AOSP build would not be compatible with Raspberry Pi hardware.
Utilize flags like -L and --create-dirs for efficient manifest download.
When downloading a local manifest using tools like 'curl', flags like '-L' (to follow redirects) and '--create-dirs' (to automatically create the necessary directory structure) streamline the process and prevent errors.
Raspberry Pi repositories include device tree and kernel source.
The downloaded local manifest for Raspberry Pi integrates repositories containing the device tree (describing hardware to Android), the specific Linux kernel patched for Raspberry Pi 4, and vendor-specific files and configurations.
Synchronizing the Source Code
'repo sync' fetches all source code from defined repositories.
The 'repo sync' command is the primary command for downloading the actual source code. It fetches content from every repository listed in both the main and any local manifests, downloading roughly 250 GB of data across over 800 repositories.
Synchronization can take hours to a full day depending on internet speed.
Due to the large amount of data and the number of repositories involved, the 'repo sync' process can take a significant amount of time, ranging from a couple of hours to an entire day, heavily dependent on the user's internet connection speed.
Repo sync can be resumed if interrupted without losing progress.
If the synchronization process is interrupted (e.g., due to connection drops or accidental terminal closure), running 'repo sync' again will resume the download from where it left off, as the tool tracks already downloaded content.
Understanding the Synced Source Tree
The synced tree contains major directories like frameworks, system, and packages.
A fully synced Android 14 source tree includes major directories such as 'frameworks' (Android application framework), 'system' (native components like init, logging), 'packages' (system apps like settings, dialer), and 'hardware' (HAL definitions).
Device-specific configurations reside in the 'device' directory.
The 'device' directory, populated by the local manifest for Raspberry Pi, contains device-specific configurations, including the 'BoardConfig.mk' for hardware settings and 'device.mk' for build rules and feature flags necessary for the Raspberry Pi 4.
The 'kernel' directory houses the specific Linux kernel source for the device.
The 'kernel' directory contains the Linux kernel source code. For Raspberry Pi, this is a specific version patched to support the hardware when running Android, as added by the Raspberry Vanilla local manifest.
Repo can list all synced projects and their status.
The 'repo projects' command can be used to list all the Git repositories that have been successfully synced. This allows for verification that each project is checked out on the correct branch and that all repositories, including those from local manifests, are present.
The total download size confirms the completeness of the sync.
Checking the total size of the downloaded source code (e.g., 239 GB) serves as a final confirmation that the synchronization process has completed successfully and all necessary data has been retrieved.
Recap and Next Steps
Process recap: Directory creation, repo init, local manifest, repo sync.
The entire process involves creating a dedicated directory, initializing the repository with 'repo init' using the correct manifest and branch, adding any necessary local manifests for specific hardware, and finally running 'repo sync' to download all source code.
Next episode will explore the structure and components of the AOSP source tree.
The upcoming episode will focus on a detailed exploration of the AOSP source tree, explaining the purpose of major directories, the location of different components, and how they interconnect, aiming to foster a deeper understanding of Android's internal structure.
Ask a Question
*Uses 1 Wisdom coin from your coin balance
