Reproducible builds

[ThexXTURBOXx]

I am currently trying to get my app uploaded to F-Droid with reproducible builds.
I could solve all the issues except for the NT_GNU_BUILD_ID within the libdartjni.so files:
https://gitlab.com/ThexXTURBOXx/fdroiddata/-/jobs/13647445748

In this diffoscope output, I have compared the signed (!) apk from GitHub with the unsigned (!) apk from F-Droid. For this reason, the certificates are also a difference here (of course they wouldn't be if the libdartjni.so issue was resolved:

Diffoscope output

--- app-release.apk
+++ studip_uni_passau.femtopedia.de.unipassaustudip_203.apk
│┄ 'androguard' Python package not installed; cannot extract V2 signing keys.
│┄ 'apktool' not available in path. Format-specific differences are supported for Android APK files. Installing the 'apktool' package may produce better output.
├── /usr/lib/android-sdk/build-tools/debian/apksigner verify --verbose --print-certs {}
│┄ error from `/usr/lib/android-sdk/build-tools/debian/apksigner verify --verbose --print-certs {}` (b):
│┄ DOES NOT VERIFY
│┄ ERROR: Missing META-INF/MANIFEST.MF
│ @@ -1,17 +0,0 @@
│ -Verifies
│ -Verified using v1 scheme (JAR signing): false
│ -Verified using v2 scheme (APK Signature Scheme v2): true
│ -Verified using v3 scheme (APK Signature Scheme v3): false
│ -Verified using v3.1 scheme (APK Signature Scheme v3.1): false
│ -Verified using v4 scheme (APK Signature Scheme v4): false
│ -Verified for SourceStamp: false
│ -Number of signers: 1
│ -Signer #1 certificate DN: CN=Nico Mexis, L=Anger, ST=Bavaria, C=83454
│ -Signer #1 certificate SHA-256 digest: f17448cfb1bd29bc4b29932de068feab87175654dfac0e1a48605f2aeecebaef
│ -Signer #1 certificate SHA-1 digest: 1d3e9ce1194d756b0a38ef26f64f00550f0eaeca
│ -Signer #1 certificate MD5 digest: 5113e3699cecd214ae2e58adcc6c8db6
│ -Signer #1 key algorithm: RSA
│ -Signer #1 key size (bits): 2048
│ -Signer #1 public key SHA-256 digest: ecf8d5b43fd5bdeecf612ad7d0dfa8242e0de3f0def52cb8e85aa116944b91b1
│ -Signer #1 public key SHA-1 digest: 161bcb2b5b9df9b545cdb2cb97fba74a08445384
│ -Signer #1 public key MD5 digest: 7ab94e51668fb248bf4bafefefa7ce79
├── zipinfo {}
│ @@ -1,8 +1,8 @@
│ -Zip file size: 64922583 bytes, number of entries: 403
│ +Zip file size: 64914391 bytes, number of entries: 403
│  -rw-r--r--  0.0 unx       57 b- defN 81-Jan-01 01:01 META-INF/com/android/build/gradle/app-metadata.properties
│  -rw-r--r--  0.0 unx       46 b- defN 81-Jan-01 01:01 META-INF/version-control-info.textproto
│  -rw-r--r--  0.0 unx      934 b- stor 81-Jan-01 01:01 assets/dexopt/baseline.prof
│  -rw-r--r--  0.0 unx      177 b- stor 81-Jan-01 01:01 assets/dexopt/baseline.profm
│  -rw-r--r--  0.0 unx  2815536 b- defN 81-Jan-01 01:01 classes.dex
│  -rw-r--r--  0.0 unx  9044912 b- stor 81-Jan-01 01:01 lib/arm64-v8a/libapp.so
│  -rw-r--r--  0.0 unx   123592 b- stor 81-Jan-01 01:01 lib/arm64-v8a/libdartjni.so
├── lib/arm64-v8a/libdartjni.so
│┄ File has been modified after NT_GNU_BUILD_ID has been applied.
│ ├── readelf --wide --notes {}
│ │ @@ -1,8 +1,8 @@
│ │
│ │  Displaying notes found in: .note.android.ident
│ │    Owner                Data size   Description
│ │    Android              0x00000084  NT_VERSION (version)       description data: 15 00 00 00 72 32 38 63 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 31 33 36 37 36 33 35 38 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
│ │
│ │  Displaying notes found in: .note.gnu.build-id
│ │    Owner                Data size   Description
│ │ -  GNU                  0x00000014  NT_GNU_BUILD_ID (unique build ID bitstring)         Build ID: 6025c922e74df201380451eb87732148b921ca58
│ │ +  GNU                  0x00000014  NT_GNU_BUILD_ID (unique build ID bitstring)         Build ID: 060cc6845256ce4a4ee9b6694b7bd8f8943576a9
├── lib/armeabi-v7a/libdartjni.so
│┄ File has been modified after NT_GNU_BUILD_ID has been applied.
│ ├── readelf --wide --notes {}
│ │ @@ -1,8 +1,8 @@
│ │
│ │  Displaying notes found in: .note.android.ident
│ │    Owner                Data size   Description
│ │    Android              0x00000084  NT_VERSION (version)       description data: 15 00 00 00 72 32 38 63 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 31 33 36 37 36 33 35 38 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
│ │
│ │  Displaying notes found in: .note.gnu.build-id
│ │    Owner                Data size   Description
│ │ -  GNU                  0x00000014  NT_GNU_BUILD_ID (unique build ID bitstring)         Build ID: 25acbaa9e18314cab399a724862f2f38c29351d0
│ │ +  GNU                  0x00000014  NT_GNU_BUILD_ID (unique build ID bitstring)         Build ID: 24f84463e5d49b718dffdc1cc7a211ebe0a4d094
├── lib/x86_64/libdartjni.so
│┄ File has been modified after NT_GNU_BUILD_ID has been applied.
│ ├── readelf --wide --notes {}
│ │ @@ -1,8 +1,8 @@
│ │
│ │  Displaying notes found in: .note.android.ident
│ │    Owner                Data size   Description
│ │    Android              0x00000084  NT_VERSION (version)       description data: 15 00 00 00 72 32 38 63 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 31 33 36 37 36 33 35 38 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
│ │
│ │  Displaying notes found in: .note.gnu.build-id
│ │    Owner                Data size   Description
│ │ -  GNU                  0x00000014  NT_GNU_BUILD_ID (unique build ID bitstring)         Build ID: 87e9919a9f5ee9bf86357b22573151ed1e16858e
│ │ +  GNU                  0x00000014  NT_GNU_BUILD_ID (unique build ID bitstring)         Build ID: dd60108c29f25d1fa2544fc71814e1f3f0387b70

As far as I can tell, this problem was not reported previously here. Is this a limitation of the jni/jnigen package or is there a way to turn off adding the build ID or something else to remedy this issue?

Thank you very much in advance!

[ThexXTURBOXx]

I was able to fix this issue by adding this to my GitHub workflow:

- name: Match F-Droid SDK Path
  run: |
    sudo mkdir -p /opt/android-sdk
    sudo chown -R $USER:$USER /opt/android-sdk
    # Symlink the default GH SDK to the F-Droid path
    ln -s $ANDROID_HOME/* /opt/android-sdk/
    echo "ANDROID_HOME=/opt/android-sdk" >> $GITHUB_ENV
    echo "ANDROID_SDK_ROOT=/opt/android-sdk" >> $GITHUB_ENV

I can make the build-id for libdartjni.so reproducible if the Android SDK (and hence NDK) is installed in the same directory as it is on F-Droid.
However, I feel like this whole process is rather brittle as a tiny change in some of the internals can change the entire build-id.
Hence, I believe that this is still a real issue and some option to completely turn off the build-id should exist.

[liamappelbe]

Hi @ThexXTURBOXx. I took over package:jni pretty recently, and I'm not across all the details yet. Can you give me some more context for this issue? Why is build reproducibility important? Do you know what NT_GNU_BUILD_ID is? How would I go about turning off the build ID?

If you're familiar with this stuff, you could also submit a PR to fix this. That would be hugely appreciated.

[ThexXTURBOXx]

No problem at all!
Alas, I am also not the most knowledgable person in this regard, but I can at least share everything I came across when trying to find solutions to my problem.
F-Droid has a short section on how to turn off build IDs: https://f-droid.org/docs/Reproducible_Builds/#ndk-build-id
I am guessing (but I have no clue about the codebase of jni/jnigen) that somewhere must be some CMake-related file that is being generated or changed by jni/jnigen and one could use add_link_options there or LOCAL_LDFLAGS in some Android.mk file along the way to turn it off.

Please, don't get me wrong: the build ID has many upsides apparently. Many use it to exactly identify which build of a given binary is being currently used.
Hence, it should not be turned off by default. There should, however, be some option to disable it, I guess.

What the build ID consists of? I am not 100% sure about this as I found quite a range of different info on the web. I think it is some kind of hash that depends on the current environment some binary is built in, such as the install location of SDKs (this was the problem in my case), versions etc. However, build date and time do not seem to influence it (as otherwise, the build ID would not be reproducible, of course).

But that's everything I really could find out about it. You can find info about it by googling both NT_GNU_BUILD_ID or just build id ndk.
Hope that helps anyway!