[RNE Rewrite] feat: on-demand native lib download and backend splitting by msluszniak · Pull Request #1283 · software-mansion/react-native-executorch

msluszniak · 2026-06-25T11:12:31Z

Description

Ports PR #1039's on-demand native-lib mechanism into the rewrite flow. Apps declare what they need via a react-native-executorch block (backends / libs / features) in package.json; the postinstall scripts/download-libs.js expands features via the forward-looking FEATURE_MAP, writes rne-build-config.json, and downloads only the requested per-backend artifacts from GitHub Releases into third-party/. The podspec and android/build.gradle.kts read that config to gate RNE_ENABLE_* and link backends as separate libs (Xnnpack/CoreML/MLX xcframeworks, lib*_executorch_backend.so); MLX is the iOS device slice only.

Adapted to the rewrite: cpp/ layout (opencv kept as an extensible source list, guarded in RnExecutorch.cpp), Kotlin-DSL gradle config read, single android/CMakeLists.txt. The scripts, FEATURE_MAP, and artifact contract are kept identical to #1039 so future rewrite work stays code-only. CI (TS-only) skips the download via RNET_SKIP_DOWNLOAD in the setup action.

Backed by software-mansion-labs/executorch@ms/separate-backends (already ET 1.3.1). Release artifacts (device-only MLX) are built/uploaded separately; committed headers / executorch.jar / ExecutorchLib wrapper are provisioned alongside.

Introduces a breaking change?

Yes
No

Type of change

Bug fix (change which fixes an issue)
New feature (change which adds functionality)
Documentation update (improves or adds clarity to existing documentation)
Other (chores, tests, code style improvements etc.)

Tested on

iOS
Android

Testing instructions

Config flow: set an app's react-native-executorch.features (e.g. ["llm"]), run yarn install, and check rne-build-config.json reflects it (enableOpencv:false, enableCoreml:false, enableXnnpack:true). Setting legacy extras errors.
Download flow: RNET_BASE_URL=https://github.com/software-mansion/react-native-executorch/releases/download/v0.0.0-rewrite-libs-test RNET_TARGET=android-arm64-v8a INIT_CWD=<app> node scripts/download-libs.js — tarballs extract (checksum-verified) into third-party/android/libs/... exactly where CMakeLists.txt/podspec expect.
CI gates: yarn lint, yarn workspace react-native-executorch prepare, yarn typecheck (all green; install with RNET_SKIP_DOWNLOAD=1).

Test artifacts uploaded to the v0.0.0-rewrite-libs-test pre-release: Android freshly built from @ms/separate-backends (ET 1.3.1, split backends, arm64-v8a + x86_64); iOS reused from the same separate-backends 1.3.1 lineage (#1039) with MLX stripped to the device slice only. Full download/extract/checksum flow verified across iOS + both Android ABIs. On-device app build still to be run.

Related issues

#1208

Checklist

I have performed a self-review of my code
I have commented my code, particularly in hard-to-understand areas
I have updated the documentation accordingly
My changes generate no new warnings

Additional notes

Draft: all of third-party/ (headers + binaries) is downloaded on demand, not committed — headers ship as a platform-independent headers.tar.gz alongside the binary artifacts (built with device-only MLX), uploaded separately.

Header provenance: headers.tar.gz is assembled from the @ms/separate-backends ExecuTorch CMake install include tree (cmake-out*/include/: executorch, pytorch/{c10,torch,headeronly}, the tokenizer third-party headers absl/re2/nlohmann, and cpuinfo/pthreadpool — all platform-independent, identical across ABIs/SDKs) plus the opencv2 headers from the OpenCV prebuilt (same source as the opencv-rne pod / Android static libopencv_*.a, not built from executorch).

Docs getting-started is deferred (no rewrite getting-started page exists yet).

Port PR #1039's on-demand native-lib mechanism into the rewrite. Apps opt in via a `react-native-executorch` block (`backends`/`libs`/`features`) in their package.json; the postinstall `scripts/download-libs.js` expands `features` via the full forward-looking FEATURE_MAP, writes `rne-build-config.json`, and downloads only the requested per-backend artifacts from GitHub Releases into third-party/. The podspec and android/build.gradle.kts read that config to gate `RNE_ENABLE_*` and link backends as separate libs (Xnnpack/CoreML/MLX xcframeworks, lib*_executorch_backend.so) — MLX device-slice only. Adapted to the rewrite: cpp/ layout (opencv gated as an extensible source list, guarded in RnExecutorch.cpp), Kotlin-DSL gradle config read, single android/CMakeLists.txt. scripts + FEATURE_MAP + artifact contract kept identical to #1039 so future rewrite work is code-only. CI (TS-only) skips the download via RNET_SKIP_DOWNLOAD in the setup action. Verified: config expansion, download/extract/checksum (local server), and CI gates (lint, bob build, typecheck). Release artifacts (executorch fork build) + committed headers/jar/ExecutorchLib wrapper are provisioned separately.

…ice-only MLX)

Rebased onto rne-rewrite incl. #1280 keypoint detection. Rename the poseEstimation FEATURE_MAP entry to keypointDetection (tracks the useKeypointDetector hook) and set backends to xnnpack+coreml+mlx (RF-DETR keypoint ships CoreML + MLX variants) + opencv. Declare it in the computer-vision demo app.

Drop the committed third-party/include header set (~200K LOC of vendored ExecuTorch/c10/torch/opencv headers). Headers are now downloaded like the binaries: a platform-independent headers.tar.gz fetched by download-libs.js and produced by package-release-artifacts.sh. Keeps the rewrite's 'no third-party in git' philosophy and avoids churn on ExecuTorch bumps.

core-android/core-ios drop the separately-shipped pthreadpool+cpuinfo (statically linked into libexecutorch.so / libthreadpool_*.a for the rewrite), and the ABI-independent executorch.jar rides in the core-android-arm64 artifact (downloaded to third-party, not committed).

Two gaps surfaced building apps/computer-vision on a physical device: - libexecutorch.so references OpenMP runtime symbols (optimized kernels), so link -fopenmp -static-openmp into libRnExecutorch.so (matches main #1039). - abiFilters was hardcoded to both ABIs, ignoring the app's reactNativeArchitectures; read it (filtered to arm64-v8a/x86_64) so device builds only compile + need the ABI they target. Build now succeeds, installs, and launches on device (arm64-v8a).

Add a Fundamentals docs tree to the rewrite (docs/docs was removed in the scaffold): - 01-getting-started: ported canonical content + a "Selecting native libraries" section covering the react-native-executorch package.json block (features/backends/libs) and the feature -> backend/lib mapping. - 02-native-libraries: how the split is produced, shipped, and linked (download flow, artifact set, header provenance, Android/iOS wiring, force-load rationale, build recipe), corrected for the rewrite (@ms/separate-backends, ET 1.3.1, device-only MLX, downloaded headers). - sidebars: current version exposes Fundamentals + API Reference; the not-yet-ported sections stay commented until their dirs are restored. Fix the docs build, broken on the rewrite independently of these pages: - restore packages/.../tsconfig.doc.json so typedoc can load the project. - convert two URL-target {@link} TSDoc tags to markdown links so the generated API reference compiles under MDX. Also fix a variable-shadowing lint error in download-libs.js and add the new technical terms to the cspell wordlist.

barhanc · 2026-06-26T12:15:25Z

Why are these scripts nested inside packages/react-native-executorch and not in the monorepo root (like the error gen scripts)?

download-libs.js must live in the package: it runs at the package's postinstall, ships in the npm tarball, and resolves third-party/ relative to itself (unlike the dev-only error-gen scripts at root). package-release-artifacts.sh is its build-side pair (produces exactly the layout download-libs.js consumes), so it's colocated. Happy to move the release script to the monorepo root if you'd prefer.

barhanc · 2026-06-26T12:28:59Z

Also, did you check the example app size with and without the split and how much it decreases it?
Would you be able to check that MLX backend gets correctly bundled for the computer-vision app (I don't have an iOS physical device available)?

Do we have some custom script for vendoring the executorch headers, because the headers downloaded using the split script do not match the ones copied directly from executorch repo used in rnet-poc, e.g. all llm runner helpers are missing?

…native-libs # Conflicts: # .cspell-wordlist.txt

- podspec: stop wrapping `$(PODS_TARGET_SRCROOT)` paths in File.expand_path (it baked a malformed `<dir>/$(PODS_TARGET_SRCROOT)/...` into the linker flags); use the literal Xcode build variable like HEADER_SEARCH_PATHS does. - download-libs.js: only send GITHUB_TOKEN to github.com hosts (not to the githubusercontent.com presigned redirect target). - third-party/.gitignore: ignore the downloaded executorch.jar. - RnExecutorch.cpp: drop the redundant comment on the opencv include guard.

msluszniak · 2026-06-26T13:40:51Z

Also, did you check the example app size with and without the split and how much it decreases it?

Would you be able to check that MLX backend gets correctly bundled for the computer-vision app (I don't have an iOS physical device available)?

Do we have some custom script for vendoring the executorch headers, because the headers downloaded using the split script do not match the ones copied directly from executorch repo used in rnet-poc, e.g. all llm runner helpers are missing?

Haven't check yet, will do this in a minute.
Unfortunately, neither do I, so we need to defer this check till Monday.
That's my fault. There is no such script and that's the reason why I missed headers. I will add such script and update published assets.

The headers.tar.gz set was assembled by an ad-hoc copy of the executorch CMake install include tree, which is incomplete: it omits the source-only headers (extension/llm/{runner,custom_ops,apple,sampler}) that the rewrite's LLM/multimodal tasks compile against directly, and the codegen'd kernels/*/Functions.h. vendor-headers.sh assembles the full, reproducible set from its four real sources — executorch C++ source headers, build-generated/installed headers, c10/torch from the xcframework, and opencv2 — laid out to satisfy both the nested tokenizer include paths (CMakeLists) and the root -I. The result is a superset of both the current RNE main header set and the CV PoC. package-release-artifacts.sh now hard-fails if third-party/include is missing the runner headers, and the native-libraries doc documents the step.

msluszniak · 2026-06-26T15:12:29Z

@barhanc I added mentioned script, now working on memory comparison for cv demo app.

msluszniak · 2026-06-26T15:42:35Z

App-size impact of the backend split

Measured on the computer-vision demo (release APK, arm64-v8a). iOS figures are artifact-derived (no IPA built yet). Native .so are stored uncompressed, so they land in the APK at full size.

CV app — split vs all-backends

	vs complete bundle	vs RNE-only files
Android (measured)	78.3 → 68.3 MB (−10.0 MB, −13%)	25.7 → 15.7 MB (−10.0 MB, −39%)
iOS (estimated)	~0	~0

iOS is ~0 here because the CV app uses every iOS backend (xnnpack + coreml + mlx via keypointDetection). Android RNE-only footprint = libexecutorch core 11.3 + libxnnpack… 2.6 + libRnExecutorch (+ statically-linked opencv) 1.9, plus Vulkan 10.0 in the no-split build.

What the split drops, per backend (the cross-app value)

Component	Platform	Dropped	Notes
Vulkan	Android	10.0 MB	dynamic `.so`, full size
MLX	iOS	~13 MB `.a` + 2.7 MB `mlx.metallib`	`-force_load` → not stripped; metallib is a non-strippable resource
CoreML	iOS	~1.8 MB `.a`	`-force_load`
XNNPACK	both	2.6 / 7 MB	almost always needed
opencv	both	~1.5 MB effective	static + dead-stripped, so ≪ its 13 MB of `.a`
phonemis	both	~1.5–2.5 MB (once Kokoro TTS lands)	source-compiled into the lib; the 22 MB espeak `data/` is stripped from the npm package

A conservative, high-signal clang-tidy config plus a runner and a (dormant) CI workflow. - .clang-tidy: bugprone-* / performance-* / clang-analyzer-* (minus the noisy easily-swappable-parameters and enum-size), analyzing only our own headers. - scripts/clang-tidy.sh + `lint:cpp`: run clang-tidy over cpp/ using compile_flags.txt, failing on any finding. - .github/workflows/clang-tidy.yml: gated behind ENABLE_CLANG_TIDY; provisions headers via the on-demand download flow (download-libs.js, #1283) since clang-tidy needs the ExecuTorch headers to parse the sources. - dtype.cpp: NOLINT the intentional identical-size switch branches.

Provisions only headers.tar.gz (no per-target native libs) when RNET_HEADERS_ONLY is set — the integration point for clang-tidy / IDE tooling that needs include paths but never links or runs the binaries. The build config is still written. Also documents the existing RNET_NO_X86_64 opt-out in the README/env-var lists.

msluszniak added 2 commits June 25, 2026 13:41

[RNE Rewrite] chore: add ExecutorchLib xcframework build wrapper (dev…

c385df2

…ice-only MLX)

msluszniak force-pushed the @ms/on-demand-native-libs branch from 94f8547 to ab793e1 Compare June 25, 2026 11:43

msluszniak added 2 commits June 25, 2026 13:49

msluszniak force-pushed the @ms/on-demand-native-libs branch from ab793e1 to 45ec2cb Compare June 25, 2026 11:53

msluszniak self-assigned this Jun 25, 2026

msluszniak added refactoring improvement PRs or issues focused on improvements in the current codebase labels Jun 25, 2026

msluszniak added 2 commits June 25, 2026 14:19

msluszniak marked this pull request as ready for review June 25, 2026 14:15

msluszniak requested a review from barhanc June 26, 2026 09:28

barhanc reviewed Jun 26, 2026

View reviewed changes

msluszniak added 2 commits June 26, 2026 14:58

Merge remote-tracking branch 'origin/rne-rewrite' into @ms/on-demand-…

c27de00

…native-libs # Conflicts: # .cspell-wordlist.txt

msluszniak mentioned this pull request Jun 26, 2026

[RNE Rewrite] chore: add clang-tidy static checks #1286

Draft

12 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[RNE Rewrite] feat: on-demand native lib download and backend splitting#1283

[RNE Rewrite] feat: on-demand native lib download and backend splitting#1283
msluszniak wants to merge 11 commits into
rne-rewritefrom
@ms/on-demand-native-libs

msluszniak commented Jun 25, 2026 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

Uh oh!

barhanc Jun 26, 2026

Uh oh!

msluszniak Jun 26, 2026

Uh oh!

Uh oh!

Uh oh!

barhanc commented Jun 26, 2026 •

edited

Loading

Uh oh!

msluszniak commented Jun 26, 2026

Uh oh!

msluszniak commented Jun 26, 2026

Uh oh!

msluszniak commented Jun 26, 2026 •

edited

Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

msluszniak commented Jun 25, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Introduces a breaking change?

Type of change

Tested on

Testing instructions

Related issues

Checklist

Additional notes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

barhanc Jun 26, 2026

Choose a reason for hiding this comment

Uh oh!

msluszniak Jun 26, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

barhanc commented Jun 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

msluszniak commented Jun 26, 2026

Uh oh!

msluszniak commented Jun 26, 2026

Uh oh!

msluszniak commented Jun 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

App-size impact of the backend split

CV app — split vs all-backends

What the split drops, per backend (the cross-app value)

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

msluszniak commented Jun 25, 2026 •

edited

Loading

barhanc commented Jun 26, 2026 •

edited

Loading

msluszniak commented Jun 26, 2026 •

edited

Loading