Deconstructing Reality Composer Pro: Logbook

An in-flux list of Things discovered during development that are worth mentioning but have yet to find a home in the series posts.

Mapping from Prims to Entities

RCP maintains a bidirectional mapping between RealityKit entities and USD prim paths. While public RealityKit doesn't easily expose this, the entity tree generated by Entity(contentsOf:) is a 1:1 structural mirror of the USD prim tree. The names match exactly, though RealityKit appends _N suffixes for sibling name collisions. For robust mapping without private APIs, the project uses a strategy of walking the hierarchy and reconstructing prim paths by name, skipping the anonymous root wrapper added during import. This allows the editor to maintain a stable link between the scene graph and the underlying USD stage.

Viewport Selection Outlines

RCP implements its signature selection outlines via a private 2-pass mask + edge detection technique.

• Pass 1 (Mask): Selected entities (tagged with a private EntityRenderEffect) are rendered to an offscreen mask texture using dedicated Metal shaders (vsOutlineMask).
• Pass 2 (Edge): A fullscreen quad pass runs an edge-detection filter on the mask texture to composite the final orange outline. This system leverages private .rerendergraph and .rematerialemitters within RealityToolsFoundation.

For Deconstructed, we tried several alternative ways, eventually settling on a custom shader-based approach that mimics this pixel-perfect, screen-space width behavior but is far from nicer in comparison to RCP, especially because the lack of a mask prevents displaying the outline over overlapping objects.

SwiftUsd (C++ interoperability)

The project serves as a practical showcase for Swift 6's C++ interoperability mode (.cxx). By enabling this mode, we can directly call into the OpenUSD C++ SDK without the need for manual C wrappers or Objective-C++ bridging. The mode is used extensively in the USDInteropAdvanced library to handle complex USD operations like stage editing, variant selection, and material binding. This approach significantly simplifies the codebase by allowing us to use native USD types like TfToken and SdfPath directly within Swift. Also, it has been a nightmare to get it compiled in different environments. This hurdle is also the main reason we can still distribute the binaries that allow the repository to be compiled out of the box.

FSEvents under strict concurrency

When we implemented the project browser, our goal was to replicate live reloading when a file on the disk changed, which we made by creating a watcher system based on FSEvents. Implementing a recursive file watcher using the macOS FSEvents API in a strict concurrency environment was challenging. Since FSEvents is a C API that uses pointers for context, we bridge it to AsyncStream using a ContinuationBox. This box is passed as an UnsafeMutableRawPointer in the FSEventStreamContext and managed using Unmanaged to ensure correct retain/release cycles. An actor-based WatcherRegistry is used to manage the lifecycle of these watchers, ensuring that file system events are processed safely without data races. And it works (surprised face).

JSONSerialization vs Codable

While Codable is the preferred tool for well-defined schemas like the main.json project index, Deconstructed had to resort to JSONSerialization for "looser" aspects of the RCP file format. Many parts of the .realitycomposerpro bundle (plugin data, per-user preferences) contain dynamic structures where keys may vary or be unknown.  JSONSerialization This approach allows the editor to read and modify specific known keys while writing back the data without stripping out metadata that a rigid Decodable struct might overlook, ensuring high compatibility with the official RCP tool. I'm still so unconvinced about this, but maybe we just have to wait for @JSONCodable.