5 min read

The Framework Next Door

Illustration of a gaming board with a grid of deformed squares and a volcano in the center, everything melted comic style

TL;DR, If you are interested in building a spatial game (or just about any immersive experience), you'll probably want to implement an architecture like TabletopKit.

A representation of a tabletop game in spatial mode in visionOS.
Create multiplayer spatial games on a virtual table surface and use FaceTime to invite players.

TabletopKit first impressions

We are creating a game called The Green Spurt that is primarily a shared, spatial, and immersive escape room experience. With a background in iOS development, some experience with RealityKit, and invested in visionOS, TabletopKit seemed to be something worth taking advantage of, at least on paper.

New Apple Vision Pro game in development with @elkraneo.bsky.social #TheGreenSpurt #AppleVisionPro @reality2713.bsky.social

Roxana Nagy (@coderox.bsky.social) 2024-12-16T18:38:30.434Z

TabletopKit was introduced this year at WWDC24 in a "lightweight" package of one session accompanied by a sample code project and the relative documentation. At first sight (maybe because the name), it appeared specialized in tabletop games, and we were not exactly doing that by constructing an escape room-like experience, so we had to evaluate it more in depth. Starting with simple questions like how many players, how many tabletops, game rules validation... to more platform (XR) related ones like object layout, ownership, physics, meshes, ... and surprisingly, one by one, our expectations were met.

TabletopKit fascinated us with its structured approach to abstracting common interactions. It provides a state-driven layout with an excellent concept for decoupling and validating sequences of gestures, making it considerably simpler to arrange and interact with objects spatially.

We are just learning how to use it, but based to our current understanding, TabletopKit is a set of high-level abstractions for common requirements for shared spatial interactions that can be employed on volumetric windows by simply adding a modifier to a SwiftUI view.

struct SomeApp: App {
  let game: TabletopGame
  let root: Entity
  
  var body: some Scene {
    WindowGroup {
      ContentView()
        .tabletopGame(game, parent: root)
    }
    .windowStyle(.volumetric)
}

This is all it takes to declare a TabletopKit game ✨

The TabletopGame instance required by the modifier must be built with a TableSetup that describes mainly the players and game layout, as well as the table and equipment (piece) positions. It will also serve as the common way of accessing game state snapshots in the future (often required to deal with hierarchies). The setup also provides logic for tracking counters (point systems), and I'm particularly attracted by how it abstracts the configuration of spatial Persona templates.

An illustration representing a surround shared activity in visionOS. Participants are gathered in a circle around shared content.

If necessary, there is a variant of the modifier that receives a closure returning a TabletopInteraction, allowing for far more fine grained control of gesture phases allowing significantly richer composition of actions, such as equipment update.

Creating a TabletopGame.Observer will prove extremely helpful, given that enables for validation of gestures and seat interactions. Also having a Tabletop.RenderDelegate is not only helpful in tracking the game's visual state, but it is also seems to be the best place to manage asset loading and UI output in general.

With this setup, you'll only have to worry about actions and layout, which is fantastic.

State-Driven Architecture and Accessibility

Tabletop Kit's behavior-based approach is ideal for meeting accessibility requirements. We can make our escape room more accessible by allowing multiple inputs, such as voice or assistance devices, to activate actions, and with the delegates we can trigger different types of output too. Also as a side effect of the context being SwiftUI, many system accessibility features simply work.

This versatility is a big advantage since it enables us to provide the required inclusive experience.

Challenges and Learning Curves

Now, despite its numerous advantages, TabletopKit is not without drawbacks (surprise emoji). As a new framework, it has some issues and a lack of comprehensive documentation, and with the current rate of releases and the number of moving parts, it feels like a moving target at times (perhaps because it is). However, the opportunity to create rich, interactive experiences exceeds these downsides by far in our opinion.

BTW, there are some options to help with the runtime debugging process

game.debugDraw(options: [.drawTable, .drawSeats, .drawEquipment])

Final Thoughts: Seizing the chance with TabletopKit

Choosing TabletopKit for our escape room project was a bold decision, but it has proven to be fruitful. Its ability to simplify complex interactions while yet providing dynamic, behavior-based experiences makes it an ideal tool for our needs. While there is still plenty to learn, we enjoy how it allows for new puzzles and the building of interactive parts. We are confident that by utilizing TabletopKit more effectively, we will be able to build an escape room experience that is both interesting and accessible to all players.

💡
Oh, did I mention that it works seamlessly with SwiftUI attachments, which I believe is one of the key reasons SwiftUI should be considered the most powerful UI framework for immersive experiences, far ahead of anything else?

some Radars


TabletopKit | Apple Developer Documentation
Create multiplayer spatial games on a virtual table surface and use FaceTime to invite players.
Customize spatial Persona templates in SharePlay - WWDC24 - Videos - Apple Developer
Learn how to use custom spatial Persona templates in your visionOS SharePlay experience to fine-tune the placement of Personas relative…

❤️ We appreciate the excitement we have seen for our upcoming Apple Vision Pro game 🙌 . We are in closed Alpha right now, and we will be accepting more testers as we go. Feel free to register here for early access: forms.gle/sa7M6h1xj86Y... @elkraneo.bsky.social @reality2713.bsky.social

Roxana Nagy (@coderox.bsky.social) 2024-12-18T19:20:22.410Z