Bringing 3D to window content

• 31 August 2024
• dev

We can add depth to windowed content in visionOS with 3D objects and 3D padding.

In the post we will look at ways to use use RealityView to generate some 3D artwork, and position it alongside some buttons and inputs, using 3D positioning.

3D theme selector #

When working on BlockStack I wanted to make a theme-selector screen that showed how the textures would look in 3D. The resulting screen looks like this:

BlockStack Theme Selector

The view includes a pair of 3D objects generated in code to which I've applied textures. Then it uses a Picker UI element to change the selected texture. Let's build something similar.

Example #

Let's build a simplified version of the above to see how 3D positioning works:

A 3D window content view

Adding a 3D object #

To start we can set up some initial code that generates and displays different shapes based on a given selectedShape:

struct ContentView: View {
    @State private var selectedShape = "cube"

    var body: some View {
        VStack {
            if selectedShape == "sphere" {
                RealityView { content in
                    content.add(generateEntity("sphere"))
                }
            } else if selectedShape == "cone" {
                RealityView { content in
                    content.add(generateEntity("cone"))
                }
            } else {
                RealityView { content in
                    content.add(generateEntity())
                }
            }
        }
        .padding()
    }

    func generateEntity(_ shape: String = "cube") -> ModelEntity {
        var modelEntity = ModelEntity(mesh: .generateBox(size: 0.1))
        if shape == "sphere" {
            modelEntity = ModelEntity(mesh: .generateSphere(radius: 0.1))
        }
        if shape == "cone" {
            modelEntity = ModelEntity(mesh: .generateCone(height: 0.1, radius: 0.1))
        }
        modelEntity.model?.materials = [SimpleMaterial(color: .red, isMetallic: true)]
        return modelEntity
    }

}

We can generate a shape and show it in the content view. Let's add some UI. Adjust the above by adding a "Done" button and a Picker to select different shapes:

var body: some View {
        VStack {
            Button("Done") {
                // Perform relevant action for "Done"
            }
            .tint(.blue)

            if selectedShape == "sphere" {
                // ... existing code
            }

            Picker("Piece texture", selection: $selectedShape) {
                ForEach(["cube", "sphere", "cone"], id: \.self) { shape in
                    Text(shape.uppercased()).tag(shape)
                }
            }
        }
    }

This adds some 2D content to our view, but the 3D shape is in the way!

We could tweak the size of the 3D object to not block the button and Picker, but then when the picker opens it'll still be behind the object. Let's instead bring those 2D elements forward to make them easier to see and interact with.

Adding UI depth #

We can use padding3D to bring the UI forward:

Picker("Piece texture", selection: $selectedShape) {
    ForEach(["cube", "sphere", "cone"], id: \.self) { shape in
        Text(shape.uppercased()).tag(shape)
    }
}
.padding3D(.back, 400)

Adding padding3D with a back value of 400 will bring it forward. Apply this to both the picker and the "Done" button makes them easier to see:

Finished code #

Here's what we have for the complete demo:

ContentView

struct ContentView: View {
    @State private var selectedShape = "cube"

    var body: some View {
        VStack {
            Button("Done") {
                // Perform relevant action for "Done"
            }
            .tint(.blue)
            .padding3D(.back, 400)

            if selectedShape == "sphere" {
                RealityView { content in
                    content.add(generateEntity("sphere"))
                }
            } else if selectedShape == "cone" {
                RealityView { content in
                    content.add(generateEntity("cone"))
                }
            } else {
                RealityView { content in
                    content.add(generateEntity())
                }
            }

            Picker("Piece texture", selection: $selectedShape) {
                ForEach(["cube", "sphere", "cone"], id: \.self) { shape in
                    Text(shape.uppercased()).tag(shape)
                }
            }
            .padding3D(.back, 400)
        }
        .padding()
    }

    func generateEntity(_ shape: String = "cube") -> ModelEntity {
        var modelEntity = ModelEntity(mesh: .generateBox(size: 0.1))
        if shape == "sphere" {
            modelEntity = ModelEntity(mesh: .generateSphere(radius: 0.1))
        }
        if shape == "cone" {
            modelEntity = ModelEntity(mesh: .generateCone(height: 0.1, radius: 0.1))
        }
        modelEntity.model?.materials = [SimpleMaterial(color: .red, isMetallic: true)]
        return modelEntity
    }

}

App View

struct ExampleApp: App {
    var body: some Scene {
        WindowGroup {
            ContentView()
        }
        .defaultSize(width: 420, height: 300)
    }
}

• Previous: BlockStack now on TestFlight