How to Build Door Visualizer Using Vision API

The idea of implementing quick visualization tools for online shopping continues to gain popularity and is likely to become a common thing in the future. It is currently particularly popular in fashion, furniture, and home decor. We remind you that Wizart Visualizer can work with many materials. It is installed on dozens of client websites, and the number of monthly users has exceeded one million per month long ago.

Let's take a look at the door market, which according to forecasts, is growing and will reach 118M USD by 2025. However, there are almost no solutions that allow visualizing doors in customers' interiors. Therefore, we want to share with you how to build such a visualization tool using the Wizart Vision API. Also, we will briefly consider the process of obtaining door panel parameters.

Are you ready to test the possibilities of Vision API? Start a free trial now!

Vision API in action

We will show you how to make the visualization process similar to the Wizart Visualizer, where API endpoints take a user's photo of the interior as input.

Using our Object Detection In Indoor Scenes algorithms, we can obtain information on the 3D dimensions (real-world dimensions) and positions of the door in the scene in the photo. For clarity, we will consider the situation when there is already a door in the interior:

Below is an example of a query to the Interior API to retrieve the door leaf parameters.

POST /vision-api/v3/interior/doors

Let's send a request, attaching a photo of our interior in the body with the key room_image.

A few seconds later, we get a processing response:

{
    "segmentation": {
        "vectorized_mask": null,
        "mask": <base_64_encoded_string>
    },       
    "analysis": {
        "camera": {
            "roll": 0.00611294539656331,
            "height": 1.5,
            "pitch": 10.0258466845234263,
            "fov": 39.619599559470416
        },
        "image_info": {
            "noisy_confidence": 0.9315704107284546,
            "bad_target_confidence": 0.0029280944727361202,
            "blurry_confidence": 0.9982958436012268,
            "dark_confidence": 0.9675150513648987
        },
        "interior_type": "living_room"
    },
    "detection": {
        "doors": [
            {
                "cx": 59.458957248263886,
                "cy": 47.01976776123047,
                "wall_id": 0,
                "door_id": 0,
                "points": [
                    {
                        "x": 0.5299999833106995,
                        "y": 0.7020000000000000,
                    },
                    {
                        "x": 0.6499999833106995,
                        "y": 0.6923333333333333
                    },
                    {
                        "x": 1,
                        "y": 0
                    },
                    {
                        "x": 0,
                        "y": 0
                    }
                ]
            }
        ]
    },
    "reconstruction": {
        "doors": [
            {
                "wall_normal": {
                    "z": 0.9969777464866638,
                    "x": -0.0776875764131546,
                    "y": 2.2667970611678356e-8
                },
                "height": 2.099999475479126,
                "width": 0.8600261974334717,
                "area": 1.8060545635085978,
                "points": [
                    {
                        "z": -3.1118643283843994,
                        "x": -1.7044533491134644,
                        "y": 2.4075446128845215
                    },
                    {
                        "z": -2.876469135284424,
                        "x": 1.3164150714874268,
                        "y": 2.4075446128845215
                    },
                    {
                        "z": -3.1118643283843994,
                        "x": -1.7044533491134644,
                        "y": -0.8924554586410522
                    },
                    {
                        "z": -2.876469135284424,
                        "x": 1.3164150714874268,
                        "y": -0.8922334313392639
                    }
                ],
                "wall_id": 0,
                "door_id": 0
            }
        ]
    }
}

On the first level of nesting, we have 4 groups of parameters:

1. "segmentation" - data obtained using the Segmentation API. The value of the mask attribute stores the RGB mask of the segmentation of the objects found in the scene. It is passed as a base-64 string.

2. "analysis" - information about the camera's tilt and height parameters, FoV, shooting quality, and type of interior. Data is obtained using the Analysis API.

3. "detection" - data obtained using the Detection API. It includes an array of objects called doors corresponding to the doors found. Each object includes 2D coordinates of polygon points describing the edges of the door.

4. "reconstruction" - data obtained using the Reconstruction API. It includes an array of objects with key doors corresponding to the doors found in the interior. Each object is described by the following parameters: 

• width, height, area - the predicted width, height, and area of the door; 
• wall_id, door_id - identifiers for linking a specific door to a wall;
• points - an array of 3D coordinates of the door in the scene.

You can find a more detailed description and a full set of the artifacts on the vision API page and in the documentation for the Wizart Vision API.

Using the door specifications in combination with the other scene parameters, you can render the desired door according to its real size and position in the interior.

At Wizart, the main goal of determining and 3D-reconstructing door panels, as well as other "working surfaces" of the interior, is the visualization of finishing materials for a wide range of product groups. Using 3D materials of the door panel, we can apply them to the door in the visualizer, replacing the original surface:

 As you can see, in just a few requests to the API, you can obtain all the necessary 3D parameters for your interior, even the description of specific objects in the scene, such as the door leaf. With Wizart Vision API, this process can be done in seconds.

You can apply the data to your own 3D interior visualization apps as well as in various AR applications or graphics engines.

Are you ready to test the possibilities of Vision API? Start a free trial now!