Pictofit / Web SDK / 7.3.1 / Mix & Match in 3D

Mix & Match in 3D

In the last chapter we talked about how one can create a custom mannequin from a set of body measurements. Now we can use this mannequin to fit garments onto it. Check out this demo to see the feature in action. You can also find the source code for it in our sample respository.

Virtual Try-On Demo

The first thing we need for this is the body model state of the customised mannequin. This state describes the pose and shape of the mannequin and can be requested as a result of the mannequin generation. The snippet below shows how to access it:

const bodyModelStateBlob = blobs[Pictofit.MannequinRequestReturnFile.BODY_MODEL_STATE];

Creating a Mix & Match Request

To perform Mix & Match, we need to make a request to the Pictofit Compute Server. The Pictofit.FittingEngine3DRequestInfo encapsulates all the required parameters and options.

// create a request object
let info = new Pictofit.FittingEngine3DRequestInfo();

The Body Model State

The body model state holds relevant information about a mannequins shape & pose. This is required for the fitting of the garments. The bodyModelState property of the request object can either be set to a Blob holding the data or a string that represents an URL to a body model state file. Therefore, it is possible directly reuse the blob that was generated from a CustomMannequin request or to load a state that is stored somewhere on your servers.

Once the users has finialised his or her mannquin, it is adviseable to store the respective assets and link them to the users profile. This allows you to provide it by URL which speeds up the request (the client doesn’t have to download it first and upload it to the Compute Server again since the server can directly fetch it from the provided location).

info.bodyModelState = bmStateBlob;
// OR
info.bodyModelState = "https://myServer/mySavedBodyModelState.bin";

The 3D Garments

We need to create a Garment3D instance for every garment that we want to virtually try on. This object encodes the identifier of the garment and optionally a custom material file which should be used for rendering. But let’s first have a look at the structure of an asset bundle for a garment:

/my-garment/mesh.obj
/my-garment/material.json
/my-garment/diffuse.jpg
/my-garment/garment.min.gm3d

This asset bundle consists of different files like a material file in our JSON configuration format or the mesh encoded as .obj file. All these files belong to a single garment 3D asset. The SDK and the Compute Server need different files from this bundle and will use the Pictofit.WebViewer.resourceProvider to generate URLs to access them. The default case is to use a Pictofit.BaseURLResourceProvider. In the example from above, the identifier of the garment would be my-garment. When resolving the URL, the resource provider will get the information that e.g. the file mesh.obj for the garment with identifier my-garment is needed. This will return a URL that look somewhat like https://cdn.customer.com/assets/garments/my-garment/mesh.obj for example. If you need more complex logic that e.g. retrieves the location from a database or generates a pre-signed URL, please have a look at custom resource providers.

The Garment3D objects are added to the request as an array as can be seen below:

info.garments = [
    new Pictofit.Garment3D("garment3"),
    new Pictofit.Garment3D("garment2", "Custom material"), // <-- Optional (preloaded) material that should be used for rendering
];

Triggering the Request

Now we are ready to execute the request and render the result with a Pictofit.WebViewer instance.

const mixMatch = new Pictofit.MixMatch3D(computeServer, viewer);
mixMatch.requestInfo = info;
await mixMatch.compute();

This creates a new cascade layer for every garment and adds it to the scene. If no custom materialName was specified it will fetch the material.json file from the garments assets and use it to rendering.

© 2014-2020 Reactive Reality AG