When do you plan to release images and VR? about simmc

[seo-95]

Hi @Franck-Dernoncourt ,
I have the same doubt before reading the paper. Real images and VR are not part of the dataset, instead they are replaced by logs annotations inside the conversations. Each log contains the ID of the furniture/clothing contained in the VR/image, in such a way the computer vision side of the challenge is totally skipped. To better understand I suggest you to read section 3.3 of the paper

[Franck-Dernoncourt]

According to an email from the organizers, no plan to release images, and see the metadata csv for the VR.

[seo-95]

Thanks @seo-95 . Just to make sure I understand: we're given some image/VR "catalog description and other attributes" (quoting section 3.3), but not the images/VR themselves?

https://github.com/facebookresearch/simmc/tree/master/data/simmc_fashion/*.json contains image_id so I thought there could be some way to retrieve the images. Also, https://arxiv.org/pdf/2006.01460.pdf's figure 4 seems to take some raw image as input ($M_t$). But reading https://arxiv.org/pdf/2006.01460.pdf's section 8 (SIMMC-Furniture (VR) and SIMMC-Fashion (Image) paragraphs) seems to confirm your previous message:

We extract the representations for each fashion item using concatenated GloVe embeddings of its attributes (similar to SIMMC-Furniture) in addition to learning the source embedding (‘memory’ or ‘current’ item), as the multimodal context $M_t$

I have just seen the image_id you mentioned and it is really strange. Maybe it's there for future possible expansion of the dataset (the introduction of images/VR would be great). I hope for an answer from Facebook organizers

[Franck-Dernoncourt]

@shubhamagarwal92 pointed me to https://github.com/facebookresearch/simmc/tree/master/data#overview-of-the-dataset-repository (mirror), which mentions:

images: ./simmc-furniture/figures/{object_id}.png

I agree, at that point we need the organizers to clarify if images/VRs will be released, and if so, when.

[Franck-Dernoncourt]

Thanks @seo-95 . Just to make sure I understand: we're given some image/VR "catalog description and other attributes" (quoting section 3.3), but not the images/VR themselves?

https://github.com/facebookresearch/simmc/tree/master/data/simmc_fashion/*.json contains image_id so I thought there could be some way to retrieve the images. Also, https://arxiv.org/pdf/2006.01460.pdf's figure 4 seems to take some raw image as input ($M_t$). But reading https://arxiv.org/pdf/2006.01460.pdf's section 8 (SIMMC-Furniture (VR) and SIMMC-Fashion (Image) paragraphs) seems to confirm your previous message:

We extract the representations for each fashion item using concatenated GloVe embeddings of its attributes (similar to SIMMC-Furniture) in addition to learning the source embedding (‘memory’ or ‘current’ item), as the multimodal context $M_t$