SHREC’16 Track Proposal: Shape Retrieval of Low-Cost RGB-D Captures

Due to the growing popularity of low-cost scanners, such as Microsoft Kinect, several RGB-D object datasets have been emerging in the research community. The recent advent of Virtual Reality technologies and applications, increase the demand for 3D objects, which can now be easily and efficiently captured with such sensing technology, which further enable their exploitation in real-time recognition scenarios. For these, it is essential to identify which 3D shape descriptors, provide better matching and retrieval of such digitalized objects [1].

One such datasets is the “RGB-D ISCTE-IUL Dataset” (http://dataset.mldc.pt/) [2]. , whose objects were captured with the Microsoft Kinect One sensor, which is freely available for the scientific community to run their experiments in different scenarios and applications, ranging from computer vision, computer graphics, image/object query and retrieval, machine learning and other domains.

The “RGB-D ISCTE-IUL Dataset” provides for each object, 90 frame pairs of RGB and Depth images, a segmented and registered point cloud and a polygon mesh model. Each captured object will also have a matching “high-quality” synthetic 3D model, acquired from other 3D dataset with rights for R&D use, such as the Princeton Shape Benchmark [3]. and the Sketch-Based 3D introduced by Li et al. [4]. . See http://dataset.mldc.pt/index.html#overview for more details on the datatset. In this track, that follows on previous SHRECK 2015 competition [5], we aim to evaluate objectively the performance of 3D shape retrieval techniques on the “RGB-D ISCTE-IUL Dataset”, which is now populated with more than 200 objects.

Dataset

Considering the SHREC track requirements, participants will be able to describe their object queries by means of raw data of the captured object, segmented point clouds of each camera view, registered point cloud of the full object, or triangle meshes of the full object too. With such possibilities, participants can use the most appropriate format for his retrieval algorithm.

The following table, describes the raw data in the dataset.

Per each frame capture by Kinect One:
  • 90 frames capture per angle
  • 3 different angles
  • 90 x 3 = 270 pairs of color and depth images
Data Description Filetype Reason
Color
image		 RGB format .png Lossless and
community standard
Depth
image		 In
Millimeters		 .png Lossless and
community standard
Bounding Box The 2 bound-box
corners in color image coordinates		 .txt Simple to Read/Write

The following table, describes the processed data in the dataset.

Processed Data available in the dataset
10 object instances per class
For each object instance:
  • 270 segmented point-clouds (90 point-clouds for each of the 3 angles)
  • 1 registered point-cloud (registration of all the 270 segmented point-clouds)
  • 1 triangle mesh, created using the Poisson Surface Reconstruction
  • 1 triangle mesh, created using Basic Triangulation (for scenarios of robustnesstest)
Data Description Filetype Reason
Segmented
Point-Clouds		 PCD
Format. In Millimeters.		 .pcd Community standard
Camera
Pose		 4x4
transformation matrix with float/double precision		 .txt Simple to read/write
Registered
Point-Cloud		 PCD
Format. In Millimeters.,PLY
Format.		 .pcd,.ply Community standard
3D
Mesh Poisson Reconstruction		 PLY
Format.,OFF
Format.		 .ply,.off Contains Vertex Color,Simple to read/Write
3D Mesh
Basic Triangulation		 PLY
Format.,OFF
Format.		 .ply,.off Contains Vertex Color,Simple to read/Write

PROCEDURE

  1. Participants must register using the form at the end of the page.
  2. The test set comprises all the objects of the dataset, and will be made available via this website in password protected zip files. The test set will be available in any of the formats described in the previous table. Participants will receive the password by e-mail, along with a list of instructions.
  3. Given a query (one object instance) from the test set, participants should return a ranked list of the remaining test data according to the similarity score. This procedure should be repeated for all objects of the test set.
  4. Up to 5 ranked lists per query may be submitted, resulting from different runs. Each run may be a different algorithm, or a different parameter setting. We ask the participants to provide the binary program that generated the results. If more than one executable or a different parametrization is used, we also ask the submission of appropriate documentation along with a one-page summary of the proposed method.
  5. Each ranked list run, representing an evaluation, should be written in an ASCII file with extension .res. Such file should be named "run id-query id", where "run id" is the number of the run and "query id" is the identifier of the object representing the query. Below we depict an example for a file 1-144.res, that is, run number 1 of a query for object number 144 (the file includes as many lines as the number of objects in the test set): 
    6. 144 1.00000
    24 0.87221
    45 0.79915
    201 0.59102
    203 0.54902
    32 0.51241
  6. The evaluation will be done automatically.
  7. The organization will release the evaluation scores of all the runs.
  8. Participants should then write a one-page description of their method and commenting the evaluation results with two figures.
  9. The track results are combined into a joint paper, published in the proceedings of the Eurographics workshop on 3D object retrieval.
  10. The description of the tracks and their results are presented at the Eurographics workshop on 3D object retrieval.

Evaluation Methodology

We will employ the following evaluation measures: Precision-recall curve, e-measure, discounted cumulative gain, nearest neighbor, first-tier (tier1) and second-tier (tier2).

Schedule

Schedule list:
February 01 A subset of the final test set will be available on line.
February 08 Please register before this date.
February 08 Distribution of the final test sets. Participants can start the retrieval contest.
March 04 Submission of results (ranked lists) and a one page description of their method(s).
March 07 Release evaluation results.
March 07 Each track is finished, results ready for a track paper.
March 07-11 Review of track paper by the participants.
March 15 Submit track papers for review.
March 22 All reviews due, feedback and notifications.
April 01 Submission of camera-ready track papers.
May 07 & 08 EG 3DOR Workshop, including SHREC'2016.

References

[1] P. F. Proença, F. Gaspar, and M. S. Dias, Good Appearance and 3D Shape Descriptors for Object Category Recognition, International Journal on Artificial Intelligence Tools, August 2015, Vol. 24, No. 04, (2015), DOI: 10.1142/s0218213015400175

[2] “RGB-D ISCTE-IUL Dataset” http://dataset.mldc.pt/index.html#overview

[3] P. Shilane, P. Min, M. Kazhdan, and T. Funkhouser, “The Princeton Shape Benchmark,” Shape Model. Appl. Int. Conf., vol. 0, pp. 167–178, 2004.

[4] B. Li, Y. Lu, C. Li, A. Godil, T. Schreck, M. Aono, M. Burtscher, H. Fu, T. Furuya, H. Johan, J. Liu, R. Ohbuchi, A. Tatsuma, and C. Zou, “Extended Large Scale Sketch-Based 3D Shape Retrieval,” pp. 121–130.

[5] P. B. Pascoal, P. Proença, F. Gaspar, M. S. Dias, F. Teixeira, A. Ferreira, V. Seib, N. Link, D. Paulus, A. Tatsuma, and M. Aono, “Retrieval of Objects Captured with Kinect One Camera,” in Eurographics Workshop on 3D Object Retrieval, 2015.

Registration