CREST Symposium Program

8th March

Session 1: Modeling Cultural Heritage Objects Overview (Chair: Masataka Kagesawa)

10:00 - 10:40

Overview of the CREST Digital Archiving Project
:Digital Archiving of Cultural Heritage Objects Using Observation and Computer Vision Techniques

Katsushi Ikeuchi, the University of Tokyo, Japan
This paper presents an overview of our project on digital archiving of objects significant to our cultural heritage. This is a five-year project that began in 2000 and involves five universities and an institute. The purpose of the project is to develop techniques to create digital contents from both tangible and intangible cultural heritage assets based on computer vision techniques. For tangible assets, such as the great Buddhas of Nara and Kamakura, some novel measurement devices were introduced, and several techniques that consider geometrical, photometrical, and environmental aspects of an object were developed. For intangible heritage assets, such as a traditional folk dance, we developed a method of symbolizing the entire dance motion acquired from a motion-capture system. We also developed techniques to make a humanoid robot dance stably while keeping the key motions of the dance the same as the original.
Invited Talk

10:40 - 11:20

Beyond Digital Archiving: A Virtual Portal to the World's Heritage

Alonzo C. Addison, UNESCO World Heritage Centre and University of California, Berkeley

With the growth in last few decades of digital technologies for sensing the world, the documentation and conservation of the world's heritage has had a renaissance. From vivid digital pictures to accurate GPS locations and detailed 3D scanner surface coordinates, we are now able to document and record at new levels of detail and precision. With the rapid evolution of digital systems however has come an equally rapid obsolescence of data. Already, some of the first digital archives have been lost as the hardware and software they were captured in have disappeared. Sadly, without careful planning, many of our digital efforts will not even survive the heritage they are meant to record and protect.

The paper presents the concept of a shared global online portal to the world's heritage. For more than ten years at Siggraph, VSMM, ICHIM, CAA and similar venues, scholars have been advocating the need for a virtual portal to the growing body of heritage information and research. Working with UNESCO (the United Nations Educational Scientific and Cultural Organization), UC Berkeley's Center for Design Visualization, and an international, multi-disciplinary team of technical, design, and heritage professionals coordinated through the Virtual Heritage Network, have begun assembling the foundations of a permanent and sustainable archive. The first manifestation, available as the UNESCO World Heritage website (http://whc.unesco.org), lays the foundations for sharing and preserving technical, statutory, and rich media content about heritage. Its evolution, structure, and planned enhancements are discussed. Although limited initially to World Heritage (as defined by UNESCO treaty), this effort defines a methodology for helping ensure the survival of digital heritage records.

11:20 - 11:40

Collaboration of geometry and photometry for virtual exhibitions of tangible and intangible cultural heritages

Takeshi Shakunaga, Okayama University, Japan
We show an overview of our research activities conducted under Ikeuchi CREST project. Two systems have been constructed in our laboratories for virtual exhibitions of tangible and intangible cultural heritages. The first system provides a virtual photometric environment for human-friendly exhibition of tangible heritages. It is implemented by collaboration of virtual photometric properties and a real geometric shape by projecting virtual photometric properties via LCD projectors onto a real object called the screen object. In order to implement the system, we have also developed some BRDF/BTF representations for controlling photometric properties of real surface patterns. The second system enables a virtual exhibition of traditional dances that have been archived by a multiple-camera system. In order to implement the system, we have developed both a fine 3d recovery method and a color blending method by combining the photometric and geometric analysis. The 3d recovery method recovers a fine shape of a dancer based on the space carving algorithm with taking multiple silhouettes into account. The color blending method generates a virtual image by blending multiple views of the dancer with the blending ratios that are determined from geometric relations among the virtual camera, real cameras and suface normals.

Session 2: Geometric Sensors and Techniques (Chair: Jun Takamatsu and Koichi Ogawara)

12:40 - 13:20

Invited Talk

Automating the 3D Modeling Pipeline

Peter K. Allen, Columbia University, USA
The advent of fast and accurate laser range scanning has brought the promise of automatic creation of 3D models. However, there is much manual effort required to move from a 3D point cloud to a complete, texture-mapped model. At the Columbia Robotics Laboratory, we have been developing methods that are aimed at automating many parts of the 3D modeling pipeline, including range-range registration, range-image registration, fusion of sensor data (images, range scans, GIS data, video), and mobile robot data acqusition. Our pipeline is being used to assist art historians and archaeologists in building rich, geometrically and photometrically accurate 3D models of large-scale sites. It also includes a sophisticated visualization tool that can be used by researchers to explore and understand the site. We present results from modeling historic and archaeological sites in New York, France, Sicily and South Africa.

13:20 - 13:40

Flying Laser Range Sensor
-A Novel Aerial Sensing System for Large-scale Heritage-

Kazuhide Hasegawa, the University of Tokyo, Japan
In a measurement operation with a laser range sensor placed on the ground, when an object becomes larger or higher, the occluded regions increase more and more. We generally put up scaffolding in such cases and measure from the top. This scaffolding technique is a practical solution but time and cost consuming. If we are unable to erect scaffolding due to some problem, its measurement operation should be discontinued. To overcome these situations, we have been developing a novel laser range sensor system that is designed to be suspended beneath a balloon platform. The movement of the sensor system causes some distortion in obtained range data. So, we have also proposed a distortion correction technique. Finally, the effectiveness of this aerial sensing system is confirmed by presenting fieldwork results.

13:40 - 14:00

Polarization-based Shape Estimation of Transparent Objects for Digitizing Cultural Assets

Daisuke Miyazaki, the University of Tokyo, Japan
Though there are many cultural assets which are made of glasses, most of the existing methods cannot estimate the shape of transparent objects with enough accuracy. We propose a novel method for estimating the surface shape of transparent objects by analyzing the polarization state of the light. Existing methods slightly consider the reflection, refraction, and transmission of the light occurring inside a transparent object. We employ a polarization ray-tracing method to compute both the path of the light and its polarization state. Our proposed iterative computation method estimates the surface shape of the transparent object by minimizing the difference between the polarization data rendered by the polarization ray-tracing method and the polarization data obtained from a real object.

14:00 - 14:20

Fast Simultaneous Alignment of Multiple Range Images

Takeshi Oishi, the University of Tokyo, Japan
This paper describes a fast simultaneous alignment method of multiple range images. The most time consuming part in alignment process is searching corresponding points. Therefore, we developed a fast searching method using "Index Images". Although this method searches corresponding points in ray directions and the complexity is O(n), where n is the number of vertices, no precise sensor's parameters are required for the search. Furthermore, this method is accelerated by graphics hardware, and can reduce memory space used for that process. For more acceleration, a parallel alignment method was developed for distributed memory system. Since the correspondence search between two range images can be preformed independently, each correspondence pair of two range images is assigned to each processor. All relations of the range images are represented by a pair node hyper-graph. Then, the graph is partitioned for obtaining an optimal pair assignment.

14:20 - 14:40

Recovery of Distorted Shapes Obtained from the Flying Laser Range Sensor for Large-Scale Cultural Heritages

Atsuhiko Banno, the University of Tokyo, Japan
For a large scale object, scanning from the air is one of the most efficient methods of obtaining 3D data. In the case of large cultural heritage objects, there are some difficulties in scanning them with respect to safety and efficiency. To remedy these problems, we have been developing a novel 3D measurement system, the Flying Laser Range Sensor (FLRS) , in which a rage sensor is suspended beneath a balloon. The obtained data, however, have some distortion due to the intra-scanning movement. In this paper, we propose a method to recover 3D range data obtained by a moving laser range sensor; this method is applicable not only to our FLRS, but also to a general moving range sensor. Using image sequences from a video camera mounted on the FLRS enables us to estimate the motion of the FLRS without any physical sensors such as gyros and GPS. At first, the initial values of camera motion parameters are estimated by perspective factorization. The next stage refines camera motion parameters using the relationships between camera images and the range data distortion. Finally, by using the refined parameter, the distorted range data are recovered. We applied this method to an actual scanning project and the results showed the effectiveness of our method.
Invited Talk

15:00 - 15:40

Enhancing processing and visualization efficiency of 3D Scanned Meshes

Roberto Scopigno, ISTI-CNR, ITALY
The construction of detailed and accurate 3D models is made easier by the increasing diffusion of 3D scanning devices. These allow to build accurate digital models of real 3D objects in a cost- and time-effective manner. The talk will present the capabilities of this technology focusing mainly on some issues which are preventing its wider use, such as the considerable user intervention required, the usually incomplete sampling of the artifact surface and the complexity of the models produced. Another emerging issue is how to support the visual presentation of the models (local or remote) with guaranteed interactive rendering rates. Some examples of the results of current projects, mainly in the Cultural Heritage field, will be shown.

15:40 - 16:00

Sunlight Illumination Simulation for Archaeological Investigation -Case Study of the Fugoppe Cave-

Nobuaki Kuchitsu, National Research Institute for Cultural Properties, Tokyo, Japan
The Fugoppe Cave, located in northern Japan, is a natural cave in tertiary tufaceous rocks. Archaeologists believe that ancient painters and sculptors worked inside the cave using artificial lights such as torches. In contrast, we argue that there is a possibility that they used natural light such as sunlight, since an accumulation of soot, which is a side e ect of using torches, is rarely found in the cave. However, our argument could be countered by the question of whether the natural light could enter and su ciently illuminate the walls inside the cave. In order to verify our theory, we attempted to simulate how natural light might illuminate the interior walls by using a 3D model of the cave which we obtained from our proposed modeling system, and to examine the possibility that ancient painters and sculptors worked under natural illumination.

16:00 - 16:20

Distortion Correction of Range Data Obtained from Floating Laser Range Sensor using Parameterized Deformation Registration

Tomohito Masuda, the University of Tokyo, Japan
In this paper, we propose the registration method among range images obtained from our floating laser range sensor in order to effectively and accurately measure the 3D shape of large-scale objects from the air. Moving a sensor during the scanning causes the shape distotion of its range image, so the conventional registration (rigid-body transformation) cannot be applied to their range images. Therefore, we propose the deformation registration method which can simultaneously correct the shape distortion at the registration step by using undistorted images obtained from the sensor stably set on the ground. We describe the accuracy and limitation of our proposed method, as well as the example of the distortion correction of corrupted range images obtained from the floating laser range sensor.

16:20 - 16:40

Virtual Kawaradera: Fast Shadow Texture for Augmented Reality

Tetsuya Kakuta, the University of Tokyo, Japan
The Kawaradera temple, formerly located in the village of Asuka, was one of the oldest temples in Japan; unfortunately, it was destroyed by fire in the latter part of the Muromachi Period. The Virtual Kawaradera Project is an outdoor on-site augmented reality application that uses a head mounted display, whose purpose is to reconstruct and represent Kawaradera virtually. We begin with a short introduction to the project. Then we suggest a method for fast shadowing of virtual objects using a set of basis images that has been rendered in advance. We first describe how, in the preparatory stages, we approximate the illumination of the scene with a number of directional lights and render the virtual objects with each light in order to obtain shadow images. We then describe how we synthesize these images with the linearity relationship between a luminance of light and a brightness of object surface, and set the synthesized shadow images onto shadowing planes, which cover objects roughly, as a texture. Finally we tell how we can compute shadings and shadows of virtual objects responding to the illumination of the real scene, and support a change of the user’s viewpoint. The proposed method is especially appropriate for static models such as buildings. Hence, we chose to apply this method to the Virtual Kawaradera Project, with the goal of improving the quality of synthesized images in augmented reality systems.

Laboratory Tour

16:40 - 18:30

9th March

Session 3: Photometric and Environmental Techniques (Chair: Daisuke Miyazaki)

Invited Talk

10:00 - 10:40

Refractive and Specular 3D Shape by Light-Path Reconstruction

Kiriakos N. Kutulakos, University of Toronto, Canada
Computer vision research on 3D reconstruction has focused almost exclusively on objects that predominantly scatter incident light. As a result, little is known about how to build 3D models of refractive or mirror-like scenes, which remain mostly beyond the reach of existing state-of-the-art 3D photography methods. Because linear light propagation is violated in such scenes, our analysis generalizes perspective projection to include "indirect projections," where 3D points project to images along unknown, piecewise-linear light paths, via the scene's interfaces. To solve the resulting scene reconstruction problem, we reconstruct the 3D path(s) traced by light as it travels from known patterns in the scene toward the image plane of one or more cameras. In this talk, I will discuss the conditions under which such reconstructions are possible and present algorithms and promising results on reconstructing complex scenes composed of water, crystal, mirrors and other highly-reflective surfaces.

10:40 - 11:00

Color Alignment for Texturing 3D Geometric Model

Hiroki Unten, the University of Tokyo, Japan
Recently, creating digital contents through measurements of the real world has become an important technique in the field of virtual reality, digital preservation of cultural heritages and so forth. One of the effective methods for creating such digital contents is texturing 3D geometric model, which maps color images on a 3D geometric model. This method has two problems: geometric and photometric. The geometric problem lies mostly in how to estimate the camera position and rotation relative to the 3D position of the target object. The photometric problem deals with color consistencies between input textures. Most studies on texturing focus on the geometric problem, while only a few works deal with the photometric problem. In this paper we propose two novel methods for color alignment in Texturing. One method is based on the fact that chromaticity of images is independent from their geometric data. The other method utilizes the basis image of a point light source on an illumination sphere. In both methods, the pseudo-albedo is estimated and the color consistency between the input images is accomplished.

11:00 - 11:20

Estimate surface color from changing illumination

Rei Kawakami, the University of Tokyo, Japan
We introduce a robust method to eliminate the illumination color in a color image from varying illumination constraint. In the context of color constancy probelm, many methods have been proposed. However, there are few methods using varying illumination as a constraint. We found Finlayson et al’s method applicable for natural images and extended it by utilizing planckian locus to detect the presence of noise in the input. We have done some experiments with natural images and also with textures mapped on 3D data, and discussed about accuracy and robbustness of our method.

11:20 - 11:40

Spectral Scene Separating: Illumination distribution and Surface Spectral reflectance

Akifumi Ikari, the University of Tokyo, Japan
A number of methods have been proposed to separate a color signal spectral into its components: illumination spectral power distribution and surface spectral reflectance. These methods usually use a minimization technique from solely a single color signal. However we found that it's not effective for real data, because of insufficiency of the constraints. To resolve this problem, we proposed unlike the existing methods, uses multiple color signals. In our implementation, we introduce three different approaches: first, color signals obtained from many different surface reflectance lit by an identical illumination; second, color signal from an identical surface reflectance lit by different illumination; and third, color signals from identical surface reflectance but with different types of reflection components (diffuse and specular reflectance) lit by identical illumination. Practically we applied our method to deal with the color signals of a scene taken by using interference filter, and separate to illumination spectral power distribution and scene of surface spectral reflectance.

Session 4: Motion Acquisition and Analysis (Chair: Takeshi Oishi)

Invited Talk

13:00 - 13:40

Computer Graphics Research at KAIST

Sung Yong Shin, KAIST, Korea
In this paper, I introduce recent research activities in computer graphics at KAIST; in particular, the research activities done by the computer graphics group led by myself. This group was chosen as one of some 400 national research laboratories (NRLs) in the year of 1999 by the Ministry of Science and Technology(MOST) of the Government of Korea, which cover all fields of science and engineering in Korea. Under the support of the government, my group has conducted a research project on real-time computer animation for five years from 1999 to 2004. I first describe the objective and scope of this project and then summarize what my group have done during this period.

13:40 - 14:00

Multimodal Dance Training System based on Motion Analysis

Yoshinori Kuno, Saitama University, Japan
Abstract:This paper presents a multimodal dance training system that targets on beginners and enables them to learn basics of dances easily. One of the conventional ways of learning dances is to watch a video about the performance of dance masters. However, by only watching video, the student cannot exactly know how long the dancers move forward or backward. Furthermore, it is difficult for him/her to know when to start moving his/her arms or legs. We solve the first issue by introducing an image display on a mobile robot. The student can learn the amount of translation just by following the robot. We also introduce active devices with some vibro-motors for the second issue. The active devices direct action-starting cues with vibration. The system analyzes the motion data of dance masters to control the robot's motion and the vibration timing. Experimental results show the effectiveness of our system.

14:00 - 14:20

Dance Motion Analysis and Synthesis using Motion Capture Data

Atsushi Nakazawa, Osaka University, Japan
In this talk, I will explain a brief overview of human motion archiving project, and the analysis and synthesis portion of it. The goal of the analysis step is detecting the common and unique compornents of dance performances. First, the dancer's motion is acquired by motion capture systems. As for common compornent analysis, the basic structures are automatically obtained by segmentating and clustring the motion data. According to this result, we can easily recognize the number of the basic movements and arrangements of them. These are mostly common to the all individuals if they are performing the same dance. As for unique compornent analysis, we developed the method to detect the basic movements and style compornents of several dance data performed by different individuals. This style compornents make individual difference (uniqueness) of the dancers. We have confirmed the varidity of our approach using Japanese folk dance capture data.

14:20 - 14:40

Structure Detection of Dance Sequence using Motion Capture and Musical Information

Takaaki Shiratori, the University of Tokyo, Japan
This paper describes a dance motion analysis method considering music rhythm. In these days, many important intangible cultural properties are being lost because of the lack of successive performers, and we have started our digital archiving project of cultural intangible heritages. Our goal for archiving dance motions is to detect ``primitive motions", which consists of dance motions. In addition, we believe these primitives must be synchronized to the musical rhythm, so our proposed method automatically segments the original motion considering with estimated musical rhythm to detect the primitive motions.

14:40 - 15:00

Balance Maintenance Model for Human-like Characters with Whole Body Motion

Shunsuke Kudoh, the University of Tokyo, Japan
Recently, interest in generating the dynamically consistent motion of human-like models has been increasing in various areas. Balance maintenance is one of the most fundamental topics in that type of research and many researchers have developed methods of balance maintenance for human-like models. Humans have great skill in balance maintenance, which they accomplish by using dynamic whole body motions, such as rotating their arms, bending down, squatting, stepping, and so on. However, with regard to controlling human-like models, these skills have seldom been utilized for developing methods of balance maintenance. In this paper, we propose a method of balance maintenance for human-like models. The method is a feedback system, in which optimization calculation for joint acceleration is performed to generate motion of balance maintenance. In this method, instead of body posture, trajectories of the body's center of mass (CM) and the zero moment point (ZMP) are mainly considered. As a result of utilizing the method, we have obtained the motion of balance maintenance in which models maintain their balance by whole body motion, e.g., rotating their arms, bending down, and taking a step just as humans do.

15:00 - 15:20

Task model of lower body motion for a humanoid robot to imitate human dances

Shinichiro Nakaoka, the University of Tokyo, Japan
We have developed a humanoid robot that can imitate human dance performances with whole body motion. In this paper, we define a model of lower body motion for a humanoid robot and propose a method of importing human motion into a robot based on that model. In general, importing human motion into a robot is difficult because a robot has many mechanical constraints and the robot must maintain its balance under its own physical properties. Based on our proposed method, human motion is recognized as a sequence of several tasks and robot motion is generated according to the recognized tasks. We successfuly realized a performance of Japance traditional dance 'Aizu-Bandaisan' by humanoid robot 'HRP2'.

Session 5: Motion Editing and Presentation (Chair: Kazuhide Hasegawa)

15:50 - 16:10

Simultaneous Object Tracking and Recognition by Nearest Neighbor Traversing Graph

Toshikazu Wada, Wakayama University, Japan
This paper presents a novel method for real-time object tracking and recognition in an image sequence. Since tracking with fixed appearance model can be easily affected by the appearance changes of an object, multiple appearance modeling has been proposed so as to represent manifold appearances of an object. Tracking problem with this model consists of two search problems: 1) search for object appearance and 2) search for object pose in the image space. These two searches should be performed simultaneously in the product space of appearance and pose spaces. Since this product space is vast, real-time tracking seems to be impossible. Our method, however, realizes the real-time tracking based on two methods: 1) PPE (pose parameter embedding) and 2) NNTG (Nearest Neighbor Traversing Graph) based nearest neighbor search. In the experiments, we have confirmed that our method can track and recognize the object in real-time.

16:10 - 16:30

Learning Everyday Manipulation Tasks from Observation

Koichi Ogawara, the University of Tokyo, Japan

This paper describes a new approach on how to teach a robot everyday manipulation tasks under the ``Learning from Observation'' framework. In this approach, human demonstrations, which are made up of mutual interactions between a grasped object and an environmental object, are observed and a reusable manipulation task model is automatically created. The task model is used to generate motions of a humanoid robot to reproduce the task. This approach can be applied to digitize intangible cultural heritage assets.

This paper presents two methods used in this approach: one that detects and reproduces essential interactions by observing multiple demonstrations of a task, and one that recognizes and reproduces the grasps used in each interaction. The methods are examined by using a humanoid robot.

16:30 - 16:50

Imitation of Assembly Tasks for Realizing Dexterous Manipulation

Jun Takamatsu, the University of Tokyo, Japan
As one of purposes of the Ikeuchi CREST project, we aim at preserving intangible cultural heritages, such as a traditional craft skill, and duplicating it by a robot. Whereby it is necessary to implement the ability for dexterous manipulation, including interaction with the external, on a robot. As case study of such interaction, we focus on assembly tasks, where not only position control (control of the internal) but also force control (control of the external) must be well considered for successfully duplicating such tasks. In this paper, we will introduce our proposed system to duplicate the tasks by imitating human demonstration.
Mail to:cvl-sympo-staff@cvl.iis.u-tokyo.ac.jp
Copyright (C) 2000-2005 CVL (Computer Vision Laboratory). All rights reserved.