The lectures at Interaction 2014 are combined with the tutorial session of 3DTV Con 2014. The lecture slides are available in the Lecture slides and supplementary material page. The password is provided during the lectures.
Lecture 1 – Lecture 2 – Lecture 3 – Lecture 4 – Lecture 5
Lecture 1: Human stereopsis and 3D visual perception
Presented by Prof Marty Banks, University of California, Berkeley
Lecture outline:
- Part 1: Human stereopsis
- Stereo geometry
- The correspondence problem
- Stereo performance in space and time
- Part 2: Stereo 3D displays
- Methods for separating images to the 2 eyes
- Temporal protocols
- Depth artifacts due to temporal protocol
- Visual discomfort
- Part 3: Picture perception
- Perspective projection
- Compensating for incorrect viewing position
- Focal length
- Spatial distortions in stereo & non-stereo imagery
Biography of the speaker:
Marty Banks attended Occidental College in Los Angeles majoring in Psychology and minoring in Physics. He received a BA in Psychology in 1970 and spent a year teaching in the German school system in Ludwigsburg, Germany. He then attended UC San Diego and received an MS in Experimental Psychology in 1973. He then attended the University of Minnesota receiving a PhD in Developmental Psychology in 1976. Marty Banks was Assistant and Associate Professor of Psychology at the University of Texas at Austin from 1976 to 1985. He then moved to UC Berkeley where he has been Associate and Full Professor of Optometry and Vision Science. He has joint appointments in Psychology, Neuroscience, and Bioengineering. He was the chair of Vision Science from 1996-2002. Prof. Banks has received a number of awards including Fellow of the American Association for the Advancement of Science, Fellow of the American Psychological Society, and Koffka Medal Awardee.
Lecture 2: 3D Audio, theory and practice
Presented by: Dr Daniel Arteaga, Barcelona Media
Lecture outline:
- Part 1: Introduction (1h)
- Introduction to 3D audio
- Stereo, surround and 3D
- Channel-independent approaches
- Part 2 and 3: Spatial audio techniques (2h)
- Binaural audio
- Panning methods
- Ambisonics
- Wave field synthesis
- Part 3: 3D audio production workflow and exhibition (1h)
- 3D audio in cinema
- 3D audio in music
- 3D audio in broadcast
Biography of the speaker:
Daniel Arteaga is the Director of the Audio Research Group at Barcelona Media, and also visiting lecturer at Universitat Pompeu Fabra. After graduating and obtaining Ph.D. in Quantum Effects in Gravitation (Summa cum laude, European Doctor, 2007), he pursued a research career in Theoretical Physics as an Associated Lecturer at the University of Barcelona also teaching special and general relativity, mathematical methods for physicists, and mechanics. Simultaneously, in 2008, he started collaborating with the group of audio at Barcelona Media, before getting a full-time position there in 2009. Since then, Arteaga has participated in European and national research projects devoted to immersive audio applications for the media industry and environmental noise prediction, among others. His current research focuses on spatial audio, acoustic imaging with sounds and ultrasounds, automatic sound recognition, and sound over ultrasound technology, among others.
Lecture 3: Speeding up MATLAB applications
Presented by: Szilard Nemeth, Gamax Laboratory Solutions / MATLAB
Lecture outline:
- Part 1: Optimizing MATLAB code
- Techniques
- Using the right class
- Vectorization
- Preallocation
- Memory constraints
- Tools
- Code analyzer
- Profiler
- Debugger
- Techniques
- Part 2: Parallel Computing – Leverage the power of multi-core hardware
- Parallel computing on CPUs
- Built-in support in Toolbox functions
- Explicit parallelization using high level constructs (parfor, spmd)
- More control using jobs and tasks
- Overview of use of MPI
- Parallel computing on GPUs
- Overview of GPU computing in MATLAB
- Built-in support in Toolbox functions
- High level constructs (gpuArray)
- Distribute computation to a cluster, grid or cloud
- Overview of grid and cluster computing
- Distributed array for large data problems
- Parallel computing on CPUs
- Part 3: Code generation
- Overview
- Overview of code generation options
- Code generation for speed up
- Code generation for interfacing with other software
- Targeting hardware
- Code generation for embedded applications (FPGA, DSP)
- Overview
- Part 4: Stream processing with System Objects
- Stream processing
- System objects for video processing and computer vision
- Stream processing vs batch processing of videos
- Code generation support
- Stream processing
Biography of the speaker:
Szilárd Németh is Application Engineer Team Leader at Gamax Laboratory Solutions, Hungary. Gamax is the authorized distributor of Matlab, Simulink, Polyspace for South-Eastern Europe and provides related consultancy services and training on different aspect of the above-mentioned software products ranging from fundamentals to advanced signal processing and parallel computing. His area of expertise is core MATLAB products, parallel computing, statistics, optimization and global optimization, data analysis, financial analysis, energy trading and risk management, image- and video processing.
Note: this lecture will include a practical demonstration of the techniques introduced during the lecture, with the active participation of students. Please check the For Students page for requirements.
Lecture 4: Scene recovery from plenoptic data
Presented by: Dr Ivana Tošić, Ricoh Innovations Corp.
Abstract:
Recent development of hand-held plenoptic cameras has brought light field acquisition into many practical and low-cost imaging applications. By inserting a micro-lens array in front of the imaging sensor, plenoptic cameras can provide additional functionalities such as single-snapshot multi-spectral imaging, re-focusing and 3D imaging.
Depending on camera design, data captured by plenoptic cameras can carry different information about a scene, such as 3D structure, object ordering, object color, etc. However, all that information is typically mixed within acquired images and recovering scene properties from plenoptic data involves hard inverse problems that depend on optics design, sensor specifications and scene statistics. This tutorial will address recent advances in using image formation models (based on geometric and wave optics) and advanced signal processing algorithms to solve some of these inverse problems.
Lecture outline:
- Part 1: The forward path – Light field capture
- Definition of the plenoptic function and light fields
- Acquisition of light fields using plenoptic cameras
- Part 2: The forward path – Plenoptic image formation
- Models based on geometric optics
- Models based on Fourier optics
- Part 3: The inverse path – Scene recovery using geometric optics models
- 3D scene segmentation using sparse representations of light fields
- Dense depth estimation from light fields
- Part 4: The inverse path – Scene recovery using Fourier optics models
- Resolution enhancement via incoherent compressive sensing
- Dictionary learning for solving inversion
Biography of the speaker:
Ivana Tošić is a Senior Research Scientist at Ricoh Innovations, Corp. in Menlo Park, California, working in the computational optics and visual processing group. She received a Ph.D. degree in computer and communication sciences from the Swiss Federal Institute of Technology (EPFL), Switzerland in 2009, and a Dipl.Ing. degree in telecommunications from the University of Niš, Serbia in 2003. From 2009 to 2011, she was a postdoctoral researcher at the Redwood Center for Theoretical Neuroscience, University of California at Berkeley. Her research interests lie in the intersection of image processing and computational neuroscience domains and include binocular vision, image and 3-D scene representation, depth perception, representation of the plenoptic function and computational photography.
Lecture 5: Multi-camera video acquisition and Human Visual System aware rendering
Presented by: Karol Myszkowski, Tobias Ritschel, MPI Informatik; Frederik Zilly, Fraunhofer IIS
Lecture outline:
- Part 1: Perceptual Displays: Exceeding Physical Limitations and Improving Apparent Qualities
- Basic dimensions of image quality
- Human Visual System (HVS): Strengths and limitations
- Illusory vs. physical effects: What we can experience and what we can physically measure?
- Contrast enhancement (Cornsweet illusion)
- Brightness enhancement (glare, temporal glare, afterimages)
- Hold-type blur reduction
- Exploiting temporal integration for apparent resolution enhancement
- Part 2: Acquisition and Rendering of Free Viewpoint video using Multi-Camera arrays
- Feature-Point-Based Multi-Camera Rectification
- Multi-Camera Disparity Estimation
- Production Rules for visually pleasant Stereoscopic 3D
- View Rendering / Depth Image Based Rendering
- Visual Effects using Multi-Camera Data: Matrix-Effect, Vertigo-Effect, Synthetic Aperture
- Part 3: Perception-based Stereo Retargeting (90 minutes)
- Disparity Model
- Global disparity operators
- Backward-compatible stereo
- Personalized stereo
- Luminance-aware disparity model
- Disparity/depth compression
- Optimizing disparity for moving objects
- 3D rendering of glossy and transparent objects
Biographies of the speakers:
Karol Myszkowski is a tenured senior researcher at the MPI Informatik, Saarbrucken, Germany. From 1993 to 2000 he served as an Associate Professor in the Department of Computer Software at the University of Aizu, Japan. In the period 1986-1992 he worked for Integra, Inc. a Japan based company, developing rendering software for customers such as Toshiba Lighting, Shiseido, Matsushita Electric, Kandenko, and others. He received his PhD. (1991) and habilitation (2001) degrees in computer science from Warsaw University of Technology (Poland). His research interests include perception issues in graphics, high-dynamic range imaging, stereovision, global illumination and rendering. Karol published and lectured on these topics widely including ACM Siggraph/Siggraph Asia Courses in 2000, 2001, 2003, 2006, and 2011. He also co-chaired the Eurographics Rendering Symposium in 2001, the ACM Symposium on Applied Perception in Graphics and Visualization in 2008, the Spring Conference on Computer Graphics in 2008, Graphicon in 2012.
Tobias Ritschel is with the Cluster of Excellence on Multimodal Computing and Interaction at Saarland University and acts a senior researcher at the MPI Informatik. He received his PhD from MPI Informatik and Saarland University in 2009, and then joined Telecom Paris Tech (CNRS), Paris, France for his postdoctoral studies. His interests include interactive and non-photorealistic rendering, human perception and data-driven graphics. He is the recipient of the Eurographics Thesis Award.
Frederik Zilly is Head of the group Computational Imaging and Algorithms in the Moving Picture Department of Fraunhofer IIS in Erlangen, Germany. He has received the Diploma degree in physics from Free University of Berlin, Germany, in 2006. Before joining Fraunhofer IIS, he was Scientific Project Manager in the Immersive Media & 3-D Video Group in the Image Processing Department at Fraunhofer HHI in Berlin where he was involved in several German and European research projects related to 3DTV. In this function he was mainly responsible for the development of the Stereoscopic Analyzer (STAN) and coordinated the activities of the Fraunhofer HHI concerning the European research project MUSCADE project. Frederik Zilly has been honored for his work on the assistance system STAN (Stereoscopic Analyzer) with the Award for Outstanding Merit in Young Scientists, the Rudolf Urtel Prize 2011. He served as reviewer for different international conferences and journals. His research focus lies on multi- camera image processing algorithms.