submitted to
Marc Abrams, Deborah Hix, Ronald D. Kriz**, Cliff Shaffer, Layne T. Watson, Robert C. Williges
** Principal Contact: R.D. Kriz
Norris Hall, ESM (0219), Virginia Tech
Blacksburg VA 24061-0219
(540) 231-4386, kriz@wave.esm.vt.edu
This proposal brings together researchers with a variety of backgrounds to focus on these problems, with our various interests integrated by the 3D CAVE environment that we propose to create. The CAVE is a 3D visual computing environment that recreates space and allows the researcher to interact and visualize complex shapes in an interactive 3D environment.
Given the availability of suitable hardware, the primary barrier to effective 3D visualization is a well designed user interface. Developing user interfaces to portray large quantities of information is rapidly becoming one of the most challenging areas of computer systems. Across all sectors of our society -- private, industrial, military, government -- the need to manipulate the ever-increasing volume of data is growing rapidly. The primary objective of our research is to dramatically improve the methods used for manipulating and visualizing vast quantities of data, especially in 3D environments. VR-based systems as the CAVE are strikingly different from the current defacto standard GUIs (graphical user interfaces) that run on desktop workstations. But there have been very few new techniques developed for users, who develop on desktop workstations, to interact with CAVEs and other 3D virtual environments in a way that is most appropriate for those environments. In fact, all too often in a virtual environment, there is a fundamental mismatch between what a device is suited for versus what is it actually being used for. Clearly the CAVE is a superior 3D visualization environment when compared to traditional desktop workstations, but desktop workstations are where engineers and scientist first visualize their data, hence linking this two environments would be of obvious benefit. Both input and output need to be studied, and techniques for enhanced user control in these environments developed and evaluated. New input technologies such as eye gaze, foot-based input, and of course speech, may offer improved interaction for both the desktop and CAVE environments. For output, such effects as haptic (tactile/pressure) and auditory feedback need to be studied. These new techniques for user control need a close coupling of input and output in order to simulate the real world (as much as is desirable and/or possible).
Human-computer interface (HCI) work in this project will be user centered. This means that the work proposed in this section will not be based not on what software or interaction techniques are already available or are easiest to code as in section 2, but rather on what tasks users want and need to perform in order to access, manipulate, and understand large amounts of data, particularly in virtual environments such as the CAVE. These user-based needs will then serve as requirements for creating supporting software for novel interaction techniques that offer an appropriate match of technique and device to user goals and tasks. Hopefully the results of this study will also provide insight into how to improve the desktop workstation environment. The full potential of promising interactive technologies can be realized only when users can easily communicate with such systems. The result of our proposed HCI research will be measurably (and documentability) improved user interfaces for visualization and control of multi-dimensional information, in terms of both functionality and usability for the user.
We propose to build an interface between 3D visual tools that work on both desktop workstations and a 3D CAVE hardware environment. Our 3D visualization software interface would be designed to allow researchers to fully develop applications on their desktop workstations and then to automatically convert their workstation application for use in the CAVE. Emphasis is placed on developing applications on the desktop workstation rather than entirely from within the CAVE. Since CAVE technology is expensive, it is difficult to justify developing a specific application entirely within the CAVE.
Researchers are more likely to use 3D CAVE hardware if they can continue to develop on their desktop workstations. This puts an emphasis on working with existing, widely used turnkey software. We propose to work with AVS, MSI-BioSym, VNI, SGI and other companies that make 3D volume visualization tools. We will create this interface both as an independent post processor and as an integral part of the source code when possible. We anticipate that while working with CAVE hardware and software interfaces design we will encounter new ideas that will also enhance the 3D visualization techniques used on desktop workstations. For this we anticipate the purchase of some stero-scopic workstation displays.
We expect researchers from a variety of disciplines on campus to participate in both designing and testing our workstation-CAVE interface tool, hence giving the HCI evaluation team the opportunity to study a much wider variety of 3D visualization applications. We plan to make our resulting software widely available to the general research community, DoD agencies and their industrial partners.
The fundamental mathematical structure of many data spaces is generally multidimensional, and the goal is to fathom the structures through 3D views of that space. The central problem is data navigation: either the data set is too massive to exhaustively visualize, or there are multiple data sets to be compared.
Many scientific disciplines produce and use large, multidimensional databases. One typical examples include the General Social Survey (GSS), widely used in the Social Sciences. Collected nearly every year for about 20 years, the GSS contains hundreds of individual questions each administered to hundreds of individuals regarding preferences and demographic statistics. Another example is the US Census, issued as hundreds of statistics on each census tract. Large scale computations such as the multidisciplinary design optimization of an aircraft or weather forecasting produce far too much data to exhaustively visualize.
Most scientific visualization software today is designed to visualize numerical data, or data that has as its domain a set whose members are totally ordered, such as integers or real numbers. Yet many databases such as the GSS or the US Census include non-numeric, or categorical data: a partial order may exist among the domain values, but a total ordering does not. Genome sequences and individuals in a demographic study are categorical data. So are the values in many computer and network traces, such as traces of which program module is currently in execution in a program used in the design of software for parallel computers. The trouble with categorical data is that there is no obvious way to visualize them, in contrast to numerical data, whose total ordering provides an obvious mapping to two or more Cartesian dimensions. A particular problem is visualizing categorical time series. The body of time series methods for numerical data, such as Fourier transforms, correlation graphs, and statistical methods such as ARIMA are defined in terms of arithmetic operations, which are meaningless for categorical data.
Just as a statistician would never make an inference based on a sample size of one, visualization systems will ultimately have to handle multiple data sets, representing multiple simulation runs, or multiple trace files, or multiple observations of a system. Doing so requires handling massive amounts of data, thereby exacerbating the scale problem in visualization tools: how to design a tool that can simultaneously analyze an arbitrary number of traces of arbitrary size, but not exceed the limited input bandwidth of a human's senses. There are three fundamental solution methods to allow analysis of multiple data sets, while addressing the scale problem:
The following are some approaches to dealing with these problems.
Real time navigation: Visualization today is mostly concerned with representing one data set, and the hardware reflects that fact--one fast workstation or a synchronous multiprocessor with a few processors. True data navigation of large and multidimensional data sets will require massive computational power, which translates to parallel supercomputers with tens to hundreds of processors (like an Intel Paragon or a Cray T3E, rather than a Power Challenge). We propose development and implementation of parallel visualization algorithms, because they are a crucial enabling technology.
Extending visualizations: Techniques for visual analysis and browsing of such multidimensional databases are beginning to emerge. Integrated environments allowing multiple views of the data can allow users to gain new insights into their data. Such views include maps (for spatial distribution), graphs for range distribution, and spreadsheets for viewing multiple criteria at a time. Scatter plots and special techniques such as parallel coordinates attempt to let users visually spot correlations and other relevant relationships. Few of these techniques have been extended to 3D visualization environments and the new visualization technology such as desktop and CAVE VR.
Machine interpretation: A methodology that helps identify "interesting" things to look at, such as outliers or steep gradients, or that intelligently condenses insignificant data, or that clusters similar data, can help in navigation. Our past work, on a system to analyze time series data called Chitra, analyzes not one but a set, or ensemble, or data sets. Chitra commands, which can manipulate the ensemble as a unit, statistically analyze, model, and visualize select members individually, or as an ensemble. Chitra provides transforms to simplify the data and models to reduce trace data to a parsimonious, dynamic characterization of system behavior. We propose building on our work with Chitra to incorporate data navigation and techniques from the emerging field of "data mining." The solution used in Chitra is to visualize some traces, test all traces, transform all, and model all. Transforms are used to control state space explosion in the resultant model; examples of transforms include state aggregation by pattern aggregation and state aggregation by filtering in frequency domain. Among its statistical tests, Chitra allows partitioning of an ensemble. Normally the user will partition into mutually exclusive and exhaustive sub-ensembles, each of which is homogeneous; and then visualize one trace in each sub-ensemble, which by definition is ``representative.'' Chitra also can generate a separate model of each sub-ensemble.
Exploiting more human input bandwidth: Naturally a significant bottleneck in visualization of multidimensional databases has been the limits of the 2D computer monitor. 3D visualization hardware allows for the development of new visualization techniques that take advantage of 3D stereoscopic displays, or true 3D support through Virtual Reality. We propose studying VR interfaces and visualization techniques for multidimensional databases.
The primary goal of this research is to provide a fundamental understanding of the major independent and dependent variables that improve human-to-human communication in computer-conferencing environments using 3D visualization. Computer conferencing can be enhanced by 3D visualization in order to facilitate electronic presence in human-to-human communication. A host of human-computer interface parameters and alternative human sensory communication variables need to be considered simultaneously in the design of these computer-augmented systems. The large number of resulting variables preclude empirical evaluation in one large experimental design. These variables need to be investigated using sequential research techniques in order to build an integrated database using the results of several studies to specify the functional relationships among these variables. A five-year research effort is proposed involving three overlapping phases of investigation. Phase 1 includes the development of the 3D computer-conferencing environment that will operate using the 3D CAVE visualization tools and the development of appropriate evaluation metrics. Phase 2 deals with sequential research planning, selection of appropriate independent variables affecting human communication, selection of dependent variables affecting communication performance, and conducting a series of sequential experiments. Phase 3 is concerned with combining all the data into an integrated database that can be queried by an empirical model based on polynomial regression to optimize the 3D computer-conferencing system.
Improved 3D visualization user interfaces that will allow the researcher to first develop 3D tools on the desktop workstation that can be transformed into a CAVE environment when necessary.
Creation of the 3D tool described in section 2. This will include
Development of a 3D computer conferencing environment, as well as a database to design 3D computer conferencing systems.
Marc Abrams Associate Professor, Computer Science, VPI; Ph.D., Computer Science, University of Maryland at College Park, 1986. Five years of work developing Chitra, a system to visualize, analyze, and model categorical and time series data sets. Areas of interest: Visualization of multidimensional and categorical data; performance modeling of communication networks and parallel, and distributed systems.
Deborah Hix Research Computer Scientist, Computer Science, VPI and Naval Research Laboratory in Washington DC. Ph.D. Computer Science and Applications, 1985. Principal investigator in Human-Computer Interaction research group at Virginia Tech, in existence since 1979 and one of the pioneering groups in HCI research and development. Areas of interest: methodologies for user interface development, multimedia applications, novel interaction techniques.
Ronald D. Kriz Associate Professor, Engineering Science and Mechanics and Materials Science and Engineering (http://www.sv.vt.edu/krizbio.html for resume), VPI; Ph.D. Engineering Mechanics, Virginia Tech, 1979. Director of the Laboratory for Scientific Visual Analysis (http://www.sv.vt.edu) and has been teaching a course on Scientific Visual Data Analysis and Multimedia with an emphasis on scientific and engineering applications since 1991 (http://www.sv.vt.edu/class/esm5984.html).
Clifford A. Shaffer Associate Professor, Computer Science, VPI; Ph.D., Computer Science, University of Maryland at College Park, 1986. Since 1987, he has been a member of the Department of Computer Science at Virginia Tech where he is presently an Associate Professor. Dr. Shaffer's primary research interests are in the areas of spatial data structures, computer graphics and computer aided education. He was project director for the award-winning GeoSim educational software.
Layne T. Watson Professor, Computer Science and Mathematics, VPI; Ph.D., 1974, Mathematics, University of Michigan, Ann Arbor; B.A., 1969, Psychology, University of Evansville. Areas of interest: Numerical analysis, nonlinear programming, mathematical software, solid mechanics, fluid mechanics, image processing, scientific computation. Associate Editor SIAM J. on Optimization and ORSA J. on Computing
Robert C. Williges Professor of Industrial and Systems Engineering, Psychology, and Computer Science at VPI; Director of Human Factors Engineering Center, VPI; M.A., Ph.D. degrees in engineering psychology from The Ohio State University, A.B. degree in psychology from Wittenberg University. Over 25 years experience managing and directing human factors engineering research dealing with topics including human-computer interaction and human factors research methodology using sequential experimentation and empirical modeling building. Fellow of the Human Factors and Ergonomics Society and the American Psychological Association; past editor of Human Factors.
Costs: (costs are only estimates that have not approved by the University)
TOTAL: $4,245,000 for 5 years; Year-1, $1,409,000; Years-2,3,4,5, $709,000/year
R.D. Kriz
Virginia Tech
1996