Dr. Kim H. Veltman

Frontiers in Conceptual Navigation 2: Interfaces for Cultural Heritage

1. Introduction
2. Emerging Interface Technologies
3. Virtual Guides and Physical Museums
4. Virtual Museums, Libraries and Spatial Navigation
5. Historical Virtual Museums
6. Imaginary Museums
7. Metadata
8. Research and Knowledge
9. Challenges
10. Two, Three and Multiple Dimensions
11. Abstracts
12. Conclusions

    The enormous rise in new information has been paralleled by an equally extraordinary rise in new methods for understanding that information, new ways of translating data into information, and information into knowledge. New fields are emerging. For instance, at the frontiers of science and in the military, scientific visualization is a thriving discipline with close connections to virtual reality, augmented, enhanced and mixed reality. In business, database materials are being linked with spreadsheets to produce new three-dimensional visualisations of business statistics (e.g. companies such as Visible Decisions). In industry, data mining is emerging as an important new field. In the field of culture, where immediate profit is less obvious, these techniques remain largely unknown.

    Interestingly enough, standard books on human computer interface by Shneiderman do not give a complete picture of techniques now available or in development, nor even recent books with promising titles. There are a few journals, organizations and some conferences devoted to the subject of which the present is the most prestigious. Meanwhile, on the world-wide-web itself, there are a series of useful sites, which offer the beginnings of a serious overview into these developments. For instance, Martin Dodge (Centre for Advanced Spatial Analysis, University College, London), has produced a useful Atlas of Cyberspace, with examples of at least four basic map(ping) techniques, namely, conceptual, geographic, information (landscapes and spaces), and topology (including ISP and web site). A more thorough survey is provided in an excellent study by Peter Young, (Computer Science, Durham University), on Three Dimensional Information Visualisation. Here he lists twelve basic techniques: surface plots, cityscapes, fish-eye views, Benediktine space, perspective walls, cone trees and cam trees, sphere-visualisation, rooms, emotional icons, self-organising graphs, spatial arrangement of data and information cube. He also has a very useful list of research visualization systems. Chris North (University of Maryland at College Park), has also produced a useful and important Taxonomy of Information Visualization User Interfaces (see Appendix 1. Cf. the list of individuals in Appendix 2). Pat Hanrahan (Stanford) has made a taxonomy of information visualization, while Mark Levoy (Stanford) also has a taxonomy of scientific visualization techniques.

    The significance of emerging interface technologies will be considered, namely, voice activation, haptic force, mobile and nomadic, video activation, direct brain control, brain implants, and alternative methods. A problem with such taxonomies and the technologies which they class, is that they are mainly from the point of view of the technology’s capabilities, as if we were dealing with solutions looking for a purpose.

    In order to arrive at a taxonomy of users’ needs, a deeper understanding of their potential purposes is required, the whys? This paper offers preliminary thoughts in that direction. It begins with an outline of five basic functions relating to cultural interfaces, namely, virtual guides, virtual museums, libraries and spatial navigation, historical virtual museums, imaginary museums and various kinds of cultural research. The role of metadata is considered briefly. Particular attention is given to the realms of research, since it is felt that the new technologies will transform our definitions of knowledge. The conclusion raises some further questions and challenges.

    Bibliographical references to Human Computer Interaction specifically with respect to Graphic User Interfaces (GUI) and Network Centred User Interfaces (NUI) are provided in the notes. The appendices provide a taxonomy of information visualization user interfaces by data type (Appendix 1), a list of individuals and their contributions (Appendix 2) and a survey of other developments mainly in Canada, Germany, Japan, the United Kingdom and the United States (Appendix 3).

Table of Contents         

2. Emerging Interface Technologies

    It is generally assumed that the two-dimensional spaces of current computers are largely a reflection of hardware limitations, which will soon be overcome. Hence there are numerous initiatives to create three-dimensional spaces. Alternative interfaces and input devices are also being developed.

Three Dimensional Spaces

    Dr. Henry Lieberman (MIT) is exploring the use of very large three-dimensional navigation spaces, with new techniques which allow "zooming and panning in multiple translucent layers." Silicon Graphics Inc. (SGI) foresees the use of landscapes. Dr. Stuart Card (Xerox PARC) and his team have been working on a series of tools for visualizing retrieved information using techniques such as a galaxy representation, spiral calendar, perspective wall, document lens and cone tree . There is an analogous project at the Gesellschaft für Mathematik und Datenverarbeitung (GMD) in Darmstadt called Lyberworld. This takes a concept, searches for related terms, links these with the concept in question and presents them spatially in a cone. Alternatively the concept in question is positioned in the centre while various related terms are placed along the circumference of a circle where they exercise the equivalent of a centrifugal gravitational force. If all these surrounding terms are equal in strength they exercise an equal force on the central concept. As one of the terms becomes more significant it exercises a greater force on the central concept. Another GMD project, SEPIA, foresees a hypermedia authoring environment with four concurrent spaces: a content space, planning space, argumentation space and rhetoric space.

    At the level of abstract ideas a series of new products are being developed. For instance, a group at Rensselaer Polytechnic is developing an Information Base Modelling System (IBMS) which allows one to visualize relationships in parallel perspective. At the Pacific Northwest National Laboratory (Richland, Washington) a team led by Renie McVeety is developing a Spatial Paradigm for Information Retrieval and Explanation (SPIRE, cf. Themescape), while John Risch is developing Text Data Visualisation Techniques as part of the Starlight project. At Carnegie Mellon University the Visualization and Intelligent Interfaces Group is creating a System for Automated Graphics and Explanation (SAGE) and related methods (Sagebrush, Visage) for Selective Dynamic Manipulation (SDM). At the Sandia National Laboratory, Chuck Myers, Brian Wylie and a team at the Computational Sciences, Computer Sciences and Mathematical Center are working on three dimensional Data Analysis Data Fusion and Navigating Science, whereby frequency of articles can be visualized as hills in an information landscape. This is part of their Advanced Data Visualization and Exploration initiative called EIGEN-VR. Another project at Sandia is an Enterprise Engineering Viewing Environment (EVE). This:

    The Sandia team is also working on Laser Engineered Net Shaping (LENS) and has been exploring the implications of these techniques for modelling and simulation in manufacturing and medicine. The implications thereof for culture are no less impressive as will be suggested in the section on research and knowledge (see below section 7).

Alternative Interfaces and Input Devices

    While monitors controlled by a mouse remain the most popular form of navigation at the moment, a number of other alternatives are being developed. Bill Buxton has, for instance, produced what appears to be the most thorough list of existing input devices. This includes: aids for the disabled, armatures, bar code readers, boards, desks and pads, character recognition, chord keyboards, digitizing tablets, eye and head movement trackers, foot controllers, force feedback ("haptic") devices, game controllers, gloves, joysticks, keyboards and keypads, lightpens, mice, MIDI controllers and accessories, miscellaneous, motion capture, speech recognition, touch screens, touch tablets and trackballs. A full assessment of the pros and cons and philosophical implications of all these devices would be a book in itself. For our purposes, it will suffice to refer to some of the main alternative human web interaction systems.

Video Interaction

    One very innovative technique entails using video cameras to capture human movements and use these as cues for manipulating virtual environments. For instance, David Rokeby, in the Very Nervous System, links human movements such as dance to acoustic environments. As one moves more slowly or quickly, a different range of sounds is produced.

    Vincent John Vincent and the Vivid Group have developed other aspects of this approach in their Mandela software, such that the video camera and a blue screen essentially allow the user’s movements in the real world to be transposed to the virtual space within the screen. This permits a person to interact as a player in a virtual space on screen. For example, at the Hockey Hall of Fame in Toronto one can stand in a real goal, see oneself standing in a virtual goal on screen and interact with other virtual players there. This complex software requires customized programming for each site or special event. By contrast, the Free Action and Human Object Reactor software of a new company called Reality Fusion, offers more simplified versions of this approach allowing persons "to interact on screen with the body using video cameras".

    Such techniques are potentially of great interest not just for physically challenged persons. One could imagine a museum or gallery carefully equipped with video cameras such that one needed only to point to an object, or part of a painting and one’s notebook computer would give one an explanation at the level desired. Hence, if one had identified oneself as a grade school child at the outset there would be an elementary explanation, whereas a research student would be given a much more thorough description.

Voice Activated Interfaces and Visualization Space

    In the 1960’s there was considerable fanfare about dictation machines which, it was claimed, would replace the need for secretaries. After more than thirty years of hype, the first reliable products for the general public have been made available in the past year through companies such as Dragon Systems and IBM. Such systems presently entail vocabularies of 10-20,000 words, but will soon expand to vocabularies of 100,000 words and more. At the same time, researchers such as Mark Lucente (IBM Watson), working in conjunction with MIT have been developing futuristic scenarios whereby a person can control a wall-sized computer screen using voice commands.

    There are related projects elsewhere. The Gesellschaft für Mathematik und Datenverarbeitung (GMD) has an Institut für Integrierte Publikations und Informationssysteme (IPSI), which is working on a Co-operative Retrieval Interface based on Natural Language Acts (CORINNA). Such methods are attracting attention within the cultural community. In the United States, the Information Infrastructure Task Force (IITF) has created a Linguistic Data Consortium to develop a Spoken Natural Language Interface to Libraries.

    Voice activation clearly opens many new possibilities. For instance, many lists are tree-like hierarchies, which means that choices inevitably require burrowing down many levels until one has the set of choices one seeks. If these choices are voice activated then one can go directly to the appropriate branch of a decision tree and skip the levels in between. The effectiveness of the technique will, however, depend, very much on the situation. In the case of public lectures voice commands can help dramatic effect. In a classroom, if everyone were talking to their computers the results might border on chaos.

    Meanwhile, there is increasing study of the ways in which visual and auditory cues can be combined. For instance, a team at the Pacific Northwest National Laboratory (Richland, Washington) is working on the Auditory Display (AD) of Information "to take advantage of known strengths of both visual and auditory perceptual systems, increasing the user's ability to glean meaning from large amounts of displayed information":

    Such combinations of visual and auditory cues, are also being studied by Richard L. McKinley (Wright Patterson Airforce Base) in the context of a new field of bio-communications. If we truly learn so much better when we see and hear things in combination or at least in certain combinations then we clearly need to find ways of incorporating such experiences within the learning framework.

Haptic Force and Tactile Feedback

    Research into artificial arms and limbs, by pioneers such as Professor Steven J. Jacobsen (University of Utah, Salt Lake City) has led to new awareness of haptic force and tactile feedback as potential aspects of input systems. Corde Lane and Jerry Smith have made a useful list of a number of these new devices. Grigore Burdea, in a recent book, offers a very useful survey of this emerging field, showing that present applications are limited mainly to the military (combat simulation, flight simulator), medicine (eye surgery and arthroscopy training simulator) and entertainment (virtual motion three-dimensional platform).

    In the military, these principles are leading to tele-presence in the sense of tele-manipulation or tele-operation, whereby one can carry out actions at a distance. In the case of a damaged nuclear reactor, for instance, from a distance a person could safely control a robot, which would enter a space lethal for humans and do a critical repair. In medicine, these same principles are leading to tele-surgery.

    In the field of culture such haptic force and tactile feedback mechanisms could well lead one day to new types of simulated conservation experiments. Before trying to restore the only extant example of a vase or painting, one creates a model and has various simulations before attempting to do so with the actual object. Not infrequently, there will only be one or two experts in the world familiar with the techniques. These could give tele-demonstrations, which advanced students could then imitate.

    In the eighteenth century, the Encyclopédie of Diderot and D’Alembert attempted to catalogue all the known trades and crafts. Within the next generations it is likely that these will be recorded in virtual reality complete with haptic simulations. These techniques will continue to change with time, such that in future one could, for instance, refer back to how things were being done at the turn of the twentieth century.

Mobile and Nomadic Interfaces

    The advent of cellular telephones and Personal Digital Assistants (PDA’s) such as the Apple Newton or Texas Instruments’ Palm Pilot has introduced the public to the general idea of mobile communications, an emerging field, which involves most of the major industry players. At the research level the Fraunhofer Gesellschaft (Darmstadt) is working on Mobile Information Visualization, which includes Active Multimedia Mail (Active M3) and Location Information Services (LOCI)

    To understand more fully the larger visions underlying mobile communications it is useful to examine Mark Weiser’s (Xerox PARC) vision of ubiquitous computing. This goes far beyond the idea of simply having a portable phone or computer. Instead of thinking of the computer as an isolated machine, he sees all the technological functions of a room integrated by a whole series of co-ordinated gadgets, which are effectively miniature computers. Employee A, for instance, might always like a big lamp shining at their desk, have their coffee promptly at 10:30 a.m. each morning and not take calls from 2-3 p.m. because that is a time when the person writes letters. Assuming that the room could "recognize" the person, say through their badge, all of these technology "decisions" could be activated automatically, without employee A needing to turn on the big lamp at 8:30, the coffee machine just before 10:30 and turn on the answering machine from 2-3 p.m. In Weiser’s vision this recognition process would continue outside one’s own office. Hence, if employee A had walked down the hall and was visiting the office of employee C, the telephone would "know" that it should not ring in their now empty office and ring instead in C’s office for employee A, using a special ring to link it with A. Such challenges are leading to an emerging field of adaptive and user modelling.

    In the military, where mobile computing is frequently called nomadic computing, this vision is taken to greater extremes. Here one of the leading visionaries is a former director of the Defence Advanced Projects Agency (DARPA), Professor Leonard Kleinrock (University of California at Berkeley). In his vision, a computer should simply be able to plug into a system without worrying about different voltage (110, 220, 240) or needing new configurations of IP addresses. A soldier on the ground with their view obstructed by a hill, could communicate with an aircraft overhead, which would then relay to the soldier a bird’s eye view of the situation. Companies such as Virtual Vision are exploring some of the non-military implications of this approach.

    While museums and galleries are far removed from the life-threatening aspects of the battlefield, one can readily see how the greatly increased interoperability of devices being developed in a military context, has enormous implications for museums and galleries. Imagine a notebook computer that "knows" which painting is in front of one, and thus downloads the appropriate information without needing to be asked. Imagine a computer that immediately sought the information one might need for a city the moment one arrived in that city. Hence, on landing in Rome, it would download an appropriate map of Rome, complete with information about the relevant museums and their collections.

Direct Brain Control and Brain Implants

    Those concerned with universal access for persons with various disabilities have developed various devices such that one can, for instance, control computers simply by eye movements or other minimal motions.

    A number of projects are moving towards direct brain control whereby intermediary devices such as a mouse are no longer necessary. In Germany, the main work is occuring at the International Foundation of Neurobionics in the Nordstadt Hospital (Hanover), at the Institute for Biomedical Technique (St. Ingbert) and at the Scientific Medical Institute of Tübingen University (Reutlingen). In Japan, Dr.

    Hinori Onishi (Technos and Himeji Institute of Technology) has produced a Mind Control Tool Operating System (MCTOS). In the United States, Masahiro Kahata (New York) has developed an Interactive Brainwave Visual Analyser (IBVA). At the Loma Linda Medical Center work is being done on controlling computers with neural signals.

    Dr. Grant McMillan (Wright Patterson Airforce Base) has been exploring the potentials of brain waves (namely, Alpha, Beta, Theta, Delta and Mu) on control mechanisms. For example, a pilot may be in a flight simulator and find themselves flying upside down. Every time one thinks, the brain produces electric pulses. By harnessing these waves a pilot has only to think and the resulting waves can act as a command to return the simulator to an upright position.

    A more futuristic and potentially disturbing trend entails direct brain implants in a manner foreseen in the film Strange Days. Part seven of a BBC television series Future Fantastic directed by Gillian Anderson, entitled Brainstorm, discusses the work on brain implants by Dr. Dick Norman and Dr. Christopher Gallen.

    Given such developments, phrases such as "I see what you mean", "sharing an idea", "look at it from my viewpoint" or "giving someone a piece of one’s mind" a may one day be more literal than we now imagine. As noted above, it is already possible to activate certain commands simply by eye movement or through bands which measure one’s thought waves. In future, instead of voice activation, there might well be thought activation. Dictation would then simply require thinking the words which could conceivably lead some to forget how to speak properly. Will we be able to let others into our dreams and daydreams? Such questions lead quickly beyond the scope of this essay and yet the problems they entail may well become central to interface design sooner than we think. In order to assess more realistically the potentials of such applications it will be useful to step back and explore some basic functions of cultural interfaces.

Table of Contents   

    At the simplest level, one can imagine a physical museum endowed with different kinds of virtual guides. Instead of having a traditional tour guide, trying to shepherd a group of twenty or thirty visitors through various rooms, standing around a painting and having to shout to make themselves heard above the noise of the crowd, a visitor could simply rent a walkman-like device and listen to descriptions of paintings as they go. At the Museum in Singapore, for instance, such a device is already available. Certain displays and paintings are specially marked and for these a virtual guided tour is available. In Italy, the National Research Council (CNR) is developing a similar device, which will function much like a push-button dial on a telephone. However, instead of dialing a telephone number, one will key in the painting or monument number to receive the desired description.

    It is foreseen that these descriptions will be on-line. Hence, when a tourist arrives in a new city such as Rome for the first time, they will simply download the appropriate tours for that city, not unlike the way one now buys cultural videos of the city in question, except that all this will be on-line over the Internet. Given new electronic billing procedures, the "rental" of the tour can be arranged to allow only one hearing, or be limited to a series of hearings, or to tours within a set time-frame of a day, a week or a month.

    The walkman-like guide is but one possibility. As notebook computers move increasingly towards electronic versions of notepads (cf. the Newton and Palmtop), much more than a pleasant description of a painting or monument is possible. The notepad computer can give a visitor images of related paintings. For instance, standing in front of Uccello’s Battle of San Romano in the National Gallery of England (London), the viewer can be reminded exactly how it differs from the two other versions by Uccello in the Louvre and the Uffizi respectively. More advanced viewers could use this technology to compare minute differences between originals, versions by students of the painter, members of their workshop, copies and so on.

    Those not able to visit an actual painting would still be able to do such comparative study from their desktops even if these were far from major centres of culture. To be sure, seeing the original has and always shall be preferable to seeing surrogates. But in the past those in remote areas were typically doomed to seeing nothing other than occasional –usually poorly reproduced images in books. Now at least they will potentially have access to an enormous array of heritage wherever they happen to be.

    For those able to visit the famous museums there are still numerous barriers to seeing the painting as directly as one might wish. In extreme cases such as the Mona Lisa the work resides in a cage behind a solid sheet of glass which often refracts light in a way that hinders careful viewing. In most cases there are ropes or other barriers to keep one from getting very close to a picture. Even if one could get as close as one would like, many of the most intriguing aspects of paintings are invisible to the naked eye. Often, for example, there are subtle variations beneath the surface (pentimenti) as a result of a painter having changed their mind: changes in the position of a figure, or sometimes its complete removal. In the past, the only way of studying such changes was by means of x-ray photographs, which were only seldom available to a general viewer. Recently (1997), a new method called infrared reflectography allows one actually to see the different layers of paint beneath the surface. For instance, in Leonardo da Vinci’s Adoration of the Magi (Florence, Uffizi) there are elephants, which he drew and were subsequently painted over. It is likely that future tourists will rent a notepad computer, which allows them to see all the layers beneath the surface, thus giving new meaning to the concept of looking closely at pictures.

    The role of virtual guides is, of course, not necessarily limited to the interfaces of hand held devices as one goes around a real museum. They can be adapted for virtual and imaginary museums. IBM’s pioneering reconstruction of Cluny Abbey, had such a virtual guide or avatar, in the form of a mediaeval nun, who took one around the virtual reality model of the famous church. If Phippe Quéau’s visions of tele-virtuality come about, then we shall, in the near future, be able to choose the kind of avatars we wish and have them take us around whichever monuments may interest us.

    In the past, a day at a museum often ended with a visit to the museum shop, where one bought postcards or posters of the images which one particularly liked. Those available were typically a small selection of the holdings of a museum, and often it seemed that these invariably omitted the ones one wanted. In future all the images of a museum can be available on line and can be printed on demand. These images will include three-dimensional objects. At the National Research Council of Canada (Ottawa), a laser camera has been developed which produces fully three-dimensional images, which can be rotated on screen. Using stereo-lithography, three-dimensional copies of such objects can be "printed" on demand.

    Virtual reality permits one to create full-scale three-dimensional simulations of the physical world. Augmented reality goes one step further, allowing one to superimpose on that reconstruction additional information or layers of information. There are a number of such projects around the world. For instance, at Columbia University, Steve Feiner has been exploring the architectural implications of augmented reality in the context of various projects. One is termed Architectural Anatomy. This allows one to view a virtual reality version of a room and then see the position of all the wires, pipes and other things hidden behind the walls.

    A second is called Urban Anatomy and entails a method aptly termed X-Ray Vision. Here one can look at a virtual reality view of a street or a whole neighbourhood, superimposed or more precisely underlying which one sees the various layers of plumbing, wires and tunnels that one would see in a Geographical Information System (GIS). Except that, in this case, it is as if the earth were fully transparent and one can see precisely how they are collocated with the actual space. Similar techniques are being developed by researchers such as Didier Stricker at the Institut für Graphische Datenverarbeitung (IGD, Munich) which is linked with the Fraunhofer Gesellschaft’s Zentrum für Graphische Datenverarbeitung e.V. (ZGDV, Darmstadt). In this case augmented reality is being used to superimpose on real landscapes, proposed designs of bridges and other person-made constructions. Other projects at the same institute are working on Multimedia Electronic Documents (MEDoc) and Intelligent Online Services to create Multimedia Extension[s] (MME).

    Of even greater direct interest for cultural applications are the research experiments of Jun Rekimoto at Sony (Tokyo). Using what he terms augmented interaction, he has created a Computer Augmented Bookshelf, with the aid of

    Navicam. This "is a kind of palmtop computer, which has a small video camera to detect real-world environments. This system allows a user to look at the real world with context sensitive information generated by a computer." Hence, looking at a shelf of magazines, the system can point out which ones arrived today, in the last week and so on. A related invention of Dr. Rekimoto for use in theatres is called the Kabuki guidance system:

    The system supplies the audience with a real time narrative that describes the drama to allow a better understanding of the action without disturbing overall appreciation of the drama. Synchronizing the narration with the action is very important and also very difficult. Currently, narrations are controlled manually, but it is possible for the system to be automated.

    Applied to libraries, versions of such a system could essentially lead a new user through the complexities of a major collection. In the case of a regular reader, it could remind them of the location of books previously consulted. The reader might know they were there last year in June and that the book was somewhere in section C. The system could then identify the books in question.

    This approach also introduces new possibilities in terms of browsing. Instead of just perusing the titles on a shelf, a person could ask their notepad computer for abstracts and reviews with respect to the book in question using the SUMS interface. Alternatively, if a person were tele-browsing from their home computer they could call up these features while sitting at their desk at home.

Table of Contents

    Complementary to the above scenarios, are cases where virtual museums and libraries create digital versions of their physical spaces. Perhaps the earliest example of such an experiment was the Micro-Gallery at the National Gallery of England (London), a small room within the physical gallery with a handful of computers, where one could view images of the paintings in the collection and plan a tour in keeping with one’s particular interests. This approach has since been copied at the National Gallery in Washington and is being adapted by the Rijksmuseum at Amsterdam.

    Some of the early experiments in the field of cultural heritage pursued one metaphor to the exclusion of others. For instance, the Corbis CD-ROM of the Codex Leicester fixed on the image of a virtual museum for both paintings and books, such that the manuscripts appeared on the walls as if they were paintings. While optically appealing, such attempts were unsatisfactory because they eliminated many of the essential characteristics of books. Physical books give important clues as to thickness, size, age and so on. Their surrogates in terms of virtual books also need to convey these characteristics.

    Present research is actively engaged in creating such surrogates. For instance, Professor Mühlhauser (Johanneum Research, Graz), is working on virtual books, which will indicate their thickness. Dr. Stuart Card and colleagues (Xerox PARC), are exploring the book metaphor in virtual space and developing ways of moving from representations of concrete books to visualisations of abstract concepts which they contain. Companies such as Dynamic Diagrams have created a simulation of file cards in axonometric perspective for the Britannica Online site and for IBM’s web site. IBM (Almaden, Visualization Lab) has developed views of pages in parallel perspective as part of their

Visualization Data Explorer, such that one can trace the number of occurences of a given term in the course of a text.

    Such virtual museums and libraries can exist at various levels of complexity and their viewing need not, of course, be limited to some ante-room of the actual museum. As noted above, a number of museums include Quick-Time Virtual Reality (VR) tours on CD-ROMS of their collections. Meanwhile, others such as the Uffizi, have recreated online a version of their entire museum complete with simple Quick Time VR models of each room, such that one can look around to each of the walls as if one were there. These relatively simple images reflect the present day limitations of Internet connectivity, which will probably be overcome within the next decades.

    At the frontiers, an Italian company, Infobyte, is developing software called Virtual Exhibitor, which will allow museums to create such virtual galleries with a minimum of effort. Although this presently requires a Silicon Graphics machine, within two years regular PCs will be powerful enough to perform the same tasks. This software, along with SUMS are part of the European Commission’s Museums over States in Virtual Culture (MOSAIC) project in the context of their Trans European Networks (TEN) intitiative.

    Such virtual visits can go much further than simply visiting the rooms of museums ahead of time. In Tarkowsky’s famous film (1972) of Stanislaw Lem’s Solaris (1961), the viewers of Breughel’s Winter Landscape (Vienna, Kunsthistorisches Museum) enter into the painting and walk around in the landscape. Professor Robert Stone (VR Solutions, Salford), in a project called Virtual Lowry, uses virtual reality to take viewers through the spaces of Lowry’s painting. In Infobyte’s version of Raphael’s Stanze, viewers are able to view the School of Athens and then enter into the space and listen to lectures by famous ancient philosophers and mathematicians. Museums and galleries typically have one or more rooms where visitors can watch slide-shows, videos, or attend lectures pertaining to some aspect of their collections. In future such virtual visits could reasonably occur in such rooms or halls.

    In the context of museums, a series of cultural interfaces thus present themselves. In the equivalent of an ante-room, viewers are able to prepare for tours using monitors or more elaborate technology. For on-site tours there will be computer notepads. Monitors linked to printers in the museum shop will allow one to print postcards and full size posters on demand. For research purposes visits will occur sometimes on a computer screen, a large display panel, an IMAX type screen (which will probably be available on-line in the next generation), on planetarium ceilings or in entirely immersive CAVE environments (cf. below in section 7), within the museum or gallery rooms. In future as bandwidth increases these materials will become available on-line such that visitors (children and adults alike), can prepare for visits to museums and galleries by studying some their highlights or their detailed contents ahead of time, either at school or in the comfort of their homes.

    Museums and galleries have traditionally been famous for their "do not touch" signs. Many visitors, especially children, want to know how things feel. This is an area where virtual reality reconstructions of objects, linked with haptic feedback, could be of great help, thus adding experiences to museum visits which would not be possible in the case of original objects. Prostheses of sculptures, statues, vases and other objects can provide visitors with a sense of how they feel without threatening the original pieces.

    In most cases, these museum interfaces increase interest in seeing the original. Their purpose is to prepare us to see the "real" artifacts. Only in the case of special sites such as the caves at Lascaux or the Tomb of Nefertari, will the new technologies serve as a substitute for seeing the actual objects in order to protect the originals. By contrast, in the case of library interfaces, virtual libraries will very probably replace many functions of traditional libraries. Instead of using card catalogues to find a title and then searching the shelves for related books on a given topic, readers will use on-line catalogues and then do tele-browsing. Having found a book of interest, they will print them on demand.

    The continuing role of libraries will be defined in part by the kind of information being sought. Much of the time readers are searching for a reference, fact, a quote or a passage. Such cases can readily be computerized and replaced by on-line facilities. On the other hand historians of palaeography and of the book are frequently concerned with the feel of the cover, details of the binding or subtle aspects of hand-painted miniatures. In such cases, electronic facsimiles may help them answer preliminary questions, but consultation of the actual manuscript or book will remain an important part of their profession which only libraries can fulfill.

    Why even print when one can read on screen? Physiological experiments have shown that one sees about a third less when light comes to the eye directly from a monitor screen rather than being reflected from the surface of a page. Hence, while computer monitors are an excellent interim measure, they are not an optimal interface for detailed cultural research. A new kind of device, similar to a slide or film projector, is needed that projects images onto a solid surface.

Spatial Navigation

    Knowing how to get there, spatial navigation, is one of the fundamental concerns in the organization and retrieval of all knowledge including culture. The use of maps for this purpose is almost as old as civilization itself. Since the Renaissance there have been enormous advances in relating different scales of maps. In the past decades rapid developments in Geographical Information Systems (GIS) have begun linking these scales electronically (as vector images). Parallel with this has been a linking of scales of satellite and aerial photographs ( as raster images). The 1990’s have seen increasing translation between raster and vector images such that there is a potential interoperability between maps and photographs (figure 1).

    Projects such as Terravision in the United States and T-Vision in Germany can be seen as first steps in this direction. This means a potentially seamless integration of all spatial data such that we could move at will from views in space down to any painting on a wall or sculpture in a room. In Powers of Ten, a famous film by Charles and Ray Eames, a viewer was taken along one such visionary set of connections using photographs alone. Today it is technically feasible to do this interactively with any object in the world.

Scales of Abstract                                                                 Scales of Concrete

                Map of-                                                                                 Satellite Photos of -    World

                                                                                                                                         -     Continent

                                                                                                                                         -     Country

                                                                                                                                         -     Province

                                                                                                             Aerial Photos of    -    City

                Plan of -                                                                                                              -     Building (GIS)

                                                                                                            Guicktime VR of    -    Room

                                                                                                                                         -     Objects in Room

Figure 1. Basic scheme of scales of abstract images (maps and plans) and concrete images (satellite photographs, aerial photos and Quick Time VR images).

    Implicit in these breakthroughs is a reconciliation of methods which earlier generations perceived as different and even potentially incompatible. For instance, Gombrich (1975), in his Royal Society lecture distinguished between the mirror (photographs) and the map. Dodge, in his Atlas of Cyberspace, distinguishes between topological maps and the photographic type maps of information landscapes. While such distinctions may continue, the breakthroughs mentioned above will increasingly permit us to move seamlessly between categories, such that we can switch from viewing a topological map to a topographical map or an aerial photograph. By this same principle it will be possible to move seamlessly between photographs of physical rooms and Computer Aided Design (CAD) reconstructions of those same rooms used for Area Management/Facilities Management (AM/FM). This will bridge many earlier oppositions between abstract and concrete, making it clear that both can be correlated with the same reality. This has implications also for temporary and imaginary tours discussed below.

    Thus far only isolated aspects of this integrated vision have been adopted in the cultural context. For instance, city guides on the Internet are beginning to list maps with major museums and galleries. CD-ROM’s of galleries such as the Louvre, Pushkin, or the Uffizi typically have Quick Time VR views of the individual rooms. The technology exists to link together all these individual elements.

    Virtual reality allows complete reconstructions of objects, archaeological sites and historical monuments in three-dimensions. Some of the best examples of these possibilities are being created by Infobyte (Rome). These include reconstructions of the Upper Church of San Francesco (Assisi), Saint Peter’s Basilica (Vatican) and more recently the Rooms (Stanze) of Raphael as part of an ongoing project which may one day recreate the whole of the Vatican museum complex and become integrated with IBM’s Vatican Library project. The enormous number of such reconstructions, listed in a very useful book by Maurizio Forte, attests that such examples are part of a much larger phenomenon and that some of the cultural implications are clearly appreciated.

    Many of these reconstructions are typically viewed on a computer monitor. Sometimes glasses are used to permit stereoscopic viewing of the images. Sometimes this effect is achieved using a Binocular Omni-Oriented Monitor (BOOM). It is of course possible to make this experience fully immersive by projecting images on all the walls of a room as in the case of CAVE environments. Alternatively, one could project them onto the hemi-spherical surface of a planetarium using multiple projectors to create a fully immersive effect as Infobyte is doing by working in conjunction with the Japanese firm GOTO. Under discussion is the possibility that Infobyte’s reconstructions could be projected onto IMAX screens.

    One of the leading pioneers in the field of virtual reality is the German Gesellschaft für Mathematik und Datenverarbeitung (GMD, Sankt Augustin), which has a section on Visualisation and Media System Design. Among its many projects is Virtual Xanthen. Besides its well-known mediaeval church, Xanthen has a famous Roman archaeological site. The GMD project transforms a regular projection screen into an entire wall. A viewer standing in front of the wall sees an entire landscape from a bird’s eye view. The small platform on which they are standing serves as a navigating instrument, permitting one to "fly" higher above the landscape or get closer to the earth. This adds a whole new dimension to virtual visits.

    Traditional blue screens permit an actor to stand in front of a screen and be projected into a scene with a completely different background, as happens, for example, with the weatherman after the evening news on television. A limitation of this technique is that the backdrop is two-dimensional whereas the actor typically moves in a three-dimensional space. The GMD’s Distributed Video Project (DVP) takes these principles considerably further. The blue screen is transformed into a blue room and the actor’s movements in three-dimensional space are accompanied by three-dimensional perspectival adjustments in the background. Some of the obvious applications of this new technique are in the field of television and film production. Suppose for example, that one wished to do a film about the Sahara desert. Instead of needing to take a crew out to extreme conditions of the North African desert, one could simply digitize views of the desert and project them onto the equivalents of four walls and then use the blue room technique for actors to be virtually transported to the Sahara. The implications of this approach for culture are considered below in section 5.

    While such a translation of physical into virtual space constitutes the most obvious application of the new technologies, it is in a sense the least exciting. For the cultural field the most fascinating challenges lie in a new series of combinations of real and virtual, some of which will now be considered.

Table of Contents

    In the case of major museums and galleries, one virtual museum will not suffice. The buildings of the Louvre, for example, have existed on the premises of the present museum since at least the eleventh century. So one will need historical virtual museums, reconstructions, which help us to understand how what began as a mediaeval fortress gradually evolved into one of the world’s great picture galleries. These reconstructions will trace not only the physical growth of various rooms and galleries but also help to trace the changing arrangements of the permanent and temporary collections of paintings therein. Where was Mona Lisa hanging in the eighteenth and nineteenth centuries, as opposed to today and what do these changing configurations tell us about the history of exhibitions, taste and so on? Such digital versions of earlier spaces and former versions will allow simulations of temporal travel.

    This principle is also being applied to urban landscapes to create historical virtual cities. For example, CINECA, as part of the MOSAIC project, is reconstructing the mediaeval city of Bologna such that one can trace the growth of the city and changes in its basic structure in the course of several centuries. This reconstruction using virtual reality modelling language (VRML) allows one to walk through the streets and watch how they change as if one were in a time machine. Traditionally, some historians have claimed that Bologna developed an elaborate water and sewage drainage system during the Middle Ages. Other historians have challenged this. The model is sufficiently detailed that it can be used to check the validity and veridity of such claims. In such cases reconstructions of cultural heritage become significant for social and even economic history.

    Similarly, in the case of archaeological sites, this approach offers further possibilities. Today, a major museum typically has a photograph and/or a small model of the Acropolis at Athens. Students studying Greek history at school typically only have access to poor black/white images. Those with Internet access can, of course, consult a number of colour images through the Perseus Project. Much more is technically possible. Most European countries have their own archaeological schools in Athens and have developed their own theories about the Acropolis. So one could theoretically call up photographs of the site as it exists today. One could then call up various historical photographs and drawings in order to appreciate how it looked before Elgin took his marbles, what the Greek temple looked like when it became a Muslim mosque and compare it with how it looks today. One could then view various reconstructions by Greek, American, British, French, German and other archaeologists. Instead of just looking at buildings as static entities, one could examine how they change in the light of different cultural and scholarly traditions. Such reconstructions could be available on-site using notepad computers, as well as on-line for study at school and home.

    Professor Iwainsky has explored further potentials in his reconstruction of the Pergamon Altar, complementing the virtual reality reconstruction of the altar now in Berlin with filmed video clips of the original landscape in Pergamon, thus helping viewers to see how it would have looked in its original context. The GMD’s Distributed Video Production initiative introduces new techniques to develop this approach. One can, for instance, theoretically film views of and from the Acropolis, then, using a blue room, combine this with virtual reality reconstructions of the Parthenon and other buildings such that one could walk through the buildings as they might have been and have realistic views of the landscape. Given sufficient bandwidth such reconstructions can be on-line, permitting students and others around the world to get a realistic sense of sites long before they have a chance to actually visit the original.

Table of Contents

    Imaginary museums in a true sense will show paintings, sculptures and other artifacts, which never physically existed together, as coherent collections. Renaissance painters such as Botticelli, Leonardo and Raphael were typically commissioned to paint works for a church, monastery or a private patron with the result that their works were dispersed from the outset. To overcome this, art historians developed the catalogue raisonnée, but the high costs of printing typically meant that these catalogues offered only black-white images of paintings and often poor ones at that. Imaginary museums will allow one to see the collected paintings of an artist. This can happen in different contexts. In an actual museum such as the Uffizi, using one’s notebook computer one could stand in front of a Madonna and Child by Raphael and ask for other images by that painter on the same theme in other collections. This same principle can be extended to apply to thematic study also. Standing in front of a Baptism of Christ by Piero della Francesca (London, National Gallery) one would ask Baptisms within a given temporal or geographical framework. Alternatively one will acquire the equivalent of a CD-ROM which allows one to make these comparisons on one’s computer at home or at school. Increasingly these materials will be available on-line and future equivalents of hard disks will function in the manner of personal libraries today. They will have subsets on topics that are of particular interest to a given scholar, amateur or member of the general public.

    Imaginary museums can also offer viewers a whole range of interpretations concerning the structure and history of paintings. Standing in front of Piero della Francesca’s Flagellation of Christ (Galleria Nazionale delle Marche, Urbino), one could see the different interpretations of its perspectival space. Standing in front of Leonardo’s Last Supper (Santa Maria delle Grazie, Milan), one could compare it with other copies, see alternative reconstructions of its perspectival space and impressions of what it once looked like as well as having access to details of restorations concerning individual figures.

    Major collections such as the National Gallery (London) have an Art in Focus series, which are effectively special exhibitions focussing each time on an in depth analysis of special effects or characteristics of a given painting. Today such materials are typically available in an exhibition catalogue, which soon goes out of print. In future, such analyses could be available using notebook computers such that one could see such special characteristics at any time. This will give extended life to the concept of special exhibitions and indeed change their significance.

    A series of basic functions for cultural interfaces thus emerge. A first is virtual guides in physical museums; a second is virtual museums; a third is virtual historical museums and a fourth is imaginary museums. A fifth basic function of cultural interfaces entails research. Before exploring this and its implications for changing definitions of knowledge, a brief note on metadata is necessary.

Table of Contents

    In seeking to find data, information and knowledge on the web, system designers and scholars have devoted increasing attention to metadata in the sense of data about data. Initial efforts in this direction were focussed on identifying basic keywords and minimal descriptors (such as those being developed in the context of the Dublin Core) in order to permit identification of an article or book. The World Wide Web Consortium introduced a potential for rating quality through their Protocol for Internet Content Selection (PICS), the scope of which is being extended within their Resource Description Framework (RDF) to include Intellectual Property Rights and Privacy Management. In a recent keynote (Brisbane, April 1998), Tim Berners-Lee outlined a considerably more ambitious goal of adding a veridity parameter to information. His vision is to develop a global reasoning system within the world wide web, whereby individual knowledge elements function as distributed axioms, which can be combined to reach truth statements.

    5. Rights (Agreements)

    4. Privacy (Copyright)

    3. Quality (Ratings, Reviews)

    2. Veridity (Truth Value, Axioms)

    1. Summaries (Keywords, Descriptors)

    0. Contents (Facts, Claims)

Figure 2. Schema showing basic levels of metadata.

    Implicit in the above is a new approach to information and knowledge whereby facts and claims will no longer exist in isolation. Knowledge packages will be surrounded by five layers of metadata (figure 2). On the basis of such added parameters it will be possible to search for various subsets of materials. If one were searching for adventure films, one could ask, for instance, for all five, four, three, two, and one star examples separately or all films irregardless of their rating and then explore what percentage belong to each of the categories. One could compare the percentages in other fields. Are there more five star films, relatively speaking, in the mystery, thriller, childrens or other category?

    One could also begin mapping relationships of texts to commentaries about texts, statements of objective truth versus subjective claims about those truths. Levels of certainty might be depicted in different diaphanous colours, such that one could visualize a given verity and all claims surrounding it becoming further removed as their levels of certainty decrease. Not all materials will be certified. So one can choose whether one wishes only certified, officially sanctioned, materials or all materials pertaining to a given book, painting or other cultural object. A cultural object will no longer be a single entity, it will have associated with it a series of attributes defining not only its physical characteristics but also its quality. In terms of object oriented programming there will be objects of objects.

    It is important to recognize that the increasing importance of metadata is part of a larger shift whereby there is a separation between knowledge and views of that knowledge. The 1960’s, for example, saw the rise of databases. These allowed one to enter basic facts into fields of information, which could then be called up in a number of different ways as reports without needing to alter the original facts. The rise of Standardized General Markup Language (SGML) took this approach considerably further by effectively devising one set of tags for the original content and a second set of tags for ways of viewing that content. To put it slightly differently an SGML knowledge object has a "content" section and a "views" section. The evolving Extensible Markup Language (XML) uses exactly the same principle with the exception that it is designed for less complex situations than SGML and according is easier to use. In the case of both SGML and XML one can change or add to the "views" section without altering the basic content. This is fundamentally different from the print world where the content and layout become inextricably mixed to the extent that any decision to alter layout requires all the work of a new edition.

    Metadata, in the sense that it is being used by the World Wide Web Consortium, takes the basic approach of databases and SGML another significant step forward. It continues the distinction between content and views, but adds to the content section basic parameters concerning veridity and value. Facts remain constant. The ways of viewing them, using them, presenting them change. This opens the way to reusable knowledge in a new sense. The same repository can be used to tailor views for a beginner and an expert, without needing to rewrite the repository each time.

Figure 3. Systematic approach to a museum or library beginning with a ground-plan, view of a room, a wall, and finally a painting or book as if in file card form with basic descriptive information.

            Figure 4a. Visualization of editions as a list, a graph, as a circle or as an undulating inverted cone.  b) Visualization of related terms as a list, as a series of surrounding terms, as a series of intersecting circles or as other undulating inverted cones.

Figure 5. Lists of editions of a work, reviews, and commentaries thereof, translated into spatial locations and then into inverted cones. In this approach unimportant writings become narrow cylinders and influential important works become broad cones.

    Marshall McLuhan characterized the history of the West as a constant shift in emphases among the three elements of the trivium: grammar (substance or structure), dialectic (logic) and rhetoric (effect). Does the multimedia world of metadata mark a return the structural dimensions of grammar or does it mark an entirely new chapter in the evolution of knowledge? One thing is certain. As will become clear in the pages that follow, metadata is changing the nature of knowledge and the horizons of study.

Table of Contents

    With respect to research, appropriate interfaces will also depend very much on the purpose at hand. Often a reader or visitor is interested in questions of Who? What and Where? In such cases requiring quick access to basic facts, they will need on-line access to the digital reference room outlined in part one of this paper. At the simplest level, this will give them factual information about persons, places and things. Standing in front of Botticelli’s Birth of Venus (Uffizi, Florence), or looking at an image of that painting on the Internet, a viewer might want to have biographical facts. This can range from wanting the most elementary listing of when he born and when he died, through a one page synopsis of his life, to reading a standard biography. Standing in front of Leonardo’s Ginevra de Benci (National Gallery, Washington), one could ask about the history of previous owners in order to learn how it got from Florence via Liechtenstein to Washington.

    Today paintings are in galleries. Cultural artifacts are in museums. What we know about those paintings and artifacts is usually in articles and books in libraries, particularly in the reference section of libraries which contain terms (classification systems), definitions (dictionaries), explanations (encyclopaedias), titles (bibliographies, catalogues), and partial contents (abstracts, reviews). Given a universally accessible, digital reference room, viewers and readers will readily be able to find definitions and explanations without having to run to dictionaries, encyclopaedias, bibliographies and the like. Such searches can readily happen on a regular PC or a portable notebook computer. In these cases simple lists and paragraphs in a coherent interface will usually suffice.

    This type of quick reference or hunting after basic facts about objects and subjects, is the most elementary level of conceptual navigation which interests us. It sounds very straightforward and yet to achieve even this will require an enormous reorganization of knowledge. It would, for instance, be highly inefficient, and very time consuming, if everyone who wanted to know about an artist such as Botticelli, had to search every database around the world. Even searching through every database relating to art would take too long.

    Using the principles of object-oriented programming, we need to develop objects of objects, a richer kind of metadata, which will contain key information concerning them. In the case of Botticelli, for instance, this will include his variant names, his date of birth and death, where he worked, and a list of all his drawings, paintings and other works. This will build on the authority files for artists’ names such as Thieme Becker’s Allgemeine Künstler Lexikon (AKL), those of museums and libraries. In addition to this key information about his primary works there will be a reference to all secondary sources about Botticelli, in books, refereed journals and elsewhere. To achieve this will require the development of individualized agents which seek extant materials, gather them and are then vetted by the leading experts on that artist, author or individual. The net result of such efforts will be a Botticelli "object," with all the metadata pertaining to a traditional "complete works."

    In the case of a given painting this will include preparatory drawings, versions by students, members of the workshop, school and followers; copies, different owners, restorers and details of their restorations; their locations and dates. In the case of a manuscript this will entail all copies, all book versions, quantities published, editions, locations, and dates.

    Once these "knowledge objects" of artists, books, paintings, sculptures and other cultural heritage exist, they can be combined in new ways. If, for example, one were beginning from the context of a virtual museum, one might zoom in from a view of the world, to the continent of Europe, to the country of Italy, the city of Florence, the ground plan of the Uffizi, to a wall in the Botticelli room, and focus on his Adoration of the Magi (figure 3a). This would bring up basic information about the painting. One could then choose to see preparatory drawings, copies, other versions, other paintings by the same artist, or other paintings on the same theme by different artists.

    Three-dimensional navigation spaces are particularly valuable for such contextualisation of knowledge. A two-dimensional title or frontispiece of a book tells us nothing about its size. A three-dimensional scale rendering helps us to recognize at once whether it is a pocket sized octavo or a folio sized book: a slender pamphlet or a hefty tome. Hence, having chosen a title one will go to a visual image (reconstruction) of the book; see, via the map function, where it appears on the shelf of a library, do virtual browsing of the titles in its vicinity or wander elsewhere in the virtual stacks (cf. figure 3b).

    In the case of such a book, one might choose to see various editions in a chronological list. One could then choose to see the same list of editions as a graph showing fluctuations over time. Alternatively one might visualize the original edition as a small circle linked with successive editions in the form of an inverted cone which sometimes contracts and then expands further (figure 4a). Or one might begin with all the keywords related to a given edition, render these spatially either as a series of concepts surrounding the original, or as circles intersecting a central circle in the manner of a Venn diagram, each of which can in turn be visualized as inverted cones (figure 4b).

    In the excursus on metadata we mentioned a trend toward creating objects of objects, which will describe their physical characteristics and their quality. There will be numerous ways of visualizing quality. An author’s primary literature could define a circle, surrounded by a larger circle of secondary literature. Influential authors would have large surrounding circles. Unimportant authors would have only their initial circle: a visualization of "no comment." Alternatively, one could have a small circle for an edition, surrounded by larger circles for reviews, commentaries and citations: effectively a section of the cone in figure 5. In some cases there will be specific ratings such that one can identify specifically the grade or rating and not just the popularity of a book, painting or cultural artifact. Not all materials on the Internet will be certified. So one can visualise an object as a circle, surrounded by a certified circle and a larger uncertified circle. Combinations of these approaches are possible, such that one might discern which portions of reviews, commentaries and citations are certified or uncertified.

    In itself the creation of such circles may seem a rather fatuous exercise. If, however, they are produced on a specific scale and applied systematically to a subject, a field, a region, a country, a period, a movement or a style, or combinations of these, then the approach can be very useful in helping us to see new patterns of development. What correlations are there between the most influential books and the most important books? Does the production of important books in a field change over time? Does it shift from country to country? Can the reasons for the shift be determined?

    The attentive reader will have perceived that the systematic approach here outlined does not pretend that computers will use artificial intelligence (AI) or other algorithms to arrive at new insights in isolation. Rather the claim is that their systematic treatment of data and information will expand the range of questions which can reasonably be answered. By providing comprehensive treatment of the four basic questions: Who?, What?, Where?, and When?, they will prepare the ground for new answers to questions of How and Why? In this sense computers will help in intelligence augmentation (IA rather than AI in the senses of Doug Engelbart and Fred Brooks).

    This quest to achieve objects of objects which contain information concerning all the physical and qualitative characteristics of the original is analogous to the quest for determining the structure of DNA and the mapping of nature in the human genome project. It is much more than just another cataloguing project. It is a quest, which will transform the very meaning of knowledge.

    On a seemingly quite different front, companies such as Autodesk have extended the notion of object-oriented programming to the building blocks of the man-made world through what they term industry foundation classes. A door is now treated as a dynamic object which contains all the information pertaining to doors in different contexts. Hence if one chooses a door for a fifty-storey skyscraper, the door object will automatically acquire certain characteristics which are very different from those of a door for a cottage or a factory warehouse. This is leading to a revolution in architectural practice because it means that architects designing buildings will automatically have at their disposal the "appropriate" dimensions and characteristics of the door, window or other architectural building block which concerns them. There is a danger that this could lead to stereotyped notions of a door or window, a McWorld effect, whereby buildings in one country are effectively copies of those in other countries, and travel loses its attractions because everywhere appears the same.

    This same technology can be used with very different consequences if one extends the concept of foundation classes to include cultural and historical dimensions. If this occurs, an architect in Nepal wishing to build a door, in addition to the universal principles of construction applying to such objects, will be informed about the particular characteristics of Nepalese doors, perhaps even of the distinctions between doors in Khatmandu or those near Annapurna. Similarly an Italian restorer will be informed about the particular characteristics of doors in Lucca in the fifteenth century.

    All this may seem exaggerated and unnecessary. During the second World War, however, some of the key historical houses with elaborate ornamental carvings in Hildesheim (e.g. the Knochenhaueramtshaus or Bone Hacker’s administrative office) were bombed and it took a small group of carpenters several decades to reconstruct the original beam by beam, carving by carving. They did so on the basis of detailed records (photographs, drawings etc.). If this knowledge is included in the cultural object-file of Hildesheim doors, windows and houses, then rebuilding such historical treasures will be much simpler in future.

    At stake is something much more than an electronic memory of cultural artefacts, which would serve as a new type of insurance against disaster. The richest cultures are not static. They change with time gradually transforming their local repertoire, often in combination with motifs from other cultures. The Romanesque churches of Northern Germany adopted lions from Italy for their entrances. The church of San Marco in Venice integrated Byzantine, Greek and local motifs. The architecture of Palermo created a synthesis of Byzantine, Norman, Arabic and Jewish motifs. The architects in Toledo and at Las Huelgas near Burgos created their own synthesis of Jewish, Arabic and Christian motifs. A comprehensive corpus of variants in local heritage thus leads to much more than a glorification of local eccentricities and provincialism. It can prove an inspiration to multi-culturalism in its deepest sense.

    The same principle, which applies to doors and windows, applies equally to ornament, decoration, various objects such as tables and stools and different building types: temples, colosseums, monasteries, cathedrals, and churches. This transforms the meaning of knowledge. According to Plato, knowledge of a temple was to recognize in some particular manifestation the "idea" of some ideal temple. Knowledge did not require knowing the exact dimensions of the Parthenon or any other temple. According to Aristotle knowledge lay in the precise characteristics of a temple such as the Parthenon. Plato was interested in a universal concept, Aristotle in a particular example. Their mediaeval successors became embroiled in philosophical debates whether knowledge lay in universals or particulars. Even in schoolbooks today this problem has not been resolved. History texts typically refer to one example, the Colosseum in Rome, as if it were the only example, as if the particular were synonymous with the universal class.

    The new object oriented approach to knowledge is very different from all of these precedents. For a "temple object" will not only contain within itself the precise description of the Parthenon, but also the exact descriptions of all the other temples including those at Segesta, Selinunte, Agrigento, and Syracuse in Sicily, at Paestum and Rome in Italy, at Ephesus, Miletus, and Uzuncaburc in Turkey and elsewhere. This new definition of knowledge resolves the age old opposition between universal and particular, for it can describe the essential characteristics which all the temples have in common (universal) and yet render faithfully all their individual differences (particular).

    Knowledge now lies in a combination of the two. The Platonic idea of a temple reduced individual complexity to common characteristics, destroyed individual differences and thereby the notion of uniqueness. The modern "temple object" centres knowledge on the fundamental significance of differences. Thus temples gain universal value through the richness of their local variation. The universal becomes a key to particular expression. Knowledge lies not in recognizing how good a copy it is but rather in how well it has created a variation on the theme.

    The Colosseums in Rome (Italy) and El-Djem (Tunisia) were built in the same style. Nonetheless their effect is profoundly different due to their spatial settings, one in the midst of the Roman Forum, the other in a near desert setting. Hence knowledge of spatial location, the co-ordinates familiar to Geographical Information Systems (GIS), and Area Management/ Facilities Management (AM/FM) will also be an essential part of a "colosseum (knowledge) object".

    The Colosseum in Rome was built at a given time. It was not, however, a static building, in the sense that it remained exactly the same in the course of the centuries. We know, for instance, that a large portion of it was dismantled in the Middle Ages to construct other buildings. Hence a "colosseum (knowledge) object" will need to include all our knowledge about changes over time: i.e. its complete history, including all restorations and interventions for its conservation. Knowledge now includes time as well as space.

    Some will say that this new approach to knowledge is merely a revival of an age-old encyclopaedic tradition. This is potentially misleading because the encyclopaedic tradition itself has undergone fundamental shifts in its goals. Aristotle was encyclopaedic but his quest was to create summaries, which were subsets of the originals such that the originals could be abandoned. That is why we have what Aristotle said about many of the ancient authors rather than the ancient authors themselves. Their works were allowed to go lost because it was assumed that the Aristotelian summary replaced them. Vitruvius was also encyclopaedic in this sense, except here there was an added goal of making the subset readily memorizable, an aide-mémoire, rather than creating a record of all that existed.

    Such decisions were not only guided by profound philosophical reflections. They were partly pragmatic reflections of the available storage media. If knowledge is writ on stone tablets, the burden of knowledge is truly great. The advent of parchment, manuscripts and then printing expanded those horizons considerably. Ironically the same Renaissance which introduced the medium of printing, introduced also a tendency to use media to separate knowledge: books were put into libraries, pictures into galleries, drawings into print galleries (cabinet de dessins), engravings into engraving galleries (Kupferstich Kabinett), maps into map rooms and cultural objects into museums. Knowledge was decontextualized.

    The 18^th century Encyclopédistes re-introduced a vision of encompassing all knowledge. But as the rate of knowledge continued to increase, even the organizers of the Encyclopaedia Britannica, decided, after 1911, to abandon the quest for universality. Recent innovations in terms of macro-paedia and micro-paedia sections have neither re-contextualized knowledge nor re-introduced a quest for an inclusive encyclopaedic approach.

    The new "knowledge objects" distinguish themselves from earlier efforts in several ways. First, computers remove both the barriers of storage capacity and any need to separate knowledge on the basis of media. Second, the "knowledge objects" require a new kind of encyclopaedic approach: one that is globally inclusive of all the variants rather than merely a local summary thereof. This will change the meaning of "objects" insomuch as we shall have collected in one place all quantitative and qualitative information about an object.

Multiple Views

    In the past scholars typically spent a majority of their time trying to locate basic facts about an object: Who painted it? Where was it made? When was it finished? For the next few generations scholars will likely be pre-occupied with assuring that the new "knowledge objects" are reliable and as comprehensive as they claim. Once all such information is at our fingertips, will scholars find themselves redundant in the face of automation as in the case of many traditional manufacturing jobs? The answer is definitely not. It is simply that the tasks will be different. In the Middle Ages it took one hundred monks ten years of full-time effort to create an index to the works of Saint Thomas Aquinas. Today that same task can theoretically be performed in minutes by a computer. Having an index spares us the need of reading the complete works everytime we are looking for some particular thought, argument, or fact. But this does not remove the challenge of choosing thoughts, arguments and facts and deciding how or why to apply them. The process of thinking remains.

    Once the basic facts have been arranged, scholars will find themselves devoting more attention to presentation. Professors will become view masters in a new sense. Their challenge may lie less in conveying basic facts, than in teaching students to look at facts and concepts in new ways: as a list, a chart, on a map or more abstractly. To take a simple example, any book has a series of key words associated with it, which provides us with some clues concerning its scope. Few keywords indicate a specialized title. Many keywords suggest a title with many applications. Such keywords can be visualized as sides of regular and semi-regular shapes and solids. In this configuration, specialized authors produce points, lines and triangles. Generalists produce increasingly many-sided solids. This introduces new possibilities for looking at the authors in a given field, or of a certain distinction. Were Nobel laureates in 1908 mainly generalists or specialists? Were there significant differences between the arts and science? Did geography play an obvious role? For instance, were the Nobel laureates from Europe more specialized than those from America, or conversely? Did this pattern change over time? Implicit in such activities is a shift from questions about substance (Who? What? Where? When?) to those of function (How?) and purpose (Why?). The old notion of scholars as philosophers may witness a revival.

    Presentation is much more than deciding whether to use overheads or slides, whiteboards or virtual reality. It is ultimately concerned with new methods of structuring knowledge, not just individual objects, but also the larger frameworks into which they can be arranged. This is the terra incognita of future scholarship. Knowledge organization will become as important as knowledge acquisition and raise many new challenges for cultural interfaces.

    One aspect of this structuration process will lie in integrating hitherto disparate knowledge elements. For instance, to continue with our earlier example, the "colosseum (knowledge) object" will entail all the physical characteristics of the colosseum at Rome and those of all the other colosseums at Arles, Nimes, El Djem, Pula and elsewhere in the empire. Using a map one will be able to see where all these places are. Linking a time line with this map one will be able to trace the order of their appearance. Were there close connections between the rise of colosseums and theatres? If so were these connections geographical and chronological or only one of the above? Or were the rise of colosseums and theatres two quite distinct phenomena? Similar questions could be posed with respect to the rise of monasteries, churches and cathedrals.

    One can imagine scholars devoting their energies to posing what they think could be fruitful or significant questions. One can also imagine a future generation of agents automatically generating questions and comparisons and only reporting on cases where some significant correlation emerged. In which case the challenge of scholarship would focus less on finding patterns and more on explaining their significance. There would be various levels of patterns, some local, others regional or national, a few international or even global. These patterns will lead to new attempts to characterize periods, movements and styles.

    How will these differ from the periods of traditional historical studies? Because they encompass a much larger sample of evidence they will frequently come to very different conclusions. By way of illustration, it is useful to cite the case of the Renaissance. In traditional studies, ancient Greece marked a period of enlightenment. During the "Dark Ages", the story goes, the lights went out. Then around 1400, someone found the light switch and there was a Renaissance, literally a rebirth. The light switch, we are told lay in the rejection of the Dark Ages and a return to the wondrous insights of Antiquity. Thus far the received wisdom.

In this stereotypical view, Renaissance art is usually reduced to the achievements of a handful of remarkable painters including Botticelli, Leonardo, Raphael and Michelangelo, and museums such as the Uffizi and San Marco are of central importance. And although passing reference is made to the importance of Assisi and the Arena Chapel of Giotto, standard books tend to ignore the predominant role played by fresco cycles in all the major churches of the Renaissance, not only in Florence and Venice, but equally in Castiglione d’Olona, Milan, Montefalco, Perugia, San Gimignano, Sansepolcro, Siena, Rome and lesser known centres such as Atri. A careful examination of these cycles reveals that they focussed on the lives of the saints, from Christ’s contemporaries such as Saint John the Baptist and the Apostles (such as Peter and Paul), through the early martyrs (Steven and Lawrence) and church fathers (Augustine, Jerome), right though the Middle Ages (including more recent saints such as Thomas Aquinas and Saint Francis of Assisi). Seen as a whole this corpus points to some very different conclusions about what was happening in the Renaissance. The artists of the Renaissance discovered an uninterupted continuity between the time of Christ and their own period provided by the lives of the saints. Far from rejecting entirely a so-called Dark Ages, one could argue that they recognized for the first time its essential role. In short the entire history of what happened in the Renaissance needs to be rewritten.

    In the longer term there is a larger challenge of finding ways to show how the complexity of cultural activities in the period 1300-1600 could be so reduced as to make the myth about rejecting the Middle Ages a temporarily convincing misrepresentation of the truth. This is another manifestation of the relationship between content and views. Let us posit, hypothetically, that there were 10,000 buildings of cultural interest during the Renaissance. Every major art historian such as a Berenson, a Chastel, or a Gombrich focusses on some subset thereof. So we should have interfaces which show us how schools of scholarship in a given country both bring into focus some aspects while at the same time obscuring many other aspects. We need to make visible the way secondary literature functions as a prism that leads us to overlook complexity while at the same time explaining other bits.

Figure 6. A term in a classification system shown as a two-dimensional list. This list is folded ninety-degrees to a plane at right angles to the screen. Another classification system is introduced in a plane parallel to the first. A third classification system is introduced in like manner. This is used to visualize links between the term in the three systems.

    An important dimension of knowledge structuration lies in classification systems. The major international systems are relatively few. They include Bliss, Dewey, Göttingen, Library of Congress, Ranganathan, Riders International for books as well as the Art and Architectural Thesaurus and Iconclass for art. To a certain extent these reflect national differences. The United States has the Library of Congress and Dewey. Germany has the Göttingen system and others. India has the Ranganathan system. In terms of lesser systems or systems specialized on some particular field there are at least 950 others. Each of these presents different ways of classing the world, with different branches, facets, alternative associations, different broader and narrower terms. These systems also change over time. Ranganathan initially had little about art compared to western systems, yet a great deal about consciousness and higher states of awareness.

    When we find a cultural object it can be classed in many ways. Traditionally museums have developed one way of classing, art galleries another and libraries another again. Yet a given painting may well represent an object which exists physically in a museum and about which there is written material in a library. This is why we need meta-data and meta-databases in order to discover the commonalities required to create integrated knowledge objects.

    To study a cultural object systematically we need authority lists to have their standard names and recognize which are their variants. Classification systems reveal how that object has been classed as a subject, topic, theme, field, discipline and so on. Such systems also reveal the hierarchies or trees within which objects have been placed. These structures change with time. So we need ways of visualizing equivalences either geographically, chronologically or both. We might begin, for example, (figure 6) by treating the term on the screen as a plane, make this transparent, rotate it downwards by 90 degrees such that it becomes the top surface of a transparent box. The x-axis now becomes the time-axis such that we can literally trace the connections between various subjects. Such an example outlines a means of moving seamlessly from two-dimensional lists to three-dimensional conceptual maps of subjects with their related topics and also offers a whole new way of seeing interdisciplinarity.

    One of the challenges in moving between different cultures lies in knowing where to look for equivalent terms. So a person from Canada familiar with the Library of Congress (LC) might choose a series of Library of Congress Subjects. If they were interested in India, the system would then find the closest related terms in Ranganathan and use these to search other catalogues and lists. At a next stage this set of terms can be used to create a cluster of closely related terms and use these for searching.

    As noted above the quest for equivalent terms leads almost inevitably to a search for related terms, objects and subjects, much in the way that browsing in a library while looking for one book, very frequently leads us to find others, which are as relevant or perhaps even more so than the book we originally set out to find. Classification systems provide another means of contextualising our search: i.e. seeing relations between one subject and another. When we are studying a subject, we typically want to know about related subjects.

    In the past we went to a library catalogue, found a title and saw the related topics at the bottom of the card. Electronic versions thereof exist. Recent software such as Apple’s Hotsauce allows us to go from a traditional two-dimensional list of terms, choose one, and then see all the related topics arranged around it. These related subjects evolve with time, so with the help of a simple time scale we can watch the evolution of a field’s connections with other subjects. This idea can easily be extended if we translate the main topic into a circle and the related subjects into other (usually smaller) circles intersecting the main one to create a series of Venn diagrams. This visualisation allows us to choose subsets common to one or more related fields, which is important if one is trying to understand connections between fields (figure 4b).

Relators

    Classification systems typically take us to broader and narrower terms in our quest for related terms. But as thinkers such as Perrault and Judge have noted there are numerous other means to acquire related terms including: alternatives, associations, complementaries, duals, identicals (synonyms as in Roget’s Thesaurus), opposites (antonyms), indicators, contextualizers and logical functions such as alternation, conjunction, reciprocal, converse, negative, subsumptive, determinative and ordinal. It is feasible that these will eventaully become part of the "knowledge objects," such that if one has a term, one can see its synonyms without needing to refer to a thesaurus. All these kinds of relations thus become different hooks or different kinds of net when one is searching for a new term and its connections.

    Such classification systems are the most familiar, important efforts at bringing order to the world in terms of subjects. But subjects in isolation are still only somewhat ordered information. Meaning which brings knowledge and wisdom requires more, namely a systematic ordering of these subjects in terms of their logical and ontological relations. Efforts in this direction go back at least to the I Ching. Aristotle, Thomas Aquinas, Ramus, Francis Bacon and Roget were among the many contributors to this tradition. In our generation, Dr. Dahlberg presented these fundamental categories in a systematic matrix. More recently these have been adapted by Anthony Judge into a matrix of nine columns and nine levels (figure 6), which generates a chart of 99 subjects. These range from fundamental sciences (00), astronomy (01) and earth (02) to freedom, liberation (97) and oneness, universality (99). Anthony Judge is using this as an "experimental subject configuration for the exploration of interdisciplinary relationships between organizations, problems, strategies, values and human development".

Matrix columns                                                                 Matrix levels

9 Condition of the whole                                                   Experiential (modes of awareness)

8 Environmental manipulation                                            Experiential values

7 Resource redistribution                                                  Innovative change (context strategies)

6 Communication reinforcement                                        Innovative change (structure)

5 Controlled movement                                                    Concept formation (context)

4 Contextual renewal                                                       Concept formation (structure)

3 Differentiated order                                                       Social action (context)

2 Organized relations                                                        Social action (structure)

1 Domain definition                                                           Biosphere

0 Formal pre-conditions                                                   Cosmosphere/Geosphere

Figure 7. An integrative matrix of human preoccupations by Anthony Judge (Union Internationale des Associations) adapted from Dr. Ingetraut Dahlberg.

    Heiner Benking, builds upon the framework of Dahlberg and Judge (as in figure 7 above), to produce his conceptual superstructure or cognitive Panorama Bridge, which is the basis of his Rubik’s Zauberwürfel [Cube of Ecology or Magic Cube]. He argues that one can use planes in order to see patterns in thought. These planes, he claims, can include continua between the animate and the inanimate on one axis and between micro-, macro- and meso-scales on another axis. Planes, he claims, can be used to compare different viewpoints at a conceptual as well as a perceptual level; to see relations among different actions, options and strategies.

    Seen in this context, it becomes evident that our discussion thus far has been rather narrow. It has dealt primarily with physical objects in the cosmosphere/geosphere (level 0) although the comments on classification have touched briefly on concept formation (level 4). From this point of view the amount of knowledge structuration that remains to be done is staggering indeed. Scholars are not about to be without work.

    If we were trying to achieve a truly big picture involving the interplay of two or more of the planes in this matrix, then a three-dimensional interface with the kinds of planes outlined earlier will be essential (cf. figure 6). Parallel planes can be used to see different levels of abstraction. A challenge remains how precisely we are to navigate between such conceptual landscapes and the knowledge structures of libraries, which have been a main focus of this paper. At a programming level this should be relatively straightforward. Each of the ninety-nine subjects is tagged with its equivalents in the main classification schemes. At the user level, this and similar matrices then become a natural extension of the system. When we use these categories as a filter to look at publications in the Renaissance or research trends in the late twentieth century, we have another means to comprehend which areas were the focus of attention and which were abandoned, or even forgotten. Search and access systems must help us to see absence as well as achievement, and possibly provoke us to look more closely at the spaces which are being ignored. Were they truly dead ends, have they surfaced in a new guise or do they now require new study?

    The third dimension has many uses beyond producing such electronic copies of the physical world. Pioneers of virtual reality such as Tom Furness III, when they were designing virtual cockpits, realised that pilots were getting too much information as they flew at more than twice the speed of sound. The challenge was to decrease the amount of information, to abstract from the myriad details of everyday vision in order to recognise key elements of the air- and land-scape such as enemy planes and tanks.

    This principle is equally important in knowledge organisation and navigation. A library catalogue gives me the works of an author. Each catalogue entry tells me under how many fields a given article or book is classed. Adding these fields together leads to an alphabetical list of that author’s intellectual activities. Producing such a list in electronic form is theoretically simple. What we need, however, is a conceptual map. To what extent did an author work as a generalist in large subject fields and to what extent as a specialist? This lends itself to three dimensions. Author A is in one plane and the subject headings of their works are on other planes. These are aligned to relative positions in classification systems such that one can see at a glance to what extent this person was a generalist or a specialist and linked with the matrix of human preoccupations to discern how they relate to this (figure 8). This principle can be extended in comparing the activities of two authors.

Figure 8. Visualisation of an author’s activities whose specialist activities touch on four fields (three of which are closely related) and whose more generalist activities are limited to one major field. This can, in turn, be linked with the matrix of human preoccupations. Further layers could be added to show how the same concepts recur in different places in various classification systems.

Figure 9. Venn diagram of a subject and its related subjects, shown as intersecting circles. In addition to regular searches by subject, this visualisation allows a user to choose subsets common to one or more related fields, which is important if one is trying to understand interdisciplinary relationships. Cf. fig. 4.

Figure 10. In this diagram the large circles again represent two fields and the smaller circles represent branches of these fields. The lines joining them represent work linking hitherto different branches. These lines thicken as the amount of articles and other research activities increase and thus become a new means of tracing the growth of an emerging field.

Figure 11. Using spatial arrangements of concepts to map problems identified and to visualise which subsets thereof were financed as research projects, which were solved in the sense of producing patents, inventions and products and which led to new predictions in the form of hypotheses and projections.

    This approach can in turn be generalised for purposes of understanding better the contributions of a group, a learned society or even a whole culture. Scholars such as Maarten Ultee have been working at reconstructing the intellectual networks of sixteenth and seventeenth century scholars based on their correspondence. A contemporary version of this approach would include a series of networks: correspondence, telephone and e-mail which would help us in visualising the complexities of remarkable individuals be it in the world of the mind, politics or business.

    The geographical aspects of these intellectual networks can be visualised using maps. Conceptually the subjects of the letters, (and the e-mials to the extent that they are kept), can be classed according to the layers outlined above such that one gains a sense of the areas on which they focussed. For instance, what branches of science were favoured by members of the Royal Society? Did these change over time? It is a truism that Renaissance artists were also engineers and scientists.

Figure 12. Diagram relating to metadatabase research at Rensselaer Polytechnic in conjunction with Metaworld Enterprises entailing a Two Stage Entity Relationship (TSER) in the context of an Information Base Modelling System.

    What particular fields did they favour? Can one perceive significant differences between artist-engineers in Siena, Florence, Rome and Venice? We could take the members of a learned society or some other group and trace how many layers in the classification system their work entailed and then study how this changed over time. Are the trends towards specialisation in medicine closely parallel to those in science or are there different patterns of development? Alternatively by focussing on a given plane of specialization we could trace which authors contributed to this plane, study what they had in common in order to understand better which individuals, networks of friends and which groups played fundamental roles in the opening of new fields. Such trends can in turn be linked with other factors such as research funding or lack thereof. In addition to universities, major companies now have enormous research labs. Nortel, for instance, has over 17,000 researchers. Hitachi has over 30,000. We need maps of Who? is doing What? and Where? In our century we could also trace where the Nobel and other prizes have gone both physically and conceptually. Navigation provides virtual equivalents of journies in the physical world. It is also a means of seeing new patterns in the conceptual world through systematic abstraction from everyday details in order to perceive new trends.

    If we were trying to trace the evolution of a new field, we could begin by using a dynamic view of classification systems described above. We could also use combinations of these intersecting Venn diagrams. For example, the last generation has seen the emergence of a new field of bio-technology. This has grown out of two traditional fields, biology and technology. These could be represented as large circles surrounded by smaller ones representing, in this case, their related branches and specialties. Any academic work would be represented in the form of a line, which thickens in proportion as the connections increase. These connections are of differing kinds. Initially they tend to be in the form of sporadic articles, conferences, or isolated research projects, which have no necessary continuity.

    Later there are books, journals, professorships, research institutes and spin-off companies which bring a conscious cumulative growth to the new field. Each of these phases could be identified with different colours so arranged that one can distinguish clearly between sporadic and cumulative activities (figure 10). We can integrate these circles within the context of frames as described above. For example, the two fields of biology and technology could be on one plane. Major universities could be on a second plane. We could then trace which universities are producing the most papers of these two fields and specifically on what sub-sections thereof. On another plane we could list the major research institutes in order to determine other trends. Are these areas being studied more in the United States than Europe or Japan? If so what are the percentages? Which companies dominate the field? What links are there between research institutes and universities? Are these increasing or decreasing?

    Experiments in the realm of metadatabase research at Rensselaer Polytechnic provide a preliinary idea of how a concept at one level can be linked via planes with a series of concepts at another level. Such a notion of planes can be extended to see further patterns. Plane one can list all the known problems or potential research areas in a given area of science. Plane two lists which subset of these problems is presently being studied. Plane three shows which problems have been solved, or rather have some solutions in the form of inventions, patents, trademarks and products. Plane four lists a further subset for which solutions are predicted or which have hypotheses for their solution (figure 12).

    Such comparative views can help scientists and decision-makers alike to understand more clearly trends in rapidly developing fields. Such matrices of problems can in turn be submitted to problem structuring methodologies whereby technical, practical and emancipatory dimensions are submitted to frameworks in order to discern where they fit into what some have called a Methodology Location Matrix.

    Returning for a moment to the framework outlined in figure 7, one can envisage the direction which a future encyclopaedia will take. For instance, level seven in this framework outlines context strategies including logic, philosophy, security, community, peace and justice. These will be related to the context of concept formation (level five) and its structure (level 4), the context of social action (level 3) and its structure (level 2).

    Earlier we discussed the spread of ancient temples, of mediaeval monasteries, churches and cathedrals. These would be linked with the growth of religious ideas and the religious orders which followed from them. Which were the ideas that led to mainstream religions? Which were the ideas that led to peripheral sects? Which ideas led to the development of significant groups, organizations, parties, political movements? Earlier we outlined the development of objects in spatio-temporal terms. The history of ideas will need to be explored in spatio-temporal-socio-conceptual terms, each represented by levels in the third-dimension, which can be translated back to two-dimensional lists and descriptions as appropriate.

    During the Renaissance the discovery of linear perspective brought new skill in visualising the physical world, but it began by illustrating episodes from the lives of saints, which none of the artists had witnessed personally. Hence it helped simultaneously in expanding the horizons of the visible world of nature and the invisible world of the mind. This dual development continues to our day. Three-dimensional visualisations, especially using virtual reality help to illustrate both the visible and invisible, and to introduce many new possibilities.

    If, for instance we take the Library of Congress classification, as above, and link each layer in its hierarchy with a different layer, then we arrive at a truncated pyramidal shape beginning with twenty initial topics at the top and increasing to many thousands as we descend. Say we are interested in total publications in the Library of Congress. At the top level, these publications can be linked to each of the twenty basic fields, such that each major subject is represented proportionately as a square or circle. We can thus see at a glance to what extent the number of books on science is greater than those in the fine arts. By going down a level in the hierarchy we can see how those figures break down, e.g. to what extent there are more books on physics than chemistry or conversely. At another level we can see whether and if so to what extent astro-physics has more publications than bio-physics or quantum physics. We are thus able to see patterns in knowledge which we could not see simply by looking at the shelves, although even shelves can give us some hint that one topic has more books than another.

    A slightly more refined version would link this approach to book catalogues such that we can trace how these trends in publications change over time. From a global point of view we could witness the rise of the social sciences in the nineteenth century. At a greater level of detail we could see the rise of psychology as a field. This same approach can also be applied to usage patterns as studied by scholars in reception theory. In future usage patterns by on-line readers will become important for scholars as well as those doing market studies.

    In our quest to see significant patterns it will sometimes prove useful to have agents examine trends and draw our attention only to cases where there are considerable changes, of say 10 or 20%. This will be another way to discover emerging subjects. This same methodology has a whole range of other applications including marketing, advertising, stock markets and even network management. Say, for example, that we want to monitor potential trouble spots on the system. Agent technologies measure usage at every major node of the system in terms of a typical throughput and usage. When these ratios change significantly the system identifies where they occur, and introduces standard adjustment measures. If these fail, the system visualises relevant points in the neighbourhood of the node such that operators can see remotely where the problem might lie and take