conanxin

Abstract

Human-computer symbiosis is an anticipated development of cooperation and interaction between humans and electronic computers. This will involve a very close coupling between humans and electronic devices. The main objectives are 1) to enable computers to facilitate formalized thinking, as they currently assist in solving formalized problems; 2) to allow humans and computers to collaborate in making decisions and controlling complex situations without relying on predetermined programs. In the anticipated symbiotic partnership, humans will set goals, formulate hypotheses, establish criteria, and conduct evaluations. Computers will perform routine tasks to prepare humans for insights and decisions in technical and scientific thinking. Preliminary analyses indicate that the symbiotic partnership will be more effective in intellectual activities than individuals alone. Prerequisites for achieving effective collaborative relationships include the development of time-sharing for computers, memory components, memory organization, programming languages, and input and output devices.

1 Introduction

1.1 Symbiosis

Only the fig wasp (Blastophaga grossorun) can help fig trees complete pollination. The larvae of this insect live in the ovaries of the fig tree, where they also find food. Thus, fig trees and fig wasps are critically dependent on each other: without fig wasps, fig trees cannot bear fruit; without fig trees, fig wasps cannot obtain food. The combination of the two not only allows them to survive but also creates a highly productive and vibrant cooperative relationship. "Two different organisms live together in intimate cooperation, even forming a close alliance," this mode of cooperation is called symbiosis.

Human-computer symbiosis is a subclass of human-machine systems. There are many human-machine systems. However, there is currently no human-computer symbiont. The purpose of this paper is to propose this concept and to draw attention to the applicable principles of human-computer engineering by analyzing some issues in human-computer interaction, pointing out some questions that need research answers, thereby promoting the development of human-computer symbiosis. We hope that, in the not-too-distant future, the human brain and computers will be closely integrated, resulting in a partnership that will be considered capable of thinking and processing data in ways that no human brain can achieve with the information processing machines known today.

1.2 Between "Machine-Enhanced Humans" and "Artificial Intelligence"

As a concept, human-computer symbiosis differs in an important way from what North refers to as "machine-enhanced humans." In past human-machine systems, the operator held the initiative, provided direction, integrated, and established standards. The mechanical parts of the system were initially the human's arms and then extensions of the eyes. These systems were certainly not composed of "different organisms living together." There was only one organism—humans—while the rest were merely aids to that person.

In a sense, any artificial system is designed to assist humans, aiding one or more individuals outside the system. However, if we focus on the operators within the system, we find that there have been tremendous changes in certain technical fields over the past few years. "Machine enhancement" has replaced humans, shifting towards automation, leaving humans more as helpers rather than being helped. In some cases, particularly in large information and control systems centered around computers, human operators are primarily responsible for functions where automation is not feasible. Such systems (which North might call "human-enhanced machines") are not symbiotic systems. They are "semi-automatic" systems, originally fully automated but failing to achieve their goals.

Human-computer symbiosis may not be the ultimate paradigm of complex technical systems. At the appropriate time, electronic or chemical "machines" seem entirely capable of surpassing the human brain in most functions we currently consider. Even now, the progress of Gelernter's IBM - 704 plane geometry theorem proving program is comparable to that of a Brooklyn high school student, making similar mistakes. In fact, there are several theoretical proof, problem-solving, chess, and pattern recognition programs that can match human intellectual performance in restricted domains; while Newell, Simon, and Shaw's "General Problem Solver" may eliminate some limitations. In short, it seems worthwhile to avoid arguing with (other) artificial intelligence enthusiasts, who believe that only machines will dominate in the distant future. However, in the meantime, major intellectual advances will be achieved through close collaboration between humans and computers, which will be a rather long transitional period. A multidisciplinary research group studying future research and development issues for the Air Force estimated that by 1980, developments in artificial intelligence would enable machines to think or solve militarily significant problems independently. This would lead, for instance, to 5 years for developing human-computer symbiosis and 15 years for its application. Those 15 years could be 10 years or 500 years, but those years should be among the most creative and exciting periods in human history.

2 Goals of Human-Computer Symbiosis

Today's computers are primarily designed to solve predetermined problems or process data according to predetermined programs. The computational process may depend on results obtained during the computation, but all alternatives must be anticipated in advance. (If unforeseen alternatives arise, the entire process halts, waiting for necessary program extensions.) The requirement for predetermined or pre-established criteria sometimes has no significant drawbacks. It is often said that programming computers forces people to think clearly, standardizing the thinking process. If users can think through their problems in advance, then a symbiotic relationship with computers is unnecessary.

However, many problems... are difficult to think through in advance, recalling the earlier description of emerging systems. If errors in reasoning can be exposed through trial and error guided by intuition in collaboration with computers, or if unexpected turns in solutions can be revealed, problems can be solved faster and better. Without computer assistance, other problems cannot be solved at all. Poincaré foresaw the frustration of a significant number of potential computer users when he said, "The problem is not, what is the answer? The problem is, what is the question?" One of the main purposes of human-computer symbiosis is to effectively incorporate computers into the formalized aspects of technical problems.

Another primary goal is closely related. It is to effectively bring computers into the thinking processes that must occur "in real-time," where the pace is too fast to allow for traditional computer use. Imagine, for example, trying to command a battle with the help of a computer on such a timeline. You pose your question today. Tomorrow you spend time with a programmer. You receive a 20-foot-long paper filled with numbers that do not provide a final solution but merely suggest a strategy that should be explored through simulation. Clearly, the battle will end before the planned second step begins. The way of thinking while interacting with a computer is the same as how you would interact with a colleague whose abilities complement your own, which would require a much closer coupling between humans and machines than this example suggests and the current state allows.

3 The Need for Computer Participation in Formalized and Real-Time Thinking

The preceding paragraphs assume the hypothesis that if effectively introduced into the thought process, the functions that data processing machines can perform will significantly improve or facilitate thinking and problem-solving. This hypothesis may require justification.

3.1 Preliminary and Informal Ergonomic Analysis of Technical Thinking

Despite the vast literature on thinking and problem-solving, including numerous historical case studies of the invention process, I found nothing better than conducting an ergonomic study analysis of the mental labor of people engaged in technological enterprises. Therefore, in the spring and summer of 1957, I attempted to record what a moderately technical person did during the time he believed he was focused on work. Although I recognized the inadequacy of the sampling, I still made my research subject. It was clear that my main activity was keeping records; if the details envisioned in the original plan were preserved, this project would turn into an infinite regression. Not so. Nevertheless, I obtained a snapshot of activities that made me stop. Perhaps my scope is atypical—I hope not, but I fear it is.

I spent 85% of my "thinking" time on thinking, decision-making, and learning things I needed to know. The time spent searching for or acquiring information was much greater than the time spent digesting information. Several hours were spent drawing charts, and other hours were spent instructing assistants on how to draw charts. Once the charts were completed, two relationships immediately became apparent, but drawing was necessary to make them a reality. At one point, it was necessary to compare the six experimental measurements of speech clarity and speech noise ratio. No two experimenters used the same definition or measurement of speech noise ratio. Several hours of computation were needed to convert the data into a comparable form. When they were in a comparable form, I spent only a few seconds determining what I needed to know.

In short, throughout the study, my "thinking" time was primarily spent on activities that were essentially clerical or mechanical: searching, calculating, drawing, transforming, determining a set of hypotheses or the logical or dynamic consequences of hypotheses, paving the way for decisions or insights. Moreover, my choices of what to try and what not to try were largely based on considerations of clerical feasibility rather than intellectual capability, which is embarrassing.

The main suggestion conveyed by the research results just described is that, for most of the time, operations referred to as technical thinking are operations that machines can perform more effectively than humans. These operations must be conducted in an unpredictable and constantly changing order across different variables, which presents serious problems. However, if these problems can be resolved in a way that establishes a symbiotic relationship between humans and fast information retrieval and data processing machines, then collaborative interaction will clearly improve the thinking process significantly.

At this point, it may be necessary to acknowledge that we are using the term "computer" to encompass a variety of computing, data processing, and information storage and retrieval machines. The capabilities of such machines are increasing almost daily. Therefore, making general statements about the functions of this class is risky. Perhaps making general statements about human capabilities is equally risky. Nevertheless, certain genotype differences in capabilities between humans and computers are indeed striking, and they have implications for the nature of possible human-computer symbiosis and the potential value of achieving such symbiosis.

As has been said in various ways, humans are noisy narrowband devices, but their nervous systems have many parallel channels that are simultaneously active. In contrast to humans, computers are extremely fast and precise, but they can only perform one or a few basic operations at a time. Humans are flexible, able to "continuously self-plan" based on newly received information. Computers are rigid, constrained by their "pre-programmed" nature. Humans naturally speak a redundant language, organized around single objects and coherent actions, using 20 to 60 basic symbols. Computers "naturally" speak a non-redundant language, typically with only two basic symbols, lacking inherent appreciation for single objects or coherent actions.

To be strictly correct, these characteristics must include many qualifiers. Nevertheless, the differences they present (and thus their potential complementarity) are essentially correct. Computers can easily, well, and quickly perform many tasks that are difficult or impossible for humans, while humans can easily and well perform many tasks that are difficult or impossible for computers, albeit not very quickly. This suggests that symbiotic cooperation, if successfully integrating the positive features of both humans and computers, will have immense value. Of course, the differences in speed and language present challenges that must be overcome.

4 The Distinct Functions of Humans and Computers in the Anticipated Symbiotic Relationship

It seems that the contributions of human operators and devices will be so completely intertwined in many operations that it will be difficult to neatly separate them in analysis. This is the case; for example, when collecting data that serves as the basis for decisions, both humans and computers find relevant precedents from experience, if the computer subsequently proposes an action plan that aligns with human intuitive judgment. (In theorem proving programs, computers find precedents in experience, while in the SAGE system, they propose action plans. The above is not a far-fetched example.) However, in other actions, the contributions of personnel and devices are somewhat separable.

Certainly, at least in the early stages, humans will set goals and provide motivation. They will formulate hypotheses. They will ask questions. They will think of mechanisms, programs, and models. They will remember that such individuals did some possibly relevant work on a topic of interest as early as 1947, or at least shortly after World War II, and they will know which journals that topic might be published in. Overall, they will make approximate, erroneous, but leading contributions; they will define standards and serve as evaluators, judging the contributions of the devices and guiding the overall thinking.

Moreover, when such situations do arise, humans will handle extremely low-probability situations. (In current human-machine systems, this is one of the most important functions of the operator. The sum of the probabilities of extremely low-probability alternatives is often too large to ignore.) When computers lack applicable patterns or programs for specific environments, humans will fill in the gaps in problem-solving or computer programming.

The information processing devices themselves will convert hypotheses into testable models and then test the models based on data (the operator can roughly specify this data and determine its relevance when the computer submits it for his approval). These devices will answer questions. They will simulate mechanisms and models, execute programs, and display results to the operator. They will transform data, draw charts (in any way "cutting the cake" as specified by the human operator, or presenting several alternatives if the human operator is uncertain about what he wants). The devices will insert, infer, and transform. They will convert static equations or logical statements into dynamic models for the operator to examine their behavior. In general, they will perform routine clerical work to fill the gaps between decisions.

Furthermore, as long as there is sufficient foundational support for formal statistical analysis, computers will serve as machines for statistical inference, decision theory, or game theory, conducting preliminary evaluations of proposed action plans. Finally, they will perform as many diagnostics, pattern matching, and correlation identifications as possible, but in these areas, they will accept a clearly subordinate position.

5 Prerequisites for Achieving Human-Computer Symbiosis

In the previous section, it was assumed that data processing devices were unavailable. Computer programs had not yet been written. In fact, there are several obstacles between the current non-symbiotic state and the anticipated symbiotic future. Let us examine some of these obstacles to gain a clearer understanding of what is needed and the feasibility of achieving this goal.

5.1 Mismatched Speeds Between Humans and Computers

Current large computers are too fast and costly for real-time collaborative thinking with a human. Clearly, for efficiency and economy, computers must allocate time among many users. Time-sharing systems are currently under active development. There are even arrangements to prevent users from "disrupting" anything other than their personal programs.

In a period of 10 or 15 years, envisioning a "thinking center" seems reasonable, combining the functions of today's libraries with the anticipated advancements in information storage and retrieval and the symbiotic functions suggested earlier in this paper. This vision can easily scale into a network of such centers interconnected through broadband communication lines and linked to individual users via leased line services. In such a system, the speed of computers would be balanced, and the costs of massive storage and complex programs would be divided by the number of users.

5.2 Hardware Requirements for Memory

When we begin to consider storing any known technical literature in computer memory, we encounter billions of bits of data, which will cost billions of dollars unless significant changes occur.

The first thing to face is that we will not store all technical and scientific papers in computer memory. We may store the most concise parts—the quantitative parts and references—but not everything. Books are one of the most exquisite and humanized components that exist, and in the context of human-computer symbiosis, they will continue to play an important role. (Hopefully, computers will expedite the searching, delivery, and return of books.)

The second point is that a very important part of memory will be permanent: part will be non-erasable memory, and part will be published memory. Computers will be able to write to non-erasable memory once and read it indefinitely, but they will not be able to erase non-erasable memory. (It may also rewrite, marking all 0s as 1s, as if marking on previously written material.) Published memory will be "read-only" memory. It will be introduced into computers that have already been built. Computers will be able to reference it repeatedly but cannot change it. As computers grow larger, these types of memory will become increasingly important. They can be more compact and much cheaper than core, thin-film, or even tape storage. The main engineering problem will involve selecting circuits.

Regarding other aspects of memory requirements, we can expect the continued development of ordinary scientific and commercial computers. Storage elements may become as fast as processing (logic) elements. This development will have revolutionary implications for computer design.

5.3 Requirements for Memory Organization

The concept of human-computer symbiosis implies that information can be retrieved by name and pattern and accessed through programs that are much faster than serial searches. At least half of the memory organization problems seem to exist in the storage process. The rest seem to be contained within the pattern recognition problems in the storage mechanisms or media. A detailed discussion of these issues is beyond the current scope. However, a brief overview of a promising idea, "trie storage," may help illustrate the general nature of anticipated developments.

Trie storage, as termed by its founder Fredkin, is designed to facilitate information retrieval, and its branching storage structure resembles a tree during development. Most common memory systems store the functions of parameters at specified locations. (In a sense, they do not store these parameters at all. In another, more realistic sense, they store all possible parameters within the framework structure of memory.) On the other hand, trie storage systems store functions and parameters. Starting from standard initial registers, parameters are introduced into memory one character at a time. Each parameter register has a cell, and each character has a cell (for example, two for binary form information), with each character cell having a storage space for the address of the next register. This parameter is stored by writing a series of addresses, each of which tells us where to find the next address. At the end of the arguments is a special "end parameter" marker. Following this is the function's instruction, which is stored in one way or another, with further trie structures or "list structures" typically being the most efficient.

Trie storage schemes are inefficient for small memories, but as memory size increases, they become increasingly efficient in utilizing available storage space. The appealing characteristics of this scheme are: 1) the retrieval process is extremely simple. Given a parameter, input the first character into the standard initial register and extract the address of the second character. Then go to the second register, obtain the address of the third register, and so on. 2) If two parameters share the same initial character, they use the same storage space for those characters. 3) The lengths of parameters do not have to be the same and do not need to be specified in advance. 4) No parameter will be retained or use storage space before actual storage. The trie structure is created when items are introduced into memory. 5) A function can serve as a parameter for another function, which can serve as a parameter for the next function. Thus, for example, by inputting the parameter "matrix multiplication," one can retrieve the entire program for performing matrix multiplication on a computer. 6) By examining the storage at a given level, one can determine which similar items have been stored thus far. For example, if there is no reference to Egan, J. P., it would only take a step or two to find Egan James's trace...

The properties just described do not encompass all desired attributes, but they resonate with human operators, and they tend to specify things through naming or pointing.

5.4 Language Issues

The fundamental differences between human language and computer language may be the most serious obstacle to true symbiosis. However, it is reassuring that significant progress has been made in adapting computers to human language forms through interpretive programs, especially through assembly or compiled programs like FORTRAN. Shaw, Newell, Simon, and Ellis's "Information Processing Language" represents another form of reconciliation. Moreover, in ALGOL and related systems, flexibility has been demonstrated by adopting standard formulas that can be easily translated into machine language.

However, to achieve real-time collaboration between humans and computers, it is necessary to utilize a rather different set of communication and control principles. This idea can be highlighted by comparing instructions typically directed at intelligent humans with those usually given to computers. The latter precisely specify the individual steps to be taken and the order in which these steps should be executed. The former propose or imply something about incentives or motivations, providing a standard by which the executor of the instructions will know when to complete the task. In short: instructions for computers specify routes; instructions for humans specify goals.

Humans seem to think more naturally and easily about goals than about routes. Indeed, they often know something about the travel or work routes, but few can start from a precisely formulated itinerary. For example, who would set out from Boston to Los Angeles with detailed route instructions? Instead, in Wiener’s words, the person heading to Los Angeles tries to continuously reduce the degree to which they are not shrouded in smoke.

There are two avenues to realize computer instructions. The first involves problem-solving, hill-climbing algorithms, and self-organizing projects. The second involves real-time chaining of pre-programmed segments and closed subroutines, which the operator can simply specify and call by name.

Along the first avenue, promising exploratory work has already been done. It is clear that working under the loose constraints of a predetermined strategy, computers will be able to design and simplify their programs at appropriate times to achieve established goals. So far, these achievements are not significant; they are merely "demonstrations in principle." However, their implications are profound.

Although the second avenue is simpler and evidently realizable sooner, it has been relatively overlooked. Fredkin's trie storage provides a promising example. We may soon see a computer program being seriously developed that can connect like words and phrases in a language, enabling any computation or control. Clearly, the considerations hindering this effort are that such an endeavor would not yield anything of significant value in the existing computer environment. Developing a language is undesirable before any computer can respond meaningfully to it.

5.5 Input and Output Devices

In terms of the requirements for human-computer symbiosis, the least advanced data processing sector seems to be that dealing with input and output devices, or from the operator's perspective, dealing with displays and controls. After stating this, it is necessary to make qualifying comments, as the engineering of devices for high-speed input and extraction of information has been excellent, and some very sophisticated display and control technologies have been developed in research laboratories such as Lincoln Laboratory. However, overall, in generally available computers, there is hardly anything more effective or immediate for human-computer communication than an electric typewriter.

Displays seem to be in a better state than controls. Many computers draw graphics on oscilloscope screens, and a few utilize the superior graphics and symbol capabilities of character display tubes. However, to my knowledge, in technical discussions, nothing comes close to the flexibility and convenience of a pencil and doodle pad, or the chalk and blackboard used by people.

1. Desktop Display and Control: Certainly, for effective human-computer interaction, both humans and computers need to draw graphics and pictures on the same display surface and write annotations and equations on the same display surface. This person should be able to show a function to the computer by drawing a chart in a rough but quick manner. The computer should read the person's writing, perhaps under the condition of clear uppercase letters, and should immediately post corresponding characters at each hand-drawn symbol's position, translating them into precise fonts. With such input-output devices, operators will quickly learn to write or print in a machine-readable way. They can write instructions and subroutines, format them appropriately, and check them before finally introducing them into the computer's main memory. They can even define new symbols, as Gilmore and Savell did at Lincoln Laboratory, and present them directly to the computer. They can roughly sketch the format of a table and then let the computer shape it precisely. They can correct the computer's data, guide the machine through flowcharts, and interact just as they would with other engineers, except that the "other engineers" will be precise draftsmen, rapid calculators, mnemonic guides, and many other valuable partners.

2. Computer-Published Wall Displays: In some technical systems, several people share responsibility for controlling vehicles whose behaviors affect each other. Some information must be presented simultaneously to everyone, preferably on a common grid, to coordinate their actions. Other information is relevant only to one or two operators. If all information is presented on one display to everyone, it will only create an incomprehensible mess. This information must be published by the computer, as manual drawing is too slow to keep up to date.

The issues outlined above are now even a critical issue, and over time, they seem certain to become increasingly critical. Some designers believe that displays with the desired characteristics can be constructed based on the principle of light valves, using pulsed light sources and time-sharing viewing screens.

Most who have thought about this believe that large displays should be supplemented by separate display control units. The latter would allow operators to modify wall displays without leaving their positions. For certain purposes, it is hoped that operators can communicate with the computer through auxiliary displays or even wall displays. At least one proposal for providing this communication seems feasible.

Of course, large wall displays and their associated systems are related to the symbiotic cooperation between computers and a group of people. Laboratory experiments have repeatedly shown that informal parallel arrangements among operators, coordinating their activities by referencing large location displays, have significant advantages over more widely used arrangements that position operators at various consoles and attempt to associate their actions through computer agents. This is one of several operational team issues that require careful study.

3. Automatic Speech Generation and Recognition: How ideal and feasible is voice communication between human operators and computers? This complex question arises whenever complex data processing systems are discussed. Engineers who work and live with computers are conservative about this desire. Engineers experienced in automatic speech recognition are cautious about its feasibility. However, there remains interest in the idea of conversing with computers. To a large extent, this interest stems from the recognition that it is difficult to pull a military commander or a corporate president away from their work to teach them typing. If computers could be used directly by high-level decision-makers, providing communication in the most natural way might be worthwhile, even at considerable cost.

Preliminary analysis of the questions and time scales for corporate presidents suggests that they are only interested in a symbiotic relationship with computers as a hobby. Business situations often progress slowly enough to allow time for briefings and meetings. Therefore, for computer experts, it seems reasonable to interact directly with computers in business offices.

On the other hand, military commanders are more likely to make critical decisions in a short time frame. It is easy to exaggerate the concept of a 10-minute war, but it is dangerous to expect more than ten minutes to make critical decisions. Therefore, as the capabilities and complexities of military systems' ground environments and control centers grow, the genuine need for automatic speech generation and recognition by computers seems likely to develop. Of course, if devices have already been developed, are reliable, and are available, they will be used.

In terms of feasibility, the technical issues posed by speech generation are less severe than those posed by automatic speech recognition. A commercial electronic digital voltmeter now loudly reads its indications one digit at a time. Over the past eight or ten years, Bell Telephone Laboratories, the Royal Institute of Technology (Stockholm), Signals Research and Development Establishment (Christchurch), Yale University's Hanskin Laboratory, and MIT, along with Dunn, Fant, Lawrence, Cooper, Stevens, and their colleagues, have demonstrated generation after generation of comprehensible automatic generators. Research at the Hanskin Laboratory has developed a digital code suitable for computer use, enabling automatic speech to be fully understandable in relevant discourse.

The feasibility of automatic speech recognition largely depends on the vocabulary of words to be recognized and the diversity of speakers and accents. A few years ago, at Bell Telephone Laboratories and Lincoln Laboratory, it was demonstrated that 98% correct recognition of natural decimal digits was achievable. To further expand the vocabulary, it can be said that it is now almost certain that an automatic recognizer for clearly pronounced alphanumeric characters can be developed based on existing knowledge. Since the speed at which untrained operators read is at least as fast as the speed at which trained operators type, such a device could be used in almost any computer installation.

However, for real-time interaction at a truly symbiotic level, a vocabulary of about 2000 words may be necessary, such as 1000 basic English words and 1000 technical terms. This is a challenging problem. There is a consensus among acoustic experts and linguists that establishing a recognizer for 2000 words cannot yet be accomplished. However, several organizations are willing to commit to developing an automatic recognition system for such a vocabulary within five years. They would stipulate that speech must be clear, with a dictation style and no unusual accents.

Although a detailed discussion of automatic speech recognition technology is beyond the current scope, it is worth noting that computers play a dominant role in the development of automatic speech recognizers. They provide the impetus for current optimism, or rather, some people's current optimism. Two or three years ago, it seemed that automatic recognition of a large vocabulary would not be achievable within 10 or 15 years; it would have to wait for the gradual accumulation of knowledge about acoustic, speech, language, and psychological processes in speech communication. However, now, many see the prospect of accelerating the acquisition of this knowledge through computer processing of speech signals, and many workers believe that complex computer programs can perform excellently in speech pattern recognition even without substantial knowledge of the underlying speech signals and processes. Combining these two considerations, the estimated time required to achieve practical speech recognition can be reduced to five years, the same five years just mentioned.