Toward Language for Intelligent Machines

Introduction This section presents a brief summary and outline of core concepts, including boundaries (what is not proposed) and some terminology. Descriptions of the main sections are collected in Section 2.5.
 A brief summary Natural language (NL) is a powerful medium for expressing human knowledge, preferences, intentions, and more, yet NL words and syntax appear impoverished when compared to the representation mechanisms (vector embeddings, directed graphs) available in modern neural ML. Taking NL as a point of departure, we can seek to develop representation systems that are strictly more expressive than natural language. The approach proposed here combines graphs and embeddings to support quasilinguistic neural representations (QNRs) shaped by architectural inductive biases and learned through multitask training. Graphs can strongly generalize NL syntactic structures, while lexical-level embeddings can strongly generalize NL vocabularies. QNR frameworks can syntactically embed and wrap non-linguistic objects (images, data sets, etc.) and formal symbolic representations (source code, mathematical proofs, etc.). Through access to external repositories (Figure  1 .1), inference systems can draw on corpora with content that spans scales that range from phrases and documents to scientific literatures and beyond. Embeddings can abstract QNR content to enable semantic associative memory at scale. Neural networks, potentially exploiting (soft) lattice operations, can process retrieved QNR content to recognize analogies, complete patterns, merge compatible descriptions, identify clashes, answer questions, and integrate information from both task inputs and repositories. "NL + " refers to aspirational QNR systems that outperform natural language as a medium for semantic expression and processing. The NL + vision aligns with and extends current research directions in NLP, and NL + implementations could build on current neural architectures and training methods. Potential applications are diverse, ranging from familiar NL-to-NL functionality (interactive search, question answering, writing, translating) to novel forms of representation and reasoning in science, engineering, software development, and mathematics. Potential advantages in scalability, interpretability, cost, and epistemic quality position QNR-based systems to complement or displace opaque foundation models  (Bommasani et al. 2021)  at the frontiers of machine learning. To facilitate skimming, brief summaries of the main sections are collected in Section 2.5. Readers who prefer to start in the middle may wish to skip ahead to Section 8: Quasilinguistic Neural Representations.
 Some (Over)simplified Descriptions An oversimplified problem framing: human intelligence : natural language :: machine intelligence : _______? An oversimplified approach: Use architectural inductive bias and representation learning in neural ML systems to upgrade language by replacing word sequences with explicit parse trees and words with embedding vectors. This is an oversimplification because it (wrongly) suggests a close, fine-grained correspondence between natural languages and QNRs. A less oversimplified description: Use architectural inductive bias and representation learning to develop models that generate and process directed graphs (that strongly generalize NL syntax) labeled with vector embeddings (that strongly generalize both NL words and phrases), thereby subsuming and extending both the syntactic structures and lexical-level components of natural languages. The resulting representation systems can surpass natural languages in expressive capacity, compositionality, and computational tractability. Further objectives and approaches: Learn to embed lexical-level vector representations in structured semantic spaces. Use inductive biases and multitask learning to associate meanings with semantic-space regions (rather than points), and exploit approximate lattice operations (soft unification and anti-unification) as mechanisms for knowledge integration, refinement, and generalization. Translate broad knowledge (e.g., from natural language corpora) into large QNR corpora and employ scalable algorithms to access and apply this knowledge to a wide range of tasks. Enable neural ML systems to write and read QNR content to enable learning that is both efficient and interpretable.
 What is Not Proposed Some contrasting negative samples from the space of related concepts can help readers refine their internal representations of the present proposal: Not a formal language. Formal languages supplement natural languages, but have never subsumed their expressive capacity; frameworks proposed here can embed but are not constrained by formal representations. Not a constructed language. Constructed languages  1  have typically sought clarity and comprehensibility, yet sacrificed expressive capacity; frameworks proposed here seek to expand expressive capacity, yet as a consequence, sacrifice full human comprehensibility. Not a system of hand-crafted representations. Products of neural representation learning typically outperform hand-crafted representations; accordingly, frameworks proposed here rely, not on hand-crafted representations, but on representation learning shaped by architectural bias and training tasks.  2  Not a radical departure from current neural ML. Frameworks proposed here are informed by recent developments in neural ML and suggest directions that are aligned with current research.
 Some Concepts and Terms • "NL" refers to natural language in a generic sense. The representational capacity of NL (in this sense) can be thought of as a sum of the representational capacities of human languages. • Representations will be vector-labeled graphs (VLGs); potential arc labels (indicating types, etc.) are not explicitly discussed. • Quasilinguistic neural representations (QNRs, implemented as VLGs) are compositional and language-like: graphs provide upgraded syntactic structure, while embeddings provide upgraded lexical components.  1  • "NL + " refers to proposed 2 QNR-based products of neural representation learning that would subsume and extend the representational capacity of natural languages.  3  • The term "lattice" and the lattice operations of "meet" (here, "unification") and "join" (here, "anti-unification", sometimes termed "generalization") have their usual mathematical meanings; in the present context, however, lattices and lattice operations will typically be approximate, or "soft" (Appendix A1).
 Motivation and Overview Several perspectives converge to suggest that high-level machine intelligence will require literacy that is best developed in a machine-native medium that is more expressive than natural language. This section concludes with an overview of the sections that follow. Because language and machine learning are broad topics intertwined with each other and with a host of disciplines and application fields, it is difficult to neatly disentangle the various "motivations and perspectives" promised by the section title. The discussion that follows (perhaps unavoidably) contains sections with overlapping conceptual content.
 Why Look Beyond Natural Language? Why seek a language-like representational medium that is more expressive and computationally tractable than natural language? The question almost answers itself. But is such a medium possible, what would it be like, how might it be developed and applied? More generally, how might we complete the analogy mentioned above, human intelligence : natural language :: machine intelligence : __________? It seems unlikely that the best answer is "natural language" (again) or "unstructured vector embeddings". Human intelligence and human societies rely on language as a primary medium for communicating and accumulating knowledge, for coordinating activities, and to some substantial extent, for supporting individual cognition. Intellectually competent humans are literate: They can read and can write content worth reading. High-level machine intelligence will surely be able to do the same and have use for that ability. Current AI research is making strong progress in reading and writing natural language as an interface to the human world, yet makes little use of language(-like) representations for communicating and accumulating knowledge within and between machines. The world's accessible information constitutes a vast, multimodal corpus in which natural language serves as both content and connective tissue. General, high-level intelligent systems must be able to use and extend this information, and it is natural to seek a medium for representing knowledge, both translated and new, that is well-adapted and in some sense native to neural machine intelligence. What might a machine-adapted language be like? It would be strange to find that the best languages for neural ML systems lack basic structural features of human language-in particular, syntax and word-like units-yet perhaps equally strange to find that machines able to share expressive vector embeddings will instead employ sequences of tokens that represent mouth noises. The present document proposes a framework for quasilinguistic neural representations (QNRs) that-by construction-could match and exceed the representational capacity of natural language. Both the potential value and general requirements for such systems seem clear enough to motivate and orient further investigation.
 Some Motivating Facts and Hypotheses The motivation for pursuing QNR approaches that are anchored in NL can be grounded both in uncontroversial facts and in contrasting plausible and implausible hypotheses.
 Key motivating facts 1) Natural language is a key element of human cognition and communication. 2) Natural language provides expressive capacity of unique breadth and flexibility. 3) Structured neural representations can be both richer and more MLcompatible  1  than sequences of words. Corresponding (and plausible) motivating hypotheses +1) Quasilinguistic neural representations of some sort will be key elements of human-level machine cognition and communication, and: +2) The abstract features of natural language (syntactic and lexical constructs) can inform the development of QNRs that subsume and extend syntax and words with graphs and embeddings, and: +3) QNRs informed by natural language constructs can be more expressive and computationally tractable than languages that translate or imitate NL-like sequences of word-like tokens. Corresponding (but implausible) demotivating hypotheses The plausibility of the above hypotheses is supported by the implausibility of contrary hypotheses: -1) That language-like representations will be of little use in human-level machine cognition and communication, or: -2) That language-like syntactic and lexical structures can be no better than flat sequences of vector representations, 2 or: -3) That embeddings in combination with language-like syntactic structures can be no more expressive than sequences of word-like tokens.
 General Approach and Goals In brief, the present line of inquiry suggests a framework that would, as already outlined in part: • Replace and generalize discrete, non-differentiable NL words and phrases with semantically rich, differentiable embeddings.  1  • Replace and generalize NL syntax with general graphs (which also have differentiable representations). • Complement flat neural representations with syntactic structure. • Move linguistic content closer to (quasi)cognitive representations. This strategy starts with NL as a point of departure, retaining generality by subsuming and extending NL piecemeal, at the level of understandable elements. The alternative-to attempt to capture the whole of NL functionality in a more formal, theory-based framework-would risk the loss of functionality that we do not fully understand. Beyond these basic features, QNR frameworks can be extended to: • Exploit abstractive embeddings of fine-grained content (Section 8.3.4). • Exploit abstractive embeddings of large-scale contexts (Section 8.3.5). • Support semantic search at scale (Section 9.1.2). • Support semantic normalization, alignment, refinement, and integration (Section 8.4). • Subsume or embed formal and non-linguistic representations (Section 9.2). What do we want from scalable high-end QNR/NL + systems? • To translate (and refine) large NL corpora into more tractable forms  2  • To combine knowledge from multiple sources, making use of recognizable concordance, clashes, and gaps • To provide comprehensive, dynamic, beyond-encyclopedic knowledge for use by machines and humans • To support the growth of knowledge through machine-aided reasoning The latter goals are worth emphasizing: A key motivation for pursuing NL + capabilities is to enable systems to learn from, apply, and extend content that ranges from informal, commonsense knowledge to mathematics and scientific 1. One direction in which language-like representations might diverge from the picture painted here is in the semantic level of embeddings: As discussed in Section 5.3, embeddings can, through representation discovery, subsume the function of syntactic units above the lexical level (e.g., relatively complex phrases and relatively simple sentences). The partial interchangeability of graph and vector representations (Section 7.3) blurs the potential significance of such a shift, however. 2. While also translating among a likely multiplicity of NL + dialects and overlapping task-oriented sublanguages. literatures. While NL + representations have potentially important roles in NL-to-NL processing  (translation, etc.) , this is almost incidental. The primary aim is to represent, not NL, but what NL itself represents, and to do so better and with broader scope. Current language-related machine representations do not provide full NL (much less NL + ) functionality: They range from opaque language models to explicit knowledge graphs and formal languages, but despite their strengths, none can match (much less exceed) human language in power and generality. Systems like these should be seen as complements-not alternatives-to QNR frameworks.  1
 Four Perspectives That Help Situate the NL + Concept Reinforcing the points above, four external perspectives may help to situate the NL + concept with respect to related research topics: • The power and limitations of natural language set a high bar to clear while suggesting directions for developing more powerful systems. • The power and limitations of symbolic systems 2 suggest a need for complementary, less formal representation systems. • The power and limitations of flat neural representations suggest the potential advantages of systems that combine the expressive power of dense vector representations with the compositional structure of NL. • The power and limitations of current NLP tools  3  suggest that current neural ML techniques can both support and benefit from QNR-based mechanisms with NL + applications.
 Section Overviews The topics addressed in this document are broad, many-faceted, and have a large surface area in contact with other disciplines. The topics are difficult to disentangle, but the following overviews provide a sketch of the organization and content of the document.  4  Section 1: Introduction This section presents a brief summary and outline of core concepts, including boundaries (what is not proposed) and some terminology. Section 2: Motivation and Overview. Several perspectives converge to suggest that high-level machine intelligence will require literacy that is best developed in a medium more expressive than natural language. Section 3: Notes on Related Work. Current developments in neural ML provide architectures and training methods that can support QNR-oriented research and development. Prospective advances are linked to work in symbolic and neurosymbolic computation, and to broad trends in deep learning and natural language processing. Section 4: Language, Cognition, and Neural Representations. Using NL as a motivation and point of departure for NL + motivates a review of its roles in communication, cognition, and the growth of human knowledge. Prospects for improving compositionality through QNR representations are key considerations. Section 5: Expressive Constructs in NL and NL + . NL + must subsume the functionality of NL constructs identified by linguists, and the shortcomings of those constructs suggest substantial scope for surpassing NL's expressive capacity. Section 6: Desiderata and Directions for NL + . Prospects for improving expressiveness in NL + representations include mechanisms both like and beyond those found in natural languages. Research directions aimed at realizing these prospects are well-aligned with current directions in neural ML. Section 7: Vector-Labeled Graph Representations. In conjunction with today's deep learning toolkit, vector-labeled graph representations provide powerful, differentiable mechanisms for implementing systems that represent and process structured semantic information. Section 8: Quasilinguistic Neural Representations. Applications of vectorlabeled graphs can generalize NL syntax and upgrade NL words to implement quasilinguistic neural representations that parallel and surpass the expressive capacity of natural language at multiple levels and scales. Section 9: Scaling, Refining, and Extending QNR Corpora. Scalable QNR systems with NL + -level expressive capacity could be used to represent, refine, and integrate both linguistic and non-linguistic content, enabling systems to compile and apply knowledge at internet scale. Section 10: Architectures and Training. Extensions of current neural ML methods can leverage architectural inductive bias and multitask learning to support the training of quasilinguistic neural systems with NL + -level expressive capacity. Section 11: Potential Application Areas. Potential applications of QNR/NL + functionality include and extend applications of natural language. They include human-oriented NLP tasks (translation, question answering, semantic search), but also inter-agent communication and the integration of formal and informal representations to support science, mathematics, automatic programming, and AutoML. Section 12: Aspects of Broader Impact. The breadth of potential applications of QNR-based systems makes it difficult to foresee (much less summarize) their potential impacts. Leading considerations include the potential use and abuse of linguistic capabilities, of agent capabilities, and of knowledge in general. Systems based on QNR representations promise to be relatively transparent and subject to correction. Section 13: Conclusions. Current neural ML capabilities can support the development of systems based on quasilinguistic neural representations, a line of research that promises to advance a range of research goals and applications in NLP and beyond.
 Appendix Overviews Several topics have been separated and placed in appendices. Of these, only the first focuses on topics that can be considered foundational. Appendix A1: Unification and Generalization on Soft Semantic Lattices. QNR representations can support operations that combine, contrast, and generalize information. These operations-soft approximations of unification and anti-unification-can be used to implement continuous relaxations of powerful mechanisms for logical inference.
 Potentially Useful Models and Tools Potentially useful models and tools for quasilinguistic processing (QLP) are coextensive with broad areas of neural ML (in particular, neural NLP), and a range of applicable tools (architectures, training data, training tasks, computational resources. . . ) can be found in current practice.
 Vector-Oriented Representations and Algorithms Flat neural representations-sets and sequences of one or more embedding vectors-are ubiquitous in modern neural ML and play key roles as components of proposed QNR architectures. The most closely related work is in natural language processing. In neural NLP, we find vector representations of words at input and (often) output interfaces; 1 some systems produce embeddings of higher-level entities such as sentences and documents.  2  Semantic structure in vector spaces emerges spontaneously in word embeddings.  3  End-to-end training can produce compatible vector representations of images and text for tasks that include image captioning and visual question answering.  4  Extensions of current NLP representations, architectures, and training methods are natural candidates for analogous roles in QLP. Transformer architectures-successful in tasks as diverse as translation, question answering, theorem proving, object recognition, and graph-based inference  5  -appear to have sufficient generality to support many (perhaps most) aspects of quasilinguistic processing. Transformer architectures have been extended to read external memories, including stores of NL text  6  and vector embeddings.  7  QLP systems could po-tentially be based on broadly similar architectures in which inference systems write and read, not flat embeddings, but QNR content.
 Graph Representations and GNNs Graph structures complement vector representations in proposed QNRs, and applications of graph representations have spurred extensive work in neural ML. Iterative, node-to-node message-passing systems-graph neural networks 1 (GNNs)-are deep, convolutional architectures that have been successful in tasks that range from scene understanding to quantum chemistry and neurosymbolic computing; 2 their functionality may be well-suited to semantic processing on QNRs. Graph-oriented models, often GNNs, are widespread in neural knowledge-representation systems.  3  Similar graphs can be aligned for comparison and processing.  4  Although classic GNNs operate on fixed graphs, both differentiable representations and reinforcement learning have been used to implement generative models in the discrete graph-structure domain  5  (graphs can, for example, be mapped to and from continuous embeddings  6  ). With suitable positional encodings, Transformers can operate not only on sequences, but on trees or general graphs.  7  The rich tool set provided by current graph-oriented neural models seems sufficient to support the development of powerful QNR-based applications.
 Computational Infrastructure Broad applications of NL + call for scaling to large corpora, first training on large NL corpora, then writing and applying QNR corpora that may be larger still. Rough analogies between NLP and QLP tasks suggest that computational costs in both training and applying large-scale systems can be in line with the costs of currently practical systems for language modeling and translation 1. Recently reviewed in J.  Zhou et al. (2020)  and  Wu et al. (2021) . 2. R.  Li et al. (2017) ,  Gilmer et al. (2017) ,  Lamb et al. (2020), and Addanki et al. (2021) . Labeled scene graphs, in particular, exemplify learnable semantic relationships among objects in which both objects and their relationships can best be represented by embeddings (see Figure  8 .1).  Zhang et al. (2019)  and  Ji et al. (2021 ) 4. Heimann et al. (2018 ,  Cao et al. (2019), and Fey et al. (2020)  5.  Yun et al. (2019) , J.  Zhou et al. (2020) ,  Kazi et al. (2020 ) 6. Cai, Zheng, and Chang (2018  and  Pan et al. (2018)  7.  Shiv and Quirk (2019)  and  Chen, Barzilay, and Jaakkola (2019)  (Section 9.1); in particular, algorithms for efficient embedding-based semantic search at scale-a key enabler for exploiting large corpora-have been demonstrated in commercial applications.  1  Accordingly, current computational infrastructure seems adequate for development, training, and potential large-scale deployment of NL + applications. Reductions in computational cost and improvements in algorithmic efficiency continue  (Hernandez and Brown 2020 ).
 Wen
 Symbolic, Neural, and Neurosymbolic AI Classic symbolic and neural approaches to AI provide further context for the proposed line of development, which has connections to combined, neurosymbolic approaches.
 Symbolic AI The early decades of AI centered on symbolic models that have little direct relevance to current neural approaches. Symbolic systems had striking successes, 2 yet produced unimpressive results in learning and perceptual tasks like vision. In NLP, symbolic AI faced persistent difficulties stemming from the interplay of word meanings, syntax, and semantic context, while in a key application-machine translation-statistical methods outperformed classic symbolic AI. The quasilinguistic approach suggested here differs from symbolic AI in two quite general ways: 1. Proposed QNRs and QLP computation are intended to support-not directly implement-mechanisms for inference and control. 2. Proposed QNRs and QLP computation are saturated with neural representations and learning mechanisms. What symbolic AI does have in common with proposed QLP is the use of graph-structured representations (in symbolic AI, typically syntax trees) that are associated with distinct lexical-level components.
 Neural ML In recent years deep learning and neural ML have advanced rapidly in both scope and performance, successfully addressing an astounding range of problems. Because the range of potential neural architectures and tasks is open ended, it would be unwise to draw a line around deep learning and propose limits to its capabilities. The present proposals are within, not beyond, the scope of modern neural ML. That said, one can point to persistent difficulties with the most common neural ML approaches, which is to say, models that employ flat neural representations (vectors, sets of vectors, sequences of vectors) that often scale poorly, lack clear compositionality, and resist interpretation.
 Neurosymbolic AI Developments in neurosymbolic AI are advancing at the intersection between symbolic and neural ML, with approaches that include the adaptation of symbolic algorithms to richer, embedding-based representations.  1  This body of work has multiple points of contact with proposed QNR approaches, but its diversity resists summarization. Appendix A1 explores connections with constraint logic programming and related reasoning mechanisms based on neurosymbolic representations. It is important to distinguish among approaches that can be called "neurosymbolic", yet differ fundamentally. Geoffrey Hinton has remarked that: Some critics of deep learning argue that neural nets cannot deal with compositional hierarchies and that there needs to be a "neurosymbolic" interface which allows neural network front-and back-ends to hand over the higher-level reasoning to a more symbolic system. I believe that our primary mode of reasoning is by using analogies which are made possible by the similarities between learned high-dimensional vectors. . .  (Hinton 2021)  The present proposal differs from those that Hinton criticizes: While both QNRs and conventional symbolic systems employ explicit syntactic structures, compositional hierarchies, and word-like units, QNRs employ highdimensional vectors, not conventional symbol-tokens, in part for the reason Hinton cites. Although higher-level reasoning seems likely to have an algorithmic character, employing conditional branches and dispatch of values to functions, 1 there is good reason to expect that those conditionals and functions will operate on neural representations through neural mechanisms. To structure the objects and operations of reasoning need not impoverish their content.
 Foundation models The term "foundation model" has been has been introduced  (Bommasani et al. 2021)  to describe systems that are "trained on broad data at scale and can be adapted (e.g., fine-tuned) to a wide range of downstream tasks". Today's leading foundation models (e.g., BERT and GPT-3 2 ) are pretrained on extensive corpora of NL next, while others (e.g., CLIP 3 ) are multimodal; all are based on Transformers. Despite their extraordinary range of applications, current foundation models have suffered from opaque representations, opaque inference mechanisms, costly scaling,  4  poor interpretability, and low epistemic quality, with consequences reviewed and explored in depth by  Bommasani et al. (2021) . QNR-oriented architectures could potentially alleviate each of these difficulties by complementing or displacing models based on stand-alone Transformers. Rather than representing knowledge in unstructured, multi-billionparameter models,  5  architectures that represent knowledge in the form of scalable QNR corpora (Section 9) could provide foundation models in which information content is compositional (Section 4.3) and substantially interpretable  (Section 9.3.3) . Questions of epistemic quality could be addressed by QNR-domain reasoning about external information sources (Section 9.5). 1. For a recent example, see  (Fedus, Zoph, and Shazeer 2021). 2. Devlin et al. (2019)  and  Brown et al. (2020 ) 3. Radford et al. (2021  4. Even relatively scalable Transformer architectures  (Beltagy, Peters, and Cohan 2020; Katharopoulos et al. 2020; Zaheer et al. 2020)  attend only to sections of text, not literatures.
 5. In which knowledge representation and inference mechanisms entangle error-prone arithmetic and information retrieval with fluent multilingual translation.
 Language, Cognition, and Neural Representations Using NL as a motivation and point of departure for NL + motivates a review of its roles in communication, cognition, and the growth of human knowledge. Prospects for improving compositionality through QNR representations are key considerations. Humans accumulate and share information through natural language, and language is woven into the fabric of human cognition. The expressive scope of NL, though limited, is unique and vast. If we seek to build artificial systems that match or exceed human cognitive capacity, then pursuing machine-oriented NL-like functionality seems necessary. The present section reviews the power and shortcomings of natural and formal languages, and from this perspective, considers prospects for quasilinguistic constructs more powerful and closer to cognitive representations than NL itself. The expressive capacity of NL has resisted formalization, and despite its familiarity, remains poorly understood. Accordingly, in seeking more powerful representations, the proposed strategy will be to upgrade the expressive capacity of the relatively well understood linguistic components of languagelexical units and means for composing them-and to thereby upgrade the less well understood whole.
 Language, Cognition, and Non-Linguistic Modalities Natural language, human cognition, and the social accumulation of knowledge are deeply intertwined. Prospects for NL + systems parallel the role of NL in supporting both reasoning and the growth of knowledge.
 The Roles and Generality of Language Through biology and culture, natural language evolved to exploit animal vocalizations, an acoustic channel that transmits information between neural systems, constrained by limitations (working memory, processing speed) of the cognitive mechanisms that encode and decode meaning. At a societal level, sequences of symbols-written language-encode and extend speech, while at a neurological level, the semantic structures of language mesh with cognition. Natural language has evolved under pressures toward comprehensive expressive capacity, yet its shortcomings are real and pervasive. We can regard NL as both a benchmark to surpass and as a template for representational architectures.
 Complementary, Non-Linguistic Modalities Not words, really, better than words. Thought symbols in his brain, communicated thought symbols that had shades of meaning words could never have. -Clifford Simak City,  1 1952  The human use of complementary, non-linguistic modalities highlights limitations of NL. It is said that a picture is worth a thousand words, but it would be more true to say that images and language each can express semantic content that the other cannot. Another modality, demonstration of skills (now aided by video), is often complemented by both speech and images. Today, artifacts such as interactive computational models provide further modalities. Like language, other modalities mesh with human cognition down to unconscious depths. Human thought relies not only on language, but on mental images, imagined physical actions, and wordless causal models. NL serves as a kind of glue between non-linguistic modalities-naming, linking, and explaining things; NL + frameworks must and can do likewise. Neural ML shows us that vector embeddings can describe much that words cannot, images and more. Expressive embeddings beyond the scope of human language can serve as integral, in some sense "word-like" parts of quasilinguistic semantic structures, stretching the concept of NL + . The ability to directly reference and wrap a full range of computational objects stretches the concept further.
 How Closely Linked are Language and Cognition? Embeddings resemble biological neural representations more closely than words do: Embeddings and neural state vectors contain far more information than mere token identities, and both are directly compatible with neural(-like) processing. Thus, embedding-based QNRs are closer to cognitive representations than are natural languages, and presumably more compatible with (quasi)cognitive processing.  2  How deep are the connections between language and human cognition? Without placing great weight on introspective access to cognitive processes, and without attempting to resolve theoretical controversies, some aspects of the relationship between language and cognition seem clear: • Language and cognition have co-evolved and have had ample opportunity to shape and exploit one another. • Language is compositional, and the success of neural models that parse scenes into distinct objects and relationships (Figure  8 .1) suggests that compositional models of the world are more fundamental than language itself. 1 • The experience of trying to "put thoughts into words" (and sometimes failing) is good evidence that there are thoughts that are close to-yet not identical with-language; conversely, fluent conversation shows that substantial cognitive content readily translates to and from language. • Externalization of thoughts in language can help to structure personal knowledge, while writing and reading can expand our mental capacities beyond the limits of memory. These observations suggest that the use of linguistically structured yet quasicognitive representations could aid the development of machine intelligence by providing a mechanism that is known to be important to human intelligence. Conversely, prospects for high-level machine intelligence without something like language are speculative at best.
 Cumulative and Structured Knowledge With some loss of nuance, speech can be transcribed as text, and with some loss of high-level structure, 2 text can be mapped back to speech. A central concern of the present document, however, is the growth (and refinement) of accessible knowledge; today, this process relies on the development of text corpora that express (for example) science, engineering, law, and philosophy, together with literatures and histories that describe the human condition. Indeed, in this document, "natural language" tacitly refers to language captured in writing. 1. Applications of compositional neural models include not only language-linked explanation and question answering  (Shi, Zhang, and Li 2019)  but non-linguistic tasks such as predictive modeling of physical systems  (Watters et al. 2017) . However, to the extent that neural ML systems can display competence in compositional tasks without language-like internal representations, this counts against the necessity of language-like representations in cognition. In communication, NL + aims to be more expressive than NL; in connection with cognition, NL + aims to be more directly compatible with (quasi)cognitive processing; on a global scale, NL + aims to be more effective in accumulating and applying general knowledge. The growth of NL + corpora can be cumulative and content can be structured for use. Returning to a cognitive perspective, humans not only read and write language, but also "read and write" long-term memories. NL + content shares characteristics of both language and memory: Like text, NL + content constitutes explicit, shareable information; like long-term memory, NL + -based systems can store (quasi)cognitive representations that are accessed through associative mechanisms.  1
 Compositionality in Language and Cognition Why does the structure of language suit it to so many tasks? Links between language and cognition-their co-evolution, their close coupling in use-are part of the story, but correspondence between compositional structures in language, cognition, and the world is another, perhaps more fundamental consideration. The case for the broad utility of NL + frameworks in AI is based in part on this correspondence:  2  Compositionality in the world speaks in favor of pursuing compositional representations of knowledge, situations, and actions. Likewise, compositionality in cognition (and in particular, deliberate reasoning) speaks in favor of strong roles for compositional representations in supporting quasicognitive processing.
 Degrees of Compositionality Here, a system-linguistic, cognitive, or actual-will be termed "compositional" if it can be usefully (though perhaps imperfectly) understood or modeled as consisting of sets of parts and their relationships.  3  Useful compositionality requires that this understanding be in some sense local, emerging from parts that are not too numerous or too remote  4  from the parts at the focus of attention or analysis. Hard, local compositionality in symbolic systems requires that the meaning of expressions be strictly determined by their components and syntactic structures; 1 in natural language, by contrast, compositionality is typically soft, and locality is a matter of degree. Strengthening the locality of compositionality can make linguistic representations more tractable, and is a potential direction for upgrading from NL.
 Compositionality in Language and the World Compositionality in language mirrors compositionality in the world-though the compositionality of the world as we see it may be conditioned by language and the structure of feasible cognitive processes. Language (and quasilinguistic systems such as mathematical notation and computer code 2 ) can represent compositionality beyond narrow notions of discrete "things" and "events". Phenomena that are distributed in space and time (e.g., electromagnetic waves, atmospheric circulation, the coupled evolution of species and ecosystems) can be decomposed and described in terms of distributed entities (fields, fluids, and populations) and relationships among their components. Entities themselves commonly have attributes such as mass, color, velocity, energy density, that are compositional in other ways. Neural representation learning confirms that compositionality is more than a human mental construct. A range of successful ML models incorporate compositional priors or learn emergent compositional representations.  3  These successes show that compositional approaches to understanding the world are useful in non-human, quasicognitive systems. Representations typically include parts with meanings that are conditioned on context.  4  In language, the meaning of words may depend not only on syntactically local context, but on general considerations such as the level of technicality of discourse, the epistemic confidence of a writer, or the field under discussion ("glass" means one thing in a kitchen, another in optics, and something more general in materials science). In images, the appearance of an object may depend on lighting, style, resolution, and color rendering, and scenes generated from textual descriptions can differ greatly based on contextual attributes like "day" or "city". Contexts themselves (as shown by these very descriptions) can to some extent be represented by NL, yet images generated by neural vision systems can be conditioned on contextual features that are substantially compositional (as shown by structure in latent spaces), and even recognizable by humans, yet not readily described by language. All of these considerations speak to the value of compositional representations in language and beyond. Taken as a whole, these considerations suggest that quasilinguistic representations can describe features of the world that elude language based on strings of words.
 Compositionality in Nonlinguistic Cognition Compositionality in cognition parallels compositionality in language and in the world. When we perceive objects with properties like "relative position", or consider actions like "dropping a brick", these perceptions and cognitive models are compositional in the present sense; they are also fundamentally nonlinguistic yet often expressible in language. Thus, visual thinking  (Giaquinto 2015 ) can be both non-linguistic and compositional; when this thinking is abstracted and expressed in diagrammatic form, the resulting representations typically show distinct parts and relationships.
 Compositionality in Natural Language The concept of language as a compositional system is woven through the preceding sections, but these have spoken of language as if compositionality in language were a clear and uncontroversial concept. It is not. Formal symbolic systems provide a benchmark for full compositionality, and by this standard, natural languages fall far short.  1  Formal concepts of compositionality have difficulty including contextual features like "topic" or "historical era" or "in Oxford", and struggle with contextual modulation of meaning at the level of (for example) sentences rather than words.  1  Neural NLP can (and to be effective, must) incorporate information that is beyond the scope of locally compositional representations of word strings. Transformer-based language models show something like understanding of text in a broad world context, yet the embodiment of that knowledge in their weights is not obviously compositional in its abstract structure, and obviously not compositional in its concrete representation. To say that the meaning of text emerges from a composition of its components-with particular Transformer computational states as one of those components-would stretch the meaning of compositionality to the breaking point.  2  To be meaningful, compositionality must be in some sense local. Compositionality in language can be strengthened by strengthening the localization of contextual information. Quasilinguistic neural representations can contribute to this in two ways: First, by substituting descriptive vector embeddings for ambiguous words and phrases,  3  and second, by incorporating embeddings that locally summarize the meaning of a syntactically broad context (for example, a context on the scale of a book), together with embeddings that locally summarize remote context, for example, the kind referred to in expressions like "when read in the context of. . . "  4  The second role calls for embeddings that are not conventionally "lexical".  5  In brief, the world, cognition, and language are substantially "compositional", and relative to natural language, quasilinguistic neural representations can improve local compositionality. As we will see, improving local compositionality can have a range of advantages. 5 Expressive Constructs in NL and NL + NL + must subsume the functionality of NL constructs identified by linguists, and the shortcomings of those constructs suggest substantial scope for surpassing NL's expressive capacity. The rich expressive constructs found in natural languages provide a benchmark and point of departure for the pursuit of more powerful capabilities. Considering linguistics in the context of neural ML suggests both challenges that must be addressed and challenges that can be set aside in pursuing the promise of NL + . Figure  6 .1 illustrates relationships among some overlapping classes of representation systems, some of which stretch the definition of "linguistic". Appendix A3 explores further aspects of the relationship between natural language and potential QNR/NL + representations, including prospects for condensing, regularizing, and extending the scope of quasilinguistic representations.
 Vocabulary and Structure in Natural Languages What linguistic constructs enable NL to serve human purposes?  1  Those purposes are broad: Natural languages are richer than "knowledge representation languages" 2 or other formal systems to date; language can describe complex things, relationships, and situations, along with goals, actions, abstract argumentation, epistemic uncertainty, moral considerations, and more. The proposed strategy for developing frameworks that subsume and extend NL is to upgrade representational functionality by upgrading both NL components and compositional mechanisms. Crucially, this strategy requires no strong, formal theory of semantics, grammar, syntax, or pragmatics, and hence no coding of formal rules. An approach that (merely) upgrades components and structure sidesteps questions that have generated decades of academic controversy and addresses instead a far more tractable set of problems.
 Grammar and Syntax It is widely agreed that sentences can usefully be parsed into trees 1 defined by grammars, yet there are several competing approaches.  2  In an NL + framework, explicit graphs can accommodate and generalize any choice, hence no choice need be made at the outset; further, because neural models can integrate information across extended syntactic regions, grammar-like choices of local graph structure need not strongly constrain semantic processing. Architectures with a QNR-oriented inductive bias together with neural representation learning on appropriate tasks should yield effective systems with NL + functionality (Section 10). Section 7 explores potential vector/graph representations in greater depth, while Section 8 considers applications of vector embeddings that subsume elements of NL syntactic structure-again, not to propose or predict, but instead to explore the potential of learned NL + representations.
 Words, Modifiers, and Lexical-Level Expressions The role of lexical-level units in NL is subsumed by embeddings in NL + frameworks, and prospects for improving NL + expressiveness depend in part on the potential advantages of representations in continuous, structured spaces. It is easy to argue that embeddings can be as expressive as words (e.g., embeddings can simply designate words), but a deeper understanding of their potential calls for considering words and word-like entities in the context of continuous semantic spaces.
 Words and word-level units When considering NL + frameworks from an NL perspective, a key question will be the extent to which multi-word expressions can be folded into compact, tractable, single-vector representations while gaining rather than losing expressive power. Two heuristics seem reliable: 1. Meanings that some languages express in a single word can be represented by a single vector.  3  2. Sets of word-level units with related meanings correspond to sets of points clustered in a semantic space. In the present context (and adopting an NL perspective), "lexical" (or "wordlevel") units will be restricted to a single noun or verb together with zero or more modifiers (e.g., adjectives or adverbs),  1  and a "simple" noun (or verb) phrase will be one that cannot be decomposed into multiple noun (or verb) phrases.  2  The phrase "a large, gray, sleepy cat" is a simple noun phrase in this sense; "a cat and a dog" is not simple, but conjunctive. As with many features of language, the single-vs.-conjunctive distinction is both meaningful and sometimes unclear: Is "spaghetti and meatballs" a single dish, or a pair of ingredients? Is "bait and switch" a deceptive strategy or a pair of actions? Is the semantic content of "ran, then halted" necessarily compound, or might a language have a verb with an inflected form denoting a past-tense run-then-halt action? Note that nothing in the present discussion hinges on the sharpness of such distinctions. Indeed, the typical flexibility and softness of mappings between meanings and representations speaks against discrete tokens and formal representations and in favor of QNR/NL + systems that can represent a softer, less formal semantics.  3  In natural language, the meaning of "word" is itself blurry, denoting a concept that resists sharp definition. In linguistics, "morphemes" are the smallest meaningful linguistic units, and include not only (some) words, but prefixes, suffixes, stems, and the components of compound words. In morphologically rich languages, words may contain morphemes that denote case or tense distinctions that in other languages would be denoted by words or phrases. Blurriness again speaks in favor of soft representations and against linguistic models that treat "word" as if it denoted a natural kind.
 Content word roles and representations Linguists distinguish "content words" (also termed "open class" words) from "function" (or "closed class") words. The set of content words in a vocabulary 1. This use of "lexical" differs from a standard usage in linguistics, where to be "lexical", a phrase must have a meaning other than what its components might indicate, making the phrase itself a distinct element of a vocabulary. The phrases "on the other hand" and "cat-andmouse game" are lexical in this sense. NLP research recognizes a similar concept, "multi-word expressions"  (Constant et al. 2017) . is large and readily extended; 1 the set of function words (discussed below) is small and slow to change. Content words typically refer to objects, properties, actions, and relationships. They include nouns, verbs, adjectives, and most adverbs, and they typically have more-or-less regular marked or inflected forms. The growth of human knowledge has been accompanied by the growth of content-word vocabularies. From the perspective of NL syntax and semantics, adjectives and adverbs can be viewed as modifying associated nouns and verbs; this relationship motivates the description of the resulting phrases as word-level (or lexical) in the sense discussed above. In exploring the potential expressive capacity of NL + frameworks, will be natural to consider semantic embedding spaces that accommodate (meanings like those of) nouns and verbs together with the lexical-level refinements provided by adjectives, adverbs, markers, and inflections.  2  One can think of content words as representing both distinctions of kind and differences in properties.  3  Numbers, animals, planets, and molecules are of distinct kinds, while magnitude, color, accessibility, and melting point correspond to differences in properties, potentially modeled as continuous variables associated with things of relevant kinds. In embedding spaces, one can think of distinctions of kind as represented chiefly by distances and clustering among vectors, and differences in properties as represented chiefly by displacements along directions that correspond to those properties.  4  Note that this perspective (kinds → clusters; properties → displacements) is primarily conceptual, and need not (and likely should not) correspond to distinct architectural features of QNR/NL + systems.
 Function word roles and representations Function (closed-class) words are diverse. They include coordinating conjunctions  (and, or, but. . . )  conjunctive adverbs  (then, therefore, however. . . ), prepositions (in, of, without. . . ) modal verbs (can, should, might. . . ), determiners (this, that, my. . . ), connectives (and, or, because, despite. . . ) , and more (see Table  A2 .1). While the set of open-class words is huge, the set of closed-class words is small-in English, only 200 or so. The vocabulary of open-class words can readily be extended to include new meanings by example and definition. Closed-class words, by contrast, typically play general or abstract roles, and linguists find that this small set of words is nearly fixed (hence "closed").  1  The still-awkward repurposing of "they" as a gender-neutral third-person singular pronoun illustrates the difficulty of expanding the closed-class vocabulary, even to fill a problematic gap-alternatives such as "ze" have failed to take hold (C. . Function words that in themselves have minimal semantic content can shape the semantics of complex, content-word constructs (such as clauses, sentences, and paragraphs), either as modifiers or by establishing frameworks of grammatical or explanatory relationships. Representations that subsume NL must span semantic spaces that include function words. The closed-class nature of NL function words suggests opportunities for enriching the corresponding semantic domains in NL + . In English, for example, the ambiguity between the inclusive and exclusive meanings of "or" in English suggests that even the most obvious, fundamental-and in human affairs, literally costly-gaps in function-word vocabularies can go unfilled for centuries.  2
 TAM-C modifiers Natural languages employ a range of tense, aspect, modality, and case (TAM-C) modifiers; 1 some are widely shared across languages, others are rare. In some languages, particular TAM-C modifiers may be represented by grammatical markers (a class of function words); in others, by inflections (a class of morphological features). Meanings that in English are conveyed by function words may be conveyed by inflections in other languages.  2  Sets of TAM-C overlap with closed-class adjectives and adverbs, and are similarly resistant to change. Some TAM-C modifiers alter the meaning of lexical-level elements (both words and phrases); others operate at higher semantic levels. They can convey distinctions involving time, space, causality, purpose, evidence, grammatical roles, and more: Note that many of these distinctions are particularly important to situated and cooperating agents. TAM-C modifiers are sufficiently general and important that NLs encode them in compact lexical representations. The role of TAM-C modifiers in NL calls for similar mechanisms in NL + , and-like adjectives and adverbs-TAM-C modifiers are natural candidates for folding into lexical-level vector representations (Section A3.3).  3  5.4 Phrases, Sentences, Documents, and Literatures NL + representations like those anticipated here condense (some) phrase-level meanings into single, hence syntax-free, embeddings. Within an NL + domain, these "phrase-level" meanings are definitional, not merely approximations of the meaning of a hypothetical phrase in NL. As outlined above, meanings of kinds that correspond to simple, lexical-level noun and verb phrases (in NL) are strong candidates for single-embedding representation, while the equivalents of noun and verb conjunctions (in NL) typically are not; nonetheless, equivalents of NL phrases of other kinds (some noun clauses?) could potentially be captured in single embeddings. The boundaries of the useful semantic scope of definitional vector embeddings are presently unclear, yet at some level between a word and a document, definitional embeddings must give way to vector-labeled graph representations.  1  5.5 At the Boundaries of Language: Poetry, Puns, and Song Discussions of potential NL + "expressiveness" come with a caveat: To say that a representation "is more expressive" invites the question "expressive to whom?" The meanings of NL text for human readers depend on potentially human-specific cognition and emotion, but NL + expressions cannot, in general, be read by humans-indeed, systems that "read" NL + (e.g., systems for which "associative memory" means scalable search and "NL + expressions" are situated within a spectrum of QNRs) are apt to be quite unlike humans. What might be called "outward-facing expressions"-descriptions, commands, questions, and so on-represent a kind of semantic content that may be accessible to human minds, but is not specific to them. The arguments above suggest that NL + representations can outperform NL in this role. However, NL text-and utterances-can convey not only outward-facing semantic content, but human-specific affective meaning, as well as NL-saturated associations, allusions, and word play. Puns and poetry translate poorly even between natural languages, and poetry merges into song which merges into pure music, far from what is usually considered semantic expression. For present purposes, these functions of text and utterances will be considered beyond the scope of "language" in the sense relevant to NL + functionality; what can be represented, however, is literal content (NL text, recorded sound) embedded in NL + descriptions of its effects on human minds (which are, after all, parts of the world to be described).  1  In the context of this document, the content (or "meaning") of natural language expressions will be equated with semantic content in the outwardfacing sense. When a poem delivers a punch in the gut, this is not its meaning, but its effect. 6 Desiderata and Directions for NL + Prospects for improving expressiveness in NL + representations include mechanisms both like and beyond those found in natural languages. Research directions aimed at realizing these prospects are well-aligned with current directions in neural ML. Building on the NL-centered considerations discussed above, and looking forward to mechanisms beyond the scope of natural language, this section outlines desiderata for NL + (and more generally, QNR) functionality. Appendix A3 further explores NL + -oriented prospects for QNR frameworks.
 Improve and Extend Fundamental NL Constructs Desiderata for NL + representations include improving and extending fundamental NL constructs:
 Subsume (but do not model) NL: The aim of research is not to model NL, but to model (and extend) its functionality. Exploit deep learning: Use neural ML capabilities to extend NL constructs without hand-crafting representations. Improve compositionality: Embeddings can provide effectively infinite vocabularies and loosen the dependence of meaning on context. Use explicit graph representations: Graphs can compose embeddings without the constraints of NL syntax and ambiguities of coreference. Exploit vector embedding spaces: Relative to words, embeddings can improve both expressive capacity and computational tractability: -Embeddings are natively neural representations that need not be decoded and disambiguated. -Embeddings, unlike words, are differentiable, facilitating end-toend training. -Embeddings provide effectively infinite vocabularies, enriching expressive capacity. Embrace (but do not impose) formal systems: NL + frameworks and formal systems are complementary, not competing, modes of representation. Formal systems can be embedded as sub-languages in NL + , much as mathematical expressions are embedded in textbooks (Section 9.2.4).
 Exploit Mechanisms Beyond the Scope of Conventional NL The discussion above has focused on features of NL + frameworks that can be regarded as upgrades of features of NL, replacing words with embeddings and implicit syntactic trees with explicit, general graphs. There are also opportunities to exploit representations beyond the scope of NL: These include vector representations that enable novel, high-level forms of expression, abstraction, and semantic search, as well as QNR-based tools for knowledge integration at scale syntactically embedded non-linguistic content far beyond the bounds of what can be construed as language (Section 9.2).
 Use Embeddings to Modify and Abstract Expressions Embeddings can perform semantic roles at the level of sentences, paragraphs, and beyond. In an adjective-like role Section 8.3.1), they can modify or refine the meaning of complex expressions; in an abstractive role, they can enable efficient, shallow processing (skimming) (Section 8.3.4).
 Use Embeddings to Support Scalable Semantic Search Abstractive embeddings can support similarity-based semantic search-in effect, associative memory-over NL + corpora. Efficient similarity search scales to repositories indexed by billions of embeddings (Section 9.1.5).
 Reference, Embed, and Wrap Everything At a syntactic level, NL + frameworks can embed not only formal systems, but also content of other kinds (Figure  6 .1): Non-linguistic lexical-level units: Neural embeddings can represent objects that differ from words, yet can play a similar role. For example, image embeddings can act as "nouns", while vector displacements in a latent space can act as "adjectives" (Section 8.2 NL + -enabled systems could contribute to current research objectives in mathematics, science, engineering, robotics, and machine learning itself, both by helping to integrate and mobilize existing knowledge (e.g., mining literatures to identify capabilities and opportunities), and by facilitating research that produces new knowledge. Efforts to harness and extend the power of natural language align with aspirations for advanced machine learning and artificial intelligence in general.
 Some Caveats The present discussion describes general frameworks, mechanisms, and goals, but the proposed research directions are subject to a range of potential (and equally general) criticisms: • Proposed NL + frameworks are templated on NL, but perhaps too closely to provide fundamentally new capabilities. • Alternatively, proposed NL + frameworks may differ too greatly from NL, undercutting the feasibility of equaling NL capabilities. • In light of the surprising power of flat neural representations, inductive biases that favor QNRs might impede rather than improve performance. • Both neural ML and human cognition embrace domains that are decidedly non-linguistic, limiting the scope of NL-related mechanisms. • Ambitious NL + applications may call for more semantic structure than can readily be learned. • Implementation challenges may place ambitious NL + applications beyond practical reach. • Current research may naturally solve the key problems with no need to consider long-term goals. • The prospects as described are too general to be useful in guiding research. • The prospects as described are too specific to be descriptive of likely developments. Most of these criticisms are best regarded as cautions: Linguistic mechanisms have limited scope; relaxing connections to NL may improve or impede various forms of functionality; implementations of working systems can be difficult to develop or fall short of their initial promise; motivations and outlines of research directions are inherently general and open-ended; generic, short-term motivations often suffice to guide developments up a gradient that leads to capabilities with far-reaching applications. Nonetheless, despite these caveats, near-term research choices informed by QNR/NL + concepts seem likely to be more fruitful than not, leading to tools and insights that enable and inform further research. Much current research is already well-aligned with QNR development and NL + aspirations, and it is interesting to consider where that research may lead.
 Vector-Labeled Graph Representations In conjunction with today's deep learning toolkit, vector-labeled graph representations provide powerful, differentiable mechanisms for implementing systems that represent and process structured semantic information. This present section examines vector-labeled graph representations (VLGs) from the perspectives of representational capacity and neural ML tools; the following section will examine prospects for applying this representational capacity to implement QNR systems that surpass the expressive capacity of natural language. 1
 Exploiting the Power of Vector Representations In a sense, the large representational capacity of typical high-dimensional embedding vectors is trivial: Vectors containing hundred or thousands of floating point numbers contain enough bits to encode lengthy texts as character strings. What matters here, however, is the representational capacity of vectors in the context of neural ML-the scope and quality of representations that can be discovered and used by neural models that are shaped by suitable architectural biases, loss functions, and training tasks. This qualitative kind of capacity is difficult to quantify, but examples from current practice are informative.
 Vector Representations are Pervasive in Deep Learning Deep learning today is overwhelmingly oriented toward processing continuous vector representations, hence the extraordinary capabilities of deep learning testify to their expressive power.
 Vector Representations Can Encode Linguistic Semantic Content Continuous vector representations in NLP shed light on prospects for expressive, tractable QNRs.  2  . The two leading roles for vector embeddings in proposed QNR systems are (1) definitional representations of lexical-level components (paralleling vector semantics in NL and word embeddings in NLP, Section 8.2) and (  2 ) abstractive representations of higher-level constructs for indexing and summarization (Section 8.3.4).
 Single Vectors Can Serve as Compositional Representations In conventional symbolic systems, compositionality enables complex meanings to be represented by combinations of components. In vector representations, meanings can be attributed to orthogonal vector components (e.g., representing different properties of something), then those components can be combined by vector addition and recovered by projection onto their corresponding axes. Condensing what in NL would be multi-component, lexicallevel syntactic structures into single embeddings can reduce the number of distinct representational elements, retain semantic compositionality, and enable facile manipulation by neural computation.
 High-Dimensional Spaces Contain Many Well-Separated Vectors In considering the representational capacity of high-dimensional vectors, it is important to recognize ways in which their geometric properties differ from those of vectors in the low-dimensional spaces of common human experience. In particular, some measures of "size" are exponential in dimensionality, and are relevant to representational capacity. Call a pair of unit-length vectors with cosine similarity ≤ 0.5 "well separated". Each of these vectors defines and marks the center of a set (or "cluster") of vectors with cosine similarity ≥ 0.86; these sets are linearly separable and do not overlap. How many such well-separated cluster-centers can be found in a high-dimensional space? In a given dimension d, the number of vectors k(d) that are well separated by this criterion is the "kissing number", the maximal number of non-overlapping spheres that can be placed in contact with a central sphere of equal radius  (Figure 7.1) . Kissing numbers in low-dimensional spaces are small (k(2) = 6, k(3) = 12. . . ), but grow rapidly with d. For d = 64 and 128, 1 k(d) > 10 7 and 10 12 ; for d = 256, 512, and 1024, an asymptotic lower bound  (Edel, Rains, and    Sloane 2002) k(d) > 2 0   .2075...d  gives k(d) >10 14 , 10 30 , and 10 62 . Thus, the number of neighboring yet well-separated cluster-centers that can be embedded in spaces of dimensionalities commonly used in neural ML is far (!) in excess of any possible NL vocabulary.  2  Note that the region around a cluster-center itself has great representation power for sub-clusters: For example, its content can be separated from the rest of the space by a linear threshold operation and then scaled and projected into a space of d-1 dimensions, where similar considerations apply recursively so long as the residual dimensionality remains large.  1  The above description is intended to provide an intuition for some aspects of the expressive capacity of high-dimensional vector spaces, not to predict or suggest how that capacity will or should be used in prospective systems: Learned representations in current neural ML may offer a better guide.
 Points Can Represent Regions in Lower-Dimensional Spaces https://www.overleaf.com/project/61183c4a1765f22abf2e3ebf An embedding of dimensionality 2d can represent (for example) a center-point in a d-dimensional semantic space together with parameters that specify a surrounding box in that space.  2  An embedding may then designate, not a specific meaning, but a range of potential meanings; alternatively, a range can be regarded as a specific meaning in a semantic space that explicitly represents ambiguity. Interval arithmetic 3 generalizes some operations on d-dimensional points to operations on d-dimensional boxes. This document will usually discuss embeddings as if they represent points in semantic spaces, with operations on embeddings described as if acting on vectors that designate points, rather than regions. The concept of semantic regions becomes central, however, in considering semantic lattice structure and constraint-based inference that generalizes logic programming (Section A1.4).
 Exploiting the Power of Graph-Structured Representations Graphs are ubiquitous as representations of compositional structure because they can directly represent things and their relationships as nodes and arcs. Graphs (in particular, trees and DAGs) are central to traditional, word-oriented natural language processing, while general graphs have found growing applications in diverse areas of neural ML. This section outlines several classes of graphs and their potential roles in representing and processing semantic information.
 Terminology, Kinds, and Roles of Graphs The components of graphs are variously (and synonymously) called edges, arcs, or links (potentially directed), which connect vertices, points, or nodes, which in turn may carry labels, attributes, or contents. The present discussion will typically refer to arcs (or links between document-scale objects) that connect nodes that carry attributes or labels (in a semantic context, contents or embeddings).  1  Here, "graph" typically denotes a directed graph with attribute-bearing nodes. Labeled arcs, multigraphs, and hypergraphs 2 are potentially useful but not explicitly discussed; weighted arcs are essential in some differentiable graph representations. (Sequences of embeddings can be viewed as instances of a particularly simple class of graphs.) In prospective QNR frameworks, vector-labeled graphs have at least two areas of application: The first area parallels the use of graphs in classic NLP, where expressions are typically parsed and represented as syntax trees or (to 1. In some formal models, arcs also carry attributes. Without loss of generality, graphs G with labeled arcs can be represented by bipartite graphs G in which labeled arcs in G correspond to labeled nodes in G . For the sake of simplicity (and despite their potential importance) the present discussion does not explicitly consider labeled arcs. In general, computational representations of graphs will be implementation-dependent and will change depending on computational context (e.g., soft, internal representations in Transformers translated to and from hard, external representations in expression-stores). represent resolved coreferences) DAGs; VLGs can represent syntactic trees explicitly, bypassing parsing, and can represent coreference through DAGs, bypassing resolution. The second area of application involves higher-level semantic relationships that in NL might be represented by citations; in an NL + context, similar relationships are naturally represented as general, potentially cyclic graphs. (These topics are discussed in Section 8.1.)
 VLGs Can Provide Capacity Beyond Stand-Alone Embeddings Fixed-length embeddings lack scalability, while sequences of embeddings (e.g., outputs of Transformers and recurrent networks), though potentially scalable, lack explicit, composable, and readily manipulated structure. Arcs, by contrast, can compose graphs to form larger graphs explicitly and recursively with no inherent limit to scale or complexity, and subgraph content can be referenced and reused in multiple contexts. Trees and graphs are standard representations for compositional structure in a host of domains, and are latent in natural language. Graphs with vector attributes can expand the representational capacity of sets of embeddings by placing them in a scalable, compositional framework. 1
 VLGs Can Be Differentiable Typical neural operations on vector attributes are trivially differentiable, while differentiable operations on representations of graph topologies require special attention. Without employing differentiable representations, options for seeking graphs that minimize loss functions include search (potentially employing heuristics or reinforcement learning) and one-shot algorithmic construction.  2  With differentiable representations, more options become available, including structure discovery through end-to-end training or inferencetime optimization. Conventional representations in which nodes and arcs are simply present or absent can be termed "hard graphs"; representations can be made "soft" and differentiable by assigning weights in the range [0, 1] to arcs. Differentiable algorithms that assign weights tending toward {0, 1} can recover conventional graphs by discarding edges when their weights approach zero, implementing structure discovery through differentiable pruning. Differentiable pruning operates on a fixed set of nodes and typically considers all pairs, impairing scalability, 1 but algorithms that exploit near-neighbor retrieval operations on what may be very large sets of nodes (Section 9.1.5) could implement scalable, differentiable, semantically informed alternatives that do not a priori restrict potential topologies. In typical graph representations, a link is implemented by a semantically meaningless value (e.g., an array index or hash-table key) that designates a target node. Through near-neighbor retrieval, by contrast, a vector associated with a source-node can serve as a query into a semantically meaningful space populated by keys that correspond to candidate target-nodes. Selecting the unique node associated with the nearest-neighbor key yields a hard graph; attending to distance-weighted sets of near neighbors yields a soft graph.  2  In this approach, query and key embeddings can move through their joint embedding space during training, smoothly changing neighbor distances and the corresponding arc weights in response to gradients. Mutable, soft-graph behavior can be retained at inference time, or models can output conventional, hard-graph VLGs, potentially retaining geometric information. Thus, models that build and update QNR corpora could provide fluid representations in which changes in embeddings also change implied connectivity, implicitly adding and deleting (weighted) arcs. In semantically structured embedding spaces, the resulting changes in topology will be semantically informed. Weighted graphs may also be products of a computation rather than intermediates in computing hard-graph representations. Substituting a set of weighted arcs for a single hard arc could represent either structural uncertainty (potentially resolved by further information) or intentional, semantically informative ambiguity.
 VLGs Can Support Alignment and Inference Neural algorithms over VLGs can support both processing within single expressions and operations that link or combine multiple expressions. These have potentially important applications within QNR frameworks. Algorithms that align similar graphs 1 can facilitate knowledge integration over large corpora, supporting the identification of clashes, concordances, and overlaps between expressions, and the construction of refinements, generalizations, analogies, pattern completions, and merged expressions, 2 as well as more general forms of interpretation and reasoning. Differentiable algorithms for graph alignment include continuous relaxations of optimal assignment algorithms that enable end-to-end learning of alignable, semantically meaningful embeddings  (Sarlin et al. 2020) .
 VLGs Can Support (Soft) Unification and Generalization If we think of expressions as designating regions of some sort 3 , then to unify a pair of expressions is to find the largest expressible intersection of their regions, while to anti-unify (or generalize) a pair of expressions is to find the narrowest expressible union of their regions. Domains that support these operations (and satisfy a further set of axioms) constitute mathematical lattices.  4  Given a set of expressions, unification may be used to infer narrower, more specific expressions, while anti-unification operations may be used to propose broader, more general expressions. As discussed above, QNR expressions that represent explicit uncertainty or ambiguity (e.g., containing vectors representing uncertain or partially constrained values) may be regarded as representing regions in a semantic space. The nature of "expressible regions" in the above sense depends on choices among representations. Notions of "soft" or "weak" unification (discussed in Section A1.4.3) can replace equality of symbols with similarity between point-like semantic embeddings, or approximate overlap when embeddings are interpreted as representing semantic regions. Soft unification supports continuous, differentiable relaxations of the Prolog backward-chaining algorithm, enabling soft forms of multi-step logical inference. Applications of soft unification include question answering, natural-language reasoning, and theorem proving; 1 Soft operations can also infer variables from sets of values (Cingillioglu and Russo 2020). As noted above, unification and generalization have further potential applications to knowledge integration in NL + corpora.
 Mapping Between Graphs and Vector Spaces Research has yielded a range of techniques for mapping graphs to and from vector representations. These techniques are important because they bridge the gap between high-capacity compositional structures and individual embeddings that lend themselves to direct manipulation by conventional (non-GNN) neural networks.
 Embeddings Can Represent and Decode to Graphs Neural models can be trained to encode graphs 2 (including tree-structured expressions 3 ) as embeddings, and to decode embeddings to graphs.  4  A common training strategy combines graph-encoding and generative models with an autoencoding objective function.  5  In the domain of small graphs, embeddings could encode representations with considerable accuracy-potentially with definitional accuracy, for graphs that are small in both topology and information content. Note that summary, query, and key embeddings (Section 8.3.4 and Section 9.1.2) can represent semantic content without supporting graph reconstruction.
 Embeddings Can Support Graph Operations Embeddings that enable accurate graph reconstruction provide differentiable graph representations beyond those discussed in Section 7.2.3. Whether directly or through intermediate, decoded representations, graph embeddings can support a full range of VLG operations. Accordingly, this class of embeddings can be considered interchangeable with small 6 VLGs, and need not be considered separately here. Potential advantages include computational efficiency and seamless integration with other embedding-based representations.
 Quasilinguistic Neural Representations Applications of vector-labeled graphs can generalize NL syntax and upgrade NL words to implement quasilinguistic neural representations that parallel and surpass the expressive capacity of natural language at multiple levels and scales. The present section discusses how vector-labeled graphs (VLGs) can be applied to implement quasilinguistic neural representations (QNRs) that improve on natural languages by upgrading expressive capacity, regularizing structure, and improving compositionality to facilitate the compilation, extension, and integration of large knowledge corpora. Appendix A3 covers an overlapping range of topics in more detail and with greater attention to NL as a model for potential NL + frameworks. As usual in this document, conceptual features and distinctions should not be confused with actual features and distinctions that in end-to-end trained systems may be neither sharp, nor explicit, nor (perhaps) even recognizable. Conceptual features and distinctions should be read neither as proposals for hand-crafted structures nor as confident predictions of learned representations. They serve to suggest, not the concrete form, but the potential scope of representational and functional capacity.
 Using Graphs as Frameworks for Quasilinguistic Representation Proposed QNRs are vector-labeled graphs.  1  Attributes can include type information as well as rich semantic content; to simplify exposition,  2  attributes of what are semantically arcs can be regarded as attributes of nodes in a bipartite graph, 3 and will not be treated separately. Like trees, general (e.g., cyclic) graphs can have designated roots.
 Roles for Graphs in NL-Like Syntax As already discussed, vector attributes can represent lexical-level components, while graphs can represent their syntactic compositions; where syntax in NL is implicit and often ambiguous, QNR-graphs can make syntactic relationships (and in particular, coreference) explicit, thereby disentangling these relationships from lexical-level semantic content. (See also Section A3.1.1.)
 Roles for Graphs Beyond Conventional Syntax In an extended sense, the syntax of NL in the wild includes the syntax of (for example) hierarchically structured documents with embedded tables, cross references, and citations.  1  QNRs can naturally express these, as well as syntactic structures that are unlike any that can be displayed in a document.  2  Stretching the concept of syntax, graphs can express networks of relationships among objects. Scene graphs provide a concrete illustration of how recognizably linguistic relationships can naturally be expressed by a nonconventional syntax (for a simple example, see Figure  8 .1).
 Figure 8.1: A neurally inferred scene graph that relates several entities through subject-verb-object relationships. An enriched graph for this scene could represent more complex relationships (e.g., man1 failing to prevent man2 from throwing a Frisbee to man3). An enriched representation of entities could replace instances of labels like "man" with embeddings that have meanings more like man-withappearance(x)-posture(y)-motion(z), replace instances of verbs like "catching" with embeddings that have meanings more like probablywill-catch-intended-pass, and so on.  3
 Using Embeddings to Represent Lexical-Level Structure At a lexical level, embedding-space geometry can contribute to expressive power in several ways: Proximity can encode semantic similarity. Large displacements can encode differences of kind. Small offsets (or larger displacements in distinguishable subspaces) can encode graded, multidimensional semantic differences. Where NL depends on context to disambiguate words, QNRs can employ lexical-level embeddings to express meanings that are substantially disentangled from context. (For further discussion, see Section A3.2.1.)
 Proximity and Similarity Neural representation learning typically places semantically similar entities near one another and unrelated entities far apart. Embedding NL words works well enough to be useful, yet its utility is impaired by polysemy and other ambiguities of natural language.  1  By construction, native embedding-based representations minimize these difficulties.
 Graded Semantic Differences The direction and magnitude of incremental displacements in embedding spaces can represent graded, incremental differences in properties among similar entities: The direction of a displacement axis encodes the kind of difference, the magnitude of the displacement along that axis encodes the magnitude of the corresponding difference, 2 and the (commutative) addition of displacements composes multiple differences without recourse to syntactic constructs. By avoiding difficulties arising from the discrete words and word-sequences of NL, the use of continuous, commutative vector offsets can substantially regularize and disentangle lexical-level semantic representations.
 Relationships Between Entities of Distinct Kinds Distinctions between entities of different kinds 1 can be represented as discrete displacement vectors, which can encode degrees of similarity in distances and cluster things of similar kinds (animals with animals, machines with machines, etc., as seen in the geometry of word embeddings). Displacement directions can also encode information about kinds. Word embeddings trained only on measures of co-occurrence in text have been found to represent relationships between entities in displacement vectors, and analogies in vector differences and sums  (Allen and Hospedales 2019) . In neural knowledge graphs, 2 relationships between entities can be encoded in geometries that are deliberately constructed or induced by learning.  3  Lexicallevel QNR labels can provide similar representational functionality.
 Expressing Higher-Level Structure and Semantics The discussion above addressed the vector semantics of embeddings that play lexical-level roles (e.g., nouns and verbs with modifiers); the present discussion considers roles for embeddings in the semantics of higher-level (supra-lexical) QNR expressions. Some of these roles are definitional: in these roles, embeddings modify the meanings of higher-level constructs.  4  In other roles, embeddings are abstractive: Their semantic content may corresponds to (quasi)cognitive results of reading higher-level constructs, and are therefore semantically optional (in effect, they cache and make available computational results). As usual, the aim here is to describe available representational functionality, not to predict or propose specific representational architectures. Representa-tion learning within a QNR framework need not (and likely will not) respect these conceptual distinctions.
 Expression-Level Modifiers (Definitional) Although lexical-level embeddings can condense (analogues of) phrases that include modifiers of (analogues of) words, some lexical-level modifiers operate on supra-lexical expressions that cannot be condensed in this way. These modifiers may resemble adjectives, but are attached to units more complex than words. Epistemic qualifiers (Section A3.4) provide examples of this functionality. Syntactically, definitional modifiers of this kind could be applied to expressions as vector attributes of their root nodes; semantically, expression-level modifiers act as functions of a single argument: an expression as a whole.
 Expression-Level Relationships (Definitional) In NL, conjunctive elements (often function words) can compose and modify the meanings of combined expressions. Some can operate on either lexicallevel units or complex expressions (examples include and, or, and/or, both-and); others (for example, however, because, and despite 1 ) typically compose meaning across substantial syntactic spans, operating not only at the level of clauses and sentences, but at the level of paragraphs and beyond. Syntactically, a relationship between expressions could be applied through vector attributes of the root node of a composition of syntactic structures (subgraphs of a tree, DAG, etc.); semantically, expression-level relationships correspond to functions of two or more arguments: the expressions they relate.
 Expression-Level Roles (Definitional) Expressions and the things they describe have roles (frequently with graded properties) in larger contexts; features of roles may include purpose, importance, relative time, epistemic support, and so on. As with expression-level modifiers and relationships, role descriptions of this kind could be applied to an expression through vector attributes of its root node. In general, however, a role may be specific to a particular context. If an expression is contained (referenced, used) in multiple contexts, it may have multiple roles, hence representations of those roles must syntactically stand 1. See also: punctuation. above root-nodes and their attributes. More concretely, multiple contexts may link to an expression through nodes that represent the corresponding contextual roles.
 Content Summaries (Abstractive) In human cognition, the act of reading a text yields cognitive representationsin effect, abstractive summaries-that can contribute to understanding directly, or by helping to formulate (what are in effect) queries that directly enable retrieval of relevant long-term memories, or indirectly enable retrieval of relevant texts.  1  In a QNR context, dynamically generated summary embeddings can play a similar, direct role at inference time. When stored, however, summaries can do more: • Amortize reading costs for inference tasks. • Enable efficient skimming. • Stretch the semantic span of attention windows.  2  • Provide keys for scalable associative memory. The nature of a summary may depend on its use, potentially calling for multiple, task-oriented summaries of a single expression.  3  Semantically, content summaries are not expression-level modifiers: Summaries approximate semantics, but modifiers help define semantics. Content summaries may depend on contextual roles, placing them in syntactic positions above the roots of summarized expressions. A natural auxiliary training objective would be for the similarity of pairs of content-summary vectors to predict the unification scores (Section A1.4.3) of the corresponding pairs of expressions.  4
 Context Summaries (Abstractive) Context summaries (e.g., embeddings that summarize context-describing QNRs) represent information that is potentially important to interpreting and using a QNR expression.  1  • What are the general and immediate topics of the context? • Is the context a formal or an informal discussion? • How does the context use the expression? • Does the context support or criticize the content of the expression? At inference time, context summaries could direct attention to relevant commonsense knowledge regarding a particular field or the world in general.  2  Interpreting an expression correctly may require contextual information that is distributed over large spans; providing (quasi)cognitive representations of contexts can make this information more readily available. Like content summaries, context summaries are semantically optional, but the potential scale and complexity of contexts (local, domain, and global) may make some form of context-summarization unavoidable, if extended contexts are to be used at all. Syntactically (and to some extent semantically), context summaries resemble contextual roles.
 Origin Summaries (Abstractive) Origin summaries (potentially summarizing origin-describing QNRs) can indicate how an expression was derived and from what information: • What inputs were used in constructing the expression? • What was their source? • What was their quality? • What inference process was applied? A structurally distinct QNR expression can be viewed as having a single source (its "author"), and hence its root node can be linked to a single origin summary. Origins and their summaries are important to the construction, correction, and updating of large corpora.
 Regularizing, Aligning, and Combining Semantic Representations Disentangling and regularizing semantic representations-a theme that runs through Section 8.1 and Section 8.2-has a range of benefits. In conjunction with basic QNR affordances, regularization can facilitate the alignment of representations, which in turn can facilitate systematic inference, comparison, combination, and related forms of semantic processing at the level of collections of expressions.
 Regularizing Representations In natural languages, expressions having similar meanings may (and often must) take substantially different forms. Words are drawn from discrete, coarse-grained vocabularies, hence incremental differences in meaning force discrete changes in word selection. Further, small differences in meaning are often encoded in structural choices-active vs. passive voice, parallel vs. non-parallel constructs, alternative orderings of lists, clauses, sentences, and so on. These expressive mechanisms routinely entangle semantics with structure in ways that impose incompatible graph structures on expressions that convey similar, seemingly parallel meanings. The nature of QNR expressions invites greater regularization: Meanings of incommensurate kinds may often be encoded in substantially different graph topologies, while expressions with meanings that are similar-or quite different, yet abstractly parallel-can be represented by identical graph topologies in which differences are encoded in the continuous embedding spaces of node attributes. Humans compare meanings after decoding language to form neural representations; natively disentangled, regularized, alignable representations can enable comparisons that are more direct. By facilitating alignment and comparison, regularization can facilitate systematic (and even semi-formal) reasoning.
 Aligning Representations Section 7.2.4 took note of neural algorithms that can align graphs in which parallel subgraphs carry corresponding attributes. Given a pair of aligned graphs, further processing can exploit these explicit correspondences. 1
 Alignment for Pattern-Directed Action By triggering pattern-dependent actions, graph alignment can establish relationships between other entities that do not themselves align. For example, reasoning from facts to actions can often be formulated in terms of production rules 2 that update a representation-state (or in some contexts, cause an external action) when the pattern that defines a rule's precondition matches a pattern in a current situation: Where patterns take the form of graphstructured representations, this (attempted) matching begins with (attempted) graph alignment. Potential and actual applications of production rules range from perceptual interpretation to theorem proving.
 Comparing, Combining and Extending Content Alignment facilitates comparison. When graphs align, natural distance metrics include suitably weighted and scaled sums of distances between pairs of vector attributes.  3  When alignment is partial, subgraphs in one expression may be absent from subgraphs in the other (consistent with compatibility), or subgraphs may overlap and clash. Clashes may indicate mutual irrelevance or conflict, depending on semantic contents and problem contexts. Overlapping graphs can (when compatible) be merged to construct extended, consistent descriptions of some entity or domain. In formal systems, this operation corresponds to unification (Section A1.1.1); in general QNR contexts, formal unification can be supplemented or replaced by soft, learned approximations (Section A1.  4.3) . Expressions that are similar and compatible yet not identical may provide different information about a single object or abstraction; unification mechanisms then can enrich representations by merging information from multiple sources. For example, unification of multiple, related abstractions (e.g., derived from uses of a particular mathematical construct in diverse domains of theory and application) could produce representations with richer semantics than could be derived from any single NL expression or documentrepresentations that capture uses and relationships among abstractions, rather than merely their formal structures. Alternatively, expressions that are similar yet not fully compatible may describe different samples from a single distribution. The dual of unification, anti-unification (a.k.a. generalization, Section A1.1.2), provides the most specific representation that is compatible with a pair of arguments, encompassing their differences (including clashes) and retaining specific information only when it is provided by both. Anti-unification of multiple samples 1 yields generalizations that can (perhaps usefully) inform priors for an underlying distribution. Like unification, generalization yields an expression at the same semantic level as its inputs, but to represent analogies calls for an output that contains representations of both input graphs and their relationships. Alignment can provide a basis for representing analogies as first-class semantic objects that can be integrated into an evolving corpus of QNR content Looking back for a moment, prospects that were motivated by the most basic considerations-upgrading words and syntax to machine-native embeddings and graphs-have naturally led to prospects that go far beyond considerations of expressiveness per se. QNR/NL + frameworks lend themselves to applications that can subsume yet are qualitatively different from those enabled by natural language. 9 Scaling, Refining, and Extending QNR Corpora Scalable QNR systems with NL + -level expressive capacity could be used to represent, refine, and integrate both linguistic and non-linguistic content, enabling systems to compile and apply knowledge at internet scale. Preceding sections discussed vector/graph representations and their potential use in constructing QNR frameworks that are powerful and tractable enough to meet the criteria for NL + . The present section explores the potential scaling of QNR/NL + systems to large corpora, then considers how scalable, fully functional systems could be applied to include and integrate both linguistic, and non-linguistic content while refining and extending that content through judgment, unification, generalization, and reasoning.
 Organizing and Exploiting Content at Scale A core application of language-oriented QNR systems will be to translate and digest large NL corpora into tractable, accessible NL + representations, a task that makes scalability a concern. Current practice suggests that this will be practical. NL → NL + translation will likely employ components that resemble today's translation systems. Potentially analogous NL → NL translation tasks are performed at scale today, while ubiquitous internet-scale search suggests that large corpora of NL + knowledge can be exploited for practical use.
 Learning by Reading Can Be Efficient at Scale To estimate the magnitude of required computational tasks, we can roughly equate the incremental costs of building a corpus by reading NL texts and translating them to NL + representations with the incremental costs of training a language model by reading NL texts and (in some sense) "translating" their content to model parameters. According to this estimate, training GPT-3 was roughly equivalent to translating ~300 billion words 1 to NL + representationsabout 30 times the content of the 1.8 million papers on the arXiv (arXiv 2021) or 10% of the 40 million books scanned by Google (H. . As an alternative and perhaps more realistic estimate, we can roughly equate the cost of NL → NL + translation to the cost of NL → NL translation; the throughput of Google Translate (>3 trillion words per year 1 ) is then an indicator of affordable throughput (~40 million books/month). These comparisons concur in suggesting the feasibility of translating NL information to NL + corpora at scale.  2  Translating NL content into accessible, quasicognitive NL + is a form of learning that is distinct from training. Unlike training model parameters, the task of reading, translating, and expanding corpora is by nature fully parallelizable and scalable. The products are also more transparent: Exploiting external, compositional representations of information can enable facile interpretation, correction, redaction, and update of a system's content. 3
 Semantic Embeddings Enable Semantic Search NL + expressions can be indexed by summary embeddings that associate similar expressions with near-neighbor points in a semantic space.  4  Using these embeddings as keys in near-neighbor retrieval can provide what is in effect "associative memory" that supports not only search, but tasks involving knowledge comparison and integration (see Section 9.4 below). Different embeddings of an expression can serve as keys suited for different search tasks. In some use-contexts, we care about the physical properties of an object; in others, we care about its cost and functionality. In some use-contexts, we care about a city's geography; in others, about its nightlife. Accordingly, 1. Reported in (Turovsky 2016). 2. Contributing to efficiency and scalability, computations that both write and read NL + corpora can, as noted above, avoid problems that stem from representing knowledge solely in trained model parameters. Large language models learn not only knowledge of language per se and general features of the world, but also a range of specific facts about people, places, etc. The striking improvements that have resulted from scaling language models have stemmed, not only from improvements in broadly applicable, immediately available syntactic, semantic, and reasoning skills (K.  Lu et al. 2021) , but from memorization of specific, seldom-used facts about the world (Yian . For example, GPT-2 uses its 1.5 billion parameters to memorize names, telephone numbers, and email addresses, as well as the first 500 digits of π  (Carlini et al. 2021) . Encoding factual information of this sort in billions of model parametersall used in computing each output step-is problematic: Both training and inference are expensive, and the results are opaque and difficult to correct or update. Parameter counts continue to grow  (Brown et al. 2020; Fedus, Zoph, and Shazeer 2021)   4.  Wang and Koopman (2017) ,  Schwenk and Douze (2017), and Tran et al. (2020) . Note that retrieval based on similarity between semantic embeddings can be effective across modalities (e.g.,  Miech et al. 2021) . Pairwise similarity between vector keys could potentially reflect the unification scores (Section A1. what might be represented as a single span of content may naturally be associated with multiple domain-oriented summaries and keys. We find this general pattern in Transformers, where multi-head attention layers project each value to multiple key and query vectors. A quite general distinction is between representing the content of an expression and representing the kinds of questions that its content can answer-between what it says and what it is about. The role of semantic search overlaps with the role of links in large-scale NL + structures (for example, generalizations of citation networks), but differs in designating regions of semantic space through query-embeddings rather than designating specific expressions through graph links. Conventional references and syntactic relationships are represented by links, but looser relationships can be represented by "query embeddings" within expressions, where these embeddings are taken to denote soft graph links to a potential multiplicity of target expressions in an indexed corpus. 1
 Semantic Search Extends to Non-Linguistic Content By proposed construction, any item that can be described by NL can be (better) described by NL + . Examples include code, engineering designs, legal documents, biomedical datasets, and the targets of current recommender systems-images, video, apps, people, products, and so on. Embeddings that represent (descriptions of) non-linguistic content (Section 9.2.2) are relevant to NL + in part because they can be directly referenced by NL + expressions, and in part because these NL + descriptions can provide a basis for semantically rich content embeddings.
 NL + -Mediated Embeddings Could Potentially Improve NL Search Systems trained to map NL to NL + can support the production of NL embeddings (NL → NL + → embedding) that are based on disentangled, contextually informed semantic representations. If so, then co-learned, NL + -mediated key and query embeddings could potentially improve kNN semantic search over corresponding NL items at internet scale.
 Semantic Search Can Be Efficient at Scale Exact k-nearest neighbor search in generic high-dimensional spaces is costly, 1 but approximate nearest neighbor search (which usually finds the keys nearest to a query) have been heavily researched and optimized; efficient, practical, polylogarithmic-time algorithms can support (for example) billion-scale recommender systems, 2 and can do likewise for retrieval of semantic content in NL + systems at scale.
 Incorporating General, Non-Linguistic Content Human knowledge includes extensive nonlinguistic content, yet we use natural language to describe, discuss, index, and provide instructions on how to create and use that content. Language-linked nonlinguistic content sprawls beyond the boundaries of an NL-centric concept of NL + -boundaries that do not constrain language-inspired QNR/NL + applications. Examples of non-linguistic information content include: • Non-linguistic lexical-level units (e.g., image embeddings) • Expressions in formal languages (e.g., mathematical proofs) • Precise descriptions of objects (e.g., hardware designs) • Formal graph-structured representations (e.g., probabilistic models) 9.2.1 Using "Words" Beyond the Scope of Natural Language "Nouns" represent things, but vector embeddings can represent things in ways that are in no sense translations of word-like entities in NL: Image embeddings, for example, can serve as "nouns",  3  though the content of an image embedding need not resemble that of an NL noun phrase. Embeddings that represent datasets or object geometries have a similar status. Similar considerations apply to verbs and relationships: Words and phrases have a limited capacity to describe motions, transformations, similarities, and differences.  1  As with nouns, embeddings (and a wide range of other information objects) can again subsume aspects of NL expressive capacity and extend representations to roles beyond the scope of practical NL descriptions. This expressive scope may often be difficult to describe concretely in NL.  2  QNR "verbs" could express transformations of kinds not compactly expressible in conventional NL: For example, displacements of physical objects can be represented by matrices that quantitatively describe displacements and rotations, and a more general range of transformations can be expressed by displacement vectors in a suitable latent space. Stretching the conceptual framework further, verb-like transformations can be specified by executable functions. 3
 Referencing Non-Linguistic Objects References to linguistic and non-linguistic objects are not sharply demarcated, but some information objects are both complex and opaque, while physical objects are entirely outside the information domain. All these (indeed, essentially anything) can nonetheless be referenced within the syntactic frameworks of QNR expressions. As a motivating case, consider how embedded hyperlinks and more general URIs expand the expressive power of online documents: The ability to unambiguously reference not only text, but arbitrary information objects, is powerful, and on the internet, this capability meshes smoothly with NL. Examples of non-linguistic information objects include images, websites, data repositories, and software. Beyond the domain of information objects, domain-appropriate reference modalities can act as proper nouns in designating physical objects, people, places, and the like. A natural pattern of use would place a reference in a QNR wrapper that might include (for example) a summary embedding, conventional metadata, descriptions, and documentation, all of which can exploit the representational capacity of NL + . QNR wrappers can facilitate indexing, curation, and the selection or rejection of entities for particular uses. Information objects include executable code, and when accessed remotely, executable code is continuous with general software and hardware services. Interactive models 1 can complement NL descriptions of systems. Access to documented instances of the computational models used to produce particular NL + items can contribute to interpreting the items themselves: Connections between products and their sources often should be explicit.
 Embedding Graph-Structured Content Human authors routinely augment sequential NL text with graph structures. Even in basically sequential text (e.g., this document), we find structures that express deep, explicit nesting and internal references. Documents often include diagrams that link text-labeled elements with networks of arrows, or tables that organize text-labeled elements in grids. These diagrams and tables correspond to graphs. Further afield, graph-structured representations can describe component assemblies, transportation systems, and biological networks (metabolic, regulatory, genetic, etc.) Text-like descriptions of statistical and causal relationships become probabilistic models and causal influence diagrams. In ML, we find formal knowledge graphs in which both elements and relationships are represented by vector embeddings, 2 while augmenting language-oriented Transformer-tasks with explicit representations of relationships has proved fruitful.  3  Distinctions between these and NL + representations blur or disappear when we consider generalizations of conventional syntax to graphs, and of symbols and text to embeddings and general data objects. Some potential use-patterns can be conceptualized as comprising restricted, graph-structured representations (e.g., formal structures that support crisply defined inference algorithms), intertwined with fully general graph-structured representations (informal structures that support soft inference, analogy, explication of application scope, and so on).
 Integrating Formal Quasilinguistic Systems Programming languages and mathematics are formal systems, typically based on quasilinguistic representations.  4  In these systems, the word-like entities (symbols) have minimal semantic content, yet syntactic structures in conjunction with an interpretive context specify semantic or operational meanings that endow these systems with descriptive capabilities beyond the conventional scope of NL. How are formal systems connected to NL, and by extension, to NL + frameworks? NL cannot replace formal systems, and experience suggests that no conventional formal system can replace NL. What we find in the wild are formal structures combined with linguistic descriptions: mathematics interleaved with explanatory text in papers and textbooks, programs interleaved with comments and documentation in source code, and so on. Experience with deciphering such documents suggests the value of intimate connections between NL-like descriptions and embedded formal expressions.  1  In one natural pattern of use, formal systems (e.g., mathematics, code, and knowledge representation languages) would correspond to distinguished, formally interpretable subsets of networks of NL + expressions. Formal languages can describe executable operations to be applied in the informal context of neural computation. Conversely, vector embeddings can be used to guide premise selection in the formal context of theorem proving. 2
 Translating and Explaining Across Linguistic Interfaces Proposed NL + content has both similarities to NL and profound differences. It is natural to consider how NL might be translated into NL + representations, how NL + representations might be translated or explained in NL, and how anticipated differences among NL + dialects might be bridged.
 Interpreting Natural Language Inputs NL + frameworks are intended to support systems that learn through interaction with the world, but first and foremost, are intended to support learning from existing NL corpora and language-mediated interactions with humans. Translation from NL sources to NL + is central both to representation learning and to key applications. It is natural to expect that encoders for NL → NL + translation will share a range of architectural characteristics and training methods with encoders for NL → NL translation and other NLP tasks (Section 10.2). Translation to NL + could be applied both to support immediate tasks and (more important) to expand corpora of NL + -encoded knowledge. Transformer-based NL → NL translation systems can learn language-agnostic representations, 1 a capability which suggests that NL → NL + translation will be tractable.
 Translating Among NL + Dialects Because NL + corpora could span many domains of knowledge-and be encoded by multiple, independently trained systems-it would be surprising to find (and perhaps challenging to develop) universally compatible NL + representations. In neural ML, different models learn different embeddings, and the representations learned by models with different training sets, training objectives, and latent spaces may diverge widely. In an NL + world, we should expect to find a range of NL + "dialects" as well as domain-specific languages. Nonetheless, where domain content is shared and representational capacities are equivalent, is reasonable to expect facile NL + → NL + translation. Further, regarding interoperability in non-linguistic tasks, the concrete details of differing representations can be hidden from clients by what is, in effect, an abstraction barrier. 2 Domain-specific languages may resist translation at the level of representations, yet contribute seamlessly to shared, cross-domain functionality. Lossless translation is possible when the semantic capacity of one representation fully subsumes the capacity of another. Given the computational tractability of NL + representations, we can expect translation between similar NL + dialects to be more accurate than translation between natural languages. In translation, spaces of lexical-level embeddings can be more tractable than discrete vocabularies in part because vector-space transformations can be smooth and one-to-one.  3
 Translating and Explaining NL + Content in NL It is reasonable to expect that, for a range of NLP tasks, conventional NL → (opaque-encoding) → NL pipelines can be outperformed by NL → NL + → NL pipelines; 1 this would imply effective translation of the intermediate NL + representations to NL. If NL + frameworks fulfill their potential, however, NL + corpora will contain more than translations of NL expressions. Content will typically draw on information from multiple sources, refined through composition and inference, and enriched with non-linguistic word-like elements. There is no reason to expect that the resulting representations-which will not correspond to particular NL expressions or any NL vocabularies-could be well-translated into NL. If NL + is more expressive than NL, it follows that not all NL + content can be expressed in NL. How, then, might NL + content be accessed through NL, either in general or for specific human uses? • Some NL + expressions will correspond closely to NL expressions; here, we can expect to see systems like conditional language models  (Keskar et al. 2019 ) applied to produce fluent NL translations. • NL + descriptions that are detailed, nuanced, complex, and effectively untranslatable can inform NL descriptions that provide contextually relevant information suitable for a particular human application. • Similarly, a general abstraction expressed in NL + might be effectively untranslatable, yet inform narrower, more concrete NL descriptions of contextually relevant aspects of that abstraction. • To the extent that NL expressions could-if sufficiently extensiveconvey fully general information (a strong condition), NL could be used to describe and explain NL + content in arbitrary detail; this approach is continuous with verbose translations. • NL + content in effectively non-linguistic domains could in some instances be expressed in diagrams, videos, interactive models, or other human-interpretable modalities. Relative to the opaque representations common in current neural ML, NL + representations have a fundamental advantage in interpretability: Because QNRs are compositional, their components can be separated, examined, and perhaps interpreted piece by piece. Even when components cannot be fully interpreted, they will often refer to some familiar aspect of the world, and knowing what an expression is about is itself informative.
 Integrating and Extending Knowledge Graph-structured representations in which some vector attributes designate regions in semantic spaces 1 lend themselves to operations that can be interpreted as continuous relaxations of formal unification and anti-unification, which in turn can support reasoning and logical inference by (continuous relaxations of) familiar algorithms. These operations can help extend corpora by combining existing representations along lines discussed from a somewhat different perspective in the preceding section.
 Combining Knowledge Through (Soft) Unification Compatible representations need not be identical: Alignment and (successful) soft unification (Appendix A1) indicate compatibility, and the successful unification of two expressions defines a new expression that may both combine and extend their graph structures and semantically narrow their attributes. Soft unification could potentially be used to refine, extend, compare, and link QNR/NL + representations. Where QNR graphs partially overlap, successful unification yields a consistent, extended description.  2  Attempts to unify incompatible representations fail and could potentially yield a value that describes their semantic inconsistencies.  3  In refining content through soft unification, relatively unspecific structures in one QNR 4 may be replaced or extended by relatively specific structures  5  from an aligned QNR. Embeddings that contain different information may commbine to yield a semantically narrower embedding. Products of successful unification (new, more informative expressions and relationships) are candidates for addition to an NL + corpus. Records of failed unifications-documenting specific clashes-can provide information important to epistemic judgment. These successful and unsuccessful unificationproducts may correspond to nodes in a semantically higher-level graph that represents relationships among expressions.
 Generalizing Knowledge Through (Soft) Anti-Unification While unification of two expressions can be regarded as combining their information content, anti-unification (generalization) can be regarded as combining their uncertainties, spanning their differences, and discarding unshared information. Generalization in this sense may represent a useful prior for generative processes within distributions that include the inputs.
 Constructing Analogies Operations based on generalization and unification could be applied to identify, construct, and apply analogies and abstractions in QNR corpora: • A range of latent analogies will be reflected in recognizably parallel structures between or among concrete descriptions. 1 • Alignment of parallel concrete descriptions can establish concrete analogies, potentially reified as graphs. • Generalization over sets of parallel descriptions can abstract their common structures as a patterns. • Unification of a concrete description with a pattern can indicate its analogy to a set of similar concrete descriptions without requiring pairwise comparison. Analogies have powerful applications. For example, if a description of A includes features of a kind absent from the analogous description of B, then it is reasonable to propose A-like features in B. Analogies among mammals, for example, underlie the biomedical value of discovery in animal models. Indeed, analogy permeates science, guiding both hypotheses and research planning. Analogy has been identified as central to cognition, 1 and reified networks of analogies can form graphs in which relationships among abstractions are themselves a domain of discourse. With suitable annotations, products of generalization and analogy-new abstract expressions and relationships-are candidates for addition to an NL + corpus.
 Extending Knowledge Through (Soft) Inference Natural language inference (NLI) is a major goal in NLP research, and recent work describes a system (based on a large language model) in which NL statements of rules and facts enable answers to NL questions  (Clark, Tafjord, and Richardson 2020) . Inference mechanisms that exploit NL + → NL + operations could potentially be useful in NLI pipelines, and in refining and extending NL + corpora NL + → NL + inference could play a central role. Regularizing and normalizing QNR representations (Section 8.4) can enable a kind of "soft formalization" based on continuous relaxations of formal reasoning (modeled, for example, on logic programming, Section A1.4.1). Rules can be represented as "if-then" templates (in logic, expressions with unbound variables) in which successful unification of an expression with an "if-condition" template narrows the values of attributes that, through coreference, then inform expressions constructed from "then-result" templates.  2  Advances in neural mechanisms for conventional symbolic theorem-proving (e.g., guiding premise selection) have been substantial.  3  It is reasonable to expect that wrapping formal expressions in NL + descriptions-including embeddings and generalizations of potential use contexts-could facilitate heuristic search in automated theorem proving.
 Credibility, Consensus, and Consilience Humans examine, compare, contrast, correct, and extend information represented in NL literatures. Machines can do likewise with NL + content, and for 1. See  Hofstadter (2009)  and  Gentner and Forbus (2011) . 2. Unification-based machinery of this kind can implement Prolog-like computation with applications to natural language inference  (Weber et al. 2019 ). 3.  Riedel (2016, 2017) ,  Minervini et al. (2018), and Minervini et al. (2019)  similar purposes, 1 a process that can exploit unification (and failure), together with generalization and analogy.
 Modeling and Extending Scholarly Literatures The strategy of using NL as a baseline suggests seeking models for NL + corpora in the scholarly literature, a body of content that includes both structures that are broadly hierarchical (e.g., summary/body and section/subsection relationships) and structures that correspond to more general directed graphs (e.g., citation networks). In abstracts, review articles, and textbooks, scholarly literatures summarize content at scales that range from papers to fields. Proposed NL + constructs can support similar patterns of expression, and can extend content summarization to finer granularities without cluttering typography or human minds. Scholarly citations can link to information that is parallel, supportive, problematic, explanatory, or more detailed; in NL + syntax, analogous citation functionality can be embodied in graphs and link contexts. Through indexing, citations in scholarly literatures can be made bidirectional, 2 enabling citation graphs to be explored through both cites-x and cited-by-x relationships. For similar reasons, NL + links in fully functional systems should (sometimes) be bi-directional.  3  In general, the structure and semantics of citations and citing contexts can vary widely (document-level remote citations are continuous with sentence-level coreference), and the structure of NL + representations makes it natural to extend cites and cited-by relationships to expressions finer-grained than NL publications. Steps have been taken toward applying deep learning to improve the integration of scholarly literatures.  4  It will be natural to build on this work using NL + tools to enrich NL + content .
 Using Credibility, Consensus, and Consilience to Inform Judgments As with NL, not all NL + content will be equally trustworthy and accurate. The origin of information-its provenance-provides evidence useful in judging epistemic quality, a consideration that becomes obvious when considering information derived from heterogeneous NL sources. Judgments of epistemic quality can reflect not only judgments of individual sources (their credibility), but also the consistency of information from different sources, considering both consensus among sources of similar kind, and consilience among sources that differ in kind. Search by semantic similarity and comparison through structural and semantic alignment provide a starting point, but where epistemic quality is in question, provenance will often be key to resolving disagreements. Some broad distinctions are important.
 Informing judgments through provenance and credibility Provenance is an aspect of context that calls for summarization. In a fully functional system, embeddings (and potentially extended QNRs 1 ) can summarize both information sources and subsequent processing, providing descriptive information that can be linked and accessed to derived content for deeper examination. Provenance information helps distinguish broad concurrence from mere repetitions-without tracking sources, repetitions may wrongly be counted as indicating an informative consensus. By analogy with (and leveraging) human judgment, systems can with some reliability recognize problematic content that can range from common misconceptions through conspiracy theories, fake news, and computational propaganda.  2  Problematic content should be given little weight as a source of information about its subject, yet may itself be an object of study.  3  Judgments of quality can be iterative: The quality and coherence of content can be judged in part by the quality and coherence of its sources, and so on, a process that may converge on descriptions of more-or-less coherent but incompatible models of the world together with accounts of their clashes. Judging source quality in generally sound NL literatures is a familiar human task. In the experimental sciences, for example, we find a spectrum of epistemic status that runs along these lines: • Uncontroversial textbook content • Reviews of well-corroborated results
 • Reports of recent theory-congruent results • Reports of recent surprising results • Reports of recent theory-incongruent results • Reports of actively disputed results • Reports of subsequently retracted results All of these considerations are modulated by the reputations of publishers and authors. Broadly similar indicators of quality can be found in history, economics, current affairs, and military intelligence. The reliability of sources is typically domain-dependent:  1  Nobel laureates may speak with some authority in their fields, yet be disruptive sources of misinformation beyond it; a conspiracy theorist may be a reliable source of information regarding software or restaurants. Although information about credibility can be propagated through a graph, credibility is not well-represented as a scalar.
 Informing judgments through consensus In judging information, we often seek multiple sources and look for areas of agreement or conflict-in other words, degrees of consensus. Relevant aspects of provenance include the quality of individual sources, but also their diversity and evidence of their independence.  2  What amount to copying errors may be indicated by sporadic, conflicting details. Lack of independence can often be recognized by close similarity in how ideas are expressed.  3  In judging information from (what are credibly considered to be) direct observations, experiments, and experience, the quality of human sources may play only a limited role. Established methods of data aggregation and statistical analysis will sometimes be appropriate, and while NL + representations may be useful in curating that data, subsequent methods of inference may have little relationship to NL + affordances. Inference processes themselves, however, constitute a kind of algorithmic provenance relevant to downstream representation and assessment of results.
 Informing judgments through consilience More powerful than consensus among sources of broadly similar kinds is con- 2. Some of this evidence is found in surface features of NL texts (e.g., uses of specific words and phrases); other evidence is found in features of semantic content. silience, agreement of evidence from sources of qualitatively different kindsfor example, agreement between historical records and radiocarbon dating, or between an experimental result and a theoretical calculation. Judgment of what constitutes "a difference in kind" is a high-level semantic operation, but potentially accessible to systems that can recognize similarities and differences among fields through their citation structures, focal concerns, methodologies, and so on. Distinguishing consilience from mere consensus is a judgment informed in part by provenance, and is key to robust world modeling. It calls for modeling the epistemic structures of diverse areas of knowledge.
 Architectures and Training Extensions of current neural ML methods can leverage architectural inductive bias and multitask learning to support the training of quasilinguistic neural systems with NL + -level expressive capacity. The preceding sections suggest that QNR frameworks can implement powerful, tractable NL + functionality, provided that suitable representations can be learned; the present section outlines potentially effective approaches to learning based on adaptations of familiar architectures and training methods. Vector-labeled graph bottlenecks can provide a strong inductive bias, while multitask learning and auxiliary loss functions can shape abstract representations that are anchored in, yet substantially decoupled from, natural languages. The final section outlines potential architectures for components that control inference strategies.
 General Mechanisms and Approaches Following neural ML practice, the development of QNR-centered systems calls, not for hand-crafting features, but for architectures that provide suitable components, capacity, and inductive bias, in conjunction with training tasks that provide suitable datasets, objectives, and loss functions. General mechanisms and approaches include: These mechanisms and approaches should be seen as facets of multitask learning in which a key goal is to develop NL → QNR → NL systems 1 that support broad applications. Pretrained NL → embedding → NL models (e.g., BERT and friends) are perhaps today's closest analogues. • Potential inputs to encoders include text, but also images, symbolic expressions, multimodal data streams, 2 and so forth. • Potential outputs from decoders include translations, summaries, answers to questions, retrieved content, classifications, and various products of downstream processing (images, engineering designs, agent behaviors, and so on). 3. It is at least questionable whether the inductive bias provided by a simple QNR-bottleneck architecture would outperform an otherwise similar but unconstrained encoder/decoder architectures in stand-alone NL → NL tasks. The use of a QNR-like bottleneck in  Bear et al. (2020)  has led to strong performance, but small-scale QNR representations can be interchangeable with vector embeddings that lack explicit graph structure (Section 7.3.1).
 Basic Information Flows Potential architectural building blocks range from MLPs and convolutional networks to Transformers and the many varieties of graph neural networks.  1  Architectures can employ building blocks of multiple kinds that collectively enable differentiable end-to-end training (Section 7.2.3 discusses differentiable representations of graph topology). The current state of the art suggests Transformer-based building blocks as a natural choice for encoding NL inputs and generating NL outputs. Transformer-based models have performed well in knowledge-graph → text tasks,  (Ribeiro et al. 2020) , and can in some instances benefit from training with explicit syntax-graph representations.  2  Encoders like those developed for scene-graph representation learning 3 are natural candidates for QNR-mediated vision tasks. In both NL and vision domains, encoders can produce vector/graph representations that, through training and architectural bias, serve as QNRs. GNNs or graph-oriented Transformers are natural choices both for implementing complex operations and for interfacing to task-oriented decoders. Simple feed-forward networks are natural choices for transforming and combining the vector components of vector-labeled graphs. Systems that read and write expressions in QNR corpora could employ scalable near-neighbor lookup in repositories indexed by QNR-derived semantic embeddings (Section 9.1.2). Information flows in generic QNR systems augmented by access to a repository of QNR content. In the general case, "QNR inference" includes read/write access to repositories, producing models that are in part pre-trained, but also pre-informed. The analysis presented in previous sections suggests that QNRs can in principle meet the criteria for representing NL + -level semantics, while the capabilities of current neural systems suggest that architectures based on compositions of familiar building blocks can implement the operations required for NL + -mediated functionality. The next question is how to train such systems-how to combine tasks and inductive bias to produce encoders, decoders, and processing mechanisms that provide the intended functionality.
 Shaping QNR Semantics Basic aspects of intended QNR semantics include non-trivial syntax in conjunction with NL-like representational capacity and extensions to other modalities. These goals can be pursued through architectural inductive bias together with training tasks in familiar domains.
 Architectural Bias Toward Quasilinguistic Representations Architectural inductive bias can promote the use of syntactically nontrivial QNRs (rather than flat sequences of embeddings) to represent broadly NL-like content. If learned representations follow the general pattern anticipated in Section 8, QNR syntax would typically employ (at least) DAGs of substantial depth (Section 8.1); leaf attributes would typically encode (at least) lexicallevel semantic information (Section 8.2), while attributes associated with internal nodes would typically encode relationships, summaries, or modifiers applicable to subsidiary expressions (Section 8.3). Encoders and decoders could bias QNRs toward topologies appropriate for expressing quasilinguistic semantics. Architectures can pass information through QNR-processing mechanisms with further inductive biases-e.g., architected and trained to support soft unification-to further promote the expression of computationally tractable, disentangled, quasilinguistic content.
 Anchoring QNRs Semantic Content in NL Although QNR representations have broader applications, it is natural to focus on tasks closely tied to language. Transformers trained on familiar NL → NL objectives (e.g., language modeling and sentence autoencoding) have produced flat vector representations (vectors and vector-sequences) that support an extraordinary range of tasks.  1  Training QNR-bottleneck architectures (NL → QNR → NL) on the same NL → NL objectives should produce comparable (and potentially superior) QNR representations and task performance. Potential tasks include: • Autoencoding NL text • NL language modeling • Multilingual translation • Multi-sentence reading comprehension • Multi-scale cloze and masked language tasks  1  It seems likely that developing NL + -level representations and mechanisms would best be served, not by a pretraining/fine-tuning approach, but by concurrent multitask learning. In this approach, optimization for individual tasks is not an end in itself, but a means to enrich gradient signals and learned representations.  If NL + is to be more than a representation of NL, the training of QNR models may require an inductive bias toward representations that are deliberately decoupled from NL. Lexical-level vector embeddings already provide a useful bias in that they decouple representations from the peculiarities of NL vocabularies. Massively multilingual tasks (translation, etc.) can further encourage the emergence of representations that abstract from the features of particular languages.  1  In current practice, combinations of multitask learning and architectural bias have been employed to separate higher-level and lower-level semantic representations.  2  It may be useful, however, to seek additional mechanisms for learning representations that are abstracted from NL.  3  Supporting this idea, recent work has found that disentangling semantics from NL syntax is practical and can provide advantages in performing a range of downstream tasks  (Huang, Huang, and Chang 2021) .
 Abstracting QNR Representations From Word Sequences Tasks and architectures can be structured to favor separation of abstract from word-level representations.  4  A general approach would be to split and recombine information paths in NL → NL tasks: An abstract QNR path could be trained to represent predominantly high-level semantics and reasoning, while an auxiliary path carries lexical-level information. To recombine these paths, the high-level semantic path could feed a decoder that is also provided with a set of words from the target expression permuted together with decoys.  5  By reducing the task of producing correct NL outputs to one of selecting and arranging elements from a given set of words, this mechanism could shift a lexical-level, NL-specific burden-and perhaps the associated low-level semantic content-away from the abstract, high-level path. To strengthen separation, the gradient-reversal trick for domain adaptation 6 could be applied to actively "anti-train" the availability of word-specific information in abstract-path representations.
 Strategies for Learning Higher-Level Abstractions Objective functions in NLP often score outputs by their correspondence to specific sequences of target words. This objective is embedded in the definitions of language modeling, masked language modeling, and typical cloze tasks, while similar objectives are standard in NL translation. However, as the size of target outputs increases-from single-word cloze tasks to filling gaps on the scale of sentences, paragraphs, and beyond-predicting specific word sequences becomes increasingly difficult or effectively impossible. When the actual research objective is to manipulate representations of meaning, lexical-level NL training objectives fail the test of scalability. Completion tasks 1 formulated in the QNR domain itself would better serve this purpose. Useful QNR-domain completion tasks require QNR targets that represent rich task-domain semantics, but we have already seen how NLP tasks can be used for this purpose (Section 10.  3.2) . Products of such training can include NL → QNR encoders that raise both inference processes and their targets to the QNR domain (Figure  10 .3). With targets raised from NL to QNR representations, it should become practical to compare outputs to targets even when the targets represent complex semantic objects with an enormous range of distinct yet nearly equivalent NL representations. While it seems difficult to construct useful semantic-distance metrics over word sequences, semantic-distance metrics in the QNR domain can be relatively smooth.  1  Ambitious examples of completion tasks could include completion of (descriptions of) code with missing functions, or of mathematical texts with missing equations or proofs.
 Training QNR × QNR → QNR Functions to Respect Lattice Structure The semantic-lattice properties discussed in Appendix A1 correspond to an algebra of information, but QNRs need not automatically respect this algebra. In particular, absent suitable training objectives, operations on QNRs may strongly violate the lattice axioms that constrain unification and generalization.  2  Learning representations and functions that approximately satisfy the lattice-defining identities (Section A1.2) can potentially act both as a regularizer and as a mechanism for training operations that support principled comparison, combination, and reasoning over QNR content. Because the existence of (approximate) lattice operations over QNR representations implies their (approximate) correspondence to (what can be interpreted as) an information algebra, we can expect that (approximately) enforcing this constraint can improve the semantic properties of a representational system. In addition, prediction of soft-unification scores (Section A1.4.3) can provide an auxiliary training objective for content summaries (Section 8.3.4), providing a distance measure with potential applications to structuring latent spaces for similarity-based semantic retrieval (Section 9.1.2).
 Processing and Inference on QNR Content The above discussion outlined coarse-grained information flows and general training considerations using block diagrams to represent units of high-level functionality. The present section examines the potential contents of boxes labeled "QNR inference". The aim here is not to specify a design, but to describe features of plausible architectures for which the implementation challenges would be of familiar kinds.
 Control, Selection, and Routing Tasks of differing complexity will call for different QNR inference mechanisms. The null case is the identity function, single-path pass-through in a QNRbottleneck architecture. A more interesting case would be a single-path system that performs QNR → QNR transformations (e.g., using a GNN) based on a conditioning input. More powerful inference mechanisms could perform QNR × QNR → QNR operations, potentially by means of architectures that can learn (forms of) soft unification or anti-unification. Toward the high end of a spectrum of complexity (far from entry level!), open-ended QNR-based inference will require the ability to learn task-and data-dependent strategies for storing, retrieving, and operating on QNRs in working memory and external repositories. This complex, high-end functionality could be provided by a controller that routes QNR values to operators while updating and accessing QNR values by means of key and query based storage and retrieval.  1  Keys and queries, in turn, can be products of abstractive operations on QNRs. (In discussions of retrieval, argument passing, etc., "a QNR" is operationally a reference to a node in a graph that may be of indefinite extent.) Note that "reasoning based on QNRs" can employ reasoning about QNR processing by means of differentiable mechanisms that operate on flat vector representations in a current task context.  2  Reinforcement learning in conjunction with memory retrieval has been effective in multi-step reasoning  (Banino et al. 2020) , as have models that perform multi-step reasoning over differentiable representations and retrieve external information to answer queries  (Bauer, Wang, and Bansal 2018) .
 Encoders QNR encoders accept task inputs (word sequences, images, etc.) and produce sparse-graph outputs. Natural implementation choices include Transformerlike attention architectures that initially process information on a fully connected graph (the default behavior of attention layers) but apply progressively sharpened gating functions in deeper layers. Gating can differentiably weight and asymptotically prune arcs to sparsen graphs that can then be read out as discrete structures. A range of other methods could be applied to this task. Optional discretization at a sparse-graph readout interface breaks differentiability and cannot be directly optimized by gradient descent. This difficulty has been addressed by means that include training-time graph sampling with tools from reinforcement learning  (Kazi et al. 2020 ) and other mechanisms that learn to discretize or sparsen through supervision from performance on downstream tasks  (Malinowski et al. 2018; Zheng et al. 2020) . Systems with potentially relevant mechanisms learn dynamic patterns of connectivity on sparse graphs  (Veličković et al. 2020)  and address problems for which the solution space consists of discrete graphs  (Cappart et al. 2021) . Because 1. In considering how this functionality might be structured, analogies to computer architectures (both neural and conventional) may be illuminating. For example, analogies with storedprogram (i.e., virtually all) computers suggest that memory stores can usefully contain QNRs that describe executable inference procedures. See related work in  Gulcehre et al. (2018) ,  Le, Tran, and Venkatesh (2020) , and Malekmohamadi, Safi-Esfahani, and Karimian-kelishadrokhi (2020). 2. A psychological parallel is the use of general, fundamental "thinking skills" in reasoning about declarative memory content. Skills in this sense can be implicit in a processing mechanism (an active network rather than a repository) and are applied more directly than explicit plans. arcs in QNRs can define paths for information flow in computation (e.g., by graph neural networks), methods for training computational-graph gating functions in dynamic neural networks 1 are potentially applicable to learning QNR construction.
 Decoders Standard differentiable neural architectures can be applied to map QNRs to typical task-domain outputs. A natural architectural pattern would employ GNNs to process sparse graphs as inputs to downstream Transformer-like attention models. Where the intended output is fluent natural language, current practice suggests downstream processing by large pretrained language models adapted to conditional text generation; 2 potentially relevant examples include models that condition outputs on sentence-level semantic graphs.  3
 Working and External Memory Stores Working memory and external repositories have similar characteristics with respect to storage and retrieval, but differences in scale force differences in implementation. In particular, where stores are large, computational considerations call for storage that is implemented as an efficient, scalable, potentially shared database that is distant (in a memory-hierarchy sense) from taskfocused computations.  4  In the approach suggested here, both forms of storage would, however, retrieve values based on similarity between key and query embeddings.
 Unary Operations Unary operations apply to single graphs. Popular node-convolutional GNNs use differentiable message-passing schemes to update the attributes of a graph, and can combine local semantic information to produce context-informed representations. Different networks could be applied to nodes of different semantic types. The values returned by unary operations may be QNRs or embeddings (e.g., keys, queries, or abstractive summaries). Unary operations may also transform graphs into graphs of a different topology by pruning arcs (a local operation), or by adding arcs (which in general may require identifying and linking potentially remote nodes).  1  Examples of neural systems with the latter functionality were noted above.  2  A local topology-modifying operation could (conditionally) pass potential arcs (copies of local references) as components of messages. 3
 Graph Alignment Graph alignment ("graph matching") is a binary operation that accepts a pair of graphs as arguments and (when successful) returns a graph that represents a (possibly partial) node-correspondence relationship between them (Section 7.2.4). Return values could range in form from a node that designates a pair of corresponding nodes in the arguments, to a representation that includes a distinguished set of arcs (potentially labeled with vector embeddings) that represent relationships among all pairs of corresponding nodes. Several neural matching models have been demonstrated, some of which are relatively scalable.  4  Graph alignment could be a pretrained and fine-tuned function.
 Lattice Operations Lattice operations (unification and generalization, Appendix A1). are binary operations that include mechanisms for graph alignment and combination. Soft lattice operations differ from matching in that they return what is semantically a single graph. Like graph alignment, lattice operations could be pretrained and fine-tuned, or could serve as auxiliary training tasks in learning QNR inference. Neural modules pretrained to mimic conventional 1. A related operation would accept a graph referenced at one node and return a graph referenced at another, representing the result of a graph traversal.  Cappart et al. (2021)  3. This is the fundamental topology-modifying operation employed by object capability systems  (Noble et al. 2018 ): A node A with message-passing access to nodes B and C can pass its node-B access (a "capability") to node C; node A may or may not retain its access to C afterward. This operation can be iterated to construct arcs between what are initially distant nodes in a graph. Intuitively, access-passing is semantically well motivated if the "need" for a more direct connection from B to C can be communicated through messages received by A. See also  Veličković et al. (2020 ). 4. Y. Li et al. (2019 ,  Sarlin et al. (2020) , Y.  Fey et al. (2020)  algorithms for unification and generalization could potentially serve as building blocks for a range of inference algorithms that operate on soft lattices and rich semantic representations.  1
 Veličković et al. (2020) and
 Potential Application Areas Potential applications of QNR/NL + functionality include and extend applications of natural language. They include human-oriented NLP tasks (translation, question answering, semantic search), but also inter-agent communication and the integration of formal and informal representations to support science, mathematics, automatic programming, and AutoML. QNR/NL + frameworks are intended to support wide-ranging applications both within and beyond the scope of natural language. The present section sketches several potential application areas: first, applications to tasks narrowly centered on language-search, question answering, writing, translation, and language-informed agent behavior-and then a range of applications in science, engineering, mathematics, software, and machine learning, including the general growth and mobilization of knowledge in human society. The discussion will assume success in developing high-level QNR/NL + capabilities.
 Language-Centered Tasks Tasks that map NL inputs to NL outputs are natural applications of NL + -based models. These tasks include internet search, question answering, translation, and writing assistance that ranges from editing to (semi)autonomous content creation.
 Search and Question Answering In search, NL + representations can provide a semantic bridge between NL queries and NL documents that employ different vocabularies. Search and question-answering (QA) models can jointly embed queries and content, enabling retrieval of NL content by semantic similarity search 2 anchored in the NL + domain; beyond comparing embeddings, direct NL + to NL + comparisons can further refine sets of potential search results. Alternatively, language models conditioned on queries (and potentially on models of readers' style preferences) can translate retrieved NL + semantic content to fluent NL answers. QA fits well with document search, as illustrated by Google's information boxes: The response to a search query can include not only a set of documents, but information abstracted from the corpus. In a broader application, NL + -based models could generate extended answers that are more comprehensive, more accurate, and more directly responsive to a query than any existing NL document. With the potential for dense linking (perhaps presented as in-place expansion of text and media), queryresponsive information products could enable browsing of internet-scale knowledge corpora through presentations more attractive and informative than conventional web pages.
 Translation and Editorial Support Translating and editing, like QA, call for interpreting meaning and producing results conditioned on corpus-based content and priors. Differences include a greater emphasis on lengthy inputs and on outputs that closely parallel those inputs, with access to specific knowledge playing a supporting rather than primary role. During training, massively multilingual translation tasks have produced language-invariant intermediate representations (interlinguas 1 ); we can expect similar or better interlingua representations-and associated translations-in systems that employ NL → NL + → NL architectures. Priors based on the frequency of different patterns of semantic content (not phrases) can aid disambiguation of NL source text. The task of machine-aided editing is related to translation: Reproducing semantic content while translating from language to language has much in common with transforming a rough draft into a refined text; stretching the notion of reproducing semantic content, a system might expand notes into text while retaining semantic alignment.  2  It is again natural to exploit priors over patterns of expression to help interpret inputs and generate outputs. Access to knowledge from broad, refined corpora could greatly enrich content when expanding notes. The graph structure of hypertext makes QNRs a good fit to NL + -supported authoring of online NL content. As a specific, high-leverage application, such tools could help contributors expand and improve Wikipedia content. Systems that compile, refine, and access a QNR translation of Wikipedia would be a natural extension of current research on the use of external information stores.  1  Human contributors could play the roles that they do today, but aided by generative models that draw on refined NL + corpora to suggest corrected and enriched content. Most of what people want to express either repeats what has been said elsewhere (but rephrased and adapted to a context), or expresses novel content that parallels or merges elements of previous content. Mechanisms for abstraction and analogy, in conjunction with examples and priors from existing literatures, can support interactive expansion of text fragments and hints to provide what is in effect a more powerful and intelligent form of autocomplete. Similar functionality can be applied at a higher semantic level. Responsible writers seek to avoid factual errors, which could be identified (provisionally!) by clashes between the NL + encoding of a portion of a writer's draft and similar content retrieved from an epistemically high-quality NL + corpus.  2  Writers often prefer, not only to avoid errors, but to inform their writing with knowledge that they do not yet have. Filling semantic gaps, whether these stem from omission or error removal, can be regarded as a completion task over abstract representations  (Section 10.4.2) . Semantically informed search and generative models could retrieve and summarize candidate documents for an author to consider, playing the role of a research assistant; 3 conditional language models, prompted with context and informed by external knowledge 4 could generate substantial blocks of text, playing the role of a coauthor.
 (Semi)Autonomous Content Creation Social media today is degraded by the influence of MIsinformed and unsourced content, a problem caused (in part) by the cost of finding good infor-mation and citing sources, and (in part) by fact-indifferent actors with other agendas. Well-informed replies are relatively costly and scarce, but mob-noise and bot-spew are abundant. As a human-mediated countermeasure, responsible social media participants could designate targets for reply (perhaps with a hint to set direction) and take personal responsibility for authorship while relying on semiautonomous mechanisms for producing (drafts of) content. As a fully autonomous countermeasure, bots created by responsible actors could scan posts, recognize problematic content, and reply without human intervention. Actors that control social media systems could use analogous mechanisms in filtering, where a "reply" might be a warning or deletion. Acceptable, fully effective countermeasures to toxic media content are difficult to imagine, yet substantial improvements at the margin may be both practical and quite valuable.
 Agent Communication, Planning, and Explanation Human agents use language to describe and communicate goals, situations, and plans for action; it is reasonable to expect that computational agents can likewise benefit from (quasi)linguistic communication.  1  If NL + representations can be strictly more expressive than NL, then NL + can be strictly more effective as a means of communication among computational agents. Internal representations developed by neural RL agents provide another point of reference for potential agent-oriented communication. Some systems employ memories with distinct, potentially shareable units of information that can perhaps be viewed as pre-linguistic representations (e.g., see  Banino et al. (2020) ). The limitations inherent in current RL-agent representations suggest the potential for gains from language-like systems in which the compositional elements express durable, shareable, strongly compositional abstractions of states, conditions, actions, effects, and strategies. QNR/NL + -based representations can combine unnamable, lexical-level abstractions with lexical-level elements that describe semantic roles, confidence, relative time, deontic considerations, and the like-in other words, semantic elements like those often expressed in NL by function words and TAM-C modifiers  (Section 5.3.3, Section 5.3.4, and Section A3.3) . The role of NL in human communication and cognition suggests that NL + representations can contribute to both inter-and intra-agent performance, sometimes competing with tightly coupled, task-specific neural representations. Communication between humans and RL agents can benefit from language. Although reinforcement learning can enable unaided machines to outperform human professionals even in complex games, 1 human advice conveyed by NL can speed and extend the scope of reinforcement learning.  2  Conversational applications provide natural mechanisms for clarification and explanationin both directions-across machine-human interfaces, potentially improving the human value and interpretability of AI actions. Given suitable NL + descriptions and task-relevant corpora, similarity search could be applied to identify descriptions of similar situations, problems, and potentially applicable plans (including human precedents); mechanisms like those proposed for knowledge integration and refinement (Section 9.4) could be applied to generalize through analogy and fill gaps through soft unification. Widely used content would correspond to "common sense knowledge" and "standard practice".  3  Like natural language, NL + representations could support both strategic deliberation and concrete planning at multiple scales. Agents with access to large knowledge corpora resemble humans with access to the internet: Humans use search to find solutions to problems (mathematics, travel, kitchen repairs); computational agents can do likewise. Like human populations, agents that are deployed at scale can learn and pool their knowledge at scale. Frequent problems will (by definition) seldom be newly encountered.
 Science, Mathematics, and System Design Although research activities in science, mathematics, engineering, and software development differ in character, they share abstract tasks that can be framed as similarity search, semantic alignment, analogy-building, clash detection, gap recognition, and pattern completion. Advances in these fields involve an ongoing interplay between: • Tentative proposals (hypotheses in science, proof goals in mathematics, design concepts in engineering and software development), • Domain-specific constraints and enablers (evidence in science, theorems in mathematics, requirements and available components in engineering and software development), and • Competition between alternative proposals judged by task-specific criteria and metrics (maximizing accuracy of predictions, generality of proofs, performance of designs; minimizing relevant forms of cost and complexity). These considerations highlight the ubiquitous roles of generative processes and selection criteria, and a range of fundamental tasks in science, mathematics, engineering, and software development can be addressed by generative models over spaces of compositional descriptions. These can be cast in terms of QNR affordances: Given a problem, if a corpus of QNRs contains descriptions of related problems together with known solutions, then similarity search on problemdescriptions 1 can retrieve sets of potentially relevant solutions. Joint semantic alignment, generalization, and analogy-building within problem/solution sets then can suggest a space of alternatives that is likely to contain solutions-or near-solutions-to the problem at hand. In conjunction with an initial problem description, such representation spaces can provide priors and constraints on generative processes, 2 and generated candidate solutions can be tested against task-specific acceptance criteria and quality metrics. These considerations become more concrete in the context of specific task domains.
 Engineering Design [T]hink of the design process as involving, first, the generation of alternatives and, then, the testing of these alternatives against a whole array of requirements and constraints. There need not be merely a single generate-test cycle, but there can be a whole nested series of such cycles. -Herbert  Simon 3  Typical engineering domains are strongly compositional, and aspects of compositionality-modularity, separation of functions, standardization of interfaces-are widely shared objectives that aid not only the design and modeling of systems, but also production, maintenance, and reuse of components across applications. Representations used in the design and modeling of engineering systems typically comprise descriptions of components (structures, circuits, motors, power sources. . . ) and their interactions (forces, signals, power transmission, cooling. . . ). In engineering practice, natural language (and prospectively, NL + ) is interwoven with formal, physical descriptions of system-level requirements, options, and actual or anticipated performance. As Herbert Simon has observed, design can be seen as a generate-and-test process-a natural application of generative models.  1  A wide range of proposed systems can be tested through conventional simulation.  2  In engineering, even novel systems are typically composed (mostly or entirely) of hierarchies of subsystems of familiar kinds.  3  The affordances of QNR search and alignment are again applicable: Embedding and similarity search can be used to query design libraries that describe options at various levels of abstraction and precision; descriptions can be both physical and functional, and can integrate formal and informal information. Semantic alignment and unification provide affordances for filling gaps-here, unfilled functional roles in system architectures-to refine architectural sketches into concrete design proposals. The generation of novelty by soft-lattice generalization and combination operations (Appendix A1) could potentially enable fundamental innovation. Because engineering aims to produce systems that serve human purposes, design specifications-requirements, constraints, and optimization criteriamust fit those purposes. The development of formal specifications is an informal process that can benefit from QNR affordances that include analogy, pattern completion, and clash detection, as well as applications of the commonsense knowledge needed to choose obvious defaults, reject obvious mistakes, and identify considerations that call for human attention. 1. See discussions in  Kahng (2018) ,  Liao et al. (2019), and Oh et al. (2019) . Machine-aided interactive design  (Deshpande and Purwar 2019)  and imitation learning can also help to generate proposals; see ,  Ganin et al. (2021) . 2. Or using ML-based simulation methods, which are of increasing scope and quality. In particular, advances in ML-based molecular simulation (reviewed in  Noé et al. 2020 ) can be expected to facilitate molecular systems engineering.
 Scientific Inquiry Science and engineering often work closely together, yet their epistemic tasks are fundamentally different: Engineering seeks to discover multiple options for achieving purposes, while science seeks to discover uniquely correct descriptions of things that exist. Science and engineering intertwine in practice: Scientists exploit products of engineering (telescopes, microscopes, particle accelerators, laboratory procedures. . . ) when they perform observations and experiments, while engineers engage in science when they ask questions that cannot be answered by consulting models. Potential applications of QNR affordances in science include: • Translating NL publications into uniform, searchable representations
 Software Development and AutoML Applications of neural ML to software development are under intense exploration.  2  Language models can support interactive, text-based code completion and repair; 3 recent work has demonstrated generation of code based on docstrings.  4  GNNs could operate on structured representations (syntactic and semantic graphs) while also exploiting function names, variable names, comments, and documentation as sources of information and targets for prediction in representation encoding and decoding. QNRs can provide affordances for enriching syntactic structures with semantic annotations and the results of static program analysis,  5  and for wrapping code objects (both implemented and proposed) in descriptions of their requirements and functionality. QNR representations have a different (and perhaps closer) relationship to automated machine learning (AutoML 1 ), because neural embeddings and graphs seem particularly well-suited to representing the soft functionality of neural components in graph-structured architectures. Again, generateand-test processes guided by examples, analogies, and pattern completion could inform search in design spaces, 2 while the scope of these spaces can embrace not only neural architectures, but their training methods, software and hardware infrastructures, upstream and downstream data pipelines, and more. 12 Aspects of Broader Impact The breadth of potential applications of QNR-based systems makes it difficult to foresee (much less summarize) their potential impacts. Leading considerations include the potential use and abuse of linguistic capabilities, of agent capabilities, and of knowledge in general. Systems based on QNR representations promise to be relatively transparent and subject to correction. Potential roles for QNR/NL + -enabled capabilities are extraordinarily broad, with commensurate scope for potential benefits and harms.  3  Channels for potential QNR/NL + impacts can be loosely divided into core semantic functionalities (applications to knowledge in a general sense), semantic functionalities at the human interface (processing and production of natural language content), and potential roles in AI agent implementation and alignment. Most of the discussion here will be cast in terms of the NL + spectrum of potential QNR functionality.
 Broad Knowledge Applications Many of the potential benefits and harms of QNR/NL + -enabled developments are linked to large knowledge corpora and their applications. Several areas of potential impact are closely related to proposed core functionalities of knowledge integration and access.  1. Real et al. (2020)  and  He, Zhao, and Chu (2021)  2. You, Ying, and Leskovec (2020),  Radosavovic et al. (2020), and Ren et al. (2021)  3. For a survey of a range of potential harms, see  Brundage et al. (2018) .
 Integrating and Extending Knowledge Translation of content from NL corpora to corresponding NL + can enable the application of QNR-domain mechanisms to search, filter, refine, integrate, and extend NL-derived content, building knowledge resources for wide-ranging applications. To the extent that improving the quality of knowledge is on the whole beneficial (or harmful), we should expect net beneficial (or harmful) impacts.
 Mobilizing Knowledge Translation of NL expressions (statements, paragraphs, documents. . . ) to corresponding NL + representations promises to improve semantic embeddings and similarity search at scale (Section 9.1.5), helping search systems "to organize the world's information and make it universally accessible and useful" (Google 2020) through higher-quality semantic indexing and query interpretation. Generation of content through knowledge integration could go beyond search to deliver information that is latent (but not explicit) in existing corpora. It is reasonable to expect beneficial first-order impacts.
 Filtering Information To the extent that NL → NL + translation is effective in mapping between NL content and more tractable semantic representations, filtering of information 1 in the NL + domain can be used to filter NL sources. Potential applications span a range that includes both reducing the toxicity of social media and refining censorship in authoritarian states. In applications of language models, filtering based on disentangled representations of knowledge and outputs could mitigate leakage of private information. 2
 Surveillance and Intelligence Analysis Surveillance and intelligence analysis are relatively direct applications of QNRenabled knowledge mobilization and integration, and the balance of impacts on security, privacy, and power relationships will depend in part on how information is filtered, shared, and applied. Mapping raw information into structured semantic representations could facilitate almost any application, with obvious potential harms. To mitigate harms, it will be important to explore how filtering of raw information could be applied to differentially enable legitimate applications: For example, disentangled compositional representations could be more easily redacted to protect sensitive information while providing information necessary for legitimate tasks.
 Producing QNR-Informed Language Outputs at Scale We should expect to see systems that translate NL + content into NL text 1 with fluency comparable to models like GPT-3, and do so at scale. Automated, NL + -informed language production, including support for human writing (Section 11.1.2), could expand quantity, improve quality, and customize the style and content of text for specific groups or individual readers. These capabilities could support a range of applications, both beneficial and harmful.
 Expanding Language Output Quantity Text generation enabled by language models has the potential to produce tailored content for social media economically and at scale: Human writers are typically paid ~0.20 US$/word, 2 ) about 1,000,000 times the cost of querying an efficient Transformer variant.  3  It is reasonable to expect that NL + -informed outputs will have broadly similar costs, orders of magnitude less than the costs of human writing, whether these costs are counted in money or time. Put differently, text output per unit cost could be scaled by a factor on the rough order of 1,000,000. Even when constrained by non-computational costs and limitations of scope, the potential impact of automated text generation is enormous.
 Improving Language Output Quality Applications of language-producing systems will depend in part on domaindependent metrics of output quality: Higher quality can both expand the scope of potential applications and decrease the costs of human supervision, while changing the nature and balance of potential impacts. Relative to opaque language models, systems informed by NL + corpora can improve abilities: • To judge, incorporate, and update factual content • To perform multi-step, multi-source inference • To apply inference to refine and expand knowledge stores Current models based on opaque, learned parameters have difficulties in all these areas; overcoming these difficulties could greatly expand the scope of potential applications.
 Potentially Disruptive Language Products The most obvious societal threats from NL + -based language capabilities stem from their ability to produce coherent content that draws on extensive (mis)information resources-content that mimics the markers of epistemic quality without the substance. The magnitude of this threat, however, must be judged in the context of other, broadly similar technologies. Systems based on large language models are becoming fluent and potentially persuasive while remaining factually unreliable: They can more easily be applied to produce plausible misinformation than informed content. Unfortunately, the current state of social media suggests that fluent, persuasive outputs based on false, incoherent information-whether from conspiracy fans or computational propaganda-can be disturbingly effective in degrading the epistemic environment.  1  This suggests that the marginal harms of making misinformation more coherent, better referenced, etc., may be relatively small.  2  To the extent that capabilities are first deployed by responsible actors, harms could potentially be mitigated or delayed. 1. Existing language models have spurred concerns regarding abuse, including scaling of social-engineering attacks on computer security and of computational propaganda in public discourse  (See Woolley and Howard (2017)  and Howard (  2021 )). In part as a consequence of such concerns  (Solaiman et al. (2019), and Brown et al. (2020) , Section 6.1), OpenAI restricted access to its GPT-3 model. 2. One may hope that influential audiences that have in the past been susceptible to documents with misleading but apparently high-quality content (e.g., academics and policymakers) would also respond to prompt, well-targeted, high-quality critiques of those documents. Responding promptly would leave less time for misleading information to spread unchecked and become entrenched as conventional wisdom.
 Potentially Constructive Language Products Generating low-quality content is easy for humans and machines, and arguments (whether for bad conclusions or good) can cause collateral damage when they inadvertently signal-boost false information; conversely, arguments (regardless of the merits of their conclusions) can produce what might be described as "positive argumentation externalities" when their content signal-boosts well-founded knowledge Although the potential harms of facilitating the production of (apparently) high-quality misinformation may be marginal, the potential benefits of facilitating the production of high-quality information seem large. It would be difficult to exaggerate the potential value of even moderate success in damping pathological epistemic spirals and enabling information to gain traction based on actual merit. Authors who employ freely available tools to produce better-written, better-supported, more abundant content (drawing audiences, winning more arguments) could raise the bar for others, driving more widespread adoption of those same tools. Epistemic spirals can be positive.  1  12.3 Agent Structure, Capabilities, and Alignment Section 11.2 discussed NL + representations as potential enablers for agent performance-for example, by supporting the composition of plan elements, retrieval of past solutions, and advice-taking from humans. In considering potential impacts, opportunities for improving transparency and alignment become particularly important.
 Agent Structure A long-standing model of advanced AI capabilities takes for granted a central role for general, unitary agents, often imagined as entities that learn much as a human individual does. The AI-services model 2 challenges this assumption, proposing that general capabilities readily could (and likely will) emerge through the expansion and integration of task-oriented services that-crucially for potential generality-can include the service of developing new services. In the AI-services model, broad knowledge and functionality need not be concentrated in opaque, mind-like units, but can instead emerge through aggregation over large corpora of knowledge and tools, potentially informed both by pre-existing human-generated corpora and by massively parallel (rather than individual) experience of interaction with the world. The AIservices model fits well with the QNR/NL + model of scalable, multimodal knowledge aggregation and integration.
 Agent Capabilities Also in alignment with the AI-services model of general intelligence, the ability of relatively simple agents to access broad knowledge and tool sets 1 could amplify their capabilities. This prospect lends credence to long-standing threat models in which agents rapidly gain great and potentially unpredictable capabilities; the mechanisms differ, but the potential results are similar. Classic AI-risk scenarios commonly focus on AI capabilities that might emerge from an immense, opaque, undifferentiated mass of functionality, a situation in which agents might pursue unexpected goals by unintended means. It may be safer to employ task-oriented agents (and compositions of agents) that operate within action-and knowledge-spaces that are better understood and do not grossly exceed task requirements.  2  Basing functionality on bounded, differentiated resources provides affordances for observing "what a system is thinking about" and for constraining "what an agent can know and do", potentially powerful tools for interpreting and constraining an agent's plans.  3  Accordingly, developers could seek to bound, shape, and predict behaviors by exploiting the relative semantic transparency of proposed QNR/NL + corpora to describe and structure the knowledge, capabilities, constraints, and objectives of task-oriented agents.
 Agent Alignment Many of the anticipated challenges of aligning agents' actions with human intentions hinge on the anticipated difficulty of learning human preferences.  4  The ability to read, interpret, integrate, and generalize from large corpora of human-generated content (philosophy, history, news, fiction, court records, discussions of AI alignment. . . ) could support the development of richly informed models of human preferences, concerns, ethical principles, and legal systems-and models of their ambiguities, controversies, and inconsistencies.  1  Conversational systems could be used to test and refine predictive models of human concerns by inviting human commentary on actual, proposed, and hypothetical actions. NL + systems that fulfill their promise could model these considerations more effectively than human language itself, in a way that is not fully and directly legible, yet open to inspection though the windows of query and translation.
 Conclusions Current neural ML capabilities can support the development of systems based on quasilinguistic neural representations, a line of research that promises to advance a range of research goals and applications in NLP and beyond. Natural language (NL) has unrivaled generality in expressing human knowledge and concerns, but is constrained by its reliance on limited, discrete vocabularies and simple, tree-like syntactic structures. Quasilinguistic neural representations (QNRs) can generalize NL syntactic structure to explicit graphs (Section 8.1) and can replace discrete NL vocabularies with vector embeddings that convey richer meanings than words (Section 5.3, Section 8.2). By providing affordances for generalizing and upgrading the components of NL-both its structure and vocabulary-QNR systems can enable neural systems to learn "NL + " representations that are strictly more expressive than NL. Machines with human-like intellectual competence must be fully literate, able not only to read, but to write things worth reading and retaining as contributions to aggregate knowledge. Literate machines can and should employ machine-native QNR/NL + representations (Section 8) that are both more expressive and more computationally tractable than sequential, mouthand-ear oriented human languages. Prospects for QNR/NL + systems make contact with a host of fields. These include linguistics (Section 5), which offers insights into the nature of expressive constructs in NL (a conceptual point of departure for NL + ), as well as current neural ML, in which vector/graph models and representation learning provide a concrete basis for potential QNR implementations (Section 10). Considerations that include local compositionality (Section 4.3) suggest that vector/graph constructs can provide computationally tractable representations of both complex expressions and the contexts in which they are to be interpreted  (Section 8.3) . Drawing on existing NL corpora, QNR-based systems could enable the construction of internet-scale NL + corpora that can be accessed through scalable semantic search (Section 9.1), supporting a powerful ML analogue of long-term memory. In addition, QNR/NL + frameworks can support unification and generalization operations on (soft, approximate) semantic lattices (Appendix A1), providing mechanisms useful in knowledge integration and refinement (Section 9.4, Section 9.5). Applications of prospective QNR/NL + functionality could support not only epistemically well-informed language production (Section 11.1), but the growth and mobilization of knowledge in science, engineering, mathematics, and machine learning itself (Section 11.3). The fundamental technologies needed to implement such systems are already in place, incremental paths forward are well-aligned with research objectives in ML and machine intelligence, and their potential advantages in scalability, interpretability, cost, and epistemic quality position QNR-based systems to complement or displace current foundation models  (Bommasani et al. 2021)  at the frontiers of machine learning.
 A1 Unification and Generalization on Soft Semantic Lattices QNR representations can support operations that combine, contrast, and generalize information. These operations-soft approximations of unification and anti-unification-can be used to implement continuous relaxations of powerful mechanisms for logical inference. A range of formal representations of meaning, both in logic and language, have the structure of mathematical lattices. Although the present proposal for QNR systems (and aspirational NL + systems) explicitly sets aside the constraint of formality, approximate lattice structure emerges in NL and will (or should, or readily could) be a relatively strong property of QNRs/NL + . Because lattices can provide useful properties, it is worth considering the potential roles and applications of lattice structure in QNR-based systems. Note that the fundamental goals of NL + -general superiority to NL in expressiveness and computational tractability-do not require lattice properties beyond those that NL itself provides. The ability to provide stronger lattice properties is a potential (further) strength of NL + , not a requirement. In other words, lattice properties are natural and useful, yet optional. The following sections begin by discussing the motivation for considering and strengthening lattice properties-supporting meet and join, a.k.a. unification and generalization-in light of their potential roles and utility. A brief review of approximate lattice structure in NL provides an initial motivation for applying lattice structure in NL + within the scope of a methodology that avoids commitment to formal models. Consideration of lattices in logic and in constraint logic programming further motivates the pursuit of approximations, and introduces a discussion, in part speculative, regarding inductive bias and prospective, emergent lattice-oriented QNR representations. This topic is adjacent to many others, creating a large surface area that precludes any compact and comprehensive discussion relationships to existing work.  1  A sketch of these relationships and pointers into relevant literatures provide starting points for further reading.
 A1.1 Motivation Typical expressions can be regarded as approximate descriptions of things, whether ambiguous (pet, rather than cat) or partial (grey cat, rather than big grey cat). Given two expressions, one may want to combine them to form either a narrower description (by combining their information) or a broader description (by combining their scope). Although many other operations are possible (averaging, perhaps, or extrapolation), narrowing and broadening are of fundamental importance, and in many semantic domains, quite useful. They can be construed as operations on a formal or approximate semantic lattice. A1.1.1 Why Unification (Meet, Intersection, Narrowing, Specialization)? In symbolic logic, expressions correspond to points in a lattice (defined below), and unification is an operation that combines two expressions to form a more specific expression by combining their compatible information.  1  In generic lattices, the corresponding operation is termed meet, which in many contexts can be regarded as an intersection of sets or regions. Unification combines compatible information; failures of unification identify clashes.  2  The Prolog language illustrates how unification and failures can enable reasoning and proof. A1.1.2 Why Anti-Unification (Join, Union, Broadening, Generalization)? Alternatively, two expressions may provide (partial) descriptions of two entities of the same kind. Here, a natural goal is to describe properties common to all things of that kind; clashes between aspects of their descriptions indicate that those aspects are not definitional. In a lattice of expressions, this form of generalization is termed antiunification,  3  which increases generality by discarding clashing or unshared information. In generic lattices, the corresponding operation is termed join, 1. More precisely, unification is an operation that yields the most general instance of such an expression. For an application-oriented overview, see  Knight (1989) . 2. When unification attempts to combine partial information about a single entity, it is natural for inconsistent information to imply failure. which in many instances corresponds to a union of sets or regions.  1  Antiunification has applications in NLP and can be used to form generalizations and analogies in mathematics.  2  In neural ML, approximations of antiunification could potentially inform priors for a distribution of unseen instances of a class.
 A1.1.3 Hypotheses Regarding Roles in QNR Processing Considerations explored in this appendix suggest a range of hypotheses regarding potential roles for approximate lattice representations and operations in QNR processing: • That lattice representations and operations ("lattice properties") can, in fact, be usefully approximated in neural systems. • That learning to approximate lattice properties need not impair general representational capacity. • That approximations of meet and join operations will have broad value in semantic processing. • That lattice properties can be approximated to varying degrees, providing a smooth bridge between formal and informal representations. • That lattice properties can regularize representations in ways that are useful beyond enabling approximate meet and join. • That approximations of lattice representations and operations are best discovered by end-to-end learning of neural functions. • That explicit, approximate satisfaction of lattice identities can provide useful auxiliary training tasks.
 A1.2 Formal Definitions A mathematical lattice is a partially ordered set that can in many instances be interpreted as representing the "inclusion" of "subsumption" of one element by another. Axiomatically, a lattice has unique meet and join operations, ∧ and ∨: the meet operation maps each pair of elements to a unique greatest lower bound (infimum), while join maps each pair of elements to a unique least upper bound (supremum). Like the Boolean ∧ and ∨ operators (or the set operators ∩ and ∪), meet and join are associative, commutative, idempotent, and absorptive. A bounded lattice will include a unique bottom element, "⊥" (in the algebra of sets, ∅; here a universally incompatible meaning, "<nil>"), and a unique top element, " " (in the algebra of sets, U; here, an all-embracing generalization, "<any>"). In a formal infix notation, operators on a bounded lattice satisfy these identity axioms: Idempotence: A ∧ A = A ∨ A = A Commutativity: A ∧ B = B ∧ A, A ∨ B = B ∨ A Associativity: A ∧ (B ∧ C) = (A ∧ B) ∧ C, A ∨ (B ∨ C) = (A ∨ B) ∨ C Absorptivity: A ∧ (A ∨ B) = A ∨ (A ∧ B) = A Boundedness: A ∧ = A, A ∨ = , A ∨ ⊥ = A, A ∧ ⊥ = ⊥ If ∧ and ∨ are implemented as functions (rather than as structural features of a finite data structure), A ∧ B → C and A ∨ B → D will necessarily satisfy the unique-infimum and unique-supremum conditions. Commutativity can be ensured by algorithmic structure, independent from learned representations and operations; idempotence, associativity and absorptivity perhaps cannot. Boundedness is straightforward. 1
 A1.3 Lattice Structure in NL Semantics The fit between lattice orders and NL semantic structures is well known,  1  and the term "semantic lattice" has been applied not only to word and symbolbased representations, but to neural representations of images.  2  In studies of NL, both "concepts" and "properties" have been modeled in lattice frameworks, applications that speak in favor of explicit lattice structure in NL + frameworks.
 A1.3.1 Models of NL Semantics: A Lattice of Concepts Formal concept analysis 3 can recover "concept lattice" structures from text corpora, and these structures have been argued to be fundamental to information representation. Set-theoretic approaches associate formal objects with formal attributes, and construct a subsumption lattice over sets of defining attributes. Formal concept analysis has been extended to fuzzy structures in which possession of an attribute is a matter of degree.  4  Note that lattice relationships depend on context: In the context of households, the join of "cat" and "dog" might be "pet", but in the context of taxonomy, the join would be "carnivora". In practice, expressions containing "cat" and "dog" would be considered not in isolation, but in some context; in NLP, context would typically be represented dynamically, as part of a computational state; in QNR processing, context could be included as an abstractive vector attribute (see Section 8.3.5).
 A1.3.2 Models of NL Semantics: A Lattice of Properties Properties of things may have values distributed over a continuous range, and properties associated with something may themselves specify not a precise value, but a range within the range: In NL, "light gray" does not denote a precise color, and inference from an description of a "light gray object" may only loosely constrain its reflectance. Relationships among descriptions that specify ranges of properties may correspond to an interval lattice (Section A1.5.3). 1
 A1.4 Logic, Constraint Systems, and Weak Unification The previous section focused on lattice relationships among individual entities, but such entities can also serve as attributes in expressions or general graphs, enabling the definition of expression-level lattice operations. Lattices over expressions in which attributes themselves have non-trivial lattice orders can provide powerful, tractable representations in logic and constraint programming. Computation over such representations can be extended to include weak unification.
 A1.4.1 Logical Expressions, Logic Programming Logic programming performs reasoning based on syntactic unification of expression-trees in which symbols represent attributes of leaf nodes (variables or constants) or interior nodes (e.g., functions, predicates, relations, and quantifiers). In Prolog, expressions are limited to a decidable fragment of first-order logic; more powerful unification-based systems include λ terms and can support proof in higher-order logics.  2  Informally, first-order terms A and B unify to yield an expression C provided that all components of A (subexpressions and their attributes) unify with corresponding features or variables of B; C is the expression that results from the corresponding substitutions. Function and predicate symbols unify only with identical symbols; constants unify with variables or identical constants; variables unify with (and in the resulting expression, are replaced by) any structurally corresponding constant, variable, or subtree.  3  Aside from variables that match subtrees, tree structures must match. As required, unification of two expressions then either fails or yields the most general expression that specializes both. Informally, expressions A and B anti-unify, or generalize, to yield C provided that C contains all features that A and B share, and contains variables wherever A and B differ in attributes or structure, or where either contains a variable. C is the unique, most specific expression that can unify with any expression that can unify with either A or B. Thus, join/anti-unification of two expressions yields the most specific expression that generalizes both.
 Relevance to QNR systems: QNR frameworks can embed (at least) first-order logic expressions and enable their unification, provided that some attributes (representing constants, functions, etc.) can be compared for equality, while others (acting as variables 1 ) are treated as features that match any leaf-attribute or subexpression. Accordingly, QNR frameworks augmented with appropriate algorithms can support logical representation and reasoning. This is a trivial consequence of the ability of QNRs to represent arbitrary expressions, in conjunction with freedom of interpretation and the Turing completeness of suitable neural models. Logical expressions and logic programming are, however, instances of richer systems-also within the potential scope of QNRs-that represent constraint systems and support constraint logic programming.
 A1.4.2 Constraints and Constraint Logic Programming In constraint logic programming, 2 representations are extended to include constraints more general than equality of components and binding to unconstrained variables, and unification is extended to (or replaced by) constraint satisfaction. The application of constraints narrows the variable domains, and unification fails when domains become empty. The attributes of constraint expressions have a lattice structure, as do the expressions that contain them, and constraint expressions can be narrowed and generalized though unification and anti-unification  (Yernaux and Vanhoof 2019) . The potential complexity of constraints spawns a vast menagerie of constraint systems, algorithms, and constraint logic programming algorithms. Provided that expressions can include both single-element domains and variables able to bind subexpressions, constraint logic programming can subsume conventional logic programming.
 Relevance to QNR systems: As with logic and logic programming, the generality of QNR representations and neural computation implies the ability to represent constraint 1. Convergent arcs in DAG expressions model the behavior of named variables that occur in multiple locations. It should be noted that unification can produce and operate on cyclic graphs unless this is specifically excluded; see  Smolka (1992) . systems and constraint-based computation, including constraint logic programming. The generalization from logic to constraints is important to semantic representational capacity: Expressions can often be interpreted as denoting regions in a semantic space,  1  and combinations of expressions can combine constraints. Constraint logic programming provides an exact formal model of computation based on this semantic foundation.
 A1.4.3 Weak and Soft Lattice Operations "Never express yourself more clearly than you are able to think" -Niels Bohr The literature describes a range of models of NL semantics and reasoning based on a range of approximate unification operations. These typically replace equality of constants (functions, etc.) with similarity: In "soft unification" (as the term is typically used in the literature) the operation succeeds if similarity (for example, cosine similarity between vectors,  Arabshahi et al. (2021) ) is above some threshold, and success may yield either conventional binding of variables to values  (Campero et al. 2018) , or merged representations of values  (Cingillioglu and Russo 2020) . "Weak unification" may produce a "unification score" that indicates the quality of unification; these scores can be carried forward and combined to score the quality of multi-step inference operations.  2  As used in the present context, the term "soft unification" subsumes both "weak" and "soft" unification as used in the literature, and entails combining representations of values in a way that approximates unification operations in constraint logic programming. Thus, "softness" allows operations that violate strict lattice identities.  3  The intended class of QNR (soft-)unification operations follows constraint logic programming in generalizing from the binding of constants to named (in effect, shared) unconstrained variables to include the narrowing of shared (in effect named), potentially constrained attributes. As in logic programming, QNR unification will permit the unification of subexpressions with variable-like attributes, but differs in that constrained attributes (unlike variables) may impose semantically non-trivial constraints on permissibility and on the content of resulting subexpressions (Section A1.6.4).
 A1.5 Exact Lattice Operations on Regions Although embeddings in vector/graph representations denote points in semantic spaces, their semantic interpretations will typically correspond to regions in lower-dimensional spaces. Comparisons to symbolic and more general constraint representations can provide insights into potential QNR representations and reasons for expecting their lattice properties to be inexact.
 A1.5.1 Conventional Symbolic Expressions In conventional expression graphs, attributes comprise symbols that represent points together with symbols that represent unbounded regions in the space of expressions. Thus, individual attribute subspaces are simple, have no spatial structure, and accordingly exhibit trivial behavior under unification and anti-unification.
 A1.5.2 General Region Representations and Operations Leaf attributes can represent regions in R n , but options for their unification and generalization may be representation-dependent. The conceptually straightforward definition is both trivial and problematic: Treating regions as sets of points (A ∨ B = A ∪ B) in effect discards spatial structure, and with it the potential for non-trivial generalization. Further, if region representations have limited descriptive capacity, then the result of generalizing a pair of attributes by set union cannot in general be represented as an attribute.  1  Alternatively, the generalization of two volumes might be defined as their convex hull. Generalization of convex regions yields convex regions, and inclusion of points outside the initial volumes reflects spatial structure and a plausible notion of semantic generalization. Unfortunately, this definition can also fall afoul of limited descriptive capacity, because the convex hull of two regions can be more complex than either. 1
 A1.5.3 Interval (Box) Representations and Operations There are region-representations for which lattice operations are exact, for example, one-dimensional intervals in R 2 and their generalization to axisaligned boxes in R n .  3  Unification of a pair of box-regions yields their intersection; anti-unification yields the smallest box that contains both. Axis-aligned box regions can be represented by vectors of twice the spatial dimensionality (for example, by pairing interval centers with interval widths), and lattice operations yield representations of the same form. Interval-valued attributes have been applied in constraint logic programming.  4  Generalization through anti-unification of intervals has a natural semantic interpretation: Points in a gap between intervals represent plausible members of the class from which the intervals themselves are drawn. 1. To illustrate, the convex hull of two spheres need not be a sphere, and the convex hull of two polytopes may have more facets than either. Intersections can also become more complex (see Jaulin 2006).  Clark et al. (2021) .
 Discussed for example in 3. Affine (and other) transformations of a space and its regions can of course maintain these properties. 4.  Benhamou and Older (1997)  and  Older and Vellino (1990)
 A1.6 Approximate Lattice Operations on Regions If box lattices are exact, why consider approximate operations on more general regions? The basic intuition is that representations entail trade-offs, and that greater flexibility of form is worth some degree of relaxation in the precision of unification and generalization. Boxes have sharp boundaries, flat facets, and corners; natural semantic representations may not. Intervals in a space of properties may correspond to natural semantic concepts, yet orthogonality and global alignment of axes may not. In the present context, it is important to distinguish two kinds of approximation: As discussed in Section A3.4, effective QNR frameworks must be able to express, not only precise meanings, but ranges or constraints on meaning-a kind of approximation in the semantic domain that differs from approximation of lattice properties. Ranges of meanings can be represented as regions in semantic spaces, while region-representations that approximate precise meanings can precisely satisfy the lattice axioms.
 A1.6.1 Continuous-Valued Functions In addition, general considerations motivate representing the membership of points in semantic classes with continuous-valued functions, rather than functions having values restricted to {1, 0}. Such representations invite further approximations of lattice properties.
 A1.6.2 Region Shapes and Alignment It is natural to exploit the flexibility of neural network functions to represent generalized region shapes, and to use the ability of fully connected layers to free representations from preferred axis alignments and thereby allow exploitation of the abundance of nearly orthogonal directions in high dimensional spaces. Learned attribute representations need not describe box-like regions or share meaningful alignment in different regions of a semantic space.
 A1.6.3 Satisfaction of Lattice Axioms as an Auxiliary Training Task Given the value of lattice structure in representations, it is natural to speculate that promoting lattice structure through inductive bias may aid performance in a range of semantic tasks. Auxiliary training tasks in which losses explicitly measure violations of lattice properties may therefore be useful components of multitask learning. 1
 A1.6.4 Unifying Attributes with Expressions As noted above, conventional symbolic expressions allow unification of unconstrained variables with subexpressions, while in the QNR context, it is natural to seek to unify subexpressions with constrained variables-attributes that represent semantic regions. The outlines of desirable behavior are clear, at least in some motivating cases: Consider a pair of graphs with subexpressions A and B in corresponding locations. Let A be a vector representation (e.g., describing a generic greystriped animal with sharp claws), while a corresponding subexpression B is a vector/graph representation (e.g. that contains components that describe a cat's temperament, ancestry, and appearance). Unification should yield a vector/graph representation in which the properties described by vector A constrain related properties described anywhere in expression B (e.g., the cat's appearance, paws, and some aspects of its ancestry) If some component of B specifies a black cat, unification fails. In this instance, generalization should yield a vector representation that does not clash with properties described in either A or B, while discarding properties that are not shared.
 A1.6.5 Learning Lattice-Oriented Algorithms Classic algorithms for unification and generalization implement particular patterns of information flow, intermediate representations, and iterative, conditional computation. Work on supervised neural algorithmic learning illustrates one potential approach to adapting such algorithms to neural computation, an approach that supervises the learning of algorithmic structure while allowing end-to-end learning of rich representations and corresponding decision criteria. 1
 A1.7 Summary and Conclusions Lattice structure is found in formal systems and (to some extent) in NL, and it seems both natural and desirable in QNR/NL + representations. Although lattice structure is not a criterion for upgrading NL to NL + representations, substantial adherence to lattice structure could potentially improve expressive capacity in a systemic sense and need not imply the rigidity of fully formal representations. Because linguistic representations and reasoning are themselves approximate, there seems little reason to sacrifice representational flexibility in order to enforce exact and universal satisfaction of the lattice axioms. A QNR framework that embraces both precise and approximate lattice relationships can enable both formal and informal applications of those relationships to formal and informal reasoning. The literatures on conventional and constraint logic programming illustrate the power and computational tractability of algorithms based on systems of this kind. The potential benefits of approximate lattice structure may emerge spontaneously, but can also be pursued by inductive bias, including training that employs satisfaction of lattice axioms as an auxiliary task in multitask learning. A2 Tense, Aspect, Modality, Case, and Function Words Tables of examples illustrate expressive constructs of natural languages that do not reduce to nouns, verbs, and adjectives. Natural languages express a range of meanings through closed-class ("function") words, and express distinctions of tense, aspect, modality, and case though both function words and morphological features. Section 5.3 discusses the roles and importance of these constructs; this appendix provides several brief tables of examples. Languages differ in the distinctions that they can compactly express, while properties like "remoteness", "completion", and "causation" invite continuous representations.) Perfect tenses -completed in past, present, or future ("had/has/will have" finished) Continuous tenses -ongoing in past, present, or future ("was/am/will be" working) Past perfect continuous -previously ongoing in the past ("had been working") Future perfect continuous -previously ongoing in the future ("will have been working") Remote perfect -completed in the remote past (Bantu languages 1 ) Resultative perfect -completed past action causing present state (Bantu languages) Section 6 discussed QNR architectures as a framework, considering potential components, syntactic structures, and their semantic roles. The present section extends this discussion to explore in more detail how anticipated QNR frameworks could subsume and extend the expressive capabilities of natural languages to fulfill the criteria for NL + . Key considerations include facilitating representation learning, upgrading expressiveness, improving regularity, and enabling more tractable reading, interpretation, and integration of content at scale. Potential NL/NL + relationships discussed here are not proposals for handcrafted representations, nor are they strong or confident predictions of the results of representation learning. The aim is instead to explore the scope and strengths of QNR expressive capacity, with potential implications for design choices involving model architectures, inductive bias, and training tasks. Where neural representation learning provides NL + functionality by different means, we should expect those means to be superior.
 A3.1 Upgrading Syntactic Structure To be fit for purpose, NL + syntactic structures must subsume and extend their NL counterparts while improving computational tractability: • To subsume NL syntactic structures, NL + frameworks can embed NL syntax; as already discussed, this is straightforward. • To extend NL syntactic structures, NL + frameworks can support additional syntactic structure; as already discussed, this can be useful. • To improve tractability in learning and inference, NL + frameworks can improve semantic compositionality, locality, and regularity. This potential NL/NL + differential will call for closer examination.
 A3.1.1 Making Syntactic Structure Explicit Explicit representations avoid the computational overhead of inferring structure from strings, as well as costs (and potential failures) of non-local disambiguation, e.g., by using DAGs to represent coreference. Improved locality and tractability follow.
 A3.1.2 Extending the Expressive Scope of Syntactic Structure Explicit graphs can enable the use of structures more diverse and complex than those enabled by natural language and accessible to human cognition.  1  Writers grappling with complex domains may employ supplementary representations (diagrams, formal notation), or may simply abandon attempts to explain systems and relationships that involve deeply nested structures, heterogeneous patterns of epistemic qualification, and complex patterns of coreference-all of which must in NL be encoded and processed as sequences. More general representations can directly describe relationships that are more complex yet readily accessible to machines.
 A3.1.3 Extending the Concept of "Syntactic Structure" "Natural language syntax" as understood by linguists is often in practice supplemented with visually parsable structures such as nested text (outlines, structured documents) and tables that represent grids of relationships. Diagrams may embed text-labels as attributes of graphs that represent networks of typed relationships (taxonomy, control, causation, enablement, etc.); program code is adjacent to NL, yet goes beyond NL concepts of syntax. Implementations that support explicit bidirectional links can model the structure of literatures that support not only "cites" but also "cited by" relationships.
 A3.1.4 Collapsing Syntactic Structure In neural-network computation, vector operations are basic, while graph operations may impose additional computational overheads. This motivates the use of embeddings in preference to graph expressions, 2 which in turn 1. What is in some sense accurate linguistic encoding does not ensure successful communication: For example, as a consequence of working-memory constraints, humans suffer from "severe limitations" in sentence comprehension  (Lewis, Vasishth, and Van Dyke 2006) . highlights the value of highly expressive lexical-level units. In addition, encoding different meanings as differences in syntactic structure can impede alignment and comparison of expressions; When individual embeddings can express a range of meanings that NL would represent with different syntactic structures, those syntactic differences disappear.
 A3.2 Upgrading Lexical-Level Expressive Capacity Where possible, we would like to replace the syntactic compositionality of words with the simpler, arithmetic compositionality of vectors. Because the best syntax is no syntax, it is worth considering what kinds of semantic content can be represented in vector spaces without relying on graph structure. The following discussion notes several kinds of semantic structure that exist in NL, can be represented by embeddings, and have emerged (spontaneously and recognizably) in neural models.
 A3.2.1 Subsuming and Extending Content-Word Denotations As careful writers know, there often is no single word that accurately conveys an intended meaning, while to unpack an intended meaning into multiple words may be too costly in word-count or complexity. Lexical-level embeddings can shift the ambiguity/verbosity trade-off toward expressions that are both less ambiguous and more concise.  1  Embeddings can place content-word meanings in spaces with useful semantic structure. Word embeddings demonstrate learnable structure in lexical-level semantic spaces. The geometry of learned word embeddings can represent not only similarity, but analogy, 2 and representations of semantic differentials across vocabularies 3 can be identified in language models. Unfortunately, the role of NL word embeddings-which must represent polysemous, context-dependent words-precludes clean representation of word-independent semantic structure. Vector spaces that represent meanings rather than words can provide semantic structure that is more reliable 1. Concise expression is of course less necessary in the context of indefatigable machine intelligence. 2. As discussed in Chen, Peterson, and Griffiths (2017). 3. Daniel Kahneman's doctoral dissertation examines semantic differentials  (Kahneman 1961) . Semantic structure embeddable in vector spaces has been extensively studied by both linguists and ML researchers (e.g., see  Messick (1957) ,  Sagara et al. (1961 ), Hashimoto, Alvarez-Melis, and Jaakkola (2016 ), and Schramowski et al. (2019 ). and useful, hence patterns observed in vector representations of natural language provide only limited insight into the potential generality and utility of semantically structured vector spaces. Embeddings can disambiguate, interpolate, and extend vocabularies of content words. Because points in an embedding space could be used to directly designate every word in every NL vocabulary-with vast capacity to spare-embeddings can be strictly more expressive than NL words. Again taking NL as a baseline, embeddings offer the advantage of avoiding polysemy, maladaptive word ambiguity, 1 and a large (even comprehensive?) range of recognizably missing word meanings (the frustrating-thesaurus problem). In suitable semantic spaces, points within and beyond the rough equivalents of NL word clusters can, at a minimum, interpolate and extend the rough equivalents of NL vocabularies.
 Embeddings can extend vocabularies by folding modifier-expressions into noun and verb spaces. The ability to collapse a range of multi-word expressions into embeddings is equivalent to extending vocabularies: In a straightforward example, adjectives and adverbs can modify the meanings of nouns and verbs to produce different lexical-level meanings. These modifiers often represent cross-cutting properties 2 that can describe things and actions across multiple (but not all) domains. Although words with modifiers can be viewed as extensions of NL vocabularies, expressions of limited size necessarily leave gaps in semantic space; continuous vector embeddings, by contrast, can fill regions of semantic space densely. As a consequence of the above considerations, a function of the form NLencode: (lexical-NL-expression) → (lexical-embedding) can exist, but not its inverse. NL-encode is neither injective nor surjective: Multiple NL expressions may be equivalent, and typical NL + expressions will have no exact NL translation.
 Directions in embedding spaces can have interpretable meanings. Linguists find that NL words can with substantial descriptive accuracy be positioned in spaces in which axes have conceptual interpretations 3 or reflect semantic differentials. In NL, modifiers commonly correspond to displacements with components along these same axes. It is natural to interpret modifier-like vector components differently in different semantic domains.  1  As noted previously, the interpretation of directions in a (sub)space that corresponds to differentials applicable to entities would naturally depend on location-on which regions of a (sub)space correspond to which kinds of entities.  2  In a semantic region that describes persons, for example, directions in a subspace might express differences in health, temperament, age, and income, while in a semantic region that describes motors, directions in that same subspace might express differences in power, torque, size, and efficiency.
 Lexical level and syntactic compositionality are complementary. As suggested above, vector addition of modifiers and other differentials can express compositional semantics within embeddings. Exploiting this capacity does not, of course, preclude syntactic compositionality: QNRs can support compositional representations through both vector-space and syntactic structure. Without predicting or engineering specific outcomes, we can expect that neural representation learning will exploit both mechanisms.
 A3.2.2 Image Embeddings Illustrate Vector Compositionality in Semantic Spaces Image embeddings can provide particularly clear, interpretable examples of the expressive power-and potential compositionality-of continuous vector representations. (Section 9.2 discusses image and object embeddings, not as examples of representational power, but as actual lexical units.) Humans are skilled in perceiving systematic similarities and differences among faces. Diverse architectures (e.g., generative adversarial networks, variational autoencoders, and flow-based generative models 3 ) can produce face embeddings that spontaneously form structured semantic spaces: These models can represent faces in high-dimensional embedding spaces that represent kinds of variations that (qualitatively, yet clearly) are both recognizable and systematic.  1  Many papers present rows or arrays of images that correspond to offsets along directions in embedding spaces, and these naturally emphasize variations that can be named in captions (gender, age, affect. . . ), but a closer examination of these images also reveals systematic variations that are less readily described by words. Words or phrases of practical length cannot describe ordinary faces such that each would be recognizable among millions. A single embedding can.  2  Similar power can be brought to bear in a wider range of lexical-level representations. 3
 A3.3 Subsuming and Extending Function-Word/TAM-C Semantics As noted in Section 5.3.4 TAM-C meanings can be encoded in either word morphology or function (closed-class) words. Some TAM-C modifiers are syntactically associated with lexical-level units; others are associated with higher-level constructs. TAM-C modifiers that represent case (e.g., nominative, accusative, instrumental, benefactive; see Table  A2 .4) can directly describe the roles of things denoted by words (e.g., acting vs. acted upon vs. used), but case also can indirectly modify the meaning of a word-a rock regarded as a geological object differs from a rock regarded as a tool.  4  Other TAM-C modifiers express semantic features such as epistemic confidence, sentiment, and use/mention distinctions. In NL, statement-level meanings that are not captured by available TAM-C modifiers may be emergent within an expression or implied by context; by compactly and directly expressing meanings of this kind, expression-level embeddings can provide affordances for improving semantic locality, compositionality, and clarity.  5  Some function words connect phrases: These include words and combinations that express a range of conjunctive relationships  (and, or, and/or, therefore, then, still, however, because, despite, nonetheless. . . )  and capability/intention related relationships  (can, could, should, would-if, could-but, would-if-could, could-but-shouldn't. . . ) . Consideration of the roles of these words and constructs will show that their meanings are distributed over semantic spaces, and the above remarks regarding the use of embeddings to interpolate and extend NL meanings apply.  1  In NL, tense/aspect modifiers express distinctions in the relative time and duration of events, and because these modifiers reference a continuous variable-time-they can naturally be expressed by continuous representations. Likewise, epistemic case markers in NL (indicative, inferential, potential) implicitly reference continuous variables involving probability, evidence, and causality. Note that much of function-word/TAM-C space represents neither kinds nor properties of things, and is sparsely populated by NL expressions. The use of embeddings to interpolate and extend meanings in these abstract semantic roles could greatly improve the expressive capacity of QNR/NL + frameworks relative to natural languages.
 A3.4 Expressing Quantity, Frequency, Probability, and Ambiguity Discrete NL constructs express a range of meanings that are more naturally expressed in continuous spaces: These include number, quantity, frequency, probability, strength of evidence, and ambiguity of various kinds. NL can express specific cardinal, ordinal, and real numbers, and absolute concepts such as none or all, but many other useful expressions are either crude (grammatical singular vs. plural forms) or ambiguous (e.g., several, few, many, some, most, and almost all, or rarely, frequently, and almost always). Note that intentional ambiguity is useful: Few does not denote a particular number, but a range of "small" numbers, either absolute (about 2 to 5, Munroe (2012)) or relative to expectations regarding some set of entities. This range of meanings (likewise for unlikely, likely, possibly, almost certainly, etc.) invites continuous representations in QNR frameworks.  2  Similar remarks apply to qualitative and probabilistic hedges (mostly, partially, somewhat, to some extent) qualifiers and often agent-centered epistemic qualifiers (presumably, if I recall correctly, it seems to me, as far as I know, in my opinion, etc.).  1  One would also like to be able to compactly express qualifiers like illustrative but counterfactual simplification, approximate description of a typical case, and unqualified statement but with implied exceptions: Today, the absence of universal idioms for expressing these meanings gives rise to gigabytes of fruitless, argumentative noise in internet discussions. The discussion above implicitly frames the expression of ambiguity (etc.) as a task for lexical units in phrases, following the example of NL. There are advantages, however, to folding ambiguity (etc.) into embeddings that represent, not points, but regions in a semantic space. A shift from pointto region-oriented semantics allows systems of representations that can approximate mathematical lattices (Appendix A1) and lattice-based inference mechanisms like Prolog and constraint logic programming (Section A1.4). These mechanisms, in turn, provide semi-formal approaches to matching, unification, and generalization of representations, with applications outlined below and explored further in Section 8.4.3.
 A3.5 Facilitating Semantic Interpretation and Comparison Relative to NL, NL + frameworks offer potential advantages that include 1. Greater expressive capacity 2. More tractable interpretation 3. More tractable comparison. The preceding sections have discussed advantages of type (1) that stem largely from improvements at the lexical level. The present section will consider how condensation, localization, regularization of expression-level representations can provide advantages of types (  2 ) and  (3) . A central theme is the use of tractable, uniform, embedding-based representations to provide expressive capacity of kinds that, in NL, are embodied in less tractable-and often irregular-syntactic constructs.
 A3.5.1 Exploiting Condensed Expressions As noted above, embeddings can condense many noun-adjective, verb-adverb, and function word constructs, facilitating interpretation by making their semantic content available in forms not entangled with syntax. Further, embeddings can be compared through distance computations, 1 potentially after projection or transformation into task-and context-relevant semantic spaces. These operations are not directly available in NL representations.
 A3.5.2 Exploiting Content Summaries Content summaries (Section 8.3.4) can cache and amortize the work of interpreting expressions. The value of summarization increases as expressions become larger: An agent can read a book (here considered an "expression") to access the whole of its information, but will typically prefer a book accompanied by a summary of its topic, scope, depth, quality, and so on. Semantically optional summaries (perhaps of several kinds) can facilitate both associative memory across large corpora (Section 9.1.2) and shallow reading ("skimming") of retrieved content. Shallow reading, in turn, can enable quick rejection of low-relevance content together with fast, approximate comparison and reasoning that can guide further exploration. Where uses of information differ across a range of tasks, useful summaries of an expression may likewise differ.
 A3.5.3 Exploiting Context Summaries Although context summaries (Section 8.3.5), like content summaries, are in principle semantically redundant, they are substantially different in practice: Expressions are bounded, but an interpretive context may be of any size, for example, on the scale of a book or a body of domain knowledge. Thus, absent summarization, contextual information-and hence the meaning of an expression-may be far from local; with context summarization, meaning becomes more local and hence more strongly compositional.  2  1. Or intersection-and union-like operations in region-oriented semantics, see Appendix A1. 2. As noted elsewhere, current language models typically encode (costly to learn, difficult to share) summaries of global context-as well as knowledge of narrower contexts and even specific facts-while their inference-time activations include (costly to infer) summaries of textually local context. Learning and sharing task-oriented summaries of both broad and narrow contexts could provide complementary and more efficient functionality. Embeddings can provide the most compact summaries, but more general QNRs could provide richer yet still abstractive information. Some use-patterns would place specific semantic content in narrow context representations, and schematic semantic content in expressions that can contribute to descriptions in a wide range of contexts. In interpreting an expression, the effective, interpreted meanings of its embeddings would be strongly dependent on its current context. A programming language analogy would be the evaluation of expressions conditioned on binding environments, but in the QNR case, employing embeddings in place of conventional values and variables, 1 and employing neural models in place of symbolic interpreters.
 A3.5.4 Aligning Parallel and Overlapping Expressions Content summaries can facilitate comparison and knowledge integration in the absence of full structural alignment. In the limiting case of a complete structural mismatch between QNR expressions, their summary embeddings can still be compared. To the extent that high-level structures partially align, comparison can proceed based on matching to some limited depth. At points of structural divergence, comparison can fall back on summaries: Where subexpressions differ, their summaries (whether cached or constructed) can be compared; likewise, a subexpression summary in one expression can be compared to a lexical embedding in the other. Appendix A1 discusses how matches can be rejected or applied through soft unification. Upgrading the expressive power of lexical-level embeddings can facilitate structural alignment by shifting burdens of semantic expressiveness away from syntax: Condensing simple expressions into embeddings avoids a potential source of irregularity, the sequential order of lexical-level elements need not be used to encode emphasis or semantic priority, and the semantic differences between active and passive voice need not be encoded through differences in syntax and grammar. Accordingly, similar meanings become easier to express in parallel syntactic forms. If expressions with similar semantic content-representing similar things, properties, relationships, roles-are cast in a parallel syntactic form, they become easier to compare. Regularizing structure need not sacrifice expressive capacity: Expression-level nuances that in NL are expressed through alternative syntactic forms can quite generally be represented by embeddings that modify expression-level meaning. Thus, structural regularization, enabled by expressive embeddings and explicit graphs, can facilitate structural alignment and semantic comparison of related expressions. In addition, however, structural regularization can facilitate transformations among alternative canonical forms, potentially facilitating translation between representational dialects in heterogeneous NL + corpora. Regularization need not adhere to a uniform standard.
 A4 Compositional Lexical Units Embeddings with explicit compositional structure may offer advantages in efficient learning and generalization. Section 7.1.3 noted that the properties of vector addition can enable semantic compositionality without recourse to syntax; the present discussion examines the potential role of explicit forms of compositionality in learning and representation. Among the considerations are: • Efficiently representing large vocabularies • Parallels to natural language vocabularies • Parallels to NLP input encodings • Inductive bias toward efficient generalization The usual disclaimer applies: The aim here is neither to predict nor prescribe particular representations, but to explore what amounts to a lower bound on potential representational capabilities. Explicit vector compositionality, would, however, require explicit architectural support.
 A4.1 Motivation and Basic Approach Because few neural models write and read large stores of neurally encoded information, prospects for building large QNR corpora raise novel questions of storage and practicality. Section A5.5 outlines an approach (using dictionaries of composable vector components) that can be compact, efficient, and expressive. The present discussion considers how and why explicit compositionality within vector representations may be a natural choice for reasons other than efficiency. A key intuition is that sets of lexical components (like morphemes in natural languages) can be composed to represent distinct lexical units (like words and phrases that represent objects, actions, classes, relationships, functions, etc.  1  ), and that composite lexical units can best be regarded and implemented as single vectors in QNRs. For concreteness, the discussion here will assume that lexical-component embeddings are concatenated to form lexical-unit embeddings, 2 then melded by shallow feed-forward transformations to form unified representations. A key underlying assumption is that discrete vocabularies are useful, whether to encode embeddings compactly (Appendix A5), or to provide an inductive bias toward compositional representations. (Note that compact encodings can combine discrete vectors with continuous scalars to designate points on continuous manifolds; see Section A5.5.4).
 A4.2 Efficiently Representing Vast Vocabularies The on-board memories of GPUs and TPUs can readily store >10 7 embeddings for fast access.  3  This capacity is orders of magnitude beyond the number of English words, yet using these embeddings as components of lexical units can provide much more. If sets of potential lexical-unit embeddings are Cartesian products of sets of lexical-component embeddings, then potential vocabularies are enormous. Cartesian-product spaces in which (for example) 2 to 4 components are drawn from 10 7 options would offer 10 14 to 10 28 potential lexical-unit embeddings; of these, one can expect that a tiny fraction-yet an enormous number-would be potentially useful in describing the world. To represent expressions as strings (Appendix A5), 3 bytes of key information per lexical component would be ample.
 A4.3 Parallels to Natural Language Vocabularies Lexical units in NL vocabularies are commonly built of multiple morphemes, including roots, affixes,  4  and words embedded in compounds or multiword units.  5  1. As already noted, this use of "lexical units" abuses standard terminology in linguistics. 2. Concatenation can be modeled as addition of blockwise-sparse vectors, and addition of dense vectors would arguably be superior. However, using addition in place of concatenation would (in application) increase storage costs by a small factor, and would (at present) incur a substantial explanatory cost. 3. See Section A5.5.5. 4. English builds on >1300 roots and affixes (prefixsuffix.com 2008).
 5. Here used in the standard linguistic sense (also termed "lexical items"). If we view NL as a model for potential NL + constructs, then it is natural to consider analogues of morphemes in embedding spaces, and to seek lexical-level semantic compositionality through explicit composition of building blocks in which the meanings of components are, as in NL, a joint result of their combination. This approach can make lexical components themselves targets of learning and thereby expand the scope of useful, accessible vocabulary. Medical terminology illustrates the role of lexical-level compositionality in building a language adapted to a rich domain.  1  Most medical terms are built of sequences of parts ("cardio+vascular") or words ("primary visual cortex").  Wikipedia (2021)  lists 510 word parts (prefixes, roots, and suffixes) used in medical terminology, while a large medical dictionary defines ~125,000 distinct, often multi-word terms  (Dorland 2007 ), a number that approaches an estimate (~200,000) of the number of words in the English language as a whole.  2  Refining and expanding the store of applicable lexical components from hundreds or thousands to millions or more would greatly increase the potential semantic resolution of medical language at a lexical level. Medicine, of course, occupies only a corner of a semantic universe that embraces many fields and extends far beyond what our words can readily describe.
 A4.4 Parallels to NLP Input Encodings There are substantial parallels and contrasts between input encodings in current NLP and compositional embeddings in potential QNR processing systems Table  A4 .1): • In the proposed mode of QNR processing, inputs are lexical components concatenated to form lexical units; in Transformer-based NL processing, inputs are words and subwords extracted from strings through tokenization. • In the QNR case, a very large vocabulary of lexical components is composed to form a vast Cartesian-product space of lexical units; in the NLP case, a smaller vocabulary of lexical units is built from word fragments and common words (in BERT, ~30,000 "wordpieces"). • In the QNR case, the composition of lexical units is determined by representation learning; In the NLP case, the decomposition of strings is determined by a tokenization algorithm. • NLP models dynamically infer (and attempt to disambiguate) lexicallevel representations from tokenized text; in QNR processing, input embeddings are explicit lexical-level products of previous representation learning. Thus, lexical-level QNR inputs are roughly comparable to hidden-layer representations in an NLP model.
 A4.5 Inductive Bias Toward Efficient Generalization The ability to represent specific lexical units as compositions of more general semantic components could potentially support both systematic generalization and efficient learning, including improved sample efficiency. An important lexical component will typically occur far more frequently than the lexical units that contain it, and learning about a component can provide knowledge regarding lexical units that have not yet been encountered.  2  Indeed, without the inductive bias provided by composition, it might be difficult to learn truly large vocabularies that form well-structured semantic spaces. As an NL illustration of this principle, consider "primary visual cortex" again: A reader who knows little or nothing of neuroanatomy will know the general meanings of "primary" and "visual" and "cortex", having encountered these terms in diverse contexts. With this knowledge, one can understand that "primary visual cortex" is likely to mean something like "the part of the brain that first processes visual information", even if this term has never before been seen. A more refined understanding can build on this. This familiar principle carries over to the world of potential QNR representations, where exploitation of compositional lexical-level semantics promises to support learning with broad scope and effective generalization.
 A4.6 A Note on Discretized Embeddings In typical applications, reading is far more frequent than writing, hence mapping continuous internal representations to discrete external representations need not be computationally efficient. This output task can be viewed as either translation or vector quantification. Lexical components that are distant from existing embeddings may represent discoveries worth recording in an expanded vocabulary. Because components populate continuous vector spaces, discretization is compatible with differentiable representation learning of components and their semantics.
 A5 Compact QNR Encodings String representations of QNRs, in conjunction with discretized vector spaces and graph-construction operators, can provide compact and efficient QNR encodings. "Premature optimization is the root of all evil." -Donald Knuth 1 NL + expressions can be implemented compactly by combining operator-based representations of graph structures with extensible dictionaries of discretized embeddings; the latter provide mechanisms for what can be regarded as lossy compression, but can also be regarded as providing a useful inductive bias (Section A4.5). The content of QNR corpora can be represented as byte  1. Knuth (1974) . Because compression is (at most) a downstream research priority, Knuth's warning against premature optimization is relevant and suggests that the value of this appendix is questionable. There is, however, good reason to explore the scope for efficient, scalable implementations: A sketch of future options can help to free exploratory research from premature efficiency concerns-or worse, a reluctance to consider applications at scale. strings approximately as compact as NL text by exploiting key-value stores of embeddings and graph-construction operators.  1  The memory footprint these stores need not strain the low-latency memory resources of current machines. The purpose of this appendix is not to argue for a particular approach, but to show that a potential challenge-the scale of QNR storage footprints-can be met in at least one practical way. Note that the considerations here have nothing to do with neural computation per se, but are instead in the domains of algorithm and data-structure design (often drawing on programming language implementation concepts, e.g., environments and variable binding). From a neural computation perspective, the mechanisms must by design be transparent, which is to say, invisible.
 A5.1 Levels of Representational Structure Prospective QNR repositories include representational elements at three levels of scale: • Embeddings: vector attributes at a level comparable to words • Expressions: graphs at a level comparable to sentences and paragraphs • References: graph links at the level of citations and document structures In brief, expressions are graphs that bear vector attributes and can include reference-links to other expression-level graphs. There is no important semantic distinction between expression-level and larger-scale graph structures; the key considerations involve interactions between scale, anticipated patterns of use, and implementation efficiency. A formal notation would distinguish between QNRs as mathematical objects (graph and attributes), QNRs as computational objects (inference-time data structures that represent graphs and attributes), and encodings that represent and evaluate to QNR objects in a computational environment. The following discussion relies on context to clarify meaning.
 A5.2 Explicit Graph Objects vs. String Encodings A relatively simple computational implementation of QNRs would represent embeddings as unshared numerical vectors,  1  and graphs as explicit data structures.  2  Repositories and active computations would share this direct, bulky representational scheme. Natural language expressions are more compact: NL words in text strings are far smaller than high-dimensional vector embeddings, and graph structures are implicit in NL syntax, which requires no pointer-like links. Inference-time representations of NL + expressions may well be bulky, but so are the inference-time representations of NL expressions in neural NLP. And as with NL, stored QNR expressions can be represented compactly as byte strings.
 A5.3 Compact Expression Strings A QNR corpus can be represented as a key-value store that contains embeddings (numerical vectors), operators (executable code), and encoded QNR expressions (strings that are parsed into keys and internal references). In this scheme, expressions encoded as byte-strings evaluate to expression-level QNR graphs that can include references that define graphs at the scale of documents and corpora. In more detail: • Keys designate operators, embeddings, or expression-strings in a keyvalue store. • Expression-strings are byte strings 3 that are parsed into keys and indices. • Indices designate subexpressions in a string. • Operators are graph-valued functions  4  of fixed arity that act on sequences of subexpressions (operands). • Subexpressions are keys, indices or operator-operand sequences. • Embeddings are graph attributes. 1. "Unshared" in the sense that each attribute-slot would designate a distinct, potentially unique vector object. 2. "Explicit" in the sense that each arc would be represented by a pointer-like reference. 3. Potentially bit strings. 4. An extended scheme (Section A5.5.4) allows vector-valued operators with vector and scalar operands. • Reference is shorthand for "expression-string key" (typically a stopping point in lazy evaluation).
 A5.3.1 From Strings to Syntax Graphs Parsing an expression-string is straightforward: Operators are functions of fixed arity, the initial bytes of an expression-string designate an operator, and adherence to a standard prefix notation enables parsing of the rest. A first level of decoding yields a graph in which operator nodes link to child nodes that correspond to embeddings, operators, and references to other expressions.  1  Thus, expression-strings represent graphs in which most intra-expression links are implicit in the composition of graph-valued operators and their products, 2 while the overhead of designating explicit, inter-expression links (several bytes per reference-key) is in effect amortized over the linked expressions. Accordingly, QNR graphs on a scale comparable to NL syntactic structures need not incur per-link, pointer-like overhead.
 A5.3.2 Lazy Evaluation Lazy evaluation enables the piecemeal decoding and expansion of QNR graphs that are too large to fully decompress. To support lazy evaluation, references to expression-strings can evaluate to graph objects (presented as a vector of nodes), or can be left unevaluated. A key-value store then can support lazy evaluation by returning either an expression-string or, when available, the corresponding graph object; a server that retrieves objects for processing can query an expression-store with a key and invoke evaluation if the store returns a string. This mechanism also supports the construction of shared and cyclic graphs.
 A5.4 Graph-Construction Operators Parsing associates each construction operator with a sequence of operands that can be evaluated to produce (or provide a decoded access to) embeddings, references, and root nodes of graph-objects. Simple construction operators can treat their arguments as atomic and opaque; more powerful operators can access the contents of evaluated graph-valued operands or the contents of the operator's evaluation-time context. QNRs encoded as expression-strings requires a similar mapping of tokens (keys) to lexical-component embeddings, but prospective vocabularies are orders of magnitude larger. The use of large vocabularies mandates the use of scalable key-value stores; with this choice, vocabulary size affects neither neural model size nor computational cost. The use of low-latency storage for frequently used embedding vectors could, however, impose substantial burdens, with requirements scaling as vocabulary size × vector dimensionality × numerical precision.  1  Baseline values for these parameters can place storage concerns in a quantitative context.
 A5.5.2 Vocabulary Size In a continuous vector space, "vocabulary size" becomes meaningful only through vector quantization, the use of identical vectors in multiple contexts.  2  Discrete vectors can be compared to NL words or word parts, and vectors produced by concatenating quantized vectors can be compared to words built of combinations of parts.
 A5.5.3 Lexical Component Embeddings Potential QNR representations and applications (Section 11) are sufficiently diverse (some far from "linguistic" as ordinarily understood) that it is difficult to generalize about the appropriate granularity of vector quantization or the extent to which it should be employed. Different applications will call for different trade-offs between compression and performance, and quantization has been found to improve rather than degrade neural representation learning in some domains.  3  More can be said about vocabulary size, however, in the context of NL + representations that constitute (merely) strong generalizations of NL. To provide a round-number baseline value, consider an NL + representation that employs a vocabulary of distinct lexical components 4 that is 50 times larger than the vocabulary of distinct words in the English language:  5  For present purposes, ~200,000 words is a reasonable estimate of the latter, 1 implying a baseline NL + vocabulary of 10 million lexical-component embedding vectors.
 A5.5.4 Composing Components As discussed in Appendix A4, a richer vocabulary of lexical units can be constructed by composition, for example, by concatenating lexical-component embeddings. This generative mechanism parallels what we see in natural languages, where many (even most) words are composed of various wordparts, TAM-C affixes, or other words. Discrete composition: Expression-strings can readily describe composite lexical units: Vector-valued operators with vector arguments can denote concatenations of any number of components. Considering only discrete combinations, operators that accept 2 to 4 arguments from a vocabulary of 10 7 embeddings can define product vocabularies of 10 14 to 10 28 composites. These are candidates for use as lexical units, and even tiny useful fractions correspond to vast vocabularies. Appendix A4 explores composite embedding representations from perspectives that include efficiency in learning and semantics in use. Weighted combination: Some entities are drawn from distributions characterized by continuous properties that include size, color, age, and probability. Discrete vocabularies are a poor fit to continuous semantics, but operators that accept numerical arguments can specify weighted combinations of vectors, and hence continuous manifolds in semantic spaces. For example, a key that designates a discrete vector embedding a could be replaced by keys designating a vector-valued operator f i , a scalar parameter w, and embeddings c and d, for example, a linear combination: f i (w, c, d) = wc + (1 − w)d. A suitable range of operators can generalize this scheme to nonlinear functions and higher-dimensional manifolds.
 A5.5.5 Dimensionality, Numerical Precision, and Storage Requirements Transformer-based models typically map sequences of tokens to sequences of high-dimensional embeddings (in the baseline version of BERT, 768 dimen-sions). Though high dimensionality is perhaps necessary for hidden states that are used to both represent and support processing of rich, non-local semantic information, one might expect that word-level semantic units could be represented by embeddings of lower dimensionality. This is empirically true: Shrinking input (word-level) embeddings by more than an order of magnitude (from 768 to 64 dimensions) results in a negligible loss of performance  (Lan et al. 2020) . In estimating potential storage requirements, 100 (but not 10 or 1000) dimensions seems like a reasonable baseline value for representations of broadly word-like lexical components. Forming lexical units by concatenation of components (desirable on several grounds; see Appendix A4) would yield larger input embeddings without increasing storage requirements. BERT models are typically trained with 32-bit precision, but for use at inference time, parameters throughout the model can be reduced to 8-bit  (Prato, Charlaix, and Rezagholizadeh 2020), ternary (Wei Zhang et al. 2020) , and even binary (H.  precision with little loss of performance. As a baseline, allowing 8-bit precision for embeddings at the input interface of a QNR-based inference system seems generous. When combined with the baseline vocabulary size suggested above (10 7 component embeddings), these numbers suggest a baseline scale for an NL + key-value store: Storage = vocabulary size × dimensionality × precision ≈ 10, 000, 000 elements × 100 dimensions × 1 bytes ≈ 1 GB. For comparison, current GPUs and TPUs typically provide on-board memory >10 GB, and are integrated into systems that provide terabytes of RAM. Given the expected power-law (Zif-like) distribution of use frequencies, implementations in which small local stores of embeddings are backed by large remote stores would presumably experience overwhelmingly local memory traffic.  1  In many tasks (e.g., fetching content used to answer human queries), occasional millisecond-range latencies would presumably be acceptable. 1. As an illustrative NL example, in its first 10 7 words, the Google Books corpus contains about 10 4 distinct words (by a generous definition that includes misspellings), a ratio of one new word in 10 3 , while in its first 10 9 words, it contains about 10 5 distinct words, a ratio of one in 10 4 .  See Brysbaert et al. (2016) .
 A5.5.6 A Note on Non-Lexical Embeddings Prospective, fully elaborated NL + systems would employ more than just lexical-level embeddings; for example, embeddings that in some sense summarize expressions can enable shallow processing (skimming), or can serve as keys for near-neighbor retrieval that implements semantic associative memory over large corpora.  1  The costs of summaries can in some sense be amortized over the expressions they summarize. Like natural language, NL + expressions can be represented by compact byte strings. Stores of discrete embeddings can support large (and through composition, vast) vocabularies of differentiable semantic representations within an acceptable footprint for low-latency memory; in other words, NL + corpora can be represented approximately as efficiently and compactly as corpora of NL text. Neither necessity nor optimality is claimed for the approach outlined here. Figure 1 . 1 : 11 Figure 1.1: Information flows in generic QNR systems supported by access to a repository of QNR content. Inputs and outputs may be multimodal.
 Figure 7 . 1 : 71 Figure 7.1: Kissing spheres, d = 2, k = 6
 1 . 1 Heimann et al. (2018) , Cao et al. (2019) , Qu, Tang, and Bengio (2019), and Fey et al. (2020)  2. Soft unification and anti-unification operations can contribute to this functionality (Appendix A1).
 .
 3 . 3 For related work, see Verga et al. (2020), Lewis et al. (2020) , Guu et al. (2020), and Févry et al. (2020) .
 4.3)  of the corresponding pairs of expressions.
 1 . 1 Domain-based assessments of credibility have been used in constructing knowledge graphs from social-media sources: Abu-Salih et al. (2021).
 • Employing NL → QNR interfaces to ensure QNR representations • Employing QNR intermediate representations in NL → NL training tasks • Decoupling QNR representations from NL encodings • Employing semantically rich training tasks with QNR objectives • Structuring QNR semantics through auxiliary, lattice-oriented training • Applying QNR-domain inference to exploit QNR repositories
 Figure 10 . 1 : 101 Figure 10.1: Information flows in minimalistic, QNR-bottleneck systems. Inputs and outputs may be multimodal.
 Figure10.2: Information flows in generic QNR systems augmented by access to a repository of QNR content. In the general case, "QNR inference" includes read/write access to repositories, producing models that are in part pre-trained, but also pre-informed.
 2 2
 10. 3 . 3 33 Extending QNR Representations Beyond Linguistic Domains A further class of tasks, X → QNR→ NL, would map non-linguistic inputs X to NL, again mediated by (and training) QNR-based mechanisms. Potential examples include: • Predicting descriptions of images • Predicting descriptions of human actions • Predicting comments in code A quite general class of tasks would encode information from a domain, decode to a potentially different domain, and train QNR → QNR components to perform intermediate reasoning steps. Potential examples include the control of agent behavior involving instruction, communication, and planning. 3 10.4 Abstracting QNR Representations from NL
 Figure 10 . 4 : 104 Figure 10.4: Outline of multitask architectures that include access to external and QNR-based information repositories (e.g., the internet). More arrows could be added.
 Figure 10 . 5 : 105 Figure 10.5: Block diagram decomposing aspects of architectures for complex, open-ended QNR inference functionality. Both working memory and an external repository store key-value pairs, and given a query, will return one or more values associated with near-neighbor keys in a semantic embedding space. Arrows labeled q and k (shown explicitly in connection with external operations) represent query and key embeddings used in storing and retrieving QNR values (v). An elaboration of Figure 10.4 would show similar QNR inference functionality in connection, not only with "QNR inference systems", but also with "reader-encoders" (which need not be distinct components).
 Figure A1. 1 : 1 Figure A1.1: A Boolean lattice over sets and subsets.
 Figure A1. 2 : 2 Figure A1.2: Exact meets and joins of interval (box) regions.
 Figure A1. 3 : 3 Figure A1.3: Approximate meets and joins of regions from a lessconstrained family of region shapes.
 In mathematical applications, proposed QNR/NL + frameworks could wrap formal, symbolic structures in soft descriptions3  that can be applied to help recognize analogies and express purpose, and these capabilities can operate at multiple levels of granularity. Pólya observes that discovery in mathematics involves generate-and-test cycles guided by soft considerations, and modern deep learning confirms the value of soft matching in guiding theorem proving. Better neural representations can improve ML-informed premise selection,1   slowing the explosive growth of deep proof trees by improving the success rate of generate-and-test cycles. Graph neural networks that operate on syntactic structures can provide useful embeddings (M., and enriching formal symbolic representations with soft semantic descriptions (e.g., of known use-contexts) should enable further gains. Pólya emphasizes the importance of analogy, a kind of soft, structured generalization (Section A1.1.2). The formal (hence more restrictive) lattice operation of generalization by antiunification has been applied to analogical reasoning in symbolic mathematics (Guhe et al. 2010) ; embedding symbolic structures in soft representations could extend the scope of potential generalizations. • Applying unification to combine and extend partial descriptions • Applying unification to identify clashes between descriptions • Applying analogies from developed fields to identify gaps in new fields • Applying analogies to suggest hypotheses that fill those gaps • Matching experimental objectives to experimental methods • Matching questions and data to statistical methods • Assessing evidence with attention to consensus • Assessing evidence with attention to consilience • Enabling ongoing updates of inferential dependency structures Applications like these need not automate scientific judgment: To provide value, they need only provide useful suggestions to human scientists. Devel- opments along these lines would extend current directions in applying ML to scientific literatures. 1 11.3.3 Mathematics You have to guess a mathematical theorem before you prove it; you have to guess the idea of the proof before you carry through the details. You have to combine observations and follow analogies; you have to try and try again. -George Pólya 2
 Table A2 . A2 1: Classes and Examples of Function words. The examples below cover about one third of the function-word vocabulary of the English language.In strongly inflected languages, the roles of some of these function words are performed by morphological distinctions. Table A2.2: Examples of Tense/Aspect Distinctions. Languages can use inflection or function words to express distinctions that describe (e.g.) relative time, duration, or causation. Determiners: the, a, this, my, more, either Prepositions: at, in, on, of, without, between Qualifiers: somewhat, maybe, enough, almost Modal verbs: might, could, would, should Auxiliary verbs: be, do, got, have Particles: up, down, no, not, as Pronouns: she, he, they, it, one, anyone Question words: who, what, where, why, how Conjunctions: -coordinating: for, so, and, nor, but, or, yet -subordinating: if, then, thus, because, however -temporal: before, after, next, until, when, finally -correlative: both/and, either/or, not/but
 Table A2 . A2 3: Examples of Modality Distinctions. Languages can express modalities by inflection or function words. The existence of graded degrees and overlaps within and between modalities suggests the potential value of continuous vector-space representations. Table A2.4: Examples of Case Distinctions. Case distinctions can express the roles of words in a sentence (in a familiar grammatical sense) or the roles of what they denote in a situation. The number of inflectional case distinctions A3 From NL Constructs to NL + varies widely among languages; English has three, Tsez has dozens, many Condensing, regularizing, and extending the scope of semantic represen- of which are locative. As with modalities, blurred boundaries and overlaps tations can improve expressive capacity and compositionality, and can between cases suggest the potential value of continuous vector-space repre-support theoretically grounded methods for comparing and combining sentations. semantic information. Nominative -subject of a verb Accusative -object of a verb Dative -indirect object of a verb Genitive -relationship of possession Comitative -relationship of accompaniment Lative -movement to something Ablative -movement away from something Orientative -orientation toward something Locative -location, orientation, direction Translative -becoming something Instrumental -means used for an action Causal -cause or reason for something Benefactive -beneficiary of something Terminative -limit or goal of an action Interrogative -Question Imperative -Command Indicative -Unqualified statement of fact Inferential -Qualified (inferred) fact Subjunctive -Tentative or potential fact Potential -Possible condition Conditional -Possible but dependent on another condition Hypothetical -Possible but counterfactual condition Optative -Desired condition Deontic -Ideal or proper condition
 Table A4 . 1 : A41 Input representations used in current NLP and prospective QNR processing. Typical NLP models Proposed QNR models Input units: wordpiece tokens component embeddings Vocabulary size: ~10 4 -10 5 ~10 7 -10 28 Embedding origins: learned representations learned representations Initial processing: multiple attention layers MoE blending layer 1
			 . This document often describes illustrative forms of representation and functionality, or describes how neural computation could potentially implement those forms and functions, but always with the implicit proviso that learned neural representations and mechanisms are apt to be surprising.
			 . Note that representations of theoretically equivalent expressive capacity need not be equivalent in, for example, computational tractability, compositionality, compactness, scalability, or inductive bias.
			 . For example, opaque Transformer-like models may be useful in QNR applications: In general, quasicognitive processing is complementary to quasilinguistic representation.2. Logic, mathematics, programming languages, knowledge representation languages, attempted formalizations of natural language, etc.3. Including systems that exploit pretrained language models. 4. Readers who don't skim will encounter redundancy provided for those who do.
			 . E.g., 128-token text pieces (Verga et al. 2020) , or more general multimodal information (Fan et al. 2021) .7. Khandelwal et al. (2020) and Yogatama, Masson d'Autume, and Kong (2021) . Models of this kind fall within the broad class of memory-augmented neural networks (Santoro et al. 2016) .
			 . Perceptions of success were (notoriously) blunted by reclassification of research results ("If it works, it isn't AI"). Highly successful automation of symbolic mathematics, for example, emerged from what had initially been considered AI research (Martin and Fateman 1971; Moses 2012 ).
			 . E.g., see Rocktäschel and Riedel (2017) ,Minervini et al. (2020), Arabshahi et al. (2021) .
			 . This does not argue that internal cognitive representations themselves have a clean, graph-structured syntax, nor that sparse, syntax-like graphs are optimal representations for externalized QNRs. The argument addresses only properties of QNRs relative to word strings.
			 . E.g., due to working-memory constraints (Caplan and Waters 1999) .
			 . Section 9.1.2 considers embedding-indexed QNR stores as associative memories accessed through near-neighbor lookup in semantic spaces.2. Goyal and Bengio propose language-inspired compositionality as a key to developing systems that more closely model human-like intelligence; see also (Y. Jiang et al. 2019) , which makes strong claims along similar lines.3. This is a softer criterion than that of formal compositionality.4. In a sense that may be neither spatial nor temporal.
			 . Scoped binding of symbols stretches but does not break notions of syntactic locality-if locality is construed in terms of arcs in identifiable graphs, it is not constrained by syntactic distances over sequences or trees.2. Interestingly, although computer languages are models of compositionality, fMRI studies show that reasoning about code is only weakly focused on brain regions specialized for natural language (Ivanova et al. 2020) . This distribution of neural function supports a broad role for compositional representations in non-linguistic cognition.3. Raposo et al. (2017) , Watters et al. (2017) , Battaglia et al. (2018) , Eslami et al. (2018) , G. R. Yang et al. (2019), and Bear et al. (2020)  4. In formal systems, definitions and bindings within a scope are examples of a precise form of contextually conditioned compositionality.
			 . The lack of full compositionally in language has been a bitter pill for formalists, and not all have swallowed it. The Principle of Compositionality, that the meaning of a complex expression is determined by its structure and the meanings of its constituents, has been taken to apply to language; although others recognize a pervasive role for context (enriching word embeddings with contextual information has been a successful strategy in neural NLP; see Liu, Kusner, and Blunsom (2020) ), some seek to apply context to determine (fully compositional) lexical-level meanings, which seems an arbitrary and perhaps unworkable choice.See Szabó  (2017).
			 . "Castle Mound gives a good view of its surroundings." Is the context of this sentence tourism in present-day Oxford, or military intelligence during the Norman conquest? The nature of the view and its significance may be clear in the context of a pamphlet or book, but cannot be found in the quoted string of words.2. The same can be said of natural language expressions in the context of human memory and neural states.3. Note that the problem is not ambiguity per se, but ambiguity that is unintentional or costly to avoid. Intentional ambiguity is expressive, and (to meet benchmark criteria) must therefore be expressible in NL + (see Section 8.3.1 and Section A3.4).4. Because expressions may appear in multiple contexts, this should be seen as information about the context of a particular citation or use of an expression.5. As discussed in Section 8.3.5.
			 . Note this question does not ask how those constructs actually operate, which is a subject of ongoing controversy among linguists.2. See Bobrow and Winograd (1977)  and McShane and Nirenburg (2012) . Knowledge representation languages typically attempt to build on unambiguous ontologies (Guarino 2009 ), yet the ability to express ambiguity is an important feature of natural languages.
			 . Or DAGs, when coreference is represented explicitly.2. See Borsley and Börjars (2011) .3. Meanings, not words: Vector representations of words perform poorly when those words have multiple meanings, but representing meanings rather than words sidesteps this problem.
			 . Linguists define "simple phrases" differently.3. Note that points in a vector space are inherently sharp, yet may be taken to represent (potentially soft) regions in a lower dimensional space (see Section 7.1.5).
			 . The vocabulary of an English speaker may include 10,000 to 100,000 or more content words; different speakers may employ different blocks of specialized (e.g., professional) vocabulary.2. Note also the potential value of explicitly compositional representations of embeddings, e.g., embeddings built by concatenation (Appendix A4).3. Distinctions of kind and differences in properties differ, yet are not entirely distinct.4. The somewhat counter-intuitive geometric properties of high dimensional spaces are relevant here (see Section 7.1.4). Note also that displacements in a particular direction need not have the same meaning in different regions of semantic space: Most properties relevant to planets differ from those relevant to music or software.
			 . The poverty of closed-class vocabulary is mitigated by the availability of compound function words (at least, because of. . . ) that can be treated as lexical entities.See Kato, Shindo,  and Matsumoto (2016).2. The constructs "X and/or Y" and "either X or Y" can express the inclusive/exclusive distinction, yet trade-offs between precision and word economy (and the cognitive overhead of instead relying on context for disambiguation) ensure frequent ambiguity and confusion in practice. As a less trivial example, the ability to compactly express "possibly-inclusive-butprobably-exclusive-or" would be useful, and in a continuous vector space of function words, would also be natural.
			 . Because tense, aspect, and modality overlap and intertwine, linguists often group them under the label "TAM"; because they also overlap with case distinctions, all their indicators (inflections, markers) will be lumped together here and simply referred to as "TAM-C modifiers" (for examples and discussion, see Appendix A2 and Section A3.3).2. Further muddling standard linguistic distinctions, punctuation can indicate case or modality (question marks, exclamation marks) and can play roles like function words that clarify syntax (commas, semicolons, periods) or express relationships of explanation, example, or reference (colons, parentheses, quotation marks). In spoken language, verbal emphasis and paralinguistic signals play similar roles.3. Linguists recognize a semantic space of modalities (Allan 2013) .
			 . Potentially accompanied by abstractive, non-definitional embeddings(Section 8.3.4). The contours of such boundaries need not be determined by an implementation: There is no need to engineer representations that neural systems can instead learn.
			 . Philosophers have dissected considerations of this sort to provide richer distinctions and more precise terminology.
			 . Unification of two expressions produces the least specific (most general) expression that contains the information of both; unification fails if expressions contain conflicting information. Generalization (anti-unification) of two expressions produces the most specific (least general) expression that is compatible with (and hence unifies) both. Unification and anti-unification are associative and commutative, and satisfy several other axioms. Soft unification and antiunification relax these constraints.(See Section A1.4.3.)
			 . For applications of language in RL, see Das et al. (2017) , Lazaridou, Peysakhovich, and Baroni (2017) , Shah et al. (2018), and Luketina et al. (2019) .
			 . Note, however, successful applications of discretized representations in which learned, finite sets of vectors are selected from continuous spaces; see, for example, Oord, Vinyals, and Kavukcuoglu (2018)  and Razavi, Oord, and Vinyals (2019)
			 . Note that the number of vectors that are all nearly orthogonal (all pairwise cosine similarities 0.5) also grows exponentially with d and becomes enormous in high dimensional spaces.
			 . The use of Euclidean distance or cosine similarity is explicitly or tacitly assumed in much of this document, but growing interest suggests that hyperbolic spaces (useful in graph and sentence embeddings) or mixed geometries may provide attractive alternatives for embedding QNR expressions; see for example Peng et al. (2021) .2. Appendix A1, Section A1.5.3, and Section A1.6 discuss both boxes and more flexible classes of representations in the context of unification and generalization operations on soft lattices.3. Arithmetic in which operands are intervals over R.
			 . Van Lierde and Chow (2019)  and Menezes and Roth (2021)
			 . To enable arbitrary graphs as outputs requires initializing with complete graphs (hence N (N − 1) directed arcs). Restricting optimization to local subgraphs, to restricted search spaces, or to algorithmically generated "rough drafts" can avoid this difficulty.2. Note, however, that efficient, highly scalable near-neighbor retrieval algorithms on unstructured spaces are typically "approximate" in the sense that nearest neighbors may with some probability be omitted. This shortcoming may or may not be important in a given application, and fast algorithms on weakly structured spaces can be exact (see Lample et al. 2019) .
			 . In logic, symbols correspond to zero-volume regions, while variables correspond to unbounded regions. 4. See Section A1.2. In generalizations of lattice representations, a "region" may be fuzzy, in effect defining a pattern of soft attention over the space, and lattice relationships may be approximate (Section A1.4.3).
			 . Rocktäschel and Riedel (2017) , Campero et al. (2018) , Minervini et al. (2018) , Weber et al. (2019), and Minervini et al. (2020) 2. Narayanan et al. (2017), Grohe (2020), and Pan et al. (2020) 3. Allamanis et al. (2017), R. Liu et al. (2017), H. Zhang et al. (2018), and Alon et al. (2019) 4. Y. Li et al. (2018) and Cao and Kipf (2018) 5. Pan et al. (2018), Simonovsky and Komodakis (2018), and Salehi and Davulcu (2020) 6. What counts as "small" is an open question.
			 . Attributes (labels) can potentially be expanded to include sets of vectors of explicitly or implicitly differing types.2. And perhaps also implementation.3. A representation that can also accommodate multigraphs and hypergraphs.
			 . Figure from Raboh et al. (2020), used with permission.
			 . When a single word (e.g., "match") has multiple unrelated meanings (homonymy), it corresponds to multiple, potentially widely scattered points in a natural semantic space; when a word (e.g., "love") has a range of related meanings (polysemy) representations that map word-meanings to points (rather than semantic regions) become problematic.See Vicente  (2018).2. When different regions of a vector space represent things of distinct kinds, the meanings of directions may vary with position: In other words, because different kinds of things have different kinds of properties, it is natural for mappings of directions to properties to be a function of kinds, and hence of location. Indeed, the literature describes discrete-word models of NL semantics in which the meanings of adjectives are a function of their associated nouns; see, for example, Baroni and Zamparelli (2010)  and Blacoe and Lapata (2012) . In this general approach, the meanings of verbs are also a function of their associated nouns, and the meanings of adverbs are functions of their associated verbs.
			 . Where "entity" is intended to include (at least) different things (horses, cows, photons, telescopes, gravitation, integers) and different actions (walk, run, observe, report).2. "[A] graph of data intended to accumulate and convey knowledge of the real world, whose nodes represent entities of interest and whose edges represent relations between these entities" (Hogan et al. 2021 ).   3. Ji et al. (2021 and Ali et al. (2021)  4. Expression-level definitional embeddings are not sharply distinguished from lexical embeddings, e.g., those corresponding to function words.
			 . In processing with access to QNR corpora, the analogues of long-term memory and text converge.2. For an analogous application of summarization in Transformers, see Rae et al. (2019) .3. E.g., representations of substantive content (potential answers to questions) vs. representations of the kinds of questions that the substantive content can answer (potentially useful as keys).4. Summaries with this property seem well suited for use as keys in search guided by semantic similarity.
			 . The considerations here are quite general; for example, context representations are also important in understanding/recognizing objects in scenes (Tripathi et al. 2019; Carion et al. 2020 ).2. StandardTransformers represent a span of immediate context in their activations, while representing both relevant and irrelevant global context in their parameters. Both of these representations are subject to problems of scaling, cost, compositionality, and interpretability. Models (potentially Transformer-based) that write and read QNRs could benefit from externalized representations of wide-ranging, semantically indexed contextual knowledge. The application of very general knowledge, however, seems continuous with interpretive skills that are best embodied in model parameters.
			 . In this connection, consider potential generalizations of the image-domain algorithm described inSarlin et al. (2020), which employs end-to-end training to recognize, represent, and align features in pairs of data objects through GNN processing followed by differentiable graph matching. Analogous processing of QNR representations would enable semantics-based alignment even in the absence of strict structural matches.2. In recent work on neural image interpretation, employed propositional knowledge in the form of production rules (condition-action pairs) that are represented by model parameters and selected at inference time by attention mechanisms based on similarity between condition and activation vectors.3. Distance metrics may result from neural computations that condition on broader semantic content, or on other relationships beyond simple pairwise sums and differences.
			 . The lattice axioms (Section A1.2) ensure that pairwise combination extends to multiple arguments in a natural way.
			 . Summing over products of training epochs and corpus sizes (Brown et al. 2020) .
			 . Note that this mechanism provides a differentiable and potentially fluid graph representation; see Section 7.2.3.
			 . Scaling as ~O(n), but learned, structured key spaces can improve scaling to O(n 1/2 ) (Lample et al. 2019). 2. J. Wang et al. (2018), Fu et al. (2018), Johnson, Douze, and Jégou (2019), Jayaram Subramanya et al. (2019), and Sun (2020)3. Note that representation of images through relationship-graphs among objects blurs the boundary between opaque image embeddings and syntactic structures; See for example Bear et al. (2020) . Image-embedding spaces can also be aligned with text (e.g., inPatashnik  et al. 2021).
			 . For example, an NL phrase can compactly say that "face_1 strongly resembles face_2", while a lexical-level embedding can compactly say that: "face_1, with a specific set of embedding-space offsets in shape, color, and expression, looks like face_2 with some quantified but approximate residual differences." 2. Hence the value of using interpretable yet non-linguistic image embeddings as examples.3. In this connection, note that QNR expressions are data, that neural functions can be of type QNR → QNR, and that "apply", "eval" and "quote" functions can play quite general roles (e.g., in languages like Scheme that can express and operationalize high-level abstractions).
			 . Szegedy (2020)  suggests an NL-translation approach to formalizing mathematics papers.2. M. and Minervini et al. (2018)
			 . Y., Tran et al. (2020), and Botha, Shan, and Gillick (2020)  2. Object-oriented programming exploits this principle.3. Even projections of sets of high-dimensional vectors into spaces of substantially lower dimensionality can preserve key geometric relationships quite well: The Johnson-Lindenstrauss lemma implies good preservation of both distances and cosine similarities between vectors.
			 . Particularly when intermediate NL + processing can draw on relevant NL + corpora. Successful augmentation of Transformers with external memory (e.g., for question answering) provides evidence for the potential power of this approach (Koncel-Kedziorski et al. 2019; Fan et al. 2021; Min et al. 2020; Thorne et al. 2020) .
			 . A simple example would be regions implied by implicit, contextual uncertainty in a vector value; richer, more formal examples include spaces in which vector values (points) explicitly correspond to regions in lower-dimensional spaces, or in which points are semantically related by taxonomic or set-inclusion relationships. In a limiting case, vector values correspond either to points (which, through comparison by equality, can model mathematical symbols) or to unknowns (which, through co-reference, can model mathematical variables).2. As a non-linguistic analogy, consider overlapping fragments of an image: Where overlaps match well enough, the fragments can be glued together to form a more comprehensive image of a scene, combining information from both fragments and potentially revealing new relationships.3. Section 10.6.7 suggests generically applicable training objectives that would favor representations and operations that (approximately) satisfy the axioms (Section A1.2) for unification and anti-unification; in this approach, operations may be performed by contextually informed neural functions.4. E.g., embeddings that represent uncertain values; nodes that lack links to optional content; leaf-level nodes that in effect summarize the content of some range of potential graph extensions.5. E.g., embeddings that represent narrower values; links to graph structure that may be modified by conditioning on a compatible leaf-level embedding.
			 . The scope of recognizable parallels will depend on learned representations and comparison operators. Regularization (Section 8.4) can make representations more comparable; useful comparison operators could potentially resemble relaxed unification and generalization operators.
			 . Argumentation mining points in this direction (Moens 2018; Galassi et al. 2020; Slonim et al. 2021) .2. In an awkward coarse-grained manner.3. Note, however, that cited-by relationships can have massive fanout, a pattern of use that may call for backward-facing structures richer than link-sets.4. Jaradeh et al. (2019), M. Jiang et al. (2020), and Cohan et al. (2020)
			 . Summaries (like other expressions) can be embodied in QNRs that provide increasing detail with increasing syntactic depth.2. See Martino et al. (2020).3. In this context, the difference between toxic text and discussions that embed examples of toxic text illustrates the importance of recognizing use-mention distinctions. Social media filters today may suppress both advocates and critics of offensive views.
			 . Here, links to NL source text can be valuable: Literal wording may convey signals of shallow repetition.
			 . Note that open-ended reasoning likely calls for conditional computation; potentially relevant architectural components and training methods are discussed in Cases et al. (2019) , Rosenbaum et al. (2019), and Banino, Balaguer, and .
			 . Reviewed in J. Zhou et al. (2020)  and Wu et al. (2021) . Transformers in effect operate on fully connected graphs, but on sparse graphs, GNNs can provide greater scalability and task-oriented inductive biases (Addanki et al. 2021) , as well as more direct compatibility with QNRs.2. and Akoury, Krishna, and Iyyer (2019)  3. Including both image and joint image-language models; see Zellers et al. (2018) , J. Yang et al. (2018) , Lee et al. (2019), and Bear et al. (2020) .
			 . X., Brown et al. (2020), and Y. Liu et al. (2020)
			 . Re. multi-scale masking, see Joshi et al. (2020) .2. See McCann et al. (2018), X. Liu et al. (2019), Alex Ratner et al. (2018), and Alexander Ratner et al. (2020).3. Analogous language-infused mechanisms are described in Shah et al. (2018) , Luketina et al. (2019), and Baroni (2020) .
			 . In a general sense, completion tasks can include not only sequence prediction and cloze tasks, but also question answering and other capabilities shown by language models in response to prompts (see Brown et al. (2020) ); prediction in abstracted (latent space) domains can also support a range of tasks.See Oord, Li, and Vinyals (2019).
			 . Aided by structural regularity (Section 8.4.) A similar approach might prove fruitful in training models that produce flat vector representations, which naturally have smooth distance metrics. This basic approach (predicting learned representations rather than raw inputs) is applied in Larsen et al. (2016)  and related work.2. E.g., they may map approximately lattice-respecting to strongly lattice-incompatible sets of representations.
			 . E.g., Keskar et al. (2019) .3. E.g., Mager et al. (2020 ).   4. Fu et al. (2018, J. Wang et al. (2018 ), Johnson, Douze, and Jégou (2019 ), and Jayaram Subramanya et al. (2019
			 .See Veličković and Blundell (2021)  and included references.2. Reviewed inYe Zhang et al. (2017)  and Mitra and Craswell (2018) .
			 . See Y. Lu et al. (2018)  and Arivazhagan et al. (2019) .2. A limiting case of this task would be semi-autonomous production of content, potentially on a large scale, guided by only the most general indications of purpose; see Section 12.2.
			 . Verga et al. (2020) , Guu et al. (2020) , and Xu et al. (2020)  discuss Wikipedia-oriented systems.2. To enable retrieval of similar yet potentially clashing content, (some) embeddings should represent, not the concrete semantic content of expressions (in effect, answers to potential questions), but the kinds of questions that the content can answer, an important distinction noted above. Relevant clashes would then be indicated by failures of partially successful attempts at soft unification between new and retrieved content.3. Because statements may provide support for other statements, providing such material is related to argumentation, where automated, corpus-based methods are an area of active research; for example, see Lawrence and Reed (2020)  and Slonim et al. (2021) .4. A process illustrated by Xu et al. (2020) .
			 . For examples of related work, see Shah et al. (2018)  and Abramson et al. (2021) . Relatively simple linguistic representations have emerged spontaneously; see Mordatch and Abbeel (2018)  and Lazaridou and Baroni (2020) .
			 . Including games that require long-term planning (Vinyals et al. 2019; OpenAI et al. 2019 ).2. Luketina et al. (2019)  reviews progress and calls for "tight integration of natural language understanding into RL".3. As noted above, it is reasonable to expect that the most general and frequently used kinds of knowledge would be encoded, not in declarative representations that enable multi-step inference, but in model parameters that enable direct decision and action; this distribution of functionality would parallel Kahneman's System-1/System-2 model of human cognition (Kahneman 2011).
			 . Along with filtering based on detailed comparisons.2. Pattern completions may suggest structures; sampling guided by embeddings may suggest components.3. Simon (1988). Note that Simon describes planning as a design process.
			 . Illustrated by work in Stump et al. (2019) , Mo et al. (2019) , and Chen and Fuge (2019).
			 . E.g., M. Jiang et al. (2020)  and Raghu and Schmidt (2020)  2. Pólya (1990)  3. Szegedy (2020) suggests deriving formal expressions from NL text.
			 . E.g., based on multi-source consistency, consensus, coherence, consilience, and provenance (Section 9.5.2). Current filtering methods appear to rely heavily on judgments of source quality (a domain-insensitive, non-content-based proxy for epistemic reliability), perhaps the simplest use of provenance.2. A problem discussed in Carlini et al. (2021) .
			 . Translation would be subject to semantic imprecision due to differences in expressive capacity.2. Approximately-range of compensation is substantial (e.g., see Tee (2021).3. An optimized ("FastFormer") model derived from BERT can perform inference at a cost of about 18 US$/100 million queries (Kim and Hassan 2020) .
			 . Effective altruists please take note.2. Drexler (2019)
			 . An application of the "Principle of Least Privilege" in system design.3. See Drexler (2019, Section 9.7).4. Bostrom (2014)  and Russell (2019)
			 . Along lines suggested by Stuart Russell (Wolchover 2015) ; see also discussion inDrexler  (2019, Section 22).
			 . In particular, studies of lattice semantics in NL, unification and generalization in symbolic computation, and lessons learned in the broader study of neuro-symbolic ML.
			 . More precisely, among potential more general expressions, anti-unification yields the most specific instance.
			 . Here, larger sets are less specific and hence provide less information.2. Guhe et al. (2010) , Martinez et al. (2017) , and Amiridze and Kutsia (2018)
			 . Clark et al. (2021)  provides a more extensive and formal presentation of lattice semantic structure in domains closely aligned with those considered here.
			 . "Feature structure" representations are particularly relevant to QNRs; Knight (1989)  reviews feature-structure unification and generalization in NL semantics.2. Tousch, Herbin, and Audibert (2008), Velikovich et al. (2018), and Wannenwetsch et al. (2019)3. Cimiano, Hotho, and Staab (2005)  4. Ganter and Wille (1997, 1999) , Cimiano, Hotho, and Staab (2005) , Belohlavek (2011), Eppe et al. (2018), and Clark et al. (2021)
			 . Paulson (1986)  and Felty and Miller (1988)  3. Excluding subtrees that contain the same symbol when cyclic graphs are disallowed.
			 . Jaffar and Maher (1994)
			 . For example, ranges of compatible meanings with respect to various properties, implying what are in effect interval constraints on those properties, a familiar (if perhaps too rigid) constraint structure (see Benhamou 1995) .2. E.g., in Sessa (2002) , Medina, Ojeda-Aciego, and Vojtáš (2004) , Weber et al. (2019) , andMinervini et al. (2020).3. In addition, lattice operations may be mixed: In combining information, differences of some kinds should lead to rejection or narrowing, while differences of other kinds should lead to generalization. For example if we are combining pieces of evidence about a cat (perhaps from two photographs), some properties should be unified (pictures of spots that differ only in visibility and angle of view should narrow possible models of coloration, while a difference of orange vs. black should lead to failure and reject the hypothesis "same cat"). By contrast, if one photo shows a sleeping cat and the other an alert cat, the combined description should represent a cat that is not always asleep. Differences between kinds of differences should be learned from and conditioned on tasks.
			 . Consider the union of disjoint volumes, each already at the limit of representable complexity. Intersections can (but do not necessarily) suffer from a similar difficulty.
			 . Note that adherence to lattice properties in representational vector spaces is important only in those regions/manifolds that are actually used for representation.
			 .See Veličković and Blundell (2021)  and included references.
			 . Bantu languages include unusually complex and expressive systems of tense and aspect (Nurse and Philippson 2006; Botne and Kershner 2008) .
			 . When feasible, of course. Another reason is the natural (and often semantically appropriate) commutativity implicit in vector representations viewed as sums of components.
			 . And conversely, maladaptive precision in the form of (for example) forced number and gender distinctions.2. E.g., color, mass, temperature, frequency, speed, color, loudness, age, beauty, and danger.3. Gärdenfors (2000)  and Lieto, Chella, and Frixione (2017)
			 . Klys, Snell, and Zemel (2018) , Kingma and Dhariwal (2018) , R. Shen et al. (2020)
			 . A good recent example is Shen et al. (2020) , which finds that diverse faces can be wellrepresented in 100-dimensional spaces (Härkönen et al. 2020 ).2. In principle, to distinguish among millions of faces requires distinguishing on the order of 10 gradations on each of 6 dimensions, but typical embeddings are far richer in both distinctions and dimensionality.3. Within the domain of concrete, interpretable images, ~100 dimensional embeddings can represent not only faces, but also diverse object classes and their attributes, thereby representing (in effect) interpretable "noun-and-adjective" combinations, few of which can be compactly and accurately described in NL; e.g., see Härkönen et al. (2020) .4. Is the rock hard and sharp, or a piece of fine-grained ultramafic basalt? 5. On the internet, emoji have emerged to compactly express expression-level sentiment (e.g., and ), and these can express meanings distributed over more than one dimension (consider , , and ).
			 . E.g., embeddings that generalize NL conjunctive/causal expressions could presumably express meanings like "object X, (probably) together with and (possibly) because of Y", and do so with graded degrees of probability or epistemic confidence.2. It is natural to want semantic spaces that express joint probability distributions as well as relationships in Pearl's do-calculus; the blurry distinction between these and the NL-like semantic spaces outlined above points to the soft boundaries of NL-centric conceptions of NL + .
			 . Expressions that are frequently reduced to abbreviations are likely to represent broadly useful, lexical-level meanings: IIRC, ISTM, AFAIK, IMO, etc.
			 . While blurring the distinction between values and variables; see Section A1.4.3, which notes potential relationships between constraint-based unification and variable binding in logic programming.
			 . A vocabulary which describes structures, functions, relationships, processes, observations, evidence, interventions and causality in systems of extraordinary complexity and human importance.2. A count that omits, for example, inflected forms (Brysbaert et al. 2016) .
			 . In addition to these considerations, note that components could potentially occupy relatively simple and well-structured semantic spaces, facilitating their interpretation even in the absence of specific training examples. Improving the interpretability of novel lexical components would feed through to improvements in the interpretability of novel elements of a lexical-unit vocabulary.
			 . The set of mechanisms outlined here is intended to be illustrative rather than exhaustive, detailed, or optimal. The discussion touches on tutorial topics for the sake of readers who may notice computational puzzles without immediately recognizing their solutions.
			 . Internal, index-encoded links require a bit of additional bookkeeping (i.e., remembering subexpression positions) but can describe DAGs and cyclic graphs.2. Indices (~1 byte?) account for the rest.
			 . Vocabulary size depends on how "distinct words" are defined; reasonable definitions and methodologies yield numbers that differ, but not by orders of magnitude.See Brysbaert  et al. (2016).
			 . To support near-neighbor indexing, embeddings should be unique, not elements of a "vocabulary".

Abstract: The views expressed herein are those of the author(s) and do not necessarily reflect the views of the Future of Humanity Institute.