Dave Doty on algorithmic self-assembly

 Dave Doty is a Senior Research Fellow at the California Institute of Technology. He proves theorems about molecular computing and conducts experiments implementing algorithmic molecular self-assembly with DNA.

Luke Muehlhauser: A couple years ago you wrote a review article on algorithmic self-assembly, and also created a video introduction to the subject. Your review article begins:
Self-assembly is the process by which small components automatically assemble themselves into large, complex structures. Examples in nature abound: lipids self-assemble a cell's membrane, and bacteriophage virus proteins self-assemble a capsid that allows the virus to invade other bacteria. Even a phenomenon as simple as crystal formation is a process of self-assembly… Algorithmic self-assembly systems automate a series of simple growth tasks, in which the object being grown is simultaneously the machine controlling its own growth.
As an example, here's a an electron microscope image of a Sierpinski triangle produced via algorithmic self-assembly of DNA molecules:

In your view, what are the most impressive examples of algorithmic molecular self-assembly produced in the lab thus far?

Dave Doty: Let me play the politician here and answer the question I wish I had been asked, which is to say, what's the difference between algorithmic and non-algorithmic self-assembly, how well does algorithmic self-assembly work as of April 2014, and what are the reasons to think we can make it work better in the future?
Non-algorithmic self-assembly
There are very impressive examples of non-algorithmic molecular self-assembly in the lab. Two very successful techniques have been DNA origami and DNA tiles. DNA origami requires a long DNA strand called a scaffold (the most commonly used one has 7249 bases). The idea is to synthesize about 200 much shorter (~32 bases) single strands of DNA, called staples, each of which binds via base-pairing (A binds to T and C binds to G) to multiple regions of the scaffold to bring the regions together, essentially folding the scaffold. Carefully designing which regions will be brought together means that different shapes can be created from the same scaffold by folding it differently.
DNA tiles use base-pairing of DNA in a different way. A DNA tile is a complex of DNA with some number (often 4) of single-stranded regions called "sticky ends" that are available to bind on 4 different sides (they are "sticky" because they will bind via base-pairing to complementary single strands on other tiles). One advantage of DNA tiles is that we are not limited by the size of a scaffold strand; potentially many more than 200 tiles could bind to form large structures.
The "default" approach in either of these techniques is to use a unique type of DNA strand for each "pixel" in the self-assembled shape: 200 DNA strands binding to form a shape means 200 different DNA sequences.
Algorithmic self-assembly
The "algorithmic" aspect of algorithmic self-assembly means that many different copies of a single type of DNA tile can be reused at several different parts of a structure. We call this "algorithmic" by analogy to the idea that small program (e.g., one printing the digits of pi) can encode a large or infinite object (the digits of pi) though local rules.
For instance, if a tile is designed to bind to copies of itself on all sides, the whole space (a plane if done in two dimensions) is filled with copies of itself in a regular square grid. We could get a little fancier: design two tile types, white and black, so that white binds only to black and black binds only to white. Then you'll get an infinite checkerboard pattern. Slightly more interesting but still periodic.
In fact, we can get much fancier if we can achieve what's called "cooperative binding": setting up the experimental conditions juuuuuust right so that a tile can bind to a grid of other tiles only if TWO sticky ends of the tile match. The trick is that some other types of tiles may match just one of the sticky ends, or the other, but ONLY the one that matches BOTH will be allowed to bind. If this rule can be enforced, then tiles can be made to assemble structures of arbitrarily high algorithmic complexity, because this gives the tiles the ability to execute any program as they grow, in turn using that program to direct further growth.
So back to your question: what are the most impressive examples of algorithmic self-assembly in the lab today? We have tiles that assemble to count in binary (tiles bind in rows representing successive positive integers in binary) and tiles that compute XOR (which in turn leads to a pattern, like the one you showed, that creates a discrete version of the Sierpinski triangle). The best published binary counter counts from 1 up to 6, and then binding errors send it to 9. There are unpublished images of counters getting much farther before making errors, but we're a long way from eliminating the errors entirely.
So what are these errors, and how do we plan to tame them? The errors stem from failed attempts at cooperative binding: a tile matched one of its sticky ends and not the other (hence was the wrong tile type to attach), but it attached anyway.
There are a number of experimental approaches to reducing error.
Lately we've been working a lot on getting certain experimental conditions right for cooperative binding to be favorable: for example, each sticky end should have the same binding strength as every other (G-C bonds are twice that of A-T bonds, for example, so this isn't trivial to achieve just by using identical sticky end lengths).
Long temperature holds seem to produce good results, but the correct temperature to choose often depends on a number of other factors, so sometimes we work hard to find the right temperature and then in the next experiment, with slightly changed conditions, need to find a new optimal temperature.
There are ways to correct errors "in software" by designing some redundancy into the tiles, in analogy to error-correcting codes that use 200 bits to represent a 100-bit message, so that instead of a single tile representing a bit, a 2×2 or 3×3 block of tiles will all represent the same bit.
Finally, there are several labs looking into other ideas from the DNA nanotechnology community to implement cooperative binding. For example, there is a very widely used technology called DNA strand displacement that has proven very useful in controlling the interaction of artificial DNA systems. It is conceivable that it could be used as a way for a DNA tile to detect the condition "both of my 'input' binding sites match" and only then to allow the tile to attach. A lot of DNA nanotechonology and synthetic biology projects take existing enzymes, particularly ones that interact with DNA and RNA, and repurpose them to help drive some part of an artificial DNA system, so this may be an idea to explore with algorithmic self-assembly as well.
I don't know whether any of these ideas I've mentioned will ultimately be what takes algorithmic self-assembly from "dreamy theoretical playground" to "reliable technology for bottom-up molecular fabrication". I have no doubt at all that something will work, however. Every day, nature uses bottom-up self-assembly to automatically assemble molecular structures of incredible complexity. What we're doing is figuring out the engineering principles needed to do it reliably and cheaply.

Luke: For those in the field of algorithmic self-assembly, do you think it has it been particularly difficult to attract funding and research talent to such a "dreamy theoretical playground"? What methods have you used to help draw funding and research talent to work on these problems, despite the fact that they are many years away from practical application?

Dave: I don't think either has been difficult.
I suppose "difficulty of funding" is always relative to other subjects we could be studying but aren't. It might be easier to get corporate funding, for instance, to study how companies can use online viral marketing in order to boost their profits, but the major risk of that project is that I would be unable to complete it after I die of boredom.
Of course, funding is always easier to secure when you can show that results are guaranteed. But big breakthroughs are made when — and only when — people decide to work on a project where it's not at all clear in advance that it's going to work.
The National Science Foundation's mission explicitly states that basic science is a goal, i.e., studying the world for the sake of understanding it better, not merely as a means to some application. I think this is a good thing for society in general. Happily for us, the National Science Foundation has been generous in funding grants related to algorithmic self-assembly, and molecular computing more generally, as have some other governmental granting agencies.
As far as attracting talent, I should say there are a lot of experimental scientists who do work on immediate practical applications: for example, fast/cheap disease diagnosis using "DNA circuits", or using self-assembled DNA structures that guide the precise placement of other structures such as carbon nanotubes or transistors with very fine resolution.
But I like to believe that a lot of people are like me and are attracted to the field precisely because it's so poorly understood and so disorganized that it's years away from (even bigger) practical applications. Once a field is so well understood that applications follow easily, we get bored and want to move on to something else that no one understands yet, so that we can be the first people to make progress in trying to understand it.
I'm inspired by someone like Alan Turing, who thought very deeply about programmable computers in 1936, years before the first one was built. Once practical electronic computers really hit their stride by the 1950s, Turing had moved on to studying, of all things, biological morphogenesis.
Of course I am frequently asked, "What are the practical applications of programming matter to manipulate itself?", and I politely oblige with guesses such as "smart drugs" and "nanoscale robots". I'm curious if Turing was ever asked in 1936, "What are the practical applications of automating computation?" and whether he had a memorized response that surely failed to include "space flight", "the internet", "smart phones", or "cruise control".
I'm not saying that every project should be funded, regardless of whether it has applications. But even the dullest imagination could brainstorm a few practical consequences that might follow from learning to program and control matter at the molecular level.
I think our field attracts smart people precisely because they don't need any convincing that this is an important goal. They need only to hear two things:

nothing in the laws of physics says that it can't be done, and
we don't know how to do it yet.

Luke: That's very interesting. Bill Hibbard and I have a forthcoming paper in CACM called "Exploratory Engineering in AI," in which I list several examples of what Eric Drexler called "exploratory engineering." I named Turing's pre-ENIAC work, pre-Sputnik astronautics, and some other examples. Looking back, I should have named algorithmic self-assembly and cited your CACM paper on that!
AI is interesting in this regard, because there are some people who think fully-general "AGI" is a reasonable thing to be studying now even though it's several decades away, precisely because physics suggests it should be feasible but we just don't know how to do it yet. But there are others who think it's kind of disreputable to write papers about AGI or even talk much about it in public, because it's too speculative and theoretical.
Does that division exist in your field as well? If it doesn't, do you have a theory as to why there is this division in the AI field but not in your own field?

Dave: This division definitely exists in my field. I've often thought of there being two flavors of theory, and I attribute the division to people judging results in the second type of theory by the standards of the first type:
1) Descriptive theory that helps predict the behavior of an existing natural system, in service of experiment. Here, if theoretical predictions differ from experiment, then something's wrong with the theoretical model, and the theoretician needs to work harder to make the model more realistic. An example is Newton's laws of motion that predict how billiard balls will move around a billiard table. When they don't move quite as predicted, it's a problem with the model, which needs to be updated with ideas like friction, air resistance, etc.
2) Prescriptive theory that serves as a programming language for talking about behavior that we want to engineer in a new system that may or may not presently exist, served by experiments that implement the programmed behavior. An example is theoretical work on Boolean circuits that abstracts away their underlying implementation with transistors (or more recently, implementation with DNA strands or genetic transcription factors). When the electronic circuit doesn't output the correct bit, it's not a problem with the Boolean circuit model, it's a problem with the experimental implementation that failed to maintain analog voltages in such a way as to correctly represent discrete bits in the Boolean circuit.
If I understand your meaning correctly, the second of these two types of theory could very well be called exploratory engineering when the system being studied doesn't exist yet. Most theory in the natural sciences is descriptive, but theoretical computer science is almost exclusively prescriptive. Theoretical computer scientists don't predict physical results that are confirmed by experiment. Rather they show what is possible (and impossible) for certain engineered systems to do, and sometimes (as in the case of algorithmic self-assembly, or quantum computing, or electronic computing in 1936) they are reasoning about systems that haven't even been built yet.
The abstract tile assembly model is a prescriptive theory (one variant of it; another variant called the kinetic tile assembly model is closer to the underlying chemistry and used more in the descriptive sense when we do experiments). The abstract tile assembly model doesn't predict what DNA tiles do, and that's not its goal. The point of the theory is to show that if DNA tiles can be built that have the idealized behavior of the model, then hey, look at all the other amazing things they could do! The fact that we currently aren't able to experimentally engineer this behavior isn't a problem with the theoretical model. It's a problem with the experiments.
Confusion results when someone familiar only with descriptive theory encounters a result in prescriptive theory. I've proved impossibility theorems (of the form, "no system exists that has so-and-so behavior") and been asked whether I intend to confirm them experimentally, a notion that doesn't even make logical sense.
For example, the uncomputability of the halting problem, proved by Turing in 1936, states that there is no computer program P that can predict whether another computer program, given as input to P, is ever going to halt. No experiment can be done to confirm this theorem, yet its truth is a fundamental law of nature.
I can't comment on the AI field and its detractors since I don't know it well. This sort of phenomenon may be one explanation.
That said, not all exploratory engineering is equally valid. It is possible to be too speculative by speculating that provably impossible tasks are possible. I think it's fine to theorize about systems that don't yet exist, so long as we have good reasons to believe they could one day exist. Unfortunately, I've seen a lot of theory done in models that are demonstrably at odds with the laws of physics, e.g., ideas like stuffing 2500 strands of DNA (more than the number of atoms in the universe) into a single test tube to "test all solutions in parallel" to some hard combinatorial problem.
But, when someone tells me that work on algorithmic self-assembly is at odds with reality because they have never seen something like that happen in an experiment, I imagine them telling electrical engineers prior to the 1930s that just because they'd never seen an electronic circuit do arithmetic on binary numbers, the notion must be at odds with reality. There are ways to prove that certain systems cannot be built, but simply lacking the imagination to build them is not one of these ways.

Luke: When doing prescriptive theory, it's very handy to have explicit, formal models of the laws which govern what you're trying to do. E.g. it would be hard to do quantum algorithm design if there was no principled way to abstract away from implementation details, but it turns out there is, so people can actually prove certain results even though the quantum computers don't exist yet. One problem in AGI theory is that nobody knows what an AGI will look like, and there's very little you can demonstrate about the ultimate capabilities of AGI with (e.g.) physics and computational complexity theory alone. At least with AIXI (and its computable variants) one can make principled arguments about what certain classes of AGIs would do or not do, but that kind of precision is rarely available in the field so far.
How is prescriptive theory in your field enabled by formal models which abstract away from implementation details?

Dave: The abstract tile assembly model is one workhorse for prescriptive theory. It abstracts away the proposed DNA implementation. In fact, it's really based more on generic crystallization theory than DNA specifically. Any crystallizing molecule forming a square lattice, held just under the temperature where the rate of monomer attachment is just barely larger than the rate of detachment of monomers held by two bonds (assuming all bond strengths are equal), should have the same effect of enforcing cooperative binding: that tiles end up permanently attached only when held by at least two bonds. There may be other physical ways to enforce this sort of cooperative binding using techniques specific to DNA, or biological molecules more generally.
There have been variants of the model that, while based on plausible physical mechanisms for implementation, abstract away the precise method of implementation. Recently there's been some work with models of signal-passing tiles, which are able to transmit information after they attach in response to receiving a signal from a neighboring tile. They can in turn send signals to other tiles, and the signals are used to do things like activate or deactivate bonds, which allows a bit more sophisticated assembly to happen than in the original model with its passive tiles. A reasonable restriction to assume is that only a fixed number of signals can be sent across a tile before they are used up; this is based on the idea that if sending a signal is thermodynamically favorable (whatever the exact mechanism), then it should put the tile in a lower energy state after sending the signal than before, and at some point it gets to a minimum energy.
There is some unpublished lab work I'm aware of implementing the beginnings of this sort of thing using DNA, but the point of the model is to say, "What's a reasonable abstract model of signalling tiles that seems plausible and likely to capture lots of different ways of doing it?" and then ask what the model can do. The hope is that the results apply not just to one potential implementation of signalling tiles, but lots of different ways.
Another example is some work on motion: what if the monomers are able to move relative to each other after they attach? Again, the physical mechanism for this could be some sort of molecular motor driven by ATP, or it could be driven by the fact that more base pairs are attached after the motion than before.
Moving outside of algorithmic self-assembly to molecular computing more generally, one of my favorite models is one of the oldest models in all of science: chemical reaction networks. These are lists of reactions like X + Y –> A + B, indicating that when a molecule of X and a molecule of Y collide, they might change into molecules of A and B. It's abstracting away the question, "What exactly are X and Y and A and B, and why would they have this effect if X and Y collide?" As long as your system is based on molecules floating around a well-mixed liquid and bumping into each other (with no control of who's going to bump into whom), and some of them might change as a result of colliding, then the model of chemical reaction networks applies to it.
The model has been around since the 1860's. It started to get fairly serious scrutiny in the 1970s by a group of mathematicians studying what sorts of long-term behaviors are possible for different classes of chemical reaction networks: oscillations, steady states, etc. More recently, a more computational perspective has been applied, asking questions like, "What sort of computation is possible if we are allowed to engineer arbitrary chemical reactions?"
This may seem like the ultimate in theoretical navel-gazing: what makes anyone think we could engineer arbitrary chemical reactions? Well, assuming a reasonable model of a physical mechanism called DNA strand displacement (one of the experimental workhorses of the field of DNA nanotechnology), it was shown a few years ago that every abstract chemical reaction network you could think of can be systematically translated into a system of DNA complexes that undergo actual reactions imitating the abstract reactions.
Of course, that's just one way to try to implement a chemical reaction network. When we do prescriptive theory with chemical reaction networks, we aren't thinking of that particular implementation. We know it's out there and it helps us sleep at night, but maybe there are other ways to engineer artificial chemical reactions, and work on abstract chemical reaction networks applies to those systems as well.

Luke: Thanks, Dave!
The post Dave Doty on algorithmic self-assembly appeared first on Machine Intelligence Research Institute.

Dave Doty on algorithmic self-assembly


Ariel Procaccia on economics and computation

 Ariel Procaccia is an assistant professor in the Computer Science Department at Carnegie Mellon University. He received his Ph.D. in computer science from the Hebrew University of Jerusalem. He is a recipient of the NSF CAREER Award (2014), the (inaugural) Yahoo! Academic Career Enhancement Award (2011), the Victor Lesser Distinguished Dissertation Award (2009), and the Rothschild postdoctoral fellowship (2009). Procaccia was named in 2013 by IEEE Intelligent Systems to their biennial list of AI's 10 to Watch. He is currently the editor of ACM SIGecom Exchanges, an associate editor of the Journal of AI Research (JAIR) and Autonomous Agents and Multi-Agent Systems (JAAMAS), and an editor of the upcoming Handbook of Computational Social Choice.

Luke Muehlhauser: Much of your work concerns mechanism design, which Leonid Hurwicz and Stanley Reiter once described succinctly:
In a [mechanism] design problem, the goal function is the main "given," while the mechanism is the unknown. Therefore, the design problem is the "inverse" of traditional economic theory, which is typically devoted to the analysis of the performance of a given mechanism.
In Brânzei & Procaccia (2014), you and your co-author go on to explain that:
Arguably, the most sought-after property [for a mechanism] is truthfulness, more formally known as incentive compatibility or strategyproofness: an agent must not be able to benefit from dishonestly revealing its private information.
But moreover, it would be nice if agents could verify the truthfulness of a mechanism for themselves in a computationally efficient way. How would you describe the current state of progress on the problem of (efficiently) verifiably truthful mechanisms?

Ariel Procaccia: The work on verifiably truthful mechanisms has been rather sporadic. Over the last decade, several groups of researchers have leveraged model checking techniques to address this problem. The main shortcoming of the model checking approach is that it is computationally intensive, and, in particular, does not lead to provable computational efficiency. Kang and Parkes (2006) studied "passive" verification algorithms that do no directly analyze the mechanism itself, but observe its sequence of inputs and outputs.
In the work with Simina Brânzei that you mentioned, we take a novel, three-step approach: (i) we provide a formalism that allows us to specify the structure of mechanisms, (ii) we construct a truthfulness verification algorithm that receives as input mechanisms specified using the formalism of step (i), and (iii) we analytically measure the quality of mechanisms whose truthfulness can be efficiently verified using the algorithm of step (ii). As a proof of concept, we applied our approach to the design of truthfully verifiable mechanisms in a specific domain (facility location on the real line). While I think our approach is quite exciting, it is too early to say whether it can be extended to richer domains.
To summarize, the study of verifiably truthful mechanisms is still in its infancy; there is still a lot of work to be done!

Luke: Some of your recent work examines "cake cutting" (fair division of a divisible good). According to your survey article for CACM, modern computational work on cake-cutting grew out of earlier work on fairness in recreational mathematics and microeconomics. To your knowledge, has there been much interaction between philosophical work on fairness (e.g. Rawls) and the fairness research in mathematics, economics, and computer science? (Along with Scott Aaronson, I'm typically curious about the dynamic in which philosophical problems are studied outside philosophy. I often feel that progress is faster when a problem is studied outside philosophy journals.)

Ariel: Work on fair division is inspired by philosophy, and, I believe, raises new philosophical questions. For example, you mentioned John Rawls, the famous philosopher who advocated (in his book A Theory of Justice) a maximin principle of fairness: society should maximize the prospects of the least fortunate members. This principle was adopted by the systems community. Indeed, Demers et al. (1989) write: "The maxmin fairness criterion … states that an allocation is fair if (1) no user receives more than its request, (2) no other allocation scheme satisfying condition 1 has a higher minimum allocation, and (3) condition 2 remains recursively true as we remove the minimal user and reduce the total resource accordingly". The CS theory community has also devoted quite a bit of attention to a related algorithmic problem – allocating indivisible goods to maximize the minimum value of any player. In fact, some theorists believe that Santa Claus uses this notion of fairness to allocate toys!
In contrast, the cake cutting literature more commonly studies "Boolean" fairness criteria. Perhaps the most compelling and intuitive axiom is envy-freeness: Each player should value his own allocation at least as much as he values any other player's allocation. But Aumann and Dombb (2010) show that the two different approaches are at odds. Specifically, they formally construct a cake cutting instance in which constraining the allocation to be envy free significantly reduces the happiness of the least happy player under an optimal allocation. I think it is fascinating that this mathematical result informs fundamental philosophical questions about fairness, e.g.,  which is preferable, a richer society where some individuals envy others, or a poorer society where individuals are content with their own shares? While fair division is being taught in philosophy courses, to the best of my knowledge the potential contribution of recent mathematical results to theories of justice and fairness has not been explored.
And, a bit more whimsically, see my blog post for an alternative connection between fair division and philosophy.

Luke: You've also worked on the fair division of indivisible goods. In Procaccia & Wang (2014), you and your co-author write:
typical real-world situations where fairness is a chief concern, notably divorce settlements and the division of an estate between heirs, involve indivisible goods (e.g., houses, cars, and works of art) — which in general preclude envy-free, or even proportional, allocations. As a simple example, if there are several players and only one indivisible item to be allocated, the allocation cannot possibly be proportional or envy free.
How would you summarize the current state of solutions to the problem of fair division of indivisible goods?

Ariel: As you noted, dividing indivisible goods is tricky, because classic notions of fairness like envy-freeness (which we discussed earlier) are clearly infeasible. We do know that if players' values for goods are drawn at random, there exists an envy-free allocation with high probability under mild assumptions (see Dickerson et al., 2014). Alternatively, for the case of two players, Brams et al. (2014) propose an interesting protocol that guarantees envy-freeness (and other properties), but may not allocate all goods.
In my view, we need a notion of fairness that can always be achieved, for any number of players. In the paper you mentioned (with Junxing Wang), we study the notion of maximin share (MMS) guarantee, proposed by Eric Budish. In a nutshell, the MMS guarantee of a given player, assuming there are n players, is the value the player can guarantee by dividing the goods into n bundles, letting other players choose bundles (in some order), and getting the remaining bundle. It would have been great if it had been possible to always divide the goods in a way that satisfies the players' MMS guarantees, but we show that this is not the case, even when valuations are additive (that is, my value for a bundle is the sum of values for individual goods in the bundle). However, we show that it is always possible to divide the goods in a way that each player gets two thirds of his MMS guarantee.
We use this theoretical result to design what is, in my highly biased opinion, arguably the most practical method for dividing indivisible goods among any number of players. First, we let the players assign points to goods. Second, in order to produce an allocation, we consider three levels of fairness: envy-freeness, proportionality (each of the n players gets 1/n of his value for the entire set of goods), and MMS guarantee; each level of fairness is stronger than the subsequent one. We return the allocation that maximizes social welfare, that is, the sum (over players) of points players assign to their allocated bundles, subject to the strongest feasible level of fairness. If the strongest feasible level of fairness is MMS guarantee, we maximize the fraction c such that we can provide to all players a c-fraction of their MMS guarantee. Our theoretical result implies a lower bound on the guaranteed level of fairness — c is at least 2/3 — but in practice (based on extensive simulations by several groups) it seems that c=1 should always be feasible with respect to realistic instances.
The method I just described is one of several practical fair division methods implemented in Spliddit, a not-for-profit fair division website that I've been working on (with Jonathan Goldman) for the past year; it is set to launch in the coming months.

Luke: From your perspective, how does computational fair division theory interact with computational voting theory? If we can't get a certain kind of satisfactory result via fair division theory, can we sometimes achieve "fairly good" results from a non-manipulable voting scheme, or vice-versa?

Ariel: Social choice and fair division share a common goal: aggregating individuals' possibly conflicting preferences in a way that achieves a desirable outcome. However, the two settings are quite different, in that fair division is typically concerned with the properties of each individual's allocation, and the fairness axioms we discussed earlier hold for each individual separately; whereas in the classic social choice setting, individuals specify their preferences over a typically unstructured space of societal outcomes. Typically, methods developed in one of the two fields should not be directly applied to the other. That said, some extensions — such as voting over combinatorial domains, and fair division with externalities — bring the two fields closer together, and their relation is explored in detail by Fleurbaey and Maniquet (2011).
As an aside (since you asked specifically about non-manipulable rules), non-manipulable fair division schemes are quite common (see, e.g., Chen et al., 2013), but we do not know how to achieve non-manipulability (a.k.a. strategyproofness) in computational voting theory. The Gibbard-Satterthwaite Theorem rules out the existence of "reasonable" voting rules that cannot be manipulated. A point that came up in your interview with Toby Walsh is that some voting rules are worst-case hard to manipulate (with respect to some formulations of the manipulation problem). However, I am more pessimistic than Toby about the degree to which such results actually provide a barrier against manipulation, and, indeed, almost a decade of work on designing voting rules that are computationally hard to manipulate on average has so far produced only negative results (see, e.g., Mossel and Racz, 2012).

Luke: What kinds of progress in fair division theory or voting theory do you expect or not-expect in the next 15 years? E.g. you seem pessimistic about finding non-manipulable "reasonable" voting rules. What else are you pessimistic or optimistic about?

Ariel: The short answer (to the first part of the question) is "applications".
In (computational) fair division, I feel that we already have an excellent understanding of how to solve real-world — even day-to-day — fair division problems (such as rent division), via clever methods developed over decades. But very few of these methods have ever been implemented or used in practice. The website Spliddit, which I mentioned earlier, aims to take a first step towards giving people access to some of the most practical methods (including ones developed in my group). I believe it will also give us a clearer understanding of which problems people want to solve, and which solutions are viewed as satisfactory (sometimes all the mathematical guarantees in the world are not enough to make people happy!). It's also worth mentioning that the Wharton Business School at Penn has recently adopted a beautiful fair division method (which they call Course Match) for the allocation of MBA courses to students. In the next 15 years, I expect to see a proliferation of usable fair division systems.
In (computational) voting theory, I am optimistic about potential applications to crowdsourcing and human computation systems, for two reasons. First, in contrast to political elections, the designer of a crowdsourcing system can easily evaluate and choose any voting rule. Second, a classical, well-studied model of voting (which dates back to the marquis de Condorcet) views voters as noisy estimators of an underlying true ranking of the alternatives by quality; and while this model is questionable in the context of political elections (where opinions are subjective), it is a perfect fit with certain crowdsourcing settings. In fact, existing systems like EteRNA already use voting in order to aggregate information; the challenge is to understand which voting rules should be used. Work done in my group (e.g., Caragiannis et al., 2013), and by others, provides some answers to this question. In the next 15 years, I believe we will see crowdsourcing systems whose design is directly informed by computational voting theory.

Luke: Thanks, Ariel!
The post Ariel Procaccia on economics and computation appeared first on Machine Intelligence Research Institute.


Suzana Herculano-Houzel on cognitive ability and brain size

Suzana Herculano-Houzel is an associate professor at the Federal University of Rio de Janeiro, Brazil, where she heads the Laboratory of Comparative Neuroanatomy. She is a Scholar of the James McDonnell Foundation, and a Scientist of the Brazilian National Research Council (CNPq) and of the State of Rio de Janeiro (FAPERJ). Her main research interests are the cellular composition of the nervous system and the evolutionary and developmental origins of its diversity among animals, including humans; and the energetic cost associated with body size and number of brain neurons and how it impacted the evolution of humans and other animals.
Her latest findings show that the human brain, with an average of 86 billion neurons, is not extraordinary in its cellular composition compared to other primate brains – but it is remarkable in its enormous absolute number of neurons, which could not have been achieved without a major change in the diet of our ancestors. Such a change was provided by the invention of cooking, which she proposes to have been a major watershed in human brain evolution, allowing the rapid evolutionary expansion of the human brain. A short presentation of these findings is available at TED.com.
She is also the author of six books on the neuroscience of everyday life for the general public, a regular writer for the Scientific American magazine Mente & Cérebro since 2010, and a columnist for the Brazilian newspaper Folha de São Paulo since 2006, with over 200 articles published in this and other newspapers.

Luke Muehlhauser: Much of your work concerns the question "Why are humans smarter than other animals?" In a series of papers (e.g. 2009, 2012), you've argued that recent results show that some popular hypotheses are probably wrong. For example, the so-called "overdeveloped" human cerebral cortex contains roughly the percentage of total brain neurons (19%) as do the cerebral cortices of other mammals. Rather, you argue, the human brain may simply be a "linearly scaled-up primate brain": primate brains seem to have more economical scaling rules than do other mammals, and humans have the largest brain of any primate, and hence the most total neurons.
Your findings were enabled by a new method for neuron quantification developed at your lab, called "isotropic fractionator" (Herculano-Houzel & Lent 2005). Could you describe how that method works?

Suzana Herculano-Houzel: The isotropic fractionator consists pretty much of turning fixed brain tissue into soup – a soup of a known volume containing free cell nuclei, which can be easily colored (by staining the DNA that all nuclei contain) and thus visualized and counted under a microscope. Since every cell in the brain contains one and only one nucleus, counting nuclei is equivalent to counting cells. The beauty of the soup is that it is fast (total numbers of cells can be known in a few hours for a small brain, and in about one month for a human-sized brain), inexpensive, and very reliable – as much or more than the usual alternative, which is stereology.
Stereology, in comparison, consists of cutting entire brains into a series of very thin slices; processing the slices to allow visualization of the cells (which are otherwise transparent); delineating structures of interest; creating a sampling strategy to account for the heterogeneity in the distribution of cells across brain regions (a problem that is literally dissolved away in the detergent that we use in the isotropic fractionator); acquiring images of these small brain regions to be sampled; and actually counting cells in each of these samples. It is a process that can take a week or more for a single mouse brain. It is more powerful in the sense that spatial information is preserved (while the tissue is necessarily destroyed when turned into soup for our purposes), but on the other hand, it is much more labor-intensive and not appropriate for working on entire brains, because of the heterogeneity across brain parts.

Luke: Your own work emphasizes importance of the brain's sheer number of neurons for cognitive ability. What do you think of other recent results (e.g. Smaers & Soligo 2013), which emphasize the apparent importance of mosaic brain reorganization?

Suzana: Mosaic brain organization is a fact. It describes the independent scaling of different parts of the brain across species in evolution, as opposed to every brain part scaling in line with every other part (what Barbara Finlay describes as "linked regularities"). Mosaic scaling in evolution is seen for example in the enormous size that some structures exhibit in some species but not others, relative to the rest of the brain: the common squirrel, for instance, has an enormous superior colliculus, involved in visual processing, that other rodents of a similar brain size do not have; moles and shrews, who rely heavily on olfaction, have even more neurons in the olfactory bulb than in the cerebral cortex – something that is quite different from rodents of a similar brain size (this is work under review).
In the context of our work, mosaic brain evolution means that the numbers of neurons allocated to different brain structures can vary independently across said structures: while, say, the superior colliculus and the visual thalamus tend to gain neurons hand in hand, a particular species can gain neurons much faster in the superior colliculus than in the visual thalamus, for instance. Mosaic brain evolution also refers to the possibility of one system (for instance, vision) expanding faster than another system (say, audition). There is the occasional surprise, however. For instance, we have found that, while primates are highly visual and have a large proportion of the cortex devoted to vision (indeed, much larger than the cortical areas devoted to audition), this proportion (as well as the relative number of cortical neurons devoted to vision) does NOT increase together with increasing brain size. Many more cortical neurons are involved in visual than in auditory processing, yes – but that proportion is stable across primate species. Still, species that rely more heavily on other sensory modalities should have a different distribution of neurons. Indeed, the mouse, contrary to primates, has a far larger percentage of cortical neurons involved in somatosensory processing than primates; and, as I mentioned above, moles and shrews have more neurons in the olfactory bulb than in the whole cortex – a pattern that is not seen in other brains of a similar size.
Even more remarkably, we have found that the apparent expansion of the cerebral cortex in mammalian evolution, varying from less than 40% of brain size in the smallest mammals to over 80% in humans and other even larger brains, is NOT the result of an expansion in numbers of neurons in the cortex: regardless of the relative size of the cortex across different species, it has about 20% of all brain neurons – even in the human brain. That's another example of how apparent mosaic evolution (of one structure taking over the others) can actually not be mosaic evolution. It all depends on the precise variable examined.

Luke: To be more specific: Do you think your view that the human brain is essentially a "linearly scaled-up primate brain" is in significant tension with Smaers & Soligo (2013)'s principal component analysis (PCA) of neural structure variation in primate species?
Smaers and Soligo claim their PCA shows that while (1) the principal component which accounts for 25.8% of the variance is closely correlated with brain size, it's also the case that (2) the remaining principal components — which account for a large majority of the variance — are not closely correlated with brain size. In particular, they claim that their phylogenetic analysis shows that "a clade-specific investment in particular brain formations (prefrontal white matter, prefronto-striatal and higher motor control) in combination with increased absolute brain size differentiates great apes (and humans) from other primates" (emphasis added).

Suzana: No, there is no tension. What we see is that the human cerebral cortex as a whole, like the human cerebellum as a whole, and the remaining areas of the brain as a whole, are linearly scaled-up in their numbers of neurons compared to the same structures in other primate brains. This means that the relationship between the particular size of a brain structure and its number of neurons is constant and shared across primate species. This does not at all imply or require that all brain areas have the same ratios of numbers of neurons relative to one another, which is what mosaic evolution states: given brain regions can become relatively enlarged or reduced compared to others, and still maintaing the same relationship between their number of neurons and mass as seen across species.
Having said that: yes, the human brain as a whole does fit the relationship between brain mass and total number of neurons that we found in other primates. As far as I understand, the relative differences that Jeroen Smaers concentrates on are very small – he is looking at the residuals of the relationships, and as many still do, using normalization to external parameters. I believe it is time that we stop assuming that things such as brain mass, or worse, body mass, are true independent parameters (which they very likely aren't; brain mass, in particular, is the result of the cellular composition of the brain and its parts, and as such cannot determine much), and start looking at the absolute values of the different parameters – which is what we have been doing in my lab, trying to keep the number of assumptions to a minimum.

Luke: What are the current estimates of neuron quantities for the largest brains, in elephants and whales? Has you isotropic fractionator process been used on those brains yet, or are their current plans to do so?

Suzana: We have a paper under review on the number of neurons in the brain of the African elephant. The elephant is a great test of our hypothesis that numbers of neurons are a strong limiting factor to cognitive abilities exactly because of its large brain, at 4-5 kg, which is about 3x the mass of the human brain: we predicted that it should have fewer neurons than the human brain, despite being larger than ours.
As it turns out, the answer was even more interesting: the elephant brain as a whole has 3 times the number of neurons of the human brain, 257 billion neurons against an average 86 billion in ours, BUT 98% of those neurons are located in the elephant cerebellum, which turns out to be a major outlier in the numeric relationship between numbers of neurons in the cerebral cortex and cerebellum. While other mammals (humans included) have about 4 neurons in the cerebellum to every neuron in the cerebral cortex, the elephant has 45 neurons in the cerebellum to every neuron in the cerebral cortex. All we can do for now is to speculate on the reason for this extraordinary number of neurons in the elephant cerebellum, and the most likely candidates right now is to me the fine sensorimotor control of the trunk, a 200-pound appendage that has amazingly fine sensory and motor capabilities, which are known to involve the cerebellum.
Despite the enormous number of neurons in the elephant cerebellum, its cerebral cortex, which is twice the size of ours, has only one third of the neurons in an average human cerebral cortex. Taken together, these results suggest that the limiting factor to cognitive abilities is not the number of neurons in the whole brain, but in the cerebral cortex (to which I would add, "provided that the cerebellum has enough neurons to shape activity in the cerebral cortex").
We don't have data on whales yet, but that research is underway in our lab – along with research on carnivores, who we predict to have more neurons than the large artiodactyls that they prey upon.

Luke: What other results in this line of research to you hope to have from your lab or other labs in the next 5 years?

Suzana: We're extending our analysis to the other mammalian branches — xenarthrans, marsupials, carnivores, chiropterans and perissodactyls — and to non-mammalian vertebrates (birds, reptiles, fish, amphibians) and even some invertebrates. The goal is to achieve a full appreciation and understanding of brain evolution, which will give us, amongst other things, a view into the mechanisms that have led to the generation of brain diversity in evolution. Such a comparative analysis also gives us insights onto the most basic features of the brain: those that are shared by all mammals. As it turns out, there are some, and they are very revealing. One of them, for instance, is the addition of glial cells to the brain, in numbers which seem to be regulated by a self-organized process that is shared across all species examined so far.
We are also focusing our analysis on the prefrontal cortex, that is, the associative areas of the cerebral cortex. While it has been very informative to compare total numbers of neurons in the cerebral cortex across species, it is supposedly those neurons in the associative areas that should really limit the cognitive abilities of the species. This more specific analysis should allow us a new glimpse into the brains of different species and how they compare to the human brain. In this regard, we have a paper in the works comparing the distribution of neurons along the human cerebral cortex with that in other, non-human primate species.
We are also moving into the spacial properties of the tissue: how neurons are distributed, and how this is related to the distribution of astrocytes and vasculature, for instance. But one large question that remains is how numbers of synapses compare across humans and other species. That is also something that we are working on.

Luke: Thanks, Suzana!
The post Suzana Herculano-Houzel on cognitive ability and brain size appeared first on Machine Intelligence Research Institute.


Martin Hilbert on the world's information capacity

Martin Hilbert pursues a multidisciplinary approach to understanding the role of information, communication, and knowledge in the development of complex social systems. He holds doctorates in Economics and Social Sciences, and in Communication, a life-long appointment as Economic Affairs Officer of the United Nations Secretariat, and is part of the faculty of the University of California, Davis. Before joining UCD he created and coordinated the Information Society Programme of United Nations Regional Commission for Latin America and the Caribbean. He provided hands-on technical assistance to Presidents, government experts, legislators, diplomats, NGOs, and companies in over 20 countries. He has written several books about digital development and published in recognized academic journals such as Science, Psychological Bulletin, World Development, and Complexity. His research findings have been featured in popular outlets like Scientific American, WSJ, Washington Post, The Economist, NPR, and BBC, among others.

Luke Muehlhauser: You lead an ongoing research project aimed at "estimating the global technological capacity to store, communicate and compute information." Your results have been published in Science and other journals, and we used your work heavily in The world's distribution of computation. What are you able to share in advance about the next few studies you plan to release through that project?

Martin Hilbert: When we first started out, we were rather surprised how little work had been done in the area of quantifying our information and communication capacity. We have statistics about everything and know how many cars and trees there are, and have estimates about the social and economic impact of shoe sales and carbon exhaust, but living in an information age, only very few pioneering studies have been done about how much information there is1 . We felt the topic would deserve a more coherent treatment. So we set up three basic stages:
First, creating the basic database: how much is there? How much is stored, how much communicated, how much can we compute? This was by far the most tedious part and resulted in a 300 page methodological appendix, where we list the more than 1,100 sources and databases we combined to create these numbers. We found some interesting things here, such as the fact that our computational capacity has grown between 2 to 3 times faster than our information and communication capacity since the 1980s. This is not only good news for the machine intelligence community, but also for human kind as a whole: while it currently seems like we are drowning in an information overload that stems from the sustained 25 – 30 % annual growth of information storage and communication capacities, we should be able to make use of the computational power (growing at 60 – 90 % per year) to make sense and eventually tame of all of this information.
Second, how can we describe it? We found several surprising things here. Some of the most basic assumptions of the digital revolution literature appeared in a totally new light. For example, usually, it is assumed that the digital revolution has increased global communication equality. The problem with this conclusion is that it based on the head-count of digital devices and subscriptions as the main indicator, so since there are more phones now than in the 1980s (with a current mobile phone penetration of 90% worldwide), the conclusion usually is that equality must have increased. However, not all phones are equal nowadays. So looking at the distribution of communication capacities, we found that communication capacity in 1986 was actually more equally distributed than in the 1990s and the 2000s! In the 1980s there were only fixed line phones, but everybody had "equally little". Afterward, the myriad of communication technologies increased the inequality among countries and within countries. Only very recently have we re-establish the pre-1990 equality levels in terms of our bits-capacity. In other words: while we are all much better off in absolute terms ("we all have more"), relative information inequality in terms of information capacity continuously opens up with each new innovation ("we are not automatically more equal"). The digital divide turns out to be a moving target! We do not yet have any idea yet about the social, economic and long-term political consequences of this ever-changing inequality in information and communication capacities among and within countries…
Another traditional assumption of the digital revolution literature is that we now live in a multimedia age, with an unprecedented share of moving videos and audio sounds. Looking at the evolution of the content of the world's information and communication capacity, we actually found that the relative share of text and still images captures a larger portion of the total amount than before the digital age! Text merely represented 0.3% of the (optimally compressed) bits that flowed through global information channels in 1986 but grew to almost 30% in 2007. Back in the pre-digital age, text mainly appeared on paper, while telephone channels were filled with audio (voice) and many homes hoarded vast amounts of video material in VHS libraries, etc. The proliferation of alphanumeric text on the web and in vast databases in a phenomena of the digital age. The fact the digital age turns out to be a "text and image age" is good news for big-data analysts who extract intelligence from more easily analyzable text and image data.
And as a last example, we were able to parse out how much of the global information and communication explosion was driven by more, and how much by better technology. We found that technological progress has contributed between two to six times more than additional technological infrastructure to our global bits capacity. While infrastructure actually seems to reach a certain level of saturation (at roughly 20 storage devices per capita and 2 to 3 telecommunication subscriptions per capita), informational capacities are still expanding quickly. We also found that additionally to progress in better hardware, software for information compression turns out to be an important and often neglected driver of the global growth of technologically-mediated information and communication capacities. We estimate that better compression algorithms alone allowed us to triple our communication capacity: in the 2000s we could send 3 times as much information through the same channel than int he 1980s, thanks to compression. This underlines the importance to measure information and communication capacities directly in bits and bytes. Traditional statistics provided by the national telecom or science authorities (such as the FCC or NTIA) merely count devices and subscriptions. But this indicator does not tell as much anymore.
As a natural third step after this rather descriptive work, we are currently working on deepening our understanding of the social, economic and political impact of this information and communication flood. The first question here is: impact on what? Per definition, a general-purpose technology (like digital technology) affects all aspects of human conduct, which gives us the free choice for the area of impact. The common theme is that this social change was produced by information, so we have to involve the [bit-metric]. With it, we can measure economic impact as [US$/kbps] or democratic participation by [participation/kbps]. These kind of measures show us that somebody makes more or less effective use of the same communication capacity than somebody else. The other way around, we can also ask about [kbps/US$] and try to understand why some have more communication capacity while starting from the same economic resources. We can then fine-tune the [bit-metric] and analyze how the communication capacity relates to additional attributes of interest of the capacity itself (e.g. mobile or fixed; individual or shared; private or public; always-on or sporadic, etc.), or to different content. It will enable us to take a more systematic approach to ideas like the information overflow: how much of which kind of content, from which kind of technology has which kind of impact on what? How does the supposed curve of "decreasing returns to information" look like empirically and in which task? More elaborate indexes and models can even integrate an arbitrary combination of these variables with communication capacity, just as economist have come up with a myriad of ways to evaluate the distribution of monetary currency with a society. In the statistical analysis of economics the unifying ingredient is naturally $, while in the statistical analysis of technologically mediated communication the unifying ingredient is naturally the bit. Obviously, bits only say "how much", not "how good". Once we understand the impact of "more" or "less" bits, we can then even go on and ask about "better" or "worse" bits (or more of less suitable kinds of bits). The "better" or "worse" will appear as an unexplained "residual" in our impact studies. In other words: instead of actively defining what is good and what is bad, we corner it by at least taking out the co-founder which stems from "more" or "less". Our main argument is that "quantity" is the lower hanging fruit, and that it must precede any question about quality. Otherwise we will helplessly confuse more- with better- information, and the other way around. So any impact must be normalized on the amount = [impact / bit], and for this we need to start measuring bits and bytes. Which brings us back to the reason why we started all of this…

Luke: In the course of your research so far, which trends related to information and information technology have you found to be roughly exponential during a certain period, and which trends have you found to be not exponential during some period?

Martin: Social systems are too complex to make identify pure distributions and dynamics, but within these limitation, they are all "roughly" exponential. Machines' application-specific capacity to compute information per capita has roughly doubled every 14 months over the past decades in our sample, whereas the per capita capacity of the world's general-purpose computers has doubled every 18 months. The global telecommunication capacity per capita doubled every 34 months, and the world's storage capacity per capita required roughly 40 months. Per capita broadcast information has only doubled roughly every 12.3 years, but still grows exponentially. One the one hand, this stems from pure technological progress, such as Moore's laws. On the other hand, this stems from the diffusion of the technology through social networks. This diffusion mechanism follows a logistic S-shaped curve, starting of with exponential growth until an inflection point, after which saturation converts the process in a reverse exponential. So actually we have two exponential processes (technological progress and social diffusion).
In a recent study I've shown that these two exponential processes combined can result in an authentic power-law distribution among the number of technological devices and their performance (so-called power-laws, scale-free-, Pareto- or Zipf distributions consist of two exponential distributions). I showed this with the distribution of supercomputers: there are exponentially few supercomputers with exponentially large computational power, and exponentially many supercomputers with exponentially lower computational capacity. Both line up in an almost spooky order, such as if some kind of super organizer subscribes the U.S. Department of Energy to order one supercomputer with performance x, and Los Alamos Laboratory, IBM and a couple of universities to order some with exactly lesser performance x-α, etc. Of course, there's no such super-organizer, but social complexity leads to this stable order, grown out of two complementary exponential processes. Recognizing such social patterns can be useful, since it provides predictive insights into the evolution of highly uncertain technology markets.

Luke: Which research investigations would you most like to see (in this line of work) over the next 5 years, whether conducted by yourself or others?

Martin: One very useful contribution would be the continuous reporting of the growth and nature of our information stock and our informational capacities. Together with my co-author Priscila Lopez, we were able to create 20 year long time series, covering over 60 technological families, but this was a two person effort, mainly driven out of curiosity. We have not counted with the resources to sustain this effort continuously. The level of detail and the scope of the inventory should also be extended.
For example, for storage and communication we normalized the amount of information on the available optimal level of compression. This allowed us to gain insights on the amount of information, not merely the available hardware infrastructure (the same bandwidth can store/communicate different amounts of information, depending on how compressed the content). For the case of computation, however, we simply had to use MIPS, which is a hardware metric. Of course, during recent decades, computational algorithms also became more efficient. The same hardware can certainly solve several of the same problems much faster now than 20 years back. We didn't have the resources to go into this distinction. The continuous effort of recording the growth and the nature of our information capacity will surely become an important corner stone of understanding reality in a digital world, and is therefore indispensable.
Besides this empirical effort, I think it will be important that we deepen our theoretical understanding of the way social organization and social dynamics are currently being "algorithmified". Social procedures, routines, habits, customs, and also laws have always been the central corner stones of civilization. These are currently being digitized, some in a more rigid, others in a less rigid fashion. Big Data is important here, as are agent-based computer simulations, and all kinds of decision support systems. This leads to profound changes what society is made of. We still lack a deeper understanding of the strengths, opportunities and threats of this ongoing process of social creative destruction.

Luke: Lack of information can be a major barrier for this kind of research. Sometimes the data you want to collect was simply never recorded by anyone, or perhaps it was recorded but never released publicly. If you could get your wish, what would change about how data about information and information technologies was recorded and disseminated? Could these changes be executed at a policy level, or an industry level, or some other level, if there was enough of a push for them?

Martin: I think this the conception of the lack of records and data is wrong.
It is true that data on ICT could be improved, and for the past 15 years I focused a large part of my effort at the United Nations Secretariat on adding ICT questions into household and business surveys worldwide. We got quite far and have achieved important improvements in this regard.2
However, after years of often fatiguing international policy dialogues (just imagine: you are not the only one lobbying for including "just one more question" into the national household survey!), and considering that the collective policy dynamic often leads to the lowest common denominator (often obsolete indicators…), I came to the conclusion that it might be easier to simply start creating an alternative database from scratch and to lead by example. That was the starting point of our undertaking that eventually included over 1,100 different databases, business records, and statistics. When I first proposed the idea of taking a 20 years inventory of the "The World's Technological Capacity to Store, Communicate, and Compute Information", we received cynic remarks even from befriended (and very recognized) colleagues in the field. The reaction included concepts like "utopian megalomania" (this also led to the Acknowledgments in the Science publication that states that we thank "colleagues who motivated us by doubting the feasibility of this undertaking", p.65). However, there is more information on ICT out there than we think, and the very same digital age often allows us to come up with proxies that enable us for very good estimates. The Big Data paradigm is not to be underestimated. The Big Data paradigm encourages us to embrace messiness of unstructured data, to look for highly correlating proxies, and to make up for it with redundancy from complementary sources. The wealth of "incomplete/ unstructured/messy" sources in a Big Data world often trumps the lack of one clean and centralized source. This also accounts for "datafying" the very own Big Data revolution!
This being said, if I "could get my wish", of course it would be nice if eventually both lines of work would concur, that is, if the global statistical machinery would start to also consider "information" as something worth of measuring continuously. Until now the UN and others have started to consider it (e.g. see Chapter 5 in Measuring the Information Society), but not yet embraced the idea fully. However, I think for this to happen it will still need much effort and we have to be proactive and show (a) that it's possible; (b) that it's worthwhile; and (c) which kind of stats are useful and which ones are not (which is still subject to an open trial and error process)

Luke: Thanks, Martin!
See this Special Section: Hilbert, M. 'How to Measure "How Much Information"? Theoretical, Methodological, and Statistical Challenges for the Social Sciences. Introduction.' International Journal of Communication 6 (2012), 1042–1055.Partnership On Measuring ICT For Development – The Global Information Society: a Statistical ViewThe post Martin Hilbert on the world's information capacity appeared first on Machine Intelligence Research Institute.

Martin Hilbert on the world’s information capacity


Why MIRI?

I wrote a short profile of MIRI for a forthcoming book on effective altruism. It leaves out many important details, but hits many of the key points pretty succinctly:
The Machine Intelligence Research Institute (MIRI) was founded in 2000 on the premise that creating smarter-than-human artificial intelligence with a positive impact — "Friendly AI" — might be a particularly efficient way to do as much good as possible.
First, because future people vastly outnumber presently existing people, we think that "From a global perspective, what matters most (in expectation) is that we do what is best (in expectation) for the general trajectory along which our descendants develop over the coming millions, billions, and trillions of years." (See Nick Beckstead's On the Overwhelming Importance of Shaping the Far Future.)
Second, as an empirical matter, we think that smarter-than-human AI is humanity's most significant point of leverage on that "general trajectory along which our descendants develop." If we handle advanced AI wisely, it could produce tremendous goods which endure for billions of years. If we handle advanced AI poorly, it could render humanity extinct. No other future development has more upside or downside. (See Nick Bostrom's Superintelligence: Paths, Dangers, Strategies.)
Third, we think that Friendly AI research is tractable, urgent, and uncrowded.
Tractable: Our staff researchers and visiting workshop participants tackle open problems in Friendly AI theory, such as: How can we get an AI to preserve its original goals even as it learns new things and modifies its own code? How do we load desirable goals into a self-modifying AI? How do we ensure that advanced AIs will cooperate with each other and with modified versions of themselves? This work is currently at a theoretical stage, but we are making clear conceptual progress, and growing a new community of researchers devoted to solving these problems.
Urgent: Surveys of AI scientists, as well as our own estimates, expect the invention of smarter-than-human AI in the 2nd half of the 21st century if not sooner. Unfortunately, mathematical challenges such as those we need to solve to build Friendly AI often require several decades of research to overcome, with each new result building on the advances that came before. Moreover, because the invention of smarter-than-human AI is so difficult to predict, it may arrive with surprising swiftness, leaving us with little time to prepare.
Uncrowded: Very few researchers, perhaps fewer than five worldwide, are explicitly devoted to full-time Friendly AI research.
The overwhelming power of machine superintelligence will reshape our world, dominating other causal factors. Our intended altruistic effects on the vast majority of beings who will ever live must largely reach them via the technical design of the first self-improving smarter-than-human AIs. Many ongoing efforts — on behalf of better altruism, better reasoning, better global coordination, etc. — will play a role in this story, but we think it is crucial to also directly address the core challenge: the design of stably self-improving AIs with desirable goals. Failing to solve that problem will render humanity's other efforts moot.
If our mission appeals to you, you can either fund our research or get involved in other ways.

The post Why MIRI? appeared first on Machine Intelligence Research Institute.

Why MIRI?


Thomas Bolander on self-reference and agent introspection

 Thomas Bolander, Ph.D., is associate professor at DTU Compute, Technical University of Denmark.
He is doing research in logic and artificial intelligence with primary focus on the use of logic to model human-like planning, reasoning and problem solving.
Of special interest is the modelling of social phenomena and social intelligence with the aim of creating computer systems that can interact intelligently with humans and other computer systems.

Luke Muehlhauser: Bolander (2003) and some of your subsequent work studies paradoxes of self-reference in the context of logical/computational agents, as does e.g. Weaver (2013). Do you think your work on these paradoxes will have practical import for AI researchers who are designing computational agents, or are you merely using the agent framework to explore the more philosophical aspects of self-reference?

Thomas Bolander: First of all, let me explain why I even started to study self-reference in the context of logical/computational agents. In my PhD I was working on how to formalise agent introspection, that is, agents representing and reasoning about their own mental states. One of the main motivations for this is that agents need introspection in order to understand their own role in social contexts, e.g. to ensure not to "be in the way of others" and to reason about how you can help other agents to achieve their goals. Another motivation is that introspection is essential in learning: realising the shortcomings of one's own knowledge and routines is required for a deliberate, goal-directed approach to gaining new knowledge and improve ones problem-solving skills.
Introspection can be into one's own knowledge, beliefs, intentions, plans, etc. In my PhD I focused on knowledge. Representing a propositional attitude such as knowledge in a logical formalisms can be done in two distinct ways, either by a predicate or by a modal operator. In most logical settings, the predicate solution is more expressive than the modal operator solution, but with high expressivity sometimes also comes trouble like paradoxical sentences leading to inconsistency. When formalising knowledge as a predicate in a first-order predicate logic, assuming a fairly reasonable, though idealised, set of properties (axioms) of knowledge leads to trivialising inconsistency (this is Montague's theorem, see e.g. my entry on Self-reference in the Stanford Encyclopedia of Philosophy). Inconsistency arises here because the logic becomes sufficiently expressive to express paradoxical self-referential sentences concerning knowledge like "this is sentence is not known". To remedy this problem, there are several possible routes to take: 1) move to a non-classical logic, e.g. a paraconsistent one; 2) restrict the set of axioms of knowledge or their applicability; 3) choose a modal operator approach instead.
In my PhD, I primarily considered option 2. Today my preferred option is 3, using modal logic instead of first-order predicate logic. Option 3 seems to be the currently most preferred option in both the logic and AI communities, with massive amounts of work on epistemic logic, temporal epistemic logic and dynamic epistemic logic based on modal logic. So why do people choose the modal logic approach? Well, one reason is certainly to avoid the problems of self-reference and inconsistency just mentioned. Another is that the modal logic approach often has lower computational complexity, which is essential for practical applications. So in a sense, to answer the question of "whether these paradoxes will have a practical import for AI researchers", a first answer could be: they already have. They already have in the sense that people shifted attention to other logics where these problems don't appear.
This answer is of course not entirely satisfactory, though. The way modal logic approaches solve problems of self-reference is by choosing logics in which self-referential sentences can not even be expressed. So the next question becomes: Are we going to miss them (the self-referential sentences)? Do they have an important role to play in reasoning that will eventually be required to build computational agents with strong introspective capabilities? This question is much harder to answer. The modal approach implies that knowledge becomes explicitly stratified. On the modal approach one can have first-order knowledge (knowledge about ontic facts), second-order knowledge (knowledge about knowledge about ontic facts), third-order knowledge (knowledge about second-order knowledge) etc., but there is no fixed-point allowing knowledge statements to refer to themselves. This stratification seems to be cognitively feasible, however. From studies in cognitive science, in particular in the so-called false-belief tasks, it is known that children only start to master second-order knowledge around the age of 4, and only third-order knowledge around the age of 7-8. And even adults seldom reason beyond third-order or at most fourth-order. This seems to talk in favour of some kind of explicit stratification also in the case of humans, and with a fairly fixed limit on the depth of reasoning. This picture fits better with the modal approach, where the stratification is also explicit (modal depth/depth of models). Furthermore, in the modal approach it is more straightforward to mimic limits on the depth of reasoning. Of course, having a fixed limit on one's depth of reasoning means that there will be certain problems you can't solve, like the muddy children puzzle with more children than your depth limit. But even within such limitations, the experience from human cognition suggests that it is still possible to do quite advanced social cognition and introspection.
So maybe the self-reference problems that are likely to follow from choosing a non-stratified approach are indeed more relevant in other areas such as the foundation of mathematics (set theory, incompleteness theorems) and the foundations of semantics (theories of truth) than for computational agents. I still find it a bit surprising, though, that I am more happy to accept an explicitly stratified approach to propositional attitudes of agents than as a solution to the self-reference problems in set theory: In set theory, I find the non-stratified approach of Zermelo-Fraenkel set theory much more reasonable than the stratified approach of type theory. But even if self-reference is not going to play a big practical role in the development of computational agents, it will probably always lurk in the background and affect people's choices of formalisms – as well as play an important foundational role in the same way it has in mathematics.

Luke: What are some examples of work that addresses (roughly) the kind of agent introspection you discuss in your thesis, but uses the modal logic approach to the challenge, or some other stratified approach?

Thomas: Most of the examples I give in the thesis involve quantifying over knowledge or beliefs, like "I believe that some of my beliefs are false" or "I believe that Alice knows more about X than I do". To quantify over knowledge or beliefs in the operator approach (modal logic approach), one needs at least first-order modal logic. When I left the predicate approach for the operator approach after my PhD, I essentially went all the way "down" to propositional modal logic, so I am not up to date with the uses of first- or higher-order modal logic for representations of introspective knowledge. I can give a couple of relevant references, but they are not very recent. One is the situation calculus extended with a modal knowledge operator as presented in the book The Logic of Knowledge Bases by Levesque & Lakemeyer (MIT Press, 2001). Another one is the book Reasoning about Rational Agents by Wooldridge (MIT Press, 2000). Both of these presents logics that extend first-order predicate logic by, among others, a knowledge operator, and they investigate the use of these logics as the basis for computational agents.
My choice of "downgrading" to propositional modal logic was not because I don't find the quantifying over beliefs non-interesting or non-important, but simply because I started to become very interested in the dynamics of knowledge, i.e., how the knowledge of an agent changes when an action is executed or an exogenous event occurs. This was essential for my goal of being able to construct planning agents with higher-order reasoning capabilities. I found a logical framework that suited my needs perfectly, dynamic epistemic logic (DEL). Since DEL is mostly studied in the context of propositional modal logic, this is where I currently find myself working. To my knowledge there is in only one paper on first-order dynamic epistemic logic, "Dynamic term-modal logic" by Barteld Kooi.

Luke: Are there examples of self-reflective computational agents (of the sort we've been discussing) in current industrial or governmental use, or is this work still entirely occurring at a more theoretical stage?

Thomas: To my knowledge it is still at the theoretical stage. However, its importance is becoming increasingly appreciated also among more applied AI researchers. As mentioned in my previous answer, I am now looking into epistemic planning: planning agents with higher-order reasoning capabilities (agents reasoning about the knowledge of themselves and other agents). The automated planning community is starting to become very interested in this. Bernhard Nebel, who is a very prominent figure in automated planning and robotics, even called epistemic planning the "next big thing". The reason is that planning formalisms and systems have to become general and expressive enough to deal with robots acting in uncertain environments, and environments containing other agents (multi-agent systems). This calls for epistemic reasoning in such systems.
There is also an increased awareness of the importance of social intelligence in AI systems, if these systems are to efficiently communicate and cooperate with humans. This could be in something like hospital robots (e.g. the TUG robot, which is very well-known for its current lack of social intelligence [Colin Barras, New Scientist, vol. 2738, 2009]), intelligent personal assistants like Siri on iPhone or Google Now on Android, and earthquake rescue robots that have to work in mixed teams with humans.
But integrating the existing formalisms for introspection and social intelligence into e.g. planning systems and then embed all of that into a robot is something which is still in its infancy. The most successful attempt up to now is probably the social robot bartender by Ron Petrick and co-authors [Petrick & Foster, ICAPS, 2013]1. They won a best paper award at the major international planning competition, ICAPS, for this last summer. Their underlying logical formalism is a restricted version of first-order epistemic logic (modal operator approach). The logic is restricted in many important ways in order to quickly get something with reasonable computational performance. The full logics of introspection and higher-order reasoning are still too expressive to be computationally feasible, and currently more work is needed to find the best trade-off between expressivity and computational efficiency that will make higher-order reasoning agents more attractive to the more applied researchers and the industry.

Luke: In general, what kinds of heuristics do you and your colleagues use to decide how to proceed with theoretical research in a broad problem category like "self-reflective reasoning in computational agents," which may be 10-30 years from practical application? How do you decide which subproblems to work on, and when to pivot to a different sub-problem or a different approach? And how willing are grantmakers to fund that kind of work, in your experience?

Thomas: Since I'm using logical formalisms to try to capture "self-reflective reasoning", any development of a new "theory" has a number of standard and clearly defined steps: 1) You define a suitable logical language and its semantics; 2) You define suitable calculi for reasoning within the logic (proof theory); 3) You give examples to show that your formalisms can capture important aspects of self-reflective reasoning; 3) You investigate the computational complexity and relative expressive strength of your logic; 4) You implement the calculi for reasoning, and optimise them if scalability is important for your proof of concept. Of course there are many choices to be made when you define your logical language, but this is usually driven by concrete examples: I want to be able to express these kinds of sentences or these lines of reasoning. Or, oppositely: I don't want my language to be able to express this, because it is known to lead to inconsistency or undecidability.
For instance, in my new work, a source of inspiration has been the false-belief tasks used in cognitive psychology to test the Theory of Mind (ToM) of humans. Theory of Mind is the ability to attribute mental states – beliefs, desires, intentions, etc. – to oneself and others. A goal for my logical formalism is that it can robustly deal with any of the existing false-belief tasks used on humans, and some suitably general closure of that set. This naturally drives research activities into finding logics that are suitable for representing that kind of reasoning in a natural way. It might be 20-50 years down the line to see e.g. hospital robots have any kind of general Theory of Mind, but this is also exactly because our understanding of Theory of Mind reasoning and how to represent it formally and in a computer is still rather limited. First we need to understand the concept, then we need to find out how to formalise it, then we need to tame the inherent computational complexity, and then finally we are hopefully able to implement it in a practical application. This is the process that e.g. description logics have been going through. Description logics are logics for representing and reasoning about terminological knowledge. Today reasoning calculi for description logic have been commercialised and are being used e.g. on large medical databases, but it was obviously a long process since the early days of description logic (description logics are in fact also a type of modal logics, more precisely a type of hybrid logics).
There is of course also a risk that what we are doing theoretically about self-reflective reasoning in computational agents will never have any practical impact. This is probably why some people start in the other end, and insist that everything they produce should from the outset be implemented in, say, a working robot. However, if you do this, you face another set of problems. Working on practical implementations of robots before the underlying concepts and mechanisms have been fully understood can often lead to solutions that are somewhat ad-hoc and far from being general and robust. There are e.g. researchers who have built robots able to pass certain false-belief tasks, but they can only pass first-order false-belief tasks, and it is not clear how the architecture can be generalised.
In terms of grants, there are always more money in applied research than in theoretical/foundational research. I don't think getting funding of theoretical work on self-reflective reasoning in computational agents is any easier or harder than other foundational work.

Luke: You give the example of description logics, which I take it were developed several decades before they saw significant commercial application? What other examples of that kind are you aware of, where there was some early theoretical work inspired by fairly abstract motivations ("In the future, we'll want systems that do X-ish type things, so let's build some toy models of X and see where things go…") that wasn't commercialized until a decade or more later?

Thomas: Description logics go back to the beginning of the 80s. Efficient reasoners with good practical performance on large databases described in expressive description logics started appearing in the beginning of the 00s. Reasoners for these expressive description logics have very bad worst-case complexities (e.g. 2EXP), so it has been quite impressive, and surprising, to see how well they scale to large databases.
With respect to other examples, I think that most of the genuinely new theories in AI start out with only being applied to toy examples. Also, almost all of mathematics falls into the category of "theoretical work inspired by fairly abstract motivations". If you e.g. look at all the mathematics needed to construct a physics engine for a modern 3D computer game, then none of this was originally made with computer games in mind, and most of it was developed without any applications in mind at all. In AI, of course, we tend to always have certain applications or sets of applications in mind, and we aim at shorter time intervals from theoretical developments to commercialisation. The example of the physics engine is about creating mathematical models of real-world physical phenomena and then implement these models in a computer. AI is not that different from this. AI is also about creating mathematical models of some part of our reality, but in this case it is mathematical models of (human) reasoning. There is no doubt that reaching this goal involves, as in the case of physics engines, significant work at all levels from the most theoretical foundations to the most applied implementation-oriented stuff.
I think it is dangerous to dismiss any new ideas in AI that at first only applies to toy examples. That might be a necessary first step, as was e.g. the case in description logics. But it is equally dangerous to forget all about scalability, and think that "this is not my business to worry about". If you don't worry at all about scalability and applicability, then you are certainly much more likely to come up with ideas that will never scale or be applied. The researchers I respect the most are those who help make the ends meet: theoreticians with strong knowledge and intuition about applications, and applied researchers with strong theoretical knowledge and interest in applying new theory.

Luke: Thanks, Thomas!
Petrick, R. and Foster, M. Planning for Social Interaction in a Robot Bartender Domain. ICAPS 2013. Slides available.The post Thomas Bolander on self-reference and agent introspection appeared first on Machine Intelligence Research Institute.