AI & Justice in 2035

 Download as PDF

The AI PULSE Project hosted the "AI and Justice in 2035" Roundtable at UCLA on February 28, 2020. This event consisted of four panels and a dozen discussion papers that examined how AI will affect the provision of justice, broadly understood, by the year 2035.
A list of participants is below:
Ifeoma Ajunwa, Cornell University, Law, Labor Relations, and History Dept.
Kevin Ashley, University of Pittsburgh School of Law
Jenna Burrell, UC Berkeley, School of Information
Anthony Casey, University of Chicago Law School
Cary Coglianese, University of Pennsylvania Law School
Aloni Cohen, Boston University School of Law
Rebecca Crootof, University of Richmond School of Law
Adnan Darwiche, UCLA, Computer Science Dept.
Julia Dressel, Recidiviz
Kristen Eichensehr, UCLA School of Law
Christine Goodman, Pepperdine University School of Law
Jerry Jacobs, University of Pennsylvania, Department of Sociology
Michael Livermore, University of Virginia School of Law
Ryan McCarl, UCLA School of Law
Susan Morse, University of Texas at Austin School of Law
Paul Ohm, Georgetown University Law Center
Davide Panagia, UCLA, Political Science Dept.
Ted Parson, UCLA School of Law
Richard Re, UCLA School of Law
Andrea Roth, UC Berkeley School of Law
Neil Sahota, UCI School of Law
Lauren Scholz, Florida State University College of Law
Andrew Selbst, UCLA School of Law
Harry Surden, University of Colorado Law School
Emily Taylor Poppe, UCI School of Law
Rory Van Loo, Boston University School of Law
Suresh Venkatasubramanian, University of Utah, School of Computing
John Villasenor, UCLA School of Law and School of Engineering
Eugene Volokh, UCLA School of Law
Andrew Wistrich, Cornell Law School
Albert Yoon, University of Toronto Faculty of Law
Elana Zeide, UCLA School of LawThe post AI & Justice in 2035 first appeared on AI Pulse.

AI & Justice in 2035


Launch of the UCLA Institute for Technology, Law and Policy

 Download as PDF

The UCLA School of Law and the UCLA Samueli School of Engineering have launched the UCLA Institute for Technology, Law and Policy.
On January 24, 2020 the Institute, in collaboration with PULSE, hosted its first symposium, "Algorithmic Criminal Justice?" The one-day event consisted of a series of interactive panels that examined the use of algorithms in policing and criminal justice, including approaches to identifying and mitigating algorithmic bias. More information and video recordings of the four panels are available here.The post Launch of the UCLA Institute for Technology, Law and Policy first appeared on AI Pulse.


Artificial Intelligence's Societal Impacts, Governance, and Ethics: Introduction to the 2019 Summer Institute on AI and Society and its rapid outputs

 Download as PDF

The works assembled here are the initial outputs of the First International Summer Institute on Artificial Intelligence and Society (SAIS). The Summer Institute was convened from July 21 to 24, 2019 at the Alberta Machine Intelligence Institute (Amii) in Edmonton, in conjunction with the 2019 Deep Learning/Reinforcement Learning Summer School. The Summer Institute was jointly sponsored by the AI Pulse project of the UCLA School of Law (funded by a generous grant from the Open Philanthropy Project) and the Canadian Institute for Advanced Research (CIFAR), and was co-organized by Ted Parson (UCLA School of Law), Alona Fyshe (University of Alberta and Amii), and Dan Lizotte (University of Western Ontario). The Summer Institute brought together a distinguished international group of 80 researchers, professionals, and advanced students from a wide range of disciplines and areas of expertise, for three days of intensive mutual instruction and collaborative work on the societal implications of AI, machine learning, and related technologies. The scope of discussions at the Summer Institute was broad, including all aspects of the societal impacts of AI, alternative approaches to their governance, and associated ethical issues.
AI and Society: The debate and its challenges
Inspired by recent triumphs in machine learning applications, issues of the societal impacts, governance, and ethics of these technologies are seeing a surge of concern, research and policy attention. These rapid linked advances – in multiple linked areas of algorithm development, data and data-handling tools, and hardware-based computational ability – are a leading current concern about technology's potential for profound and disruptive societal transformation.
In part, current concerns about AI reprise familiar themes from other areas of high-stakes technological advance, so the existing body of research on these other technology areas offers insights relevant for AI. A few of these insights are especially prominent. For example, the rate and character of technological change are shaped not just by scientific knowledge but also by the economic, policy/legal, and social/cultural conditions that determine relevant actors' incentives and opportunities. Societal impacts are not intrinsic to characteristics of technology, but depend strongly on how it is developed, integrated into products and services, and used – and how people adjust their behavior around it: As Kranzberg's first law of technology tells us, "Technology is neither good nor bad; nor is it neutral."1 Together, the conjunction of rapid technical change and uncertain uses and responses challenge efforts to govern the associated impacts, so governance often merely aims to mitigate the worst impacts after the fact. Even when societal impacts are profound, they tend to emerge gradually in response to repeated adaptations of technology, deployment, and behavior, and are thus difficult to project, assess, or manage in advance.
These broad parallels with prior areas of technological advance and associated societal concerns are real, but there are also reasons to expect that AI may be different, and more serious, in its impacts and implications. What is popularly called "AI" is not one thing, but a cluster of multiple algorithmic methods, some new and some old, which are linked to parallel advances in the scale and management of data, computational capacity, and multiple related application areas. This set of advancing capabilities is diffuse, labile, and hard to define – a particular challenge to governance, since the ability to workably define something is normally a precondition for any legal or regulatory response. AI is also foundational, potentially able to transform multiple other technologies, research fields, and decision areas – to the extent that its impact has been credibly compared to that of electricity or fossil fuels in prior industrial revolutions.
AI's societal impacts thus present deep uncertainties, for good or ill. Expert views of what it will do, and how fast, span a broad range: from the cumulation of many incremental changes, to existential transformations of human capabilities, prospects, societies, and identities. Even setting aside "singularity" issues – potential general or super-intelligent AI that might threaten (or in some accounts, transcend) human survival and autonomy – multiple mechanisms of impact have been identified by which even continued development of AI short of these landmarks could have transformative societal impacts. Examples include large-scale displacement of human livelihoods, disruption of geopolitical security relationships, transforming (or undermining) collective decision-making processes through democratic governments or other institutions, extreme concentration of wealth and power (perhaps based on new mechanisms of power), and large-scale changes in human capabilities and identities. Even limiting attention to present and near-term developments, there are a host of concerns raised by current AI applications – e.g., safety and security of systems, bias in algorithmic decision-making, threats to privacy, and inscrutability of decisions – some of which may also give early warning signs of coming larger-scale impacts.
Relative to the scale and gravity of potential impacts, present debate on AI and Society presents a seeming paradox. The issue is receiving a flood of attention, with dozens of new programs, a rapid flow of resources, and meetings and conferences seemingly every week. Yet well-founded insights remain scarce on the nature and mechanisms of impacts, effective and feasible means of governing them, and associated ethical issues. There has been relatively little convergence or progress on major questions, which in many cases remain not just unanswered but also subject to wide uncertainty and disagreement, or even not yet clearly posed.2 Because AI is so labile and weakly defined, studying its impacts has been likened to the ancient Buddhist parable of the blind men and the elephant: each observer feels that part of the unfamiliar thing that is closest to them, so each thinks they know it; yet their views are all partial and mutually contradictory. As with the elephant, it is possible to approach AI impacts from any discipline or field of inquiry (e.g., corporate law, anthropology, Marxist social history), any area of interest (education, finance, climate change), any political or ethical concern (racial justice, social mobility, privacy, due process), or any prior technological analogy, and find something resonant. Pulled by these centrifugal forces, the debate is thus unhelpfully sub-divided along multiple dimensions and lacks a coherent core.
There is also continued disagreement over where the most important impacts sit in time and scale, yielding a distribution of present attention and concern that is bi-modal. To be a little glib, those whose disciplinary perspectives make them most comfortable with speculative reasoning – often technical AI researchers and philosophers – are attracted to endpoint, singularity-related issues, which lend themselves to elegant, analytically rich theoretical inquiries. Most other researchers, on the other hand, gravitate to current concerns and historical precedents, because their disciplines frown on speculation and favor arguments based on observable (i.e., present or past) data and evidence. These areas of inquiry are both valuable and important, yet they leave disturbingly empty the large middle ground of impacts and challenges lying between these endpoints – where AI might transform people and societies by vastly reconfiguring capabilities, information, and behavior, while still remaining (mostly) under human control.3 At the same time, while there is a widespread sense that early action is needed to assess and limit risks of severe harmful impacts, there is little knowledge, and even less agreement, on what that action should consist of or how it should be developed.
The 2019 Summer Institute on AI and Society
This description of the range of issues posed by AI and the state of present debate underpin the aims of the Summer Institute. Just as AI and its impacts present a huge societal challenge, so too does mobilizing existing bodies of experience, knowledge, and methods to effectively inform the assessment and management of its impacts. These challenges will not be surmounted by any single insight, study, or activity. The summer institute aimed to point, tentatively, at a direction of efforts that can advance and expand the debate, establish an early model of the kind of collective engagement needed, and – by seeding cross-disciplinary networks for continued collaborations – contribute to the long-run project of building the needed capacity.
The summer institute pursued this aim in two ways. First, it sought to convene the needed broadly interdisciplinary dialog, with the ability to integrate knowledge and experience from multiple technical, scientific, and humanistic domains, and to resist widespread tendencies to converse mainly within existing disciplinary communities. In seeking this breadth of expertise contributing to the discussions, the summer institute benefited from its co-convening between a program on AI and Society based at a leading law school, and the CIFAR Deep Learning and Reinforcement Learning Summer School – a vehicle for advanced technical AI training with a distinguished international group of faculty and advanced students. Yet even with the right breadth of expertise in the room, making such interdisciplinary interactions productive takes sustained hard work to understand each other, clarify key concepts and methods, and build new conceptual and communication skills. These aims are better advanced by sustained collaborative work on problems of commonly recognized importance than by discussions that lack such common goals, which tend toward superficiality. Second, it is clear that understanding and addressing AI-related impacts is a long-term, even inter-generational project, which must combine mutual instruction with advancing inquiry, aiming to both advance the debate and broaden its participation by engaging more junior and more senior thinkers on collegial terms.
An Experimental Process
To pursue these aims, the summer institute experimented with a novel two-part structure, with the first part tightly programmed and structured by the organizers and the second part left almost entirely to the collective, bottom-up authority of the group. The first part aimed to provide the essential foundation of common knowledge and concepts to enable people from a wide range of fields and career stages to participate effectively and confidently in the discussions. To this end, the institute opened with a day of short, focused briefings by faculty experts, each covering elements from their expertise they judged essential for anyone to be an informed participant in the debates. These briefings were grouped into four sessions organized by broad subject-matter:
– Recent advances and current technical issues in AI and Machine Learning (briefings by Graham Taylor on Deep Learning, Rich Sutton on Reinforcement Learning, and Dirk Hovy on Natural Language Processing);
– Current issues and controversies in AI societal impacts (briefings by Elizabeth Joh on use of AI in policing and criminal justice; Michael Karlin on military uses of AI; Trooper Sanders on biased data, its implications and potential correctives; and Elana Zeide on use of predictive analytics in education and employment);
– Alternative approaches to governance of AI and its impacts (briefings by Geoffrey Rockwell on the historical trajectory of concerns about automation and proposed responses; Gary Marchant on limits to hard-law approaches, and potential soft-law and international alternatives; Brenda Leong on corporate AI ethics boards and their limitations; and Craig Shank on internal corporate controls and multi-stakeholder governance processes);
– Larger-scale and medium-term issues (briefings by Jason Millar on embedded values in navigation and mobility systems; Evan Selinger on facial recognition and its implications; Osonde Osoba and Casey Bouskill on technology-culture interactions in AI impacts and governance; and Robert Lempert on large-scale societal implications of alternative approaches to algorithm design).
Following the briefings, the rest of the Summer Institute was dedicated to collaborative work on projects that were not pre-specified, but instead were developed and proposed by individual participants, then selected in real time by all participants choosing which of the proposed projects they wanted to work on. Any participant, regardless of seniority, was invited to propose a workgroup project via a statement posted online and a short oral "pitch" presentation to the group, followed by brief clarifying discussion. In selecting projects, participants were urged to consider a few explicit criteria – that the projects address interesting and important issues related to AI and society, that they not duplicate existing work, and that they offer the prospect of meaningful progress in the limited time available. Otherwise, there was no central control of projects proposed or chosen. The form of proposed projects was completely unconstrained, explicitly including making a start on collaborative research projects, drafting op-eds or other non-specialist publications, developing proposed contributions to policy or governance, developing instructional material, or creating a story or other work of art on the theme of AI and society.
From twelve proposals, the group selected eight highly diverse projects to work on. The resultant eight groups worked intensively over a day and a half, in a process that several participants likened to a hack-a-thon. The analogy is suggestive but only partly accurate, in that that each SI workgroup included a wide range of disciplinary skills and expertise, and each pursued a different project, all generated by participants rather than pre-specified by organizers. The entire group convened briefly in plenary at half-day intervals to hear short reports from each workgroup summarizing what they were doing, what progress they had achieved, what completed output they targeted by the end of the SI, and what help they needed from the rest of the group. All eight workgroups achieved substantial progress by the end of the summer institute, even within the extremely limited time available. All eight also expressed the intention and developed concrete plans to continue their collaborative work after the Summer Institute – with some continuing that work immediately afterwards.
Workgroups' Provisional Outputs
The contributions published here represent the initial outputs of these eight work groups' collaborative efforts, as achieved during the intensive work period of the summer institute plus a little further polishing over the following few weeks. One consequence of the decentralized, bottom-up model, with each workgroup defining its own project, is that the resultant outputs are too diverse for any single publication or communication outlet to be suitable for them all. Yet in order to have a single vehicle that captures the collective energy and themes of the SI – and moreover, to communicate these while the experience is still fresh in participants' minds – all workgroups agreed to disseminate interim outputs from their work for this fast web publication. This quickly distributed – but explicitly half-baked – publication model was variously likened to theatrical workshopping or rapid prototyping in product development, in addition to hack-a-thons.
This experimental early publication model is very much in line with the exploratory and experimental spirit of the SI, taking the risk of trying different models to advance and broaden the debate. It also, of course, has the unavoidable consequence that these works – while they reflect remarkable achievements in the short time available – are all provisional and not yet fully developed. With some variation among the workgroups, they are presented here with the aim of being starting points for needed discussions, and providing concrete resources, background information, and proposals to move those discussions forward with specificity. They are not completed or polished products.
We provide below a brief synopsis of the aims and outputs of each of the eight workgroups. Each workgroup is continuing to develop its project, aiming for publication in various outlets in line with the groups' diverse aims and intended audiences. As the outlet for each workgroup's completed work is finalized, we will identify it and, as available, add links to the discussions below.
Mobility Systems and Embedded Values
This group used the example of the now-ubiquitous, AI-driven, turn-by-turn navigation systems to illustrate the range of values affected by these systems, whether explicitly or not. They then considered the resultant implications for societal impacts of projected large-scale expansion and integration of these systems, moving from separate navigation apps used by individual drivers, to complete urban mobility systems integrating signaling and multiple types of human-driven and autonomous vehicles, private and public. Navigation apps may at first glance seem prosaic, but the exploration was surprisingly rich. Present implementations of these systems seek to minimize individual drivers' travel time between a given origin and destination, with limited options to tune results to individual preferences such as avoiding freeways. But since their early deployment, a collateral impact of these systems has been increased traffic in residential neighborhoods – an impact well known to the planners who design streets, signals, and signage, but not recognized as implicated in individual navigation systems until large numbers of drivers began taking the same recommended shortcuts through side streets. The group identified several additional values affected by mobility design systems, which will require explicit consideration as the scope and integration of systems increases. In addition to travel time and neighborhood character, these include safety (at the individual level for drivers, pedestrians, and other street users, and collectively); allocation and prioritization of mobility access among types of users (now implemented simply, through right-of-way for emergency vehicles, HOV or toll lanes, etc., but potentially generalizable in multiple ways with fully integrated systems); and policing strategy and resource allocation, among others – including an unexpected linkage to the important role presently played by traffic fines in some local government budgets. In this initial published collection, the workgroup presents a taste of their discussions in the form of a fictitious press release, announcing the release of a new navigation app that generates routes based on minimizing drivers' cognitive burden.
This group's discussion illustrates a widespread phenomenon related to automation of decision processes. Societal institutions and processes often serve multiple values, only some of which are explicitly articulated as their mission or objective. Just as urban transport systems advance multiple values in addition to efficient mobility, so too do other organizations. A prominent example is provided by military services, in the United States and to different degrees in other countries. While their explicit missions are all broadly related to national defense and security, one of their most important social impacts – almost unrelated to their explicit missions – has long been to provide training and life skills to young people from disadvantaged backgrounds, making these organizations one of the most powerful drivers of social mobility. Many institutions serve such multiple corollary or implicit societal values. Automation or codification of decisions – typically with a single objective function that aligns with the institution's explicit, official mission – can put these other implicated values at risk, either from the automated decisions themselves or from related organizational changes. (In military organizations, the concern arises from the higher level of technical skills and education required of even entry-level recruits in AI-rich environments.) Yet these corollary values are challenging to integrate into explicit algorithmic decision-making – because they are ambiguous, hard to integrate into an objective function that trades them off against core organizational missions, and potentially contestable – such that they may only flourish while flying under the radar. As Joni Mitchell sang in another context, "You don't know what you've got till it's gone." The loss of corollary, emergent, or ambiguously defined organizational values may be a systematic consequence of automating decisions, which typically requires explicitly codifying what before was ambiguously embedded in organizational practice.4
Output of the Mobility Systems group
Meaningful Human Control
This group considered the problem of coupled human and algorithmic decision-making in high-stakes settings, using as initial examples the domains of weapons, aviation, and medicine. Noting the definitional ambiguity and difficulty operationalizing widely repeated concepts such as "humans-in-the-loop," their initial ambition was to unpack the meaning of "meaningful human control" (MHC) and identify processes and criteria to operationalize it across these diverse decision domains. But the group adjusted mid-course, recognizing that this was a longer project and that they needed first to engage the prior question of why – and with what conditions and limitations – meaningful human control is judged desirable, or even essential, in such decision contexts. They argue that retaining meaningful human control carries both costs and benefits, and that both the costs and benefits include distinct components, some related to system performance and some to issues of legal and moral responsibility. In general, greater human control may improve system performance by increasing redundancy and adaptability to novel conditions, and may be necessary to ensure moral and legal accountability. Yet it may also degrade performance by requiring uncoupling of complex autonomous systems and increase the risk of human error, carelessness, or other forms of improper human decisions. The group noted that the optimal balancing of these factors, and hence the preferred degree and form of human control, are likely to vary substantially even among the three decision domains they consider. The group is continuing work on the larger project generating guidelines how to implement the desired degree and form of human control in particular decision types.
Output of the Meaningful Human control group
AI Without Math
This group began a project to develop non-technical instructional materials on key AI and machine-learning concepts. They recognized that as deployed AI-based products and services continue to expand, many decisions will be required about how to control, explain, and manage these. These decisions will include many by various professionals who not only lack specific training in AI and Machine Learning, but may also lack training in the underlying mathematical and statistical concepts that provide the core of even introductory instruction in AI/ML. In view of this need, the group began development of an online instructional resource that would provide introductory explanations of key AI/ML concepts with no use of formal mathematical notation. As illustrative audiences toward whom to target their explanations, they took journalists and judges. Their short contribution here presents a start on this project and an illustration of their targeted level of explanation, including explanations for four key concepts: rational agents, naïve Bayes classifiers, linear regression, and convolutional neural networks. Their more extensive resource will be an ongoing project, to be available at https://www.aiwithoutmath.com.
Output of the AI Without Math group
Siri Humphrey:5 Design Principles for an AI Policy Analyst
There are many studies underway of the potential for AI tools to take on various functions of government – legislative, executive, judicial, and electoral – asking how the use of AI in specific functions would work, what it would require, with what attendant benefits and risks, and whether (and how) it could align with applicable legal, democratic, and moral principles. This group looked at a previously unexamined piece of this landscape, the potential for AI systems to take over, partly or wholly, the functions of policy analysts who advise senior officials or political leaders. Starting from recent scholarship that has identified several distinct functions that policy analysts perform, they examined how AI systems – either current ones or reasonably projected extensions – could serve these functions, with what implications for the policy-making process and the multiple public values implicated in policy decisions.
The group argues that AI systems could substantially replace the "synthesis" function of policy analysis: the gathering, curating, and synthesis of publicly available information relevant to an issue or decision. At least initially, use of AI in this role would have to be subject to specific limitations on the tasks delegated, and also subject to review and revision of the resultant briefing notes or other documents before they go to Ministers or other senior decision-makers. The group also argues that repetition of this synthesis and review process, with feedback from both human policy analysts and decision-makers (such as Ministers routinely provide on briefing materials prepared by their human staff) could serve as high-order training for the AI, allowing progressive reduction – although not elimination – of the amount of oversight and input needed from human policy analysts. In contrast to the "synthesis" function, they argue that certain other policy analysis functions depend more strongly on the essentially human interaction between decision-makers and their advisors. This militates against the wholesale replacement of analysis and advising functions by AI systems, suggesting instead a model of "Artificial-intelligence-amplified policy analysis," in which AI systems augment and amplify the skills of human policy analysts.
Output of the AI Policy Analyst group
Assessment Tool for Ethical impacts of AI products
The next two workgroups form a complementary pair, both concerned with the problem of what to do with the multiple sets of AI ethical principles being advanced to provide guidance for individuals or organizations engaged in AI development and application. These sets of principles pose two widely noted problems. First, the proliferation of large numbers of similar, but not quite identical, lists of principles raise questions about the relationships between them, the normative foundations of any of them, and the basis for adopting any of them over the others.6 Second, all these principles are stated at high levels of generality and abstraction, so their implied guidance for what to do, or what not to do, in the actual development, design, training, testing, application, and deployment of AI-enabled systems is indirect, non-obvious, and contestable.
In an unplanned piece of serendipity, these two groups approached the same problem from nearly opposite perspectives, one operational and one critical, yielding a rich and instructive counterpoint. This group took an operational, constructive approach rooted in engineering. Boldly (and practically) going where no one has gone before, they reasoned step by step through the process of operationalizing a particular set of ethical principles for any AI-related product or project. They first reduced each principle to a list of specific areas of concern, then to operational questions about observable practices and procedures relevant to each area of concern, and finally to a numerical scoring system for alternative answers to each question. Subject to some remaining ambiguities about appropriate weighting, the resultant component scores can then be aggregated to generate an overall numerical score for conformity of a system or project with the specified principle. The group stresses that such reductive scoring systems are prone to various forms of misinterpretation and misuse – such as imputing false precision or prematurely closing discussions. They also highlight that this heroic, first-cut effort is incomplete. Yet at the same time, they vigorously defend the approach as providing a stimulus, and a concrete starting point, for the discussions of impacts and ethical implications that are needed in the context of specific projects and systems.
Output of the Ethical Impact Assessment Tool group
Shortcut or Sleight-of-Hand: Why the checklist approach in the EU guidelines does not work
This group took as their starting point a different set of ethical principles, the "Ethics Guidelines for Trustworthy Artificial Intelligence" issued by the EU Commission's High-Level Expert Group on Artificial Intelligence in April and June 2019, including an "assessment checklist." This checklist is intended to help technology developers consider ethical issues in their policies and investments, and thus to create more trustworthy AI. In effect, this EU expert group undertook an exercise quite similar to that conducted by the Summer Institute "Tools" group summarized above, except that the EU expert group's exercise is more limited: it consists only of a checklist of yes/no questions (with extensive supporting discussion), and does not pursue a numerical scoring system.
This workgroup conducted a detailed critical assessment of the guidelines and checklist, aiming to assess their implications – and in particular, their limitations – as a tool to guide AI development. They argued that these guidelines are a fair target for such critical scrutiny because of their likely influence and importance, based on their ambition to articulate a broadly applicable standard of care for AI development and their prospect of influencing EU regulatory development – especially given the EU's emerging role as a world leader in this regulatory area.
The group finds the proposed approach problematic in several ways, most of them related to intrinsic limitations of checklists in this context rather than problems specific to this particular checklist. Using the analogy of safety procedures in aviation and space flight, they argue that checklists are an appropriate technology to manage human-factors risks in complex environments whose operations, salient risk mechanisms, and implicated values are well known, but that these conditions do not apply to development of safe or ethical AI systems. The group argues that many items on the checklist are seriously ambiguous but lack the additional explanation or documentation needed to reduce the ambiguity; and that the checklist thus risks conveying false confidence that needed protections are in place, when the conditions for this to be the case are in fact subtle, context-specific, and evolving over time.
Although the EU expert group's report includes extensive discussions of caveats and limitations, the workgroup finds these insufficient to mitigate the risks they identify, in view of the likely uses of the checklist in real-world, operational settings. They worry that enterprises are likely to treat the checklist either reductively or opportunistically – perhaps delegating responses to their legal teams to seek defensible markers of regulatory compliance or fulfilling some relevant duty of care. Used in such ways, the checklist would fail to stimulate the serious, organization-wide reflection on the concrete requirements of ethical conduct in their setting that should be the aim. Moreover, the group argues, the checklist is unlikely ever to yield a decision not to pursue an otherwise attractive project due to irreducible risks of unacceptable outcomes, when a meaningful and effective ethical filter must be capable – at least occasionally – of generating this outcome. Finally, the group argues that checklists are likely to be proposed or used as safe harbors – by enterprises, or even worse, by regulators, judges, citizen groups, or political leaders – with the resultant risk of reducing the pursuit of ethical AI to empty "ethics-washing" or "ethics theatre."
In contrast to their sharp criticism of the checklist, the group finds the expert group's higher-level "guiding questions" to be of great value, in helping to identify issues and problems that require sustained attention and so to promote an organizational culture of heightened ethical awareness. But they find the pursuit of simplification and codification embodied in the checklist approach to be premature, promoting misleading, too-optimistic assessments of risks and the subsequent prospect of broad, destructive backlash against the AI and related technologies broadly.
Output of the Ethical Guidelines group
AI and Agency
This group examined the deep, and deeply contested, concept of "agency," as it applies to and is modified by the context of AI development. Working both individually and collectively, they wrote a set of short, provocative essays that approach the concept of agency from multiple disciplinary perspectives, including philosophy, political science, sociology, psychology, economics, computer science, and law. The essays also lay out a set of deep questions and tensions inherent in the concept. They ask how agency is defined; whether humans have it, and if so, whether and how this distinguishes humans from present and prospective AI (and also from other animals); and what are the implications of alternative conceptions and ascriptions of agency – for human behavior, identity, welfare, and social order.
The definitions they consider for agency cluster around two poles, one positive and one negative. At the positive pole, agency is defined by the capacity for goal-directed behavior, and thus identified by observing robust pursuit of a goal in response to obstruction. At the negative pole, agency is defined by not being subject to causal explanation without introducing conceptions of intention or subjectivity. The group notes that conventional conceptions of agency as being unique to humans are increasingly challenged on two fronts: by human inequity in diverse social contexts, and hence wide variation in individual humans' capacities to exercise effective agency; and by scientific advances that suggest both that subjectively perceived agency may be illusory, and that to the extent humans do have agency, so too may other animals.
Present and projected developments of AI raise the stakes of these inquiries. The increasing complexity of AI performance implies, at a minimum, the lengthening of causal chains connecting behavior to proximate or instrumental goals and thence to higher-order goals, shifting the location of agency and casting doubt on simple claims that people have it but AI's do not or cannot. Yet the connection between this causation-driven notion of agency, and thus the validity of societal ascription of responsibility and deployment of incentives, are obscure, in the context of both human and AI decision-making. Does accountability always pass back to the human designer or creator, no matter how many layers of intermediate goals are generated within an AI? If human behavior is increasingly understood as subject to causation, does this reduce moral problems to correctible, technical ones – and if so, correctible by whom, in terms of both effectiveness and legitimacy? Finally, even if strong human-other or subject-object distinctions in ascribing agency become untenable under further advance of scientific knowledge and AI technology, might agency nevertheless be a useful fiction, a myth that is useful or even necessary to believe – for stable conceptions of human identity, and for effective collective regulation of human behavior?
Output of the AI and Agency group
Can AI be an instrument of transformative social and political progress? The "levelers" group
This group took its inspiration from a strain of political thought early in the industrial revolution, which identified markets and technological innovation as powerful engines of political progress, holding the prospect of large gains in both liberty and equality. Looking forward to the transformative possibilities of AI, the group took a perspective at odds with the dystopian gloom that marks much discussion of AI impacts – and also, for that matter, at odds with the mixed outcomes that attended the actual technological and economic transformations of the industrial revolution. Instead, the group asked whether advances in AI could drive transformative social and political progress – and if so, what conditions would be necessary or helpful in promoting such progressive impacts. The group considered technical and socio-political conditions separately. Are there particular technical characteristics of deployed AI systems that would be most compatible with the aim to increase rather than decrease broad human liberty, equality, and agency? And what social, political, and economic conditions – including the need for viable business models – would be most conducive to AI systems with these beneficial characteristics being successfully developed, deployed, scaled, and sustained over time?
Regarding technical characteristics, the group identified two areas that might promise greater, and more broadly distributed, societal benefits than present and projected AI development patterns, one related to the structure of decision-making and one related to the scale, decision scope, and number of separate AI systems. Most methods of algorithmic decision-making, whether modern machine-learning or earlier approaches, structure their decision-making with the aim of optimizing a single-valued objective or scoring function under a single characterization, deterministic or probabilistic, of conditions in the world. An alternative approach, rooted in concepts of satisficing, bounded rationality, and multi-criteria decision making, instead pursues decisions that perform acceptably well under a wide range of possible realizations of uncertainties – and also under a wide range of plausible objectives and associated values. The group speculated that such robustness to diverse conditions is likely to be associated with greater pluralism of values, and with a tentative approach to decisions that recognizes uncertainty and limited knowledge, makes informed guesses, and seeks additional guidance – and thus, perhaps, with more inclusive and more equitable AI-driven decision-making.
Regarding scale and scope, most present AI-based products are developed by for-profit enterprises and marketed to users – individual consumers, businesses or other organizations, government agencies, etc. – under conditions of asymmetric information and substantial market power. Moreover, users' values and preferences implicated by the AI systems are often under-specified, ambiguous, and manipulable, and may also exhibit systematic disparities between immediate impulses and considered longer-term values and welfare. The relationships between AI systems and users are thus ripe for exploitation to benefit the dominant party, e.g., by bundling attractive services with subtle, hard-to-observe costs such as loss of privacy or autonomy, or by manipulating users' adaptive and labile preferences to their detriment.
Many alternative models for AI deployment are plausible, at a wide range of scales in terms of people served and decision scope, and are potentially compatible with better advancing the pursuit of individual well-being and shared values. But achieving this alignment will require certain conditions, once again mainly related to the specification of objective functions but now with additional complexities that arise when multiple actors' interests and values are implicated. Such complexities include, for example, typical mixtures of shared, rival, and conflicting interests among actors, as well as collective-action problems and other pathologies of collective choice. In all such settings, AI systems must be faithful servants – which aim to advance as best they can the values and interests of the individual or collection they serve, even when these are tentative, imperfectly understood, and require continual adjustment – but with no consideration of the interests of the agent who developed or applied the AI.
Even if or when the associated technical requirements are clear, systems with these attributes may well not be compatible with present AI development business models. Such systems will need contextual conditions that allow them to be developed, deployed, adopted, and scaled – while maintaining fidelity to the progressive aims and principles of the endeavor. The group worked through various scenarios of conditions that could enable such development, allowing the desired systems to gather initial development resources; secure the ongoing inputs needed to scale and progress; avoid being destroyed or corrupted by competition or attack from incumbents whose rents are threatened; and operate sustainably over time. Promising directions included a mix of strategic identification of initial targets; strategic early deployment of philanthropic or crowd-sources resources using open-source development; building strong early competitive positions through aggressive exploitation of IP advantages, coupled with binding pre-commitments to relinquish these at some certain future date; and compatible public policies regarding data ownership, IP, antitrust, and related matters. The group recognized that they were engaged in hopeful speculation about potential technical capabilities and associated societal conditions and impacts, when these conditions remain largely unexamined at present. They concluded, however, that in view of stakes and plausibility, these development directions merit high-priority investigation.
Output of the Transformative Social Progress group
Concluding reflections: Routes to progress in understanding and governing AI impacts
As these short previews suggest, the discussions and outputs of the summer institute's work groups were too broad-ranging and diverse to admit any single summary or synthesis characterization. Still, a few salient themes emerged across multiple groups, including the following:
– The pluralism and ambiguity of values often embedded in current procedures, practices, and institutions, which may be put at risk by automation or codification of decision-making that exclusively optimizes for a single value – whether this single value is efficiency or cost-minimization as often proposed, or something else;
– The rapidity with which considerations of AI deployment and impacts moves from seemingly prosaic considerations of system and application characteristics, to engage deep, even foundational questions of social values, political organization, and human identity;
– The frequency with which new configurations of responsibility and authority, in which AI-based systems augment and partner with human decision-makers rather than replacing them, appear superior on multiple dimensions to either human or machine decision-makers operating alone;
– The value, in considering ill-posed problems marked by deep uncertainty, of taking a dialectical approach – or alternatively, an adversarial or "red team-blue team" approach. This was clearest in the work of the two groups that struggled, from nearly opposite perspectives, with the thorny problems posed by the widely proliferating sets of AI ethical principles. The rich counterpoint between these two groups was unplanned good luck that emerged from the process of proposing and selecting workgroup projects. These groups have not yet had the opportunity to respond to each other directly: they were aware of each other's work from the brief plenary check-ins, but given the intensely compressed schedule of the summer institute there was little opportunity for substantive interaction between groups. Each group's work is limited and incomplete, in line with the aims of this rapid-output publication – as indeed are the outputs of all the workgroups. Yet they are also powerfully mutually enriching, offering complementary perspectives on the urgent question of how to inject ethical considerations into AI system development in practice, each hinting at potential correctives to the limitations of the other. They thus provide great heuristic value informing concrete early actions on a problem that defies resolution in any single step.
– The urgent imperative of finding footholds for progress in efforts to assess and govern mid-term developments and impacts. This is the place where immediate concerns and conflicts that suggest obvious – if unavoidably incomplete – responses shades into potentially transformative impacts. Yet this is also where early interventions hold the possibility of high-leverage benefits, even despite the relative scarcity of attention now being directed to these problems and the profound methodological challenges of developing disciplined and persuasive characterizations of risks and responses.
In addition to the substantive richness of the discussions and outputs, the Summer Institute also represented an experiment in process that greatly exceeded our expectations, which we believe offers significant insights into how to stimulate effective conversations and collective activities that deliver real progress on wicked problems like AI impacts, governance, and ethics. We noted above the conditions that make understanding or practical guidance on these issues so difficult to achieve, despite the flood of attention they are receiving – including deep uncertainty, rapid technical progress, and fragmented knowledge and expertise. Given that the familiar approach of waiting until impacts are determinate is insufficiently precautionary, what types of activity or process might promise useful insights for assessment or governance action? There is obviously no determinate checklist available, but a few conditions and criteria appear likely.
– It is necessary to mobilize multiple areas of relevant knowledge, expertise, and method – both across research and scholarly disciplines, and between academia and multiple domains of practice – because the problems' tentacles extend far broader than any single community of inquiry or practice;
– It is not sufficient to bring suitably broad collections of relevant expertise together; it is also necessary to facilitate sustained, intensive interaction, in which people dig hard into each other's concepts, methods, terminologies, and habits of thought – to avoid the common failure mode of interdisciplinary activities, superficial agreement without actual advance of understanding;
– The problem of AI impacts and governance is both fast and slow: rapid pieces of technical progress and reactions to them add up to transformative changes over decades. There is news every week, yet the problem is not going away any time soon. There is thus a need to broaden debate and build expertise along generational lines as well as on other dimensions, to integrate instruction and professional development with parallel efforts to advance understanding. (This is one respect where the parallels between AI and climate change are instructive: both issues combine processes that operate on a wide range of time-scales, although in climate change there is much better knowledge of the long-term behavior of the relevant systems.)
– Knowledge in the field is diffuse, provisional, rapidly evolving. There is not an established and bounded body of knowledge sufficient to create an expert community. Plenty of expertise is relevant, but little that is on-point, certainly none that provides clear guidelines for progress.
These conditions suggest there is a need to encourage collaborative discussion and shared work along multiple parallel lines, which in turn suggests a decentralized approach to convening collaborative groups with the range of expertise needed to generate and pursue specific promising questions and ideas. The same conditions also suggest a need for communication vehicles to share questions, insights, arguments, and ideas, which is informed by relevant research and scholarship but proceeds faster, and more provisionally, than normal conventions of research and scholarship allow. Thus, even with the existence of such communication vehicles, there is also a need to develop a culture and practice of substantively rich but quick exchange of ideas, even provisional and incomplete. It does defy academic convention, but on issues like this, rigor and completeness may be the enemy of progress.
These requirements suggest that the Summer Institute was, with small exceptions, a nearly ideal model to advance understanding and capacity – on AI and society issues, and on issues that exhibit similar characteristics. Indeed, the power of the model for similar issues was substantiated by the success of another summer institute convened two weeks later by one of the organizers here on a different issue, the governance of geoengineering. It appears to be a powerful model, subject to various conditions related to selection or participants, available time, etc., which clearly merits further development and application. We wish we could claim to have been prescient in designing this process, but there were large elements of luck in the outcomes of the Summer Institute. Still, the results – both those experienced by participants within the Summer Institute, and those marked by these first-round outputs – strike us as astonishing, given their origin as outputs of less than two full days of intensive focused work by newly formed groups. This was an exciting exercise to be a part of, and we are deeply grateful to our faculty and participants – for the intensity, intelligence, and good will of their shared explorations, and also for their participation in this experiment and the significant intellectual courage they have exhibited in allowing their work to be disseminated here in this provisional, incomplete form.The post Artificial Intelligence's Societal Impacts, Governance, and Ethics: Introduction to the 2019 Summer Institute on AI and Society and its rapid outputs first appeared on AI Pulse.

Artificial Intelligence’s Societal Impacts, Governance, and Ethics: Introduction to the 2019 Summer Institute on AI and Society and its rapid outputs


Mob.ly App Makes Driving Safer by Changing How Drivers Navigate

 Download as PDF

A group of multi-disciplinary researchers from across North America today announced the launch of a new app, Mob.ly, that reduces the incidents of road rage by promoting a driver's sense of well-being and safety without sacrificing efficiency and access.
The researchers, who gathered at the Summer Institute for Artificial Intelligence, Ethics and Society, came together to discuss new modalities for AI assisted turn-by-turn navigation. Current navigation offerings focus on speed and efficiency, leaving a large portion of the population underserved. Many drivers, including young or new drivers, the elderly, and parents of young children, often have different needs when navigating city streets. Those drivers, though licensed and fully capable of driving in all traffic scenarios, benefit from navigation decisions that reduce their overall cognitive burden, that is, reduce the level of attention needed to safely operate an automobile. Reducing cognitive burden has been identified as a navigation strategy that can enable a greater sense of well-being and increase safety by avoiding unprotected left turns, major highways, confusing intersections and roundabouts, and other difficult-to-navigate road features that demand higher levels of concentration and focus.
"We saw the need for a navigation app that is designed for values other than simply finding the fastest route. We designed Mob.ly to minimize the cognitive burden placed on drivers by their route," said lead developer Jason Millar. "Some traffic interactions are simply more demanding than others. Our system allows drivers to specify which high-cognitive-load situations they want to avoid, and we route them accordingly."
The Mob.ly app interface. (left) Current routing options as determined by an algorithm that minimizes time-to-destination; (middle) Mob.ly allows drivers to select various high-cognitive-load situations they wish to avoid; (right) Mob.ly's algorithms optimize for low cognitive load.
Experts say this app could fill an important need in society. "With road rage, you're basically driving under the influence of impaired emotions," says Leon James, PhD, a professor of psychology at University of Hawaii and co-author of Road Rage and Aggressive Driving. Managing those emotions by providing options to avoid uncomfortable situations will reduce agitation and could improve outcomes in many ways.
Mob.ly is currently available worldwide on iOS and Android devices.
About Mob.ly Group*
Mob.ly Group is interested in creating navigational aids that intersect the fields of artificial intelligence, ethics and society. The group is led by Dr. Jason Millar, Canada Research Chair in the Ethical Engineering of Robotics and Artificial Intelligence (University of Ottawa) and is comprised of:
Nicholas Novelli, PhD student, University of Edinburgh;
Anne Boily, PhD Candidate, Université de Montréal;
Carlos Ignacio Gutierrez, Doctoral Fellow, The Rand Corporation;
Courtney Doagoo, AI and Society Fellow, University of Ottawa Centre for Law Technology & Society;
Kathryn Bouskill, Researcher, The Rand Corporation;
Elizabeth Wright, MA Candidate, George Washington University;
Brent Barron, CIFAR;
Elizabeth Joh, Martin Luther King Jr. Professor of Law, UC Davis;
Thomas Gilbert, PhD Candidate, University of California, Berkeley;
Leilani Gilpin, PhD Candidate, MIT;
Graham Taylor, Canada Research Chair in Machine Learning, University of Guelph and Vector Institute;
Nicolas Rothbacher, Master's student, Technology and Policy Program, MIT; and,
Margaret Glover-Campbell, Alberta Machine Intelligence Institute.
*Mob.ly is the result of a Design for Human Values mini-workshop conducted at the CIFAR-funded Summer Institute on Artificial Intelligence and Society. The app described above is one prototype among many that participants (listed above) imagined as alternatives to the current regime of turn-by-turn navigation apps, all of which focus on minimizing time-to-destination as the primary value embedded in the system. The workshop demonstrated that we can realize interesting alternatives when we focus on alternative values in design.The post Mob.ly App Makes Driving Safer by Changing How Drivers Navigate first appeared on AI Pulse.


On Meaningful Human Control in High-Stakes Machine-Human Partnerships

 Download as PDF

Reflections on the Process:
Our team at the Summer Institute was diverse in both skills (including technical computer science, cognitive science, systems innovation, and radiology expertise) and career stage (including faculty, graduate students, and a medical student). We were brought together at the 'pitch' stage by a mutual interest in human-machine partnerships in complex, high-stakes domains such as healthcare, transport, and autonomous weapons. We began with a focus on the topic of "meaningful human control" – a term most often applied in the autonomous weapons literature, which refers broadly to human participation in the deployment and operation of potentially autonomous artificial intelligence (AI) systems, such that the human has a meaningful contribution to decisions and outcomes.
We began from an applied perspective, with a how question: how might meaningful human control be created in high-stakes domains beyond autonomous weapons, and what threats might be faced during implementation? We worked backwards, and markers in hand we filled the windows of the large AMII boardroom with necessary preconditions for meaningful human control – focusing on challenges that could render human control meaningless, cause meaningful control to be inhuman, or cause control to be lost altogether. We continued to cover windows with specific design issues related to human factors, machine systems, and deployment environments that would influence "meaningful human control" in human-machine partnerships.
As the hours passed, our conversations grew more productive and contentious. We had made progress in understanding the design principles behind the a priori goal of meaningful human control, but questions remained. What did the human add to the equation? What did the human take away? As incredible progress is made in the space of AI, is meaningful human control something we even want at all? If so, what is the continuing value of meaningful human control? Amidst the discussion a clear consensus emerged: the core of our paper should focus on the importance and implications of meaningful human control, rather than the conditions required to attain it.
We are still committed to the framework that so painstakingly painted the windows, but in this initial paper from the Summer Institute we seek to answer the why before we get to the how. For this rapid web publication (in line with the concrete impact focus of the Summer Institute that kept us motivated all along), we have generated a preliminary draft of the former. We will define the concept of meaningful human control more precisely, explore background literature, and briefly outline our framework for the upsides and downsides of the concept in high-stakes environments. This preliminary publication will be updated with a link to the full work as it emerges in the fall.
Introduction:
The autonomy of artificial agents is an important aspect of defining machine-human partnerships. There is no clear consensus definition for the concept of meaningful human control (MHC) (Santoni de Sio et al., 2018). Broadly it connotes that in order for humans to be capable of controlling – and ultimately responsible for – the effects of automated systems, they must be involved in a non-superficial or non-perfunctory way. In particular, the concept of MHC emphasizes the "threshold of human control that is considered necessary" (Roff et al., 2016, p.1), to go beyond the ambiguous concept of humans "in-the-loop", or merely setting initial parameters and providing no ongoing control. Furthermore, the concept of MHC rests on the assumption that humans have exercised control over systems such as weapons in the past and present, and is concerned with maintaining this assumed human control (Ekelhof, 2018). Within this paper we aim to explore the concept of meaningful human control and its value as a concept for evaluating the balance of autonomy throughout the broader landscape of human-machine partnerships in high-stakes environments.
Background:
The concept of "meaningful human control" is most closely associated with lethal autonomous weapons systems, where there is general agreement that autonomous weapons capable of taking human life should not operate without human participation in or oversight of the decision-making process (see, e.g., Crootof, 2016; Roff and Moyes, 2016; Santoni de Sio et al., 2018). The term first appeared in this literature in 2001, in an article that discussed the inevitable rise of autonomous weapons systems and the accompanying challenge to meaningful human control of those systems (Adams, 2001). There are three broad and inter-related classes of reasons for requiring human participation in machine decision making. The first is rooted in the (arguably) unique capacity of human beings to make moral/ethical decisions, based in empathy and compassion (e.g., Asaro, 2006; Docherty, 2018). The second focuses on the ascription of legal responsibility or accountability (Hubbard, 2014; Calo, 2015; Scherer, 2016). The third class of reasons concerns performance – specifically system redundancy, error detection, and recovery – on the premise that humans can (at least for now) do some things, or do things in some ways, that machines cannot. These issues, of course, are not limited to lethal autonomous weapons: the issue of meaningful human control arises in other, typically high-stakes, contexts. Examples include the operation of autonomous vehicles (Heikoop et al., 2019) and surgical robots (Ficuciello et al., 2019). MHC has also been cited as a key challenge in the continuing development of robust and humane artificial intelligence systems (Russell et al., 2015; Stephandis et al., 2019).
Meaningful Human Control: Pros
The reasons meaningful human control is desirable in automated systems can be broadly divided into the categories of performance-related and responsibility-related – concerned with how well the autonomous system is able to perform the desired action, as opposed to the process of action selection itself.
Humans are adaptable, which can improve performance particularly on unusual inputs. Because machine learning-based systems are built to perform well on a pre-specified training set, they may underperform on novel or atypical inputs. These inputs may be benign (simply out-of-distribution), yet still yield harmful outcomes. For instance, an automated debt calculation system ("robo-debt") run by the Australian government frequently overestimated debts for people with highly variable income streams, who were not considered in the algorithmic design (Henriques-Gomes, 2019). Inputs can also be atypical in malicious ways – adversarial examples are a known vulnerability of computer vision systems (Akhtar and Ajmal, 2018). These are intentionally constructed to "fool" computer vision models into making incorrect classifications, yet appear unremarkable to the human eye (Goodfellow, 2014). In both these cases, human control over the system's response to the "atypical" input would allow for superior performance of the human-machine partnership.
Another important motivation for meaningful human control is adding redundancy to an otherwise automated system. Even on tasks where machine errors are highly infrequent, the character of their errors may differ greatly from human errors, in ways that can lead to catastrophic outcomes. Human oversight introduces heterogeneity to the decision-making progress, which can mitigate these risks. Airplane flight provides an example of a well-studied human-machine partnership which displays this characteristic. Airplanes are mostly guided by highly effective automated systems. Yet it is widely considered essential to have pilots "behind the wheel" to oversee the autopilot, who are able to select between different levels of control in case of system failure (Sheridan, 1987; Mindell 1999). This is the case despite the downsides: human pilots are a frequent cause of accidents (Shappell, 2017), and can lose skills over time if they are infrequently used (Arthur Jr et al., 1998), as they may be under a regime of widespread automation.
The most technologically intractable reasons for meaningful human control are moral. Human decisions are imbued with a moral weight that we do not accord to machines, and we commonly rely on humans to interpret vague rules in determining real-world actions in a way that is sensitive to context and human values (Russell, 2015). Humans are seen as having a capacity for moral judgment and empathy beyond any advanced AI. Domains such as legal decision-making (e.g. sentencing, bail, and parole) call for meaningful human control due to their moral dimension, despite some evidence that algorithms can predict recidivism as well as or better than expert human decision-makers (Kleinberg, 2017).
Automated systems with no human control also raise concerns about legal liability and accountability. For example, if a robot harms a person, who should be held responsible and liable for compensation? Possibilities include the manufacturer, the programmer(s), the user, and the robot itself. This is a real-world scenario, which courts have already addressed to some extent (Calo, 2015), but the prospect of increasingly intelligent, autonomous, interacting systems – especially those capable of ongoing learning from their environment – will create many legal and financial uncertainties. Under American law, for example, the situation of an autonomous system causing harm in a way that was not intended or foreseeable falls into a lacuna, in which it is unclear who, if anyone, would be liable (Hubbard, 2014; Calo, 2015; Scherer, 2016).
Meaningful Human Control: Cons
Meaningful human control also has costs, which again can be divided into performance-related and responsibility-related types. These costs, which can range from the minor to the substantial, must be weighed against the benefits when considering implementing MHC within a given context. The "handoff problems" associated with moving from a fully autonomous system to a human-machine partnership may be substantial (Mindell, 2015). It may also be possible that, in certain cases, human decisions are consistently inferior to or more biased than the machine's choices.
Many tasks are designed to be machine-driven precisely because of their superior performance or efficiency. Adapting a system for meaningful human control requires creating a monitoring apparatus, and potentially pausing automated routines to insert decision points. This paper focuses on "high-stakes" domains, where the consequences of errors can be substantial. Yet there are also many tasks for which each decision is so trivial that the loss of performance or efficiency outweighs the potential benefits of human involvement. Networking equipment, for example, autonomously performs repetitive tasks rapidly and accurately, with little perceived need for meaningful human control.
The variability and adaptability of human input interferes with predictability and consistency. This is particularly true in highly coupled, tightly interacting systems. Consider the case of an integrated autonomous driving network, in which vehicles hurtle past each other through an intersection at high speed. Safety and predictability are tightly linked in such a scenario, and the uncertainty introduced by the possibility for human control would have cascading effects. Instead of knowing what each other actor will do and planning accordingly, agents would be forced to monitor, project, and react to others' behavior under uncertainty. Contexts can be conceived in which, even if any given human decision was more appropriate than its automated counterpart, the downsides of this decoupling far outweigh the benefits.
There are also cases where human decision-making may be undesirable due to humans' risk of bias (intentional or unintentional) or ulterior motive. Certain autonomous systems – such as autonomous arbitration systems or escrow services – could derive their usefulness precisely from the lack of human control. The potential for bias in human decision-making may provide an additional impetus for developing autonomous systems without MHC – though precaution must be taken to ensure that the system does not merely perpetuate and mask existing biases with a veneer of algorithmic objectivity.
Preliminary Thesis:
Meaningful human control is important to consider in the context of machine human partnerships in high-stakes domains. Human involvement may improve system performance by way of redundancy and increased adaptability, and plays an important role in ensuring ethical and legal responsibility. These benefits do not come without downsides, however, including both the potential for improper human choices and the efficiency losses associated with decoupling complex autonomous systems. Finding the context-specific balance between these trade-offs is essential to ensuring effective, robust, and ethical performance in cases of autonomous human-machine partnership.
References:
Adams, T. K. (2001). Future warfare and the decline of human decision-making. Parameters, 31(4), 57-71.
Akhtar, Naveed, and Ajmal Mian. "Threat of adversarial attacks on deep learning in computer vision: A survey." IEEE Access 6 (2018): 14410-14430.
Arthur Jr, Winfred, et al. "Factors that influence skill decay and retention: A quantitative review and analysis." Human performance 11.1 (1998): 57-101.
Asaro, P. M. (2006). What should we want from a robot ethic?. International Review of Information Ethics, 6 (12):9-16.
Calo, R. (2015). Robotics and the Lessons of Cyberlaw. California Law Review, 103(3).
Crootof, R. (2016). A Meaningful Floor for Meaningful Human Control. Temp. Int'l & Comp. LJ, 30, 53.
Docherty, B. (2018). Statement on meaningful human control, CCW meeting on lethal autonomous weapons systems, April 22, 2018. Retrieved from https://www.hrw.org/news/2018/04/11/statement-meaningful-human-control-ccw-meeting-lethal-autonomous-weapons-systems, July 31, 2019.
Ekelhof, M. (2018). Autonomous weapons: Operationalizing meaningful human control.
Ficuciello, F., Tamburrini, G., Arezzo, A., Villani, L., & Siciliano, B. (2019). Autonomy in surgical robots and its meaningful human control. Paladyn, Journal of Behavioral Robotics, 10(1), 30-43.
Goodfellow, Ian J., Jonathon Shlens, and Christian Szegedy. "Explaining and harnessing adversarial examples." arXiv preprint arXiv:<PHONE> (2014).
Heikoop, D. D., Hagenzieker, M., Mecacci, G., Calvert, S., Santoni De Sio, F., & van Arem, B. (2019). Human behaviour with automated driving systems: a quantitative framework for meaningful human control. Theoretical Issues in Ergonomics Science, 1-21.
Henriques-Gomes, L. Labour calls on government to scrap 'malfunctioning' robodebt scheme, July 30, 2019. Retrieved from https://www.theguardian.com/australia-news/2019/jul/30/labor-calls-on-government-to-scrap-malfunctioning-robodebt-scheme, August 8, 2019.
Hubbard, F. P. (2014). 'Sophisticated Robots': Balancing Liability, Regulation, and Innovation, Florida Law Review, 66(5).
Keeling, G., Evans, K., Thornton, S. M., Mecacci, G., & de Sio, F. S. (2019, July). Four perspectives on what matters for the ethics of automated vehicles. In Automated Vehicles Symposium (pp. 49-60). Springer, Cham.
Kleinberg, Jon, et al. "Human decisions and machine predictions." The quarterly journal of economics 133.1 (2017): 237-293.
Mecacci, G., & de Sio, F. S. (2019). Four Perspectives on What Matters for the Ethics of Automated Vehicles. Road Vehicle Automation 6, 49.
Mindell, David A. (2015). Our Robots, Ourselves: Robotics and the Myths of Autonomy. Viking.
Roff, H. M., & Moyes, R. (2016). Meaningful human control, artificial intelligence and autonomous weapons. In Briefing Paper Prepared for the Informal Meeting of Experts on Lethal Autonomous Weapons Systems, UN Convention on Certain Conventional Weapons.
Russell, S., Dewey, D., & Tegmark, M. (2015). Research Priorities for Robust and Beneficial Artificial Intelligence. AI Magazine, 36.
Santoni de Sio, F., & Van den Hoven, J. (2018). Meaningful human control over autonomous systems: a philosophical account. Frontiers in Robotics and AI, 5, 15.
Scherer, M. U. (2016). Regulating artificial intelligence systems: risks, challenges, competencies, and strategies. Harvard Journal of Law & Technology, 29(2).
Sheridan, Thomas B. "Supervisory control." Handbook of human factors (1987): <PHONE>.
Shappell, Scott, et al. "Human error and commercial aviation accidents: an analysis using the human factors analysis and classification system." Human Error in Aviation. Routledge, 2017. 73-88.
Stephanidis, C. C., et al., (2019) Seven HCI Grand Challenges, International Journal of Human–Computer Interaction, 35(14), <PHONE>, DOI: 10.<PHONE>.<PHONE>The post On Meaningful Human Control in High-Stakes Machine-Human Partnerships first appeared on AI Pulse.


AI Without Math: Making AI and ML comprehensible

 Download as PDF

Abstract
If we want nontechnical stakeholders to respond to artificial intelligence developments in an informed way, we must help them acquire a more-than-superficial understanding of artificial intelligence (AI) and machine learning (ML). Explanations involving formal mathematical notation will not reach most people who need to make informed decisions about AI. We believe it is possible to teach many AI and ML concepts without slipping into mathematical notation.
Introduction1
Artificial intelligence (AI) and machine learning (ML) are transforming industries, societies, and economies, and the pace of change is accelerating. Businesspeople, lawyers, policymakers, and other stakeholders will increasingly face practical questions (e.g., "Should my firm adopt this AI-related product?") as well as political, ethical, and legal questions (e.g., "What limits should we place on law enforcement's use of facial recognition technology?") related to AI.
If we want nontechnical stakeholders to respond to AI developments in an informed way, we must develop ways to help them acquire a reasonable understanding of what AI and ML are and how different techniques work. Many institutions and individuals have been developing teaching materials, but this remains an open problem.
General overviews are often insufficient for the purposes of sophisticated stakeholders such as judges and regulators. However, in-depth explanations risk presuming a shared basic literacy about AI and ML. And even introductory technical explanations often fall into the trap of overestimating the learner's prior knowledge, particularly the learner's knowledge of math. That may be a mistake. Explanations involving mathematical notation will not reach many people who need to make informed decisions about AI.
Sophisticated but nontechnical stakeholders should be empowered to think about AI and ML at a more-than-superficial level even if they lack the mathematical and technical background of computer scientists. In some cases, curiosity will drive them to "dig deeper"; in other cases, a superficial understanding will be inadequate for the decision-making task they face. A judge faced with dispositive motions in a software patent case—or a judge considering whether to overturn a conviction on the grounds that the police used some algorithm or ML technique in their investigation—will have an ethical duty to try to understand the technology at issue in the case if doing so is necessary to achieve a just and legally correct outcome.
We believe it is possible to teach many AI and ML concepts without slipping into advanced mathematical notation. As Steven Skiena has written, "[t]he heart of any algorithm is an idea."2 Mathematical notation is rarely the only way to communicate such an idea, and it is possible to explain many ML concepts by analogies and examples while avoiding the terminology of calculus and linear algebra.
We have therefore developed a prototype website to communicate fundamental AI and ML concepts to an educated but nontechnical audience. This website can be found at https://www.aiwithoutmath.com.3 The site's name is a bit of a misnomer, as some machine learning concepts (such as "gradient descent") are inherently mathematical, and sometimes simple math examples make a concept easier to understand. Nevertheless, we intend to make the site's explanations as accessible as possible to people lacking mathematics backgrounds beyond high school algebra.
We aim to promote accessibility by providing simple but rigorous explanations of AI and ML concepts. These explanations go beyond high-level overviews and glossaries and attempt to teach the intuition behind various algorithms. To that end, they avoid formal mathematical notation and reduce the time commitment and cognitive effort required to learn the material.
We hope to create articles and learning materials that are as clear, concise, and self-contained as possible. We want to enable busy professionals to get the understanding they need without wading through details that may be relevant only to computer scientists or machine learning developers.
Further, instead of providing a linear walkthrough characteristic of a textbook or college course, we want to create modular, self-contained explanations of each concept that teach (or at least link to) the necessary background concepts. This will encourage non-linear navigation of the material that is driven by the reader's needs and curiosity. Many professionals lack the time or resources to complete an entire course on machine learning, but need a level of understanding that goes beyond that provided by high-level overviews or glossary entries.
Examples: Four AI/ML concepts explained without advanced math
In the following subsections, we illustrate our idea by explaining four AI concepts without relying on advanced math. Note that we also define all technical vocabulary as it is introduced. We try to give the reader the intuition behind each idea while assuming as little as possible about the reader's prior knowledge.
A. Rational agents
In AI, an agent is something that acts (such as a software "bot" or a robot). A rational agent is an agent that tries to achieve the best outcome. Agents are programmed to view some outcomes as better than others. The measuring stick by which the agent determines the "best outcome" is called the agent's objective function.
The idea that an agent tries to achieve the best outcome is often stated in technical vocabulary. For example, some might say that the agent tries to maximize utility, maximize expected utility, or maximize its objective function.
B. Naive Bayes classifier
A Naive Bayes classifier uses probability to classify (categorize) an object.
What is a classifier?
A classifier is a program that categorizes items as one type of thing or another. For example, a picture might be classified as a "cat picture" or a "dog picture." If you feed (input) an image into a classifier designed to distinguish cat pictures from dog pictures, it would output the label "cat" or "dog."
Spam filter example
Email systems include "spam filters" that automatically determine whether each incoming email is likely to be spam. Their input is an email and their output is the probability that the email is spam.
Each time we estimate the probability that a particular email is spam, we should take into consideration the overall probability that an email is spam. Suppose that 45% of emails are spam (this is called a base rate). If we know that an email arrived but we do not know anything else about it, we should conclude that it is probably not spam, because most emails (55%) are not spam.
However, each incoming email may have certain features that make it more likely to be spam. For example, every email has a certain number of exclamation points; "number of exclamation points" is a feature. We could say that an email containing more than two exclamation points is 80% likely to be spam. We can keep identifying more features and comparing them across the emails we see.
After considering many emails which are already labeled as "spam" or "not spam," the algorithm will know that some features have certain values for spam emails and other values for non-spam emails. This knowledge is called a "model"; generating that knowledge is called "training the model." Once we have a trained model, we can compare the feature values of any new email to the values in the model to estimate whether the new email is spam or not.
What makes Naive Bayes different from other classification methods?
A major advantage of the Naive Bayes method is that Naive Bayes models are relatively simple and can be trained quickly with a small set of data.
One potential disadvantage is that, in the Naive Bayes algorithm, each feature is considered independently of the others. This simplifying assumption is why the algorithm is called "naive." In reality, many variables "travel together" and are not independent. For example, if the weather is rainy, it is probably also cloudy. But in a Naive Bayes analysis, "raininess" and "cloudiness" might be treated as two separate and independent features. For many tasks, however, treating these features as independent does not affect the outcome.
C. Linear regression
What is it?
Regression is a way of predicting an output value based on one or more input variables. For example, a regression model might attempt to predict house prices (an output value) based on input variables such as number of rooms, school district, and proximity to the ocean.
Input variables are also called explanatory variables because they attempt to "explain" the reasons for the output value. In the housing prices example, the input variables "explain" why each house is sold at a certain price.
What are some business use cases for regression?
• Given past prices and economic indicators, should we expect copper prices to rise or fall?
• How might sales change if, instead of investing $100k in TV advertising, $50K is invested in TV advertising, and $50K is invested in social media advertising?
• If we hire additional doctors, how much can we decrease patient wait time?
• What are the top five factors that can cause a customer to default on their loan payment?
When should you use linear regression as opposed to another machine learning algorithm?
Linear regression should be used when (1) your target output value is a continuous numerical value, and (2) you expect a linear relationship between the input variables and the output value. A linear relationship means that if the input variable (e.g., number of rooms) goes up, the output value (e.g., housing price) should go up as well.
Note that one strength of machine learning and neural networks is that they can learn complex, nonlinear relationships between input data and the target variable. However, more complex algorithms are not necessarily better for a particular task. It is important to choose algorithms whose complexity is similar to the complexity of the underlying data relationship. If there is reason to suspect that there is not a linear relationship between input data and a target variable, however, a linear regression model will be too simple, and a neural network or alternative algorithm (such as polynomial regression) will be a better fit.
How does linear regression work?
A linear regression model works by fitting a line (when there is one input variable) or a plane (when there are two or more input variables) to historical data. "Fitting" means that the algorithm finds the line or plane which best explains the output value, given the historical data.
The image below shows a linear regression model with a line fit between one input variable (x) and one output value. The x-axis shows the value of a single input variable (such as a neighborhood's safety rating), while the dots show the historical data—the value of different output value and input variable pairs (such as house price and neighborhood safety rating). The linear regression model determines the fit between these variables by finding a line which minimizes the distance between the line and the data points. The values of y on the line give the prediction of the output value. For example, a neighborhood safety rating of 20 would predict a house price of 8.
Image source: Wikipedia.org
D. Convolutional neural networks
Researchers designed convolutional neural networks (CNNs) because they needed better tools to process images. Most systems that "see" the world—self-driving cars, medical diagnostics, etc.—use a convolutional neural network to do so.
Neural networks are a class of algorithms that are used in many machine learning problems. Their basic building block is a neuron, which performs a simple operation on its input. These neurons are arranged into layers, which are connected in a network that can perform complex tasks. Engineers have lots of flexibility when structuring layers and connections, making these algorithms suitable for many problems.
Convolution is a special type of operation that answers the question, "How much of B is in A?" where A is often an image, and B is often a pattern. For instance, if A is an image of a house, and B is a horizontal edge, the convolution might return an abstract image of the house showing only its horizontal lines.
By connecting many such operations into a neural network, CNNs are able to detect increasingly complex features. For example, in a CNN for face detection, early layers look for edges, intermediate layers look for facial components, and later layers look for full faces.
Future work
We hope to expand the AI Without Math website to include many more technical topics as well as nontechnical concepts (such as "explainability") that form the shared vocabulary of AI researchers. Other ideas for the website include offering alternative explanations for complicated topics (perhaps with some voting mechanism, similar to that used on StackOverflow and Quora, in which readers can upvote and downvote explanations); linking to off-site explanations; and illustrating concepts with multimedia resources such as videos, games, and demonstrations.
Conclusion
Most resources on AI and ML are either too general or too technical. There are many high-level overviews that can give stakeholders a sense of what these concepts refer to, but many stakeholders will need more than just a broad overview or glossary. Some will want (or need) to peek "under the hood" of AI and ML technologies to get a basic understanding of how they work and why one might use one technique as opposed to another. There is an urgent need for educational resources that will help more people participate in decisions requiring a more-than-superficial understanding of AI and ML. We hope that the AI Without Math website can become a hub for those resources.The post AI Without Math: Making AI and ML comprehensible first appeared on AI Pulse.