No Permits, No Fabs The Importance of Regulatory Reform for Semiconductor Manufacturing

Executive Summary The ongoing global chip shortage, coupled with China's heavy investments in indigenizing semiconductor manufacturing capabilities, has brought attention to the importance of semiconductors to the U.S. economy, the fragility of semiconductor supply chains, and the decline of U.S. chipmaking capacity over the past three decades. In part as a result of this attention, Congress has advanced legislation to appropriate $52 billion in funding for the CHIPS for America Act. Approximately $39 billion will likely go toward incentivizing semiconductor manufacturers to build new chipmaking capacity in the United States. But more can be done to improve the resiliency of U.S. access to microelectronics beyond manufacturing incentives. This report outlines infrastructure investments and regulatory reforms that could make the United States a more attractive place to build new chipmaking capacity and ensure continued U.S. access to key inputs for semiconductor manufacturing.
 Key Findings: The United States currently builds fewer fabs * at a slower rate than the rest of the world. Part of the reason for this is permitting regulations which require long assessment timelines. Fabs have extensive infrastructure requirements, which interact with federal, state, and local regulations in complex ways. Modern fabs require access to (1) large plots of (2) seismically inactive land with a reliable, affordable, and stable supply of (3) water, (4) electricity, (5) talent,  (6)  transportation infrastructure, and  (7)  nearby land for co-locating with suppliers essential for constructing and operating a modern fab. The CHIPS Act correctly aims to increase the number of fabs constructed in the United States, but regulatory support and infrastructure investments are needed to ensure that these new fabs are built on time and on budget. The United States government should prioritize regulatory support at the local, state, and federal level to expedite fab construction. In particular, full implementation of several recommendations from the 2017 President's Council of Advisors on Science and Technology related to environmental review and permitting of high technology facility construction are essential.  1  The United States should also make infrastructure investments targeting utilities, transportation, and supply chain networks that will assist semiconductor manufacturers. Engagement with allies will be essential for increasing resilience in the semiconductor materials, gases, and chemicals supply chain. In the medium to long term, increasing domestic production and/or stockpiling should be considered. Increasing domestic United States production of many raw materials and chemicals is contingent on opening new mining and/or refining operations, which would require extensive permitting. Thus, coordination with allies who already have existing production and refining capacity may be more expeditious than attempting to establish new capacity in the United States. Though many of these materials have a limited shelf life, stockpiling of certain materials, modeled after existing United States government programs like those operated by the Defense Logistics Agency Strategic Materials, may be an option. The United States should quantify demand for key material inputs, identify potential alternatives, and support their development. Ongoing efforts supported by the Environmental Protection Agency and the semiconductor industry to develop substitutes for environmentally harmful greenhouse gases used in semiconductor manufacturing could serve as a template for further work to identify substitutes for certain materials, chemicals, and gases used in semiconductor manufacturing for which there is no commercially viable domestic supply.
 Introduction: Fab Infrastructure Requirements and Federal Permitting This paper, in concert with forthcoming companion papers on reshoring semiconductor manufacturing, argues that current semiconductor reshoring efforts should prioritize construction of leading-edge fabrication facilities and increase the resilience of the associated supply chain necessary to support these facilities. The U.S. Department of Commerce's 100-day review of the semiconductor supply chain in response to Executive Order 14017 on Securing America's Supply Chains generally aligns with this argument, finding "federal incentives to build or expand semiconductor facilities are necessary to counter the significant subsidies provided by foreign allies and competitors." 2 However, the United States has many regulations in place which may effectively counteract the purpose of CHIPS Act funds, slowing construction of new leading-edge fabrication facilities. In addition, the semiconductor industry must contend with myriad environmental, health, and safety (EHS) regulations that serve important purposes, but will inevitably slow the development of a more resilient semiconductor supply chain in the United States. Finally, the simple reality is that there are very few leading-edge semiconductor manufacturers in the world, and most of them are headquartered outside the United States. In practice, this means that the United States must craft policies that convince specific foreign companies to build outside of their headquarters country, where presumably they face significant political pressure to build domestically and enjoy easier access to policymakers to facilitate build-out in regulatory environments they can navigate adeptly. Local, state, and federal permitting processes in the United States are beneficial to the general public but present tradeoffs for semiconductor manufacturers. A 2017 report from the White House found that these permitting processes "minimize[d] environmental and community impact, which companies are not always economically incentivized to do. The combination of the current Federal and state permitting and review processes, however, can be slow, unpredictable, and lacking in transparency."  3  The report goes on to note that semiconductor factories are particularly vulnerable to permitting-related delays due to their long construction times, significant geographic footprint, and complex supply chains. In addition, due to the many specialized chemicals and gases used in the semiconductor manufacturing process, unique permits must be acquired, which can further delay, or even halt, operations. Adding to this challenge, the semiconductor industry's transition to a fabless-foundry operating model (in which semiconductor design and semiconductor fabrication are done by separate firms) in the past 20 years largely caught U.S. firms flat-footed. Many leading U.S. companies maintain an Integrated Device Manufacturer (IDM) operating model of doing both design and manufacturing in-house, an increasingly costly proposition, especially given the success of Taiwanese foundries that focus solely on manufacturing. In part as a result of both the U.S. regulatory environment and this ongoing structural shift in the industry, the United States builds fewer fabs at a much slower rate than other countries, and at a greater cost to companies. The United States should recognize the nature and value of indirect EHS regulatory support provided by other countries interested in attracting semiconductor manufacturers, and adopt policies that make it equally attractive to build fabs in the United States. This paper begins by reviewing fab rates of construction worldwide from 1990 to 2020, finding that the United States builds fabs far more slowly than other competitor countries and regions, notably Taiwan and China. Next, this paper reviews fabs' myriad unique infrastructure requirements. Finally, this paper discusses how these infrastructure requirements must contend with U.S. environmental, health, and safety regulatory permitting processes, potentially slowing the construction timeline for fabs and the associated supply chain necessary to support semiconductor manufacturing.
 The United States Builds Fewer Fabs More Slowly Than the Rest of the World It takes a long time (typically two to four years) to build a fab in any country. But analysis in this section shows that fab construction takes much longer in the United States than in the East Asian countries where most chipmaking currently takes place. There are many factors underlying the decline in U.S. chipmaking capacity, but one underappreciated factor may be the longer construction timelines associated with building new ("greenfield") American fabs. In part because of the unique infrastructure requirements of fabs and the regulatory processes these large construction projects must navigate, construction of semiconductor fabs takes several years. Between 1990 and 2020 there were approximately 635 greenfield semiconductor fabs built around the world. The average time between the construction start date and the beginning date of production was 682 days or roughly 1.86 years. This timeline does not include pre-permitting and pre-construction considerations, indicating that fab construction times exceed two years on average. There is considerable regional variation in the time it takes to build a new fab. As Figure  1  shows, Japan (584 days) and South Korea (620 days) build fabs significantly faster than the rest of the world on average. The Americas, of which the United States is the primary site of semiconductor fabrication facilities, build fabs at a significantly reduced speed, taking an average of 736 days, or roughly five months longer than Japan. Only construction of fabs in Southeast Asia takes longer than in the Americas. Until the recent slowing of Moore's Law, this industry introduced a new generation of chips every 12 to 18 months, meaning a five month delay in construction could be damaging to a firm's competitiveness. For context, in a period of five months, leading edge foundries like those operated by Samsung and TSMC could produce roughly 500,000 wafers. This trend is particularly troubling as the U.S. industry's position as a leading-edge manufacturer has been ceded to firms in Taiwan and South Korea. Both of these countries maintain the ability to build the world's most advanced fabs at rates that exceed the United States' ability to build trailing-edge fabs. Source: World Fab Forecast. Sample consisted of 635 greenfield fab construction projects with a "construction start" date between 1/1/1990 and a "production date" no later than 12/1/2020. The difference between "construction start" and "production date" was calculated for each fab project and then averaged by region. Figures  2-4  follow this methodology and further break the analysis out by decade. The United States' ability to expeditiously construct fabs has declined at the same time as the total number of fab projects in the United States has declined. Some of this is due to changes in the global semiconductor value chain, which has concentrated resources in Asia as foundries have risen in prominence, and countries like Taiwan, South Korea, and China have established significant market share in the industry from 1990 to 2020. However, during this same 30-year period, the time required to build a new fab in the United States increased 38 percent, rising from an average of 665 days (1.8 years) during the 1990 to 2000 time period to 918 days (2.5 years) during the 2010-2020 time period (Figure  2 ). At the same time, the total number of new fab projects in the United States was halved, decreasing from 55 greenfield fab projects in the 1990-2000 time period to 22 greenfield fab projects between 2010 and 2020.  The decline in the total number of new fab projects in the United States, as well as the speed with which those projects are completed, is striking when compared to other countries. In China, for example, the total number of new fab projects has increased from 14 during the 1990-2000 time period to 95 during the 2010-2020 time period (Figure  3 ). At the same time, China has seen the average number of days from construction start date to production date for these fab projects decrease from a high of 747 days (2 years) during the 2000-2010 time period to 675 days (1.85 years) during the 2010-2020 time period. China is building more fabs and building them faster.   Comparing average fab construction time across regions while controlling for the size of the fab construction project further illustrates the U.S. deficit in advanced semiconductor manufacturing. As Figure  5  shows, from 1990 to 2020, China built 32 fabs that produce 100,000 or more wafer starts per month (WSpM), while the rest of the world only built 24 during the same time period. * The United States had no greenfield fab projects that involved construction of factories with capacity greater than or equal to 100,000 WSpM during this time period. * High WSpM correlates strongly with advanced fabrication. The newest and most advanced fabs in the world can have WSpM greater than 200k. There are several important caveats to this data. First, this analysis is limited to greenfield projects and thus does not take into account fab expansion projects frequently undertaken by leading firms like Samsung and TSMC. Second, while China builds large advanced fabs faster than any other region in the world, most of these advanced fabs are owned and operated by non-Chinese headquartered firms. For example, Samsung and SK Hynix (South Korea) and TSMC (Taiwan) operate high volume fabs in China. This is also the case in Southeast Asia, where firms like Micron (USA) have built their most advanced and highest-WSpM fabs. The fact that some U.S.-headquartered firms are choosing to build their most advanced fabs outside the United States speaks to the challenge U.S. policymakers face in crafting subsidies, regulatory support, and an infrastructure environment that is competitive with allied and adversary countries in Asia.
 Fab Infrastructure Requirements This section reviews some of the infrastructure constraints facing chipmakers deciding where to establish greenfield fabs. Modern fabs require access to (1) large plots of (2) seismically inactive land with a reliable, affordable, and stable supply of (3) water, (4) electricity, (5) talent, (6) transportation infrastructure, and (  7 )
 Number of Fab Projects by Region China Japan Korea SE Asia Taiwan nearby land for co-locating with suppliers essential for operating a modern fab. These infrastructure requirements touch on construction and permitting processes in complex and sometimes costly ways. This section concludes with a discussion of infrastructure investments that other countries with strong semiconductor manufacturing industries have made, which could provide a model for future U.S. infrastructure investments. (1) Large plots of land. Fabs require large plots of land on which to locate their operations. While the cost of land is not as expensive an input in the semiconductor supply chain as the capital expenditures required to purchase and install semiconductor manufacturing equipment, it can present an obstacle to prospective semiconductor manufacturers. Fab "shells," the physical structures that house semiconductor manufacturing equipment (SME) and all associated materials and supporting operations, can occupy hundreds of acres and account for between 20 and 40 percent of the total capital expenditures associated with a greenfield semiconductor manufacturing facility.  6  For example, Samsung's Austin, TX-based fab occupies 640 acres (1 square mile).  7  Semiconductor manufacturers frequently purchase more land than they initially need in anticipation of future expansion. (2) Low seismic activity. Fabs and the semiconductor manufacturing equipment inside them are extremely sensitive to ambient vibration, meaning that they must be located in regions that are not seismically active, and on plots of land relatively isolated from highways, airports, and rail links.  8  While there are mitigation options available, and the industry has pioneered novel seismic isolation techniques, these unique requirements further limit the overall geographic supply of ideal building sites. (3) Stable supply of water. Some estimates indicate a modern fab consumes around five million gallons of water per day. 9 This reliance on a consistent supply of water was recently highlighted in Taiwan, when a record drought forced the government to institute measures to ensure the country's semiconductor manufacturers had access to adequate supplies of water to continue operations. 10 This intensive water consumption frequently necessitates that individual companies take action. For example, TSMC recently announced plans to establish on-site water storage and treatment facilities to prevent potential disruptions stemming from droughts and impurities.  11  (4) Stable supply of electricity. Fabs rely on a stable electrical grid to sustain their 24/7/365 operations. As wafer processing has increased in complexity and automation, energy consumption has gone up proportionally. Leading edge fabs can now consume as much electricity in one year as 50,000 homes.  12  Once a fab has reached volume production, energy costs can account for up to 30 percent of the fab's operating costs.  13  For smaller countries that are home to significant fab operations like Singapore, Taiwan, and South Korea, these factory operations consume a notable percentage of overall electricity in the country. The OECD estimates that TSMC alone accounted for 7.7 percent of total industrial electricity consumption in Taiwan in 2017.  14  The cost of an electrical grid failure that disrupts fab operations is substantial. A February 2021 power grid failure in Texas shut down Samsung, NXP, and Infineon chip factories for several weeks, costing Samsung alone over $270 million.  15  (5) Talent. Fab sites need to be strategically located in an area nearby a skilled workforce. This has resulted in fab clusters in close proximity to university systems with a consistent talent pool and in relative proximity to metropolitan areas. In the United States, these clusters include California's Silicon Valley, New York's Tech Valley, and Oregon's Silicon Forest, all of which draw technical talent from nearby cities, universities, and firms.  16  (6) Transportation infrastructure. Transportation infrastructure also drives the cost of fab construction around the world. The vast majority of semiconductor-affiliated commerce occurs by air freight, necessitating that fabs be close to an international airport. However, there are some chemicals and materials used in the semiconductor production process which, due to their hazardous nature, low value, or extreme weight, ship via ocean freight. In addition, fabs need adequate last-mile road infrastructure to take delivery of particularly large pieces of semiconductor manufacturing equipment and associated material handling devices. For example, ASML indicates one of its Extreme Ultraviolet Lithography systems "ships in 40 freight containers, spread over 20 trucks and three cargo planes."  17  This combination of air, ocean, and land transportation logistics presents myriad supply chain and regulatory bottlenecks. Specialty logistics services have been developed to facilitate semiconductor transportation. One supplier of transportation logistics services targeting the semiconductor industry advertises that they handle everything from SME and hazardous chemical delivery to air freight distribution of finished products (Figure  6 ).  18  Center for Security and Emerging Technology | 14 (7) Nearby land for co-location with key suppliers. Semiconductor fabrication facilities rely on a vast supply chain. Intel, for example, reports that it has identified over 16,000 suppliers.  19  However there are some suppliers whose products or services are so essential to the operations of modern fabs that they must co-locate production and warehousing facilities with the fabs they are supplying. Suppliers of rare gases and specialty chemicals used intensively in fabs need proximate land to establish production and warehousing facilities. Following the announcement that TSMC will be constructing a new fab in Arizona, several Taiwanese suppliers of specialty chemicals and gases to TSMC indicated they will be establishing operations nearby the new facility in addition to their current locations near TSMC's fabs in Taiwan.  20  In this instance, having additional land nearby prospective greenfield semiconductor fabrication sites is another important consideration.
 Government Policies to Reduce Infrastructure Costs and Accelerate Fab Construction Recognizing the costs that these infrastructure requirements impose on semiconductor manufacturers, many countries offer incentives to offset some of the price of constructing new factories. A joint report from the Semiconductor Industry Association and the Boston Consulting Group found that some countries provide both direct support to individual semiconductor companies and also incentives that are designed to facilitate the creation of a semiconductor ecosystem in close proximity to fabs.  21  The Japanese government has offered tax breaks, subsidies, and investments to attract semiconductor companies to establish joint ventures with Japanese firms, mirroring the sorts of incentives proposed by South Korea.  22  Similarly, the Taiwanese government's Ministry of Economic Affairs offers tax and tariff incentives as well as research and development (R&D) subsidies to attract semiconductor companies. Specifically, Taiwan maintains a business tax rate of 17 percent, allows semiconductor firms to credit up to 15 percent of their research & development expenses against their income tax bill annually, and permits firms to import semiconductor manufacturing equipment tariff-free.  23  Subsidies are also available for up to 50 percent of total spending by semiconductor firms who establish an R&D center in Taiwan. International semiconductor firms are clearly responsive to these incentives. The Taiwanese government notes these incentives have successfully attracted Micron, the U.S. memory chip company, to build a fab in Taiwan, while equipment suppliers like ASML from the Netherlands, as well as Applied Materials and Lam Research from the United States, have all set up R&D centers or training headquarters in Taiwan.  24  In addition, some countries focus on minimizing pre-and postfabrication operating costs for semiconductor manufacturers. A recent OECD report found that some governments support semiconductor manufacturers through the provision of water and electricity at below-market rates via state utilities.  25  Furthermore, the provision of land at below-market prices to semiconductor manufacturers was observed by the OECD to be a form of investment incentive. The OECD highlighted the case of Tsinghua Unigroup, a Chinese semiconductor firm which "purchased land for its foundry in Chengdu for CNY 240 per m^2, while the official average price for industrial land in second-tier cities was CNY 724 per m^2."  26  The Government of Israel also made use of similar land-specific incentives which were used by Intel to expand its operations in the country.  27  Finally, many countries in Asia also provide infrastructure support in the form of utilities and logistics investments, as well as providing for expedited procedural consideration and eased regulations associated with semiconductor factory construction.  28  When Micron first considered establishing a fab in Taiwan, the country's investment authority "assisted Micron in terms of land acquisition…accelerated the administrative process…eliminated investment barriers (such as coordinating underground pipelines and sidewalk construction)  [and]  organized job fairs to help the company recruit talent." 29 Taiwan has also established a series of free trade zones designed to facilitate efficient trading, warehousing, transport, and customs clearance processes that are critical to international semiconductor manufacturers. 30 The government of Singapore has provided myriad infrastructure investments designed to attract semiconductor manufacturers. Through government agencies like the Economic Development Board and JTC Corporation, Singapore has established four industrial estates that provide shovel-ready plots of land for semiconductor manufacturers and their suppliers that come preequipped with basic infrastructure like power, electricity and roads.  31  These estates also include ready-built facilities that feature chilled water, bulk industrial gas supply, high ceilings to accommodate SME, and incorporate vibration-control construction techniques.  32  The results of the Singaporean government's efforts are clear, having successfully attracted 14 global semiconductor firms employing 18,600 workers in the industry across these estates.  33  The value of this infrastructure support is not easily quantified but lowers the cost of doing business for semiconductor manufacturers and decreases construction timelines. And the success of this regulatory facilitation and infrastructure investment is clear: GlobalFoundries, one of the leading foundries in the United States, recently chose to substantially expand its semiconductor manufacturing facility in Singapore rather than doing so at its factory in New York.  34
 Semiconductor Fabrication Regulatory Permitting Considerations Fabs' many infrastructure requirements touch on local, state, and federal EHS regulations. These regulations implicate agencies at each level of government, sometimes with overlapping jurisdictions. As a result, construction must carefully navigate arcane regulatory processes to develop greenfield semiconductor fabs. Recognizing the time delays that these regulations and permitting processes place on semiconductor manufacturers, other countries provide incentives and indirect subsidies to expedite fab construction timelines. To date, the United States government has not provided sufficient local, state, and federal regulatory support to match the efforts offered by peer semiconductor-producing countries. Regulatory support steps could include fully implementing 2017 recommendations from the President's Council of Advisors for Science and Technology, such as identifying where federal permitting regulations for high-technology facilities are redundant with state rules and might therefore be modified or removed.  35  The U.S. Environmental Protection Agency (EPA) could also consider creating a "fast track" process for preconstruction and operating permits related to the Clean Air Act (CAA). State and local environmental agencies perform a first review of these permit applications, but the EPA retains the right to review any draft permit and provide comments to state or local authorities. The EPA has experimented with the use of flexible air permits in the past and should create a program specifically tailored to the needs of U.S. semiconductor firms that would accelerate construction of new fabs and re-tooling of existing fabs. For example, following receipt of a flexible air permit, Intel Corporation used this permits' advance approval to make 150-200 equipment changes and process modifications to its Aloha, Oregon fab that would have otherwise required EPA New Source Review permits and resulted in concurrent permits from the Oregon Department of Environmental Quality.  36  This flexible permit allowed Intel to make changes to its fab without notifying environmental regulators, so long as the changes did not result in the fab's emissions exceeding previously agreed-upon levels. This saved the company "hundreds of business days associated with making operational and process changes to ramp up production."  37  Representatives from Intel stated that, had they not received the flexible air permit, continued permitting-related delays would likely have pushed Intel to redirect its production investment and operating facilities to locations where changes could be made within existing environmental regulations (e.g. other U.S. states or the company's fabs in Ireland or Israel). 38
 United States Federal EHS Regulatory Processes Federal EHS laws and regulations govern both the construction and operation of fabs. These laws and regulations are generally overseen by the EPA, the U.S. Army Corps of Engineers, and the U.S. Department of the Interior. These agencies are tasked with auditing proposed construction projects for compliance with relevant federal regulations that touch on fab utility, transportation, and supply chain infrastructure requirements. For example, a regional economic development association in the state of Washington that sought to increase fab construction in the Pacific Northwest observed that "among local and federal agencies, the Bonneville Power Administration (BPA), the United States Army Corps of Engineers (USACE), and the Environmental Protection Agency (EPA), would be key players in coordinating site reviews with parallel agencies in Washington State."  39   At the federal level, there are many laws designed to maintain environmental quality that are also known to often significantly delay major construction projects. Notably, the National Environmental Policy Act (NEPA) review process governs construction projects deemed to be a "major federal action." 40 If the provision of CHIPS Act incentives are determined to be a "major federal action" 41 then the construction of new semiconductor fabs could be significantly delayed.  42  In 2020 the White House Council on Environmental Quality compiled data on timelines for 1,276 Environmental Impact Statements (EIS) filed between 2010 and 2018 and found that NEPA reviews averaged 4.5 years.  43  This permitting process does not include the average of 1.86 years it takes to physically construct a semiconductor fab. This number also does not reflect other federal environmental reviews, some of which may happen concurrent with the NEPA process or entirely separately. A 2017 report from the White House also identified preconstruction permits and operating permits required under the Clean Air Act as "the primary barrier to responsible and timely facility permitting" finding that "for some large projects
 Suppliers In addition to constructing fabs, a process which takes a minimum of two years, the process of establishing a more resilient semiconductor supply chain in the United States necessitates increased domestic production of critical minerals, materials, chemicals, and gases, all of which require lengthy permitting processes. As a result, ongoing collaboration with international allies may be a more expeditious means of increasing semiconductor supply chain resiliency than attempting to increase domestic production in the short term. Some of the semiconductor supply chain, such as providers of specialty chemicals, would need to construct production facilities that would implicate many of the same regulatory considerations identified in Table  1 . Other parts of the supply chain, such as suppliers of raw minerals, would encounter entirely new regulatory considerations such as mining permits that may slow the ability to ramp up domestic production. The June 2021 White House supply chain report found "establishing strategic and critical material production is an extremely lengthy process. Independent of permitting activities, a reasonable industry benchmark for the development of a mineralbased strategic and critical materials project is not less than ten years." 48 United States-based production of materials, chemicals and gases used in semiconductor manufacturing is, with few exceptions, limited. The United States Geological Survey's 2020 Mineral Commodities survey indicates materials like Arsenic, Beryllium, Bismuth, Boron, Cadmium, Helium, Indium, Rhenium, and Silicon are all used in the semiconductor manufacturing process.  49  The National Institute of Standards and Technology also maintains an index of 49 semiconductor process gases used in semiconductor production.  50  Except for a few materials like Helium and Silicon, U.S. firms do not produce or refine these materials in the United States. For example, Intel reports that it relies on smelters and refiners in China, Japan, Germany, Russia, Vietnam, the Philippines, Austria, and the United States for its supply of Tungsten, a key metal that is used to produce tungsten hexafluoride and tungsten sputter targets, both of which are used in the majority of semiconductor devices.  51  One reason that United States firms are not major suppliers of semiconductor materials is the increased cost of doing business in the United States. A Taiwanese supplier of specialty gases estimates that it costs five to six times more for them to build a factory in the United States than in Taiwan.  52  This price disparity is driven by several factors. In addition to contending with some of the construction permitting described above in Table  1  and higher labor market rates,  53  Taiwanese suppliers of semiconductor materials identified "the role of transportation systems" and a need for "seamless dual supply of electricity and natural gas at a favorable rate for the operation of purification and solvent recovery plants" as important factors, and requested federal and state government help to improve transportation system connectivity and utility provider reliability.  54  Both of these companies are single source suppliers of chemicals for TSMC in Taiwan and expressed interest in supplying TSMC's United States facility, but observe a clear increase in the cost of doing business in the United States. United States-based production of materials, chemicals and gases used in semiconductor manufacturing is also limited because many of these goods are derived from, or contribute to, environmentally harmful practices. For example, the semiconductor industry intensively consumes hydrofluorocarbons in the manufacturing process, a type of gas that is recognized by the EPA as a high global warming potential (GWP) greenhouse gas (GHG).  55  HFCs for semiconductor use are generally not produced in the United States. Recently, The Biden Administration's support for the Kigali Amendment to the Montreal Protocol on Substances that Deplete the Ozone Layer committed the United States to cap, and ultimately reduce, the semiconductor industry's use of HFCs.  56  However, until the semiconductor industry can innovate a substitute for the use of HFCs and qualify suppliers, there is no short-term alternative except to continue importing HFCs. Increasing the domestic resilience of this part of the supply chain in the medium to long term requires finding an international supplier willing to navigate the regulatory bureaucracy to establish new operations in the United States. Several companies and industry associations have highlighted how United States government regulatory decisions may increase the costs of doing business in the United States for semiconductor manufacturers and their suppliers. For example, the EPA has undertaken risk evaluations of several chemicals under the Toxic Substances Control Act that are used in the semiconductor manufacturing process.  57  Depending on the findings of these risk assessments, new regulations may restrict the production and supply of these chemicals in the United States  58  In addition, Executive Order 140083 (January 27, 2021) and the subsequent implementation of a temporary ban on new oil and gas leases under Department of Interior Order No. 3395 may reduce future opportunities for domestic helium development. Helium is recovered from natural gas deposits but exists in economic quantities in only a few places within the United States, mainly on federal lands, and is intensively used in semiconductor manufacturing.  59
 Conclusion This report demonstrates that the United States is building fewer fabs at a slower rate than the rest of the world. CHIPS Act manufacturing incentives correctly aim to increase the number of fabs constructed in the United States, but more policy work is needed to ensure these new fabs are built on time and on budget. Prioritize regulatory support at the local, state, and federal level to expedite fab construction. The United States should make infrastructure investments targeting utilities, transportation, and supply chain networks that will assist semiconductor manufacturers. These policies would make the United States competitive with the "foreign allies and competitors" identified in the June 2021 White House supply chain report, and would align United States incentives to match those already offered abroad. Fully implement several of the recommendations from the 2017 PCAST report: 60 1. The Federal government should review permitting for technology facilities to identify areas where regulations are redundant with state rules and might therefore be modified or removed. 2. The EPA should create additional "fast track" permitting options that allow fabs to make some operational changes without filing environmental permit applications, potentially modeled after the State of Oregon's Plant Site Emissions Limit (PSEL) program. 61 Engage with allies and partners to increase semiconductor supply chain resiliency in materials, gases, and chemicals. Because increasing domestic United States production of many raw materials and chemicals would be contingent on lengthy development of mining and/or refining capacity, coordination with allies who already have existing production and refining capacity is essential. Quantify current and forecasted demand for materials, gases, and chemicals used in United States semiconductor manufacturing. Stockpiling of certain materials, modeled after existing United States government programs like those operated by The Defense Logistics Agency Strategic Materials branch, 62 may be an option. Identify substitutes for environmentally harmful or strategically concentrated resources. The EPA is currently supporting semiconductor industry efforts to develop substitutes for environmentally harmful greenhouse gases used in semiconductor manufacturing. These efforts could serve as a template for further work to identify substitutes for certain materials, chemicals, and gases used in semiconductor manufacturing for which there is no commercially viable domestic supply.  63   Figure 1 . 1 Figure 1. Average Number of Days from Fab Construction Start Date to Production Date, by region 1990-2020
 Figure 2 . 2 Figure 2. Time Required to Build New Fabs in the United States (L) and Total Number of New Fab Projects (R), 1990 -2020
 Figure 3 . 3 Figure 3. Time Required to Build New Fabs in China (L) and Total Number of New Fab Projects (R), 1990 -2020
 Figure 4 . 4 Figure 4. Time Required to Build New Fabs in Taiwan (L) and Total Number of New Fab Projects (R), 1990 -2020
 Figure 5 . 5 Figure 5. Number of >100k WSpM Greenfield Fab Projects (L) and Average Number of Days from Fab Construction Start Date to Production Date for those Projects (R) by Region, 1990-2020
 Figure 6 . 6 Figure 6. Transportation Infrastructure Requirements for Semiconductor Production
 Table 1 . 1 Table 1 describes federal environmental requirements for construction in greater detail. Federal Environmental Requirements for Construction Relevant Law Agency Associated Requirements Regulation Jurisdiction Air Quality Clean Air Act (CAA) Environmental Permit for construction- Protection related pollutant emissions Agency Asbestos Clean Air Act (CAA) Environmental Report of asbestos releases Protection above a threshold Agency Dredged and Section 404 of the CWA Army Corp of Permit for discharge of Fill Engineers or dredged material Material/Water state regulator s Environmental National Environmental Environmental Submit Environmental Impact Policy Act Protection Assessment Agency Submit Environmental Impact Assessment Hazardous Comprehensive Environmental Permit to excavate Substances Environmental Response, Protection contaminated soil Compensation, and Agency Liability Act (CERCLA) Historic National Historic United States Pre-construction Properties Preservation Act Department of consultation for historic the Interior property considerations Polychorinated Toxics Substances Environmental Storage and disposal biphenol (PCB) Control Act (TSCA) Protection requirements Wastes Agency Solid and Resource Conservation Environmental Transporters of hazardous Hazardous and Recovery Act (RCRA) Protection waste must register with Wastes Agency EPA Spill Reporting Maintain a material safety data sheet (MSDS)
 ,[this]   permitting process can take 12-18 months."44  More recently, the EPA has announced a goal to make permitting decisions within six months of receipt. But as discussed earlier, given the tight timelines on which this industry operates, any delay can be costly to a firm's competitiveness.45    U.S. State and Local EHS Regulatory ProcessesState and local EHS regulations and agencies also have a bearing on semiconductor manufacturers. In some cases, the EPA delegates authority to implement regulatory programs to states and other agencies. For example, the U.S. Army Corps of Engineers (USACE) administers Section 404 of the Clean Water Act, which regulates the discharge of dredged and fill material into all waters of the United States. The USACE's Fort Worth, TX District reviewed an application of Samsung Austin Semiconductor LLC in 2014 to expand their fabrication facility in compliance with Section 404 of the CWA. However, this application also invoked reviews being conducted concurrently by the Texas Commission on Environmental Quality and a City of Austin Development Permit.46   Even after federal and state-level concerns are addressed, semiconductor building sites may run into local regulatory barriers. When GlobalFoundries was considering building a new fab in Malta, NY there were lingering zoning changes at the local level that needed to be made by the Town Boards of Malta and Stillwater, NY which delayed the site's initial development.47     United States Regulatory Processes Affecting Semiconductor
			 Stephanie Yang, "The Chip Shortage Is Bad. Taiwan's Drought Threatens to Make It Worse," The Wall Street Journal, April 16, 2021,

Abstract: The author would also like to thank Melissa Deng, Dale Brauner, and Danny Hague for editorial support. The author is solely responsible for any mistakes.


Key Concepts in AI Safety: Interpretability in Machine Learning CSET Issue Brief

Introduction Interpretability, also often referred to as explainability, in artificial intelligence (AI) refers to the study of how to understand the decisions of machine learning systems, and how to design systems whose decisions are easily understood, or interpretable. This way, human operators can ensure a system works as intended and receive an explanation for unexpected behaviors. Modern machine learning systems are becoming prevalent in automated decision making, spanning a variety of applications in both the private and public spheres. As this trend continues, machine learning systems are being deployed with increasingly limited human supervision, including in areas where their decisions may have significant impacts on people's lives. Such areas include automated credit scoring, medical diagnoses, hiring, and autonomous driving, among many others. 1 At the same time, machine learning systems are also becoming more complex, making it difficult to analyze and understand how they reach conclusions. This increase in complexity-and the lack of interpretability that comes with it-poses a fundamental challenge for using machine learning systems in high-stakes settings. Furthermore, many of our laws and institutions are premised on the right to request an explanation for a decision, especially if the decision leads to negative consequences. 2 From a job candidate suing for discrimination in a hiring process, to a bank customer inquiring about the reason for receiving a low credit limit, to a soldier explaining their actions before a court-martial, we assume that there is a process for assessing how a decision was made and whether it was in line with standards we have set. This assumption may not hold true if the decisionmaker in question is a machine learning system which is unable to provide such an explanation. In order for modern machine learning systems to safely integrate into existing institutions in high-stakes settings, they must be interpretable by human operators.
 Why Are Modern Machine Learning Systems Not Interpretable? Many modern machine learning systems use statistical models called deep neural networks which are able to represent a wide range of complex associations and patterns. To understand why decisions by deep neural networks are hard to interpret, consider two types of systems that are interpretable. One example is an earlier generation of AI systems which use human-specified rules instead of relying on data. As a simplified example, the autopilot system on an airplane uses a set of if-thisthen-that rules to keep the plane on course-if the nose drops, lift it; if the plane banks left, roll a little right, and so on. While the rules in real systems are more complicated, they nonetheless allow humans to look back on system behavior and recognize which this triggered which that. A second example is a linear model, a simple kind of machine learning model. Like all machine learning systems, a linear model uses number values called parameters to represent the relationship between inputs and outputs. For example, one could create a model to predict someone's salary from their age and years of schooling, two explanatory variables. In a linear model, the main parameters would be one number value to be multiplied by the explanatory variable "age," and one number to be multiplied by the other explanatory variable "years of schooling." Determining what value those two parameters should take is the learning part of machine learning. For a linear model, good parameter values can be found by simple calculations that may take a computer less than a second to perform. More importantly, because each parameter in a linear model is directly associated with one explanatory variable, understanding how the model works is simple. If, say, the parameter that gets multiplied by "age" is much higher than the parameter for "years of schooling," then the model is predicting age is a more important determinant of salary. Deep neural networks are different. By design, they have far more parameters than linear models, and each parameter is connected in complex ways with inputs and other parameters, rather than directly linking explanatory variables to the outcome that the model seeks to predict. This complexity is a double-edged sword. On one hand, models can represent highly complex associations and patterns, allowing them to solve problems previously considered out of reach for computers, including image recognition, autonomous driving, and playing the game of Go. On the other hand, unlike older or simpler computer systems, the internal functioning of each model is very difficult to understand. At this point it is worth noting why the term "black box," often used in this context, is not quite right to describe why deep neural networks are hard to understand. Machine learning researchers understand perfectly well how the mathematical operations underlying these systems work, and it is easy to look at the parameter values that make up the model. The problem lies in understanding how these millions (or even billions) of number values connect to the concepts we care about, such as why a machine learning model may erroneously classify a cat as a dog. In other words, interpreting deep neural networks requires both the ability to understand which high-level features in the data-such as a certain part of an image or a specific sequence of wordsaffect a model's predictions and why a model associates certain high-level features with a corresponding prediction-that is, how deep neural networks "reason" about data.
 How to Make Modern Machine Learning Systems More Interpretable Researchers are pursuing a range of different approaches to improving the interpretability of modern machine learning systems. A fundamental challenge for this work is that clear, well-defined concepts have yet to be developed around what it would mean for different types of systems to be interpretable. So far, interpretability research seeks to build tools making it somewhat more possible for a human operator to understand a system's outputs and inner workings. Saliency maps are one popular set of tools for making modern machine learning systems used for computer-vision applications more interpretable. Broadly speaking, saliency maps visualize which areas of an image led to a model's classification of the same image.  3  For example, we might investigate why a deep learning model has learned to identify images of cats and dogs from a large dataset of cat and dog images labeled as such. If we wish to understand why the model classified an image of a German Shepherd as a dog, a saliency map may highlight the parts of the image containing features present in dogs, but not in cats (for example, a large muzzle). In this way, a saliency map communicates to a human operator which part of the image prompted the machine learning system to classify the image as it did. Another popular method for making deep neural networks more interpretable is to visualize how different components of the model relate to high-level concepts that may affect the model's predictions-concepts such as textures and objects in image classifiers, grammar and tone in language models, or short-term vs. long-term planning in sequential decision-making models.  Unfortunately, existing methods to make modern machine learning systems interpretable fall short; they typically only provide one angle from which to view the system instead of taking a holistic view. To fully understand how a machine learning system works, we must understand how the data and learning algorithm affected training, whether training could have resulted in a different model under modified training conditions, and how all of these factors ultimately affect the system's predictions. At this time, our understanding of these questions is still very limited and the insights obtained from existing methods are fallible, require human supervision, and only apply to a small subset of application areas. For example, the saliency maps in Figure  1  do shed some light on how the model in question works. One can see, for instance, that the model focuses on the dog to classify it as such, but identifies the sailboat in part by looking at the ocean. Saliency maps, however, do not help a human observer understand what might have led to different outcomes. Likewise, Figure  2  shows a method for understanding what the different parts of an image classifier detect. Unfortunately, looking at this type of visualization does not help a human operator evaluate whether the system is likely to be accurate, fair, or reliable. The lack of a common, well-defined vocabulary for interpretable machine learning systems further exacerbates these shortcomings.  5  Key concepts, such as trustworthiness, reliability, transparency, or verifiability, are often used loosely or interchangeably rather than referring to standardized or generally accepted technical definitions of such terms, making it difficult to measure progress and to accurately and reliably communicate research results to the public.
 Outlook Interpretability will allow us to understand potential failure modes for machine learning systems, enable regulatory compliance and audits, and reduce the risk of models with algorithmic or datainduced bias being deployed. Rendering modern machine learning systems like deep neural networks interpretable will help us ensure that any such system deployed in safety-critical settings works as intended. Unfortunately, while an active and ongoing area of research, existing approaches to achieving interpretability do not yet provide satisfactory solutions. It remains unclear when-or even if-we will be able to deploy truly interpretable deep learning systems. In the meantime, our best option may be to stick with simpler, more inherently interpretable models whenever possible.   Figure 1 . 1 Figure 1. Examples of images and their corresponding saliency maps indicating which parts of the images contribute most to how a machine learning system trained on a large set of images would classify them.
 4 4
 Figure 2 . 2 Figure 2. A visualization showing examples of what different layers of an image classification network "see." The left-most column, depicting an early layer, is picking up lines; middle layers are detecting textures and patterns of increasing complexity; later layers, shown on the right, are looking for objects.
 6
 Endnotes1 Danny Yadron and Dan Tynan, "Tesla Driver Dies in First Fatal Crash While Using Autopilot Mode," The Guardian, June 30, 2016, https:// www.theguardian.com/technology/2016/jun/30/tesla-autopilot-death-selfdriving-car-elon-musk; Forough Poursabzi-Sangdeh, Daniel G. Goldstein, Jake M. Hofman, Jennifer Wortman Vaughan, Hanna Wallach, "Manipulating and Measuring Model Interpretability," arXiv [cs.AI] (November 8, 2019), arXiv, preprint, https://arxiv.org/abs/1802.07810; and Jennifer Valentino-DeVries, "How the Police Use Facial Recognition, and Where It Falls Short," The New York Times, January 12, 2020, https://www.nytimes.com/2020/01/12/ technology/facial-recognition-police.html.

			 For a more in-depth discussion of what an "explanation" is in a legal context, why explanations are necessary, and how AI explanations compare to human
			 Cynthia Rudin, "Stop Explaining Black Box Machine Learning Models for High Stakes Decisions and Use Interpretable Models Instead," Nature Machine Intelligence 1 (May 2019): 206-215, https://www.https://www.nature.com/ articles/s42256-019-0048-x.

Abstract: This paper is the third installment in a series on "AI safety," an area of machine learning research that aims to identify causes of unintended behavior in machine learning systems and develop tools to ensure these systems work safely and reliably. The first paper in the series, "Key Concepts in AI Safety: An Overview," described three categories of AI safety issues: problems of robustness, assurance, and specification. This paper introduces interpretability as a means to enable assurance in modern machine learning systems.


Owain Evans
Andreas Stuhlmüller
Chris Cundy
Ryan Carey
Zachary Kenton
Thomas Mcgrath
Andrew Schreiber

Predicting Human Deliberative Judgments with Machine Learning

There is great demand for automating tasks that require deliberative judgment. Current ML approaches can be unreliable: this is partly because such tasks are intrinsically difficult (even AI-complete) and partly because assembling datasets of deliberative judgments is expensive (each label might take hours of human work). We consider addressing this data problem by collecting fast judgments and using them to help predict deliberative (slow ) judgments. Instead of having a human spend hours on a task, we might instead collect their judgment after 30 seconds or 10 minutes. These fast judgments are combined with a smaller quantity of slow judgments to provide training data. The resulting prediction problem is related to semi-supervised learning and collaborative filtering. We designed two tasks for the purpose of testing ML algorithms on predicting human deliberative judgments. One task involves Fermi estimation (back-of-the-envelope estimation) and the other involves judging the veracity of political statements. We collected a dataset of 25,000 judgments from more than 800 people. We define an ML prediction task for predicting deliberative judgments given a training set that also contains fast judgments. We tested a variety of baseline algorithms on this task. Unfortunately our dataset has serious limitations. Additional work is required to create a good testbed for predicting human deliberative judgments. This technical report explains the motivation for our project (which might be built on in future work) and explains how further work can avoid our mistakes. Our dataset and code is available at https: //github.com/oughtinc/psj.
 Introduction
 Fast and slow judgments Machine Learning has been successful in automating mental tasks that are quick and effortless for humans. These include visual object recognition, speech recognition and production, and basic natural language prediction and comprehension  [1, 2, 3, 4] . Andrew Ng states the following heuristic  [5] : If a typical person can do a mental task with less than one second of thought, we can probably automate it using AI either now or in the near future.  1  In this technical report we refer to judgments that are quick (roughly 30 seconds or less) and easy for most humans as fast judgments. Fast judgments contrast with slow judgments, which may involve lengthy processes of deliberate reasoning, research, experimentation, and discussion with other people.  2  Many important real-world tasks depend on slow judgments: • Predict the verdict of jury members in a criminal trial. • Predict which engineers will be hired by a company with an extensive interview process. • Predict whether experts judge a news story to be fake or intentionally misleading. • Predict a doctor's advice to an unwell patient after a thorough medical exam. • Predict how a researcher will rate a new academic paper after reading it carefully. • Predict how useful a particular video lecture will be for someone writing a thesis on recent Chinese history. There is great demand for Machine Learning (ML) and other AI techniques for predicting human slow judgments like these, especially in hiring workers, detection of fake or malicious content, medical diagnosis, and recommendation  [8, 9, 10, 11] . However ML approaches to predicting these slow judgments are often unreliable  [12, 13, 14] : even if they do reasonably well on a majority of instances, they may have large errors on more demanding cases (e.g. on inputs that would be tricky for humans or on inputs chosen by humans to try to fool the algorithm). One source of the unreliability for ML algorithms is optimizing for a subtly wrong objective. Suppose a student gets video lecture recommendations from YouTube.  3  These recommendations may be optimized based on video popularity ("Do users click on the video?") and engagement ("Do users Like or share the video?"). These metrics are mostly based on fast judgments. Yet the student would prefer recommendations based on slow judgments, such as the evaluation of another student who has carefully checked the lecture for factual accuracy by tracking down the sources. Fast and slow judgments will sometimes coincide, as when a lecture is inaudible or off-topic. Yet a lecture may seem useful on first glance while turning out to be riddled with inaccuracies that careful research would expose.  4  Predicting slow judgments in the tasks above is challenging for current ML algorithms. This is due in part to the intrinsic difficulty of the tasks; predicting how a student evaluates a lecture is arguably AI-complete  [16] . Another difficulty is that collecting slow judgments is inherently expensive: if it takes five hours of fact-checking to recognize the errors in a lecture then a dataset of millions of such evaluations is impractically expensive. Big datasets won't solve AI-complete problems but will improve ML performance. Predicting slow judgments is also related to long-term AI Safety, i.e. the problem of creating AI systems that remain aligned with human preferences even as their capabilities exceed those of humans  [17, 18, 19, 20, 21] . Rather than create AI that shares only human goals, a promising alternative is to create AI that makes decisions in the way a human would at every timestep  [22, 23] . This approach of imitating human decision-making is only promising if it imitates human deliberate (slow) judgments  [24, 25] . A system making thousands of human-like slow judgments per second could have super-human capabilities while remaining interpretable to humans  [26, 27] .
 Using Fast Judgments to Predict Slow Judgments How can we get around the challenges of predicting slow judgments? One approach is to tackle the AI-completeness head on by trying to emulate the process of human deliberate reasoning.  5  A second approach (which complements the first) is to tackle the data problem head on and find ways to collect huge datasets of real or synthetic slow judgments  [38] . This technical report explores an indirect approach to the data problem. Instead of collecting a big dataset of slow judgments, we collect a small dataset of slow judgments along with a larger quantity of side information related to the slow judgments. What kind of side information would help predict a person's slow judgment? If Alice makes a slow judgment about a question, then Alice's fast judgment about the same question is relevant side information. As noted above, in predicting a student's thorough evaluation of a video lecture, it is helpful to know their fast judgment (e.g. after watching the video for only 30 seconds). Likewise, a doctor's guess about a diagnosis after 30 seconds will sometimes predict their careful evaluation. Another kind of side information for predicting Alice's slow judgment about a question are the judgments of other people about the same question. This is the idea behind collaborative filtering  [39] , where a person's rating of a song is predicted from the ratings of similar people. While collaborative filtering has been widely studied and deployed, there is little prior work on using fast judgments as side information for slow judgments. The motivation for using fast judgments is that they are often easily available and their cost is much lower than slow judgments. Human cognition is like an "anytime" iterative algorithm: whereas our slow judgments are more discerning and reliable, our fast judgments are well-typed and provide increasingly good approximations to slow judgments over time.  6  For most judgment tasks we can collect fast judgments from humans and these fast judgments will come at a cost orders of magnitude cheaper than slow judgments. Where fast judgments sometimes coincide with slow judgments and include uncertainty information 7 , it may be better to spend a fixed budget on a mix of slow and fast judgments than on slow judgments alone. Often fast judgments are not just cheaper to collect but are essentially free. YouTube, Facebook, Twitter and Reddit have vast quantities of data about which content people choose to look at, share with others, "Like", or make a quick verbal comment on. Using fast judgments as side information to predict slow judgments requires modifying standard ML algorithms. While the objective is predicting slow judgments, most of the training examples (e.g. most video lectures) only come with fast judgments. This is related to semi-supervised learning  [40, 41] , distant supervision  [42]  (where the training labels are a low-quality proxy for the true labels), learning from privileged information  [43] , as well as to collaborative filtering.
 Contributions and caveats This tech report describes a project applying machine learning (ML) to predicting slow judgments. We designed tasks where slow judgments (deliberate thinking and research) are required for doing well but quick judgments often provide informative guesses. We collected a dataset of fast and slow human judgments for these tasks, and formulated a set of prediction problems (predicting held-out slow judgments given access to varying quantities of slow judgments at training time). We applied ML baselines: standard collaborative filtering algorithms, neural collaborative filtering specialized to our tasks, and a Bayesian hierarchical model.  8  . Unfortunately our dataset turned out to be problematic and is unlikely to be a good testing ground for predicting slow judgments. This report describes parts of our project that may be usefully carried over to future work and summarizes what we learned from our efforts to create an appropriate dataset. The problem of predicting slow judgments (and of creating a dataset for the purpose)
 Fermi comparisons weight of a blue whale in kg < 50,000 population of Moscow * smaller angle in degrees between hands of clock at 1.45 < 15,000,000 driving distance in miles between London and Amsterdam < 371 weight of $600 worth of quarters in pounds * area of Canada in square miles < 328,000,000
 Politifact truth judgments Rob Portman: "Since the stimulus package was passed, Ohio has lost over 100,000 more jobs." Republican Party: "Charlie Crist is embroiled in a fraud case for steering taxpayer money to a de facto Ponzi scheme."
 Mark Zaccaria: "James Langevin voted to spend $3 billion for a jet engine no one wants." was harder than expected. We hope to stimulate future work that avoids the problems we encountered. Our main contributions are the following: 1. We designed two slow judgment tasks for humans: Fermi Comparisons involve mental reasoning and calculation, and Politifact Truth Judgments involve doing online research to assess the veracity of a political statement.  9  We created a web app to collect both fast and slow judgments for the task. These tasks and the app could be used in future work. 2. We relate predicting slow judgments with side information to collaborative filtering and we implement simple baselines based on this relation. 3. We diagnose problems with our dataset. First, while slow judgments were significantly more accurate than fast judgments, the difference was smaller than expected. Second, variability among subjects was hard to distinguish from noise; so ML algorithms could not exploit similarities among users as in collaborative filtering. Third, while there is clear variation in how users respond to different questions, this variation is very hard for current ML algorithms to exploit.
 Tasks and Datasets for Slow Judgments Our overall goal was (A) to define the ML problem of predicting slow judgments given a training set of fast and slow judgments, and (B) to find or create datasets which can be used to test algorithms for this problem. The domain that humans are making fast/slow judgments about should be AI-complete or closely related to an AI-complete problem.  10  Solving such a task will (for some problem instances) depend on patterns or structures that current Machine Learning algorithms do not capture. So while we cannot hope for highly accurate predictions of slow judgments, we can seek ML algorithms that "know what they know"  [44] . Such algorithms exploit patterns they are able to learn (producing confident accurate predictions) and otherwise provide well-calibrated uncertain predictions  [45, 46] . This section describes the AI-complete tasks we designed for testing algorithms for predicting slow judgments. Before that we first review related datasets in this area.
 Existing Datasets Many ML datasets record human slow judgments for AI-complete tasks. These include datasets of movie reviews [47], reviewer scores for academic papers  [48] , and verdicts for legal cases  [49] . There are also datasets where the ground-truth labels are the product of extensive cognitive work by humans (e.g. theorem proving  [38] , political fact-checking  [14] ) and these could potentially be used to study human slow judgments. However, these datasets do not contain fast judgments. In particular, for each task instance in the dataset, there's a slow judgment by a particular individual but there's no corresponding fast judgment. Moreover the datasets do not explicitly record information about the time and resources than went into the slow judgment  11  . For example, the reviewers of academic papers do not divulge whether they spent ten minutes or two hours reviewing a paper. Due to the lack of existing datasets, we created a new dataset recording human slow and fast judgments for AI-complete tasks. We designed two tasks for this purpose, which may be useful for future work in this area (even if our dataset is not).
 Two Judgment Tasks: Fermi and Politifact We created two tasks for humans that require deliberate, multi-step thinking and rely on broad world knowledge. In the Fermi Comparisons task (abbreviated to "Fermi") users determine which of two quantities is larger. This often involves simple arithmetic, factual knowledge, and doing back-of-the-envelope estimates to determine the order of magnitude of the quantity. Example questions are show in Figure  1 . Note that human subjects (who we refer to as "users") are not allowed to look up the quantities online as this would trivialize the task. In the Politifact Truth task (abbreviated to "Politifact") users evaluate whether a statement by a US politician or journalist is true or false (see Figure  1 ). They have access to metadata about the statement (see Figure  2 ) and are allowed to do research online. For both tasks, users assign a probability that the statement is true (with "0" being definitely false and "1" being definitely true) and they enter their probability via the interface in Figure  2 . Their goal is to minimize mean squared error between their probabilistic judgment and the ground-truth answer. Questions and ground-truth answers for Fermi were constructed and computed by the authors. Political statements and ground-truth (i.e. the judgments of professional fact-checkers) for Politifact were downloaded from the politifact.com API (following  [14] ). The Fermi and Politifact tasks satisfy the following desirable criteria: • A generalized version of each task is AI-complete. Fermi questions depend on mathematical reasoning about sophisticated and diverse world knowledge (e.g. estimate the mass of the atmosphere). If novel Fermi questions are presented at test time (as in a job interview for high-flying undergraduates), the task is AI-complete. Politifact questions can be about any political topic (e.g. economics, global warming, international relations). Answering them requires nuanced language understanding and doing sophisticated research on the web. • Fast judgments are informative about slow judgments. Given a question (as in Figure  2 ), fast judgments of the probability (e.g. less than 30 seconds) will be informative about a user's slow judgment (e.g. 5 or 10 minutes). However some questions are too difficult to solve with a fast judgment. • It is possible to make progress through analytical thinking or gathering evidence. For Fermi, breaking down the problem into pieces and coming up with estimates for each of the pieces is a useful cognitive strategy. For Politifact (but not Fermi), participants are allowed to use a search engine and to read web pages to check political facts. • The ground-truth is available for both Fermi and Politifact questions. The problem of predicting slow judgments we discussed in the Introduction is not limited to human judgments about objective facts (as the list of examples in Section 1.1 makes clear). However, if the ground-truth is available then this makes certain experiments possible.  12
 Data collection Human participants (who we refer to as "users") were recruited online  13  and answered questions using the UI in Figure  2 . Users see a question, a form for entering their probability judgment, a progress bar that counts down the time remaining, and (for Politifact only) a search engine tool. For each question, users provides fast, medium and slow probability judgments. In Fermi the user is presented with a question and has 15 seconds to make the fast judgment. Once the 15 seconds have elapsed, they get an additional 60 seconds in which they are free to change their answer as they see fit (the medium judgment). Finally they get another 180 seconds to make the slow judgment. The setup for Politifact is identical but the time periods are 30, 90 and 360 seconds. (Users are free to use less than the maximal amount of time. So for particularly easy or difficult questions, the "slow" judgment may actually only take 30 seconds or less.)  12  In Politifact, the ground-truth is just an especially slow judgment of an expert in political fact-checking and could be modeled as such. We did not try this in our experiments.  13  We used Amazon's Mechanical Turk (MT) for a pilot study. For the main experiment we used volunteers recruited via social media with an interest in improving their probabilistic judgments. We found it hard to design an incentive scheme for MT such that the users would make a good-faith effort to do well (and not e.g. declaring 50% on each judgment) while not cheating by looking up Fermi answers.  The Fermi data we collected consisted of 18,577 judgments, with a third of these being slow judgments. There were 619 distinct users and 2,379 Fermi estimation questions (with variable numbers of judgment per question). Each user answered 12.9 distinct questions on average (median 5). The Politifact dataset was smaller, containing 7,974 judgments, from 594 users, covering 1,769 statements. Each user answered 6.4 distinct questions on average (median 5).
 Descriptive Statistics Our goal was to use the dataset to train ML algorithms to predict slow judgments and such a dataset should have the following features: 1. Large, clean, varied dataset: as usual a large dataset with minimal noise is desirable. In particular there should be many judgments per user and many distinct questions. Possible sources of noise should be minimized (e.g. users are noisier when learning the task and if they pay less attention due to tedious or overly difficult questions). 2. Fast-Slow Correlation: fast judgments are informative about slow judgments but not too much. For some users and some questions, fast judgments may be similar to slow judgments. In other cases, the fast judgment will be uncertain or wrong. 3. User Variation: individual users vary in their overall performance and their performance in different kinds of question. We want to predict the slow judgments of an individual (not the ground-truth). In some tasks listed in Section 1.1 users vary because of different preferences. In Fermi and Politifact users might vary in their areas of knowledge (e.g. science vs. sport questions in Fermi). 4. Question Variation: individual questions vary in terms of how users answer them. For example, some questions are hard (producing random or systematically incorrect answers) and some are easy. This allows algorithms to predict user answers better based on features of the question. Did our data have the features above? As noted in Section 2.3 the dataset was relatively small (especially Politifact) and many users only answered a small number of questions. Correlations between fast and slow judgments are shown in Figure  3 . They show that judgments started out uncertain but became more confident (in either direction) given time, and that confident fast judgments were less likely to change. Figure  4  shows that slow judgments were more accurate than fast on average. Nevertheless, the differences between slow, medium and fast judgments are not as large as we had hoped. Did users vary in their accuracy? We expected users to vary in overall accuracy and accuracy for different kinds of question (e.g. some users do better on science than sports questions). However the level of variation among users was not high enough to rule out possibility that the best two-thirds of users did not vary (see Figure  5 ). This is a problem with the design of our experiment and we discuss it further in Section 4.  Histogram shows number of users who obtained a particular level of performance (measured by MSE). Note that 0.25 is the MSE achieved by a user who always says 50%. While users have widely varying performance, much of this is due to noise and we can't rule out the possibility that most users do not vary in actual skill (see Figure  5 ). The graph suggests we can't rule out the null hypothesis that the best two-thirds of users don't vary in their accuracy. (We removed users with less than k=6 judgments; the graphs were similar when k was set higher.) Questions in both Fermi and Politifact did vary significantly in difficulty for human users (see Figure  6 ). However, there are less than 2500 questions for each task. This makes it hard for algorithms to generalize from the question text or meta-data without substantial pre-training.
 Predicting Slow Judgments with ML This section presents a problem definition for predicting slow judgments using ML and results for baseline algorithms on our dataset. The ML task and algorithms are independent of the dataset and may be useful for future work in this area.
 Task: Predicting Slow Judgments with ML
 Task Definition For both Fermi and Politifact the data consists of user probability judgments about questions. Each probability judgment h(q, u, t) ∈ [0, 1] depends on the question text and meta-data features q, the user id u, and the time t ∈ {0, 1, 2} (where the indices correspond to fast, medium, and slow judgments respectively). We let ĥ(q, u, t) be an algorithm's prediction of the user judgment h(q, u, t). The task is to predict slow judgments h(q, u, 2) from the test set. While only slow judgments appear in the test set, the training set consists of fast, medium and slow judgments.  14  The loss function is mean-squared error over all slow judgments in the test set. Let T = {(q, u)} be the set of question-user pairs that appear in the test set. Then the objective is to minimize the mean squared error over slow judgments in the test set: 1 |T | (q,u)∈T h(q, u, 2) − ĥ(q, u, 2) 2 (1) Note that this problem setup is analogous to content-based collaborative filtering, where the task is to predict a user's held-out rating for an item (e.g. a movie), given the user id and a feature vector for the item  [51] .
 Train on Fast Judgments, Predict Slow Slow judgments are much more expensive to collect than fast judgments. Yet for questions that are either very easy or very difficult for a user, the fast and slow judgments will be very similar. So if a model can learn when fast judgments are a good proxy for slow judgments, it can predict slow judgments reasonably well from a much cheaper dataset (as discussed in Section 1.1). In our dataset we have the user's slow judgment for a question whenever we have the fast judgment. Yet we can simulate the strategy of mostly collecting fast judgments by masking many of the slow judgments from our training set.  15  Our results are computed for three different levels of masking. "Unmasked" means that all slow judgments in the training set are included. "Heavy" means 90% of question-user pairs have the medium and slow removed and 7% have just the slow removed. "Light" means we removed slow and medium judgments for 60% of question-user pairs and removed just slow for 30%.  16
 Algorithms and Results We ran the following baseline algorithms on our datasets: 1. Collaborative Filtering (KNN and SVD): The prediction task is closely related to collaborative filtering (CF). The main difference is that instead of predicting a user's judgment about an item, we predict a user's judgment about a question at a given time (where we expect users to generally converge to zero or one given more time). To reduce our task to CF, we flatten the data by treating each user-time pair (u, t) as a distinct user. We applied the standard CF algorithms K-nearest-neighbors (KNN CF) and singular value decomposition (SVD CF), using the implementation in the Surprise library  [52] .
 Neural Collaborative Filtering: We adapt the Neural CF algorithm  [53]  to our task. A neural net is used to map the latent question and user embeddings to a sequence of judgments for each time. Linear Neural CF forces the judgments to change linearly over time.
 Hierarchical Bayesian Linear Regression: To predict the user judgment for a question we pool all user judgments for the same question (ignoring user identity) and regress using a Bayesian linear regression model. This model exploits the temporal structure of judgments but it discards the question features and user identity. 4. Clairvoyant Model : Since user slow judgments are correlated with the ground-truth, algorithms will do better on new questions to the extent they can predict the ground-truth. Predicting ground-truth without sideinformation is difficulty for Fermi and Politifact. However, we can investigate how well a model would do if it had access to the ground-truth. The "Clairvoyant" model simply predicts that each user will respond with the ground-truth answer with full confidence. The "Clairvoyant Mean" model predicts the base-rate probability for a user given the ground-truth of a question.
 Results Table  1  shows results for Fermi and Politifact with different levels of masking of slow judgments. Hierarchical Linear Regression and SVD perform best. Since Hierarchical Linear regression ignores both question features and user identity, this suggests it is difficult for algorithms to improve predictions by modeling questions and users. This is additional evidence that our dataset is problematic. We had expected the Neural CF algorithms would do well, as they learn latent representations for both questions and users and (unlike SVD and KNN) they do not discard the temporal structure in the data. However, their poor performance is partly explained by the difficulty (discussed in Section 2.4) of distinguishing user variation from noise.  17  The performance of Clairvoyant Mean suggests that non-clairvoyant algorithms could be improved with a strong language and reasoning model that accurately predicts the ground-truth.  Problem: Fast and slow judgments were too similar There weren't big differences between fast and slow judgments. This can be addressed by choosing a task that requires more thinking and research than the Fermi and Politifact tasks. The task should also be incremental, such that a small amount of additional work yields a slightly better answer.  19  If additional work is unlikely to help, users will be averse to doing it (without big compensation). Problem: Variation between users was hard to distinguish from noise Lack of discernible variation can be addressed by collecting substantially more data per user and by trying to reduce noise (e.g. having practice questions, having less ambiguous questions, having tasks for which users are consistently fully engaged). A related issue is that human responses were low in information content. Users assigned probabilities (from 20 discrete options) to binary ground-truth outcomes. Many questions were easy and were answered with 95-100% confidence, while others were very difficult and answered with 50-55% confidence. Furthermore, users were rarely anti-correlated with the ground truth, so slow judgments for true statements were generally answered somewhere in the range 60-100%, which is a small region of the response space in terms of mean squared error. This problem of low information content could be addressed by having a task where user responses are richer in information: e.g. scalar valued, categorical (with many categories), or a text response such as a movie review. The cost of this change is that the ML modeling task becomes harder. Another way to address this issue is to have a task in which the goal for users is something other than guessing the ground-truth. There are many situations in which people make slow judgments for questions that are not about objective matters of fact. For example, when someone reviews a non-fiction book for their blog, they consider both objective qualities of the book ("Does it get the facts right?") and also subjective qualities ("Did I personally learn things from it? Will it help me make decisions?"). As noted above, there is nothing about our task definition or modeling that assumes the questions have an objective ground-truth.  20  Problem: Question features were hard for algorithms to exploit The questions in our tasks varied substantially in difficulty and content. Our models couldn't really exploit this variation, probably because (a) there were less than 2500 questions, and (b) predicting whether a question is challenging for humans is an intrinsically difficult task (drawing on NLP and world knowledge). Having models that can make use of question and meta-data features is an important goal for our research. If models can only predict slow judgments based on other human judgments, they will not be able to generalize to new questions that no humans have yet answered. The most obvious fix for this problem is to collect a much larger dataset. If the dataset contained millions of questions then language models would be better at learning to recognize easy vs. difficult questions. While collecting such a big dataset would be expensive, the cost could be mitigated by mainly collecting fast judgments.  21  An alternative to collecting a large dataset is to choose some object task (instead of Fermi and Politifact) for which pre-trained language models are useful (e.g. something related to sentiment analysis rather than judging the ground-truth of statements).
 Conclusion Machine Learning will only be able to automate a range of important real-world tasks (see Section 1.1) if algorithms get better at predicting human deliberative judgments in varied contexts. Such tasks are challenging due to AI-completeness and the difficulty and cost of data. We tried to create a dataset for evaluating ML algorithms on predicting slow judgments. The previous section discussed problems with our dataset and potential remedies. It's also worth considering alternative approaches to the challenge we outline in Section 1.1. First, some tasks may be more fruitful to model than Fermi estimation and political fact checking. Second, deliberation can be modeled explicitly by building on recent deep learning approaches  [28, 29, 30, 31, 32, 33, 34, 35, 36, 37] . To more precisely capture how humans deliberate, we could also record the individual steps people take during deliberating (e.g. by requiring users to make their reasoning explicit or by recording their use of web search). Finally, we acknowledge that while predicting slow judgments is an important task for the future of AI, it may be difficult to make progress on today. 6 Acknowledgements Figure 1 : 1 Figure1: Example questions. In Fermi, people guess whether the left-hand side is smaller than right-hand side. In Politifact they guess whether the speaker's statement is true.
 Figure 2 : 2 Figure 2: Question-answering UI for Fermi Comparisons (above) and Politifact Truth (below) tasks. Users are not allowed to do online research for Fermi but are for Politifact.
 Figure 3 : 3 Figure3: Correlation between fast and slow judgments for a given question-user pair for Fermi (above) and Politifact (below). Blue markers are proportional in size to number of instances and orange line is a linear regression through the data. Fast user judgments are often 50% but then become more confident in either direction given more time to think.
 Mean squared error of user's answers, FermiFast Slow
 Figure 4 : 4 Figure 4: Variation in User Performance in Fermi for fast and slow judgments.Histogram shows number of users who obtained a particular level of performance (measured by MSE). Note that 0.25 is the MSE achieved by a user who always says 50%. While users have widely varying performance, much of this is due to noise and we can't rule out the possibility that most users do not vary in actual skill (see Figure5).
 Figure 5 : 5 Figure 5: Mean performance on Fermi for different user quantiles. Users were divided into quantiles based on performance on a random half of the data and the figure shows their performance on the other half. Error bars show standard error from 5 random divisions into quantiles.The graph suggests we can't rule out the null hypothesis that the best two-thirds of users don't vary in their accuracy. (We removed users with less than k=6 judgments; the graphs were similar when k was set higher.)
 Figure 6 : 6 Figure 6: Variation in question difficulty for human users. Histograms show number of questions with a given MSE across users. Many questions had an MSE near zero (most users got them confidently right), while others were difficult (MSE near 0.25) or counter-intuitive (MSE greater than 0.25).
 Table 1 : 1 Mean squared test error for various algorithms and different levels of masking slow judgments at training time. Note that Clairvoyant models had access to groundtruth and other models did not. Politifact Fermi Unmasked Light Heavy Unmasked Light Heavy KNN CF 0.127 0.130 0.133 0.115 0.125 0.134 SVD CF 0.124 0.126 0.127 0.102 0.113 0.112 Linear Neural CF 0.131 0.135 0.135 0.137 0.141 0.141 Neural CF 0.130 0.131 0.129 0.136 0.136 0.138 Hierarchical Lin. Reg. 0.123 0.126 0.127 0.098 0.107 0.114 Clairvoyant 0.242 0.242 0.242 0.216 0.216 0.216 Clairvoyant Mean 0.112 0.112 0.112 0.111 0.111 0.111 Always Guess 50% 0.137 0.137 0.137 0.138 0.138 0.138 4 Discussion: Mitigating Dataset Problems Some problems with our dataset were described in Section 2.4. How could these problems be mitigated in future work?
			 Ng intends it as a heuristic rather than a rigorous scientific conclusion. 2  The distinction is similar to that between System 1 (fast) and System 2 (slow) cognition [6, 7] . However, in this work we distinguish judgments by how long they take and whether they make use of external information and not by the underlying cognitive process. For instance, fast judgments can depend on quick application of analytical reasoning.
			 This is just meant as an example of a recommender system and is not a comment on the actual YouTube algorithm. A paper [15]  on YouTube recommendations states that they optimize for whether users follow a recommendation (weighted by user watch-time). This will mostly depend on fast judgments. 4  It could be argued that YouTube, Facebook and other sites are optimizing for being entertaining and keeping users on the site and that these are well predicted by fast judgments. Yet it's clear that users sometimes seek content that they would rate highly after a slow judgment (e.g. for educational purposes, for help making a business decision). So the question remains how to build ML algorithms for this task. 5  There is a large and varied literature aiming to create algorithms that perform sophisticated reasoning. Here are some recent highlights: [28, 29, 30, 31, 32, 33, 34, 35, 36, 37]
			 In doing a long calculation we might have no idea of the answer until we solve the whole problem. For many real-world problems our quick guesses are somewhat accurate and gradually improve with time.7 That is, the human provides both a guess and a measure of confidence. 8  Code is available at https://github.com/oughtinc/psj
			 Both tasks require humans to decide some objective matter of fact. Yet there is no requirement that slow judgments be about objective facts: e.g. a student's judgment about a lecture is partly based on their own preferences and interests. 10  The important real-world problems of predicting slow judgments in Section 1.1 are plausibly AI-complete.
			 A paper [50]  on adversarial examples for humans does record human judgments under different time constraints. But this domain is not AI-complete and the slower judgments are still pretty fast.
			 The test set is randomly sampled from the entire dataset and so the same users and questions can appear both in train and test.
			 This is like the standard practice in semi-supervised learning, where researchers remove all but 5% of the labels from the training set to simulate a situation where most data is unlabeled16  In order to achieve clear-cut separation of the training data and the held-out test set we also implemented a masking procedure before doing the additional masking for Light and Heavy. For each judgment in the test set, we removed the medium judgment made by the same user about the same question from the training set. We also stochastically remove the corresponding fast judgment with 80% probability.
			 Like Hierarchical Linear Regression, the Linear Neural CF assumes that judgments evolve linearly over time. However, Linear Neural CF does not build in which data-points to regress on and wasn't able to learn this. 18  We experimented with various language models (results not shown). We found it difficult to train or fine-tune these models on our small dataset without overfitting.
			 This would also allow a series of intermediate times between fast and slow. 20  We collected data for a third task (not discussed elsewhere in this report), where the aim was to evaluate Machine Learning papers. We asked researchers to judge papers subjectively ("How useful is this paper for your research?") rather than objectively ("Should the paper be accepted for a conference?"). Unfortunately we did not a sufficient amount of data from volunteers. But we think some kind of variant on this task would be worth doing.
			 It remains to be seen how well this can work if the goal is to predict slow judgments.

Abstract: Machine Learning (ML) has been successful in automating a range of cognitive tasks that humans solve effortlessly and quickly. Yet many realworld tasks are difficult and slow : people solve them by an extended process that involves analytical reasoning, gathering external information, and discussing with collaborators. Examples include medical advice, judging a criminal trial, and providing personalized recommendations for rich content such as books or academic papers.