ABSTRACT. Background and rationale The Chips Act of 2022 highlights topics of critical importance, including the “cyber workforce”, microelectronics workforce, bioeconomy, and artificial intelligence. It is widely known, as the National Science Board has stated, that “foreign-born individuals have long been major contributors to science and engineering (S&E) in the United States”. For example, the Center for Security and Emerging Technology states that “approximately 40% of high-skilled semiconductor workers in the United States were born abroad”. Nationally representative surveys can quantify the share and movement of foreign-born individuals as they move through the U.S. S&E education and training system to the S&E workforce in various fields of interest. For example, most U.S. S&E doctorates who are non-U.S. citizens at graduation expect to remain in the United States after graduation according to the Survey of Earned Doctorates conducted by the National Center for Science and Engineering Statistics within the National Science Foundation; however, when surveyed later, a considerable proportion of foreign-born U.S. S&E PhD recipients have left the United States to pursue future research and employment. This has a significant impact on the S&E ecosystem and empirical evidence can inform various levers employed by government, industry, and academia.

This work aims to quantify the movement of foreign-born U.S. S&E PhD recipients and investigate correlates using nationally representative surveys and bibliometric data in effort to produce high quality evidence related to the participation of foreign-born U.S. S&E research PhD recipients in the U.S. S&E ecosystem.

Initial results The proportion of foreign-born S&E workers in the United States has steadily increased in the past decade, representing 24% of the U.S. S&E workforce in 2019. Among the highly trained U.S. S&E research doctorate holders, the share of foreign-born is larger. It has increased from 25% of the 1971-1975 cohort to 48% of those who graduated in the period of 2016-2020. For the field of semiconductors, foreign-born individuals accounted for more than 50% of S&E research doctorate graduates since the late 1980s, reaching 66% in the latest cohort of graduates from 2016-2020. About 27% of foreign-born U.S. S&E PhD recipients who were on a temporary U.S. visa when graduating resided abroad in 2015, representing a significant amount of talent flow from the United States to other countries.

Methods and further analysis Mobility is studied through the use of nationally representative surveys of science, engineering, and health research PhD recipients. The NCSES Survey of Earned Doctorates, a census of all research PhDs from U.S. institutions, provides respondent-reported demographics and educational history, including birthplace, citizenship status, doctorate field of study, the 2005 Carnegie classification of the doctoral institution, post-graduation employment, sex, race, ethnicity, and type of support for graduate study. It is linked to the NCSES Survey of Doctorate Recipients (SDR), which provides respondent-reported employment outcomes, including occupation, sector of employment, job activities, and location information for the survey reference year. The longitudinal data file of the SDR (2015-19) provides longitudinal data on mobility (geographical and employment) over the period of the surveys. The respondents to the cross-sectional SDR 2015 are linked to the bibliometric database Scopus, providing information on publication history. The author affiliation, co-author networks, and citation topic information contained in the metadata of publications provide unique longitudinal data to explore the mobility of PhD scientists and engineers throughout their publication careers. These datasets will be used to produce analyses of characteristics of U.S. Ph.D. recipients with regards to country of origin, mobility, and various correlates. Multi-variate regression will explore the significance of the correlates between those who remain in the United States after graduation and those who do not.

Significance A fundamental belief of government investment in higher education and training is that U.S. universities play a critical role in training the future workforce in key industries. This data set provides descriptive information about training, publication, and employment as well as self-reported demographics. These rich, detailed data sources allow policymakers to develop evidence for interventions that support critical fields, such as those listed in the CHIPS Act of 2022.

ABSTRACT. Limited research has focused on the experiences of foreign born/foreign trained (FB/FT) faculty compared to the U.S. born/U.S. trained (USB/UST faculty), and even fewer have used the intersectional lens of gender to explore issues faced by FB/FT women faculty compared to either FB/FT men faculty or USB/UST women faculty in STEM departments. This systematic review and meta-synthesis of extant qualitative literature on FB/FT faculty using intersectional lenses should provide valuable information to develop policies and practices tailored to better enhance inclusion, career success and retention of both FB/FT and USB/UST faculty of all genders and race/ethnicities. Since the U.S. academic STEM workforce is highly dependent upon FB/FT faculty, understanding their experiences and how their retention might be enhanced is crucial for innovation and science.

ABSTRACT. Background In most academic fields, women remain substantially underrepresented among tenured and tenure-track faculty compared to the U.S. population [1]. Despite broad interest in measuring, explaining, and mitigating gendered retention in faculty careers, evidence for its magnitude, ubiquity, and causes remains controversial. Past work has tended to examine separate aspects of retention dynamics, using either large-scale statistical analyses [2-4] that report within- or cross-disciplinary gendered retention rates but cannot explain them, or small-scale surveys or ethnographies [5, 6] that provide detailed explanations for why women leave academia, but do not provide large-scale evidence or systematic conclusions. Moreover, many studies report conflicting conclusions, depending on discipline, study design, and sample size. A deeper understanding of cross-disciplinary gendered faculty retention patterns would help illuminate the social processes that drive systemic underrepresentation, and would inform policies to improve retention and to mitigate the induced asymmetric loss of talent and concomitant scientific discoveries [7].

Data & Methods We describe the results of a unique large-scale study of women’s retention in academia that combines the generality of a multidisciplinary, census-scale statistical analysis of administrative retention data with the explanatory power of deep survey data on gendered reasons for leaving faculty jobs. We quantify gendered retention using detailed employment data from a census of 282,759 tenure-track faculty who were active in their roles between 2011-2020, spanning all domains of academia, including science, technology, engineering, and mathematics (STEM) fields, the social sciences, the humanities, and medicine, and representing all 391 PhD-granting institutions in the U.S. First, we consider retention across academia as a whole, then analyze large groups of disciplines, then each discipline itself. We then identify detailed explanations for these retention patterns using 10,071 responses to a new survey of former and current tenure-track professors across 17 disciplines, sampled from the larger census dataset. Finally, we complement these findings with a qualitative analysis of nearly 7,000 free-text responses by the survey respondents, which highlight the issues faculty identify as driving gendered retention, and specific policy recommendations to improve retention.

Results I. Longitudinal Analysis We find that across 10 years of observation, at every career age, women faculty are more likely to leave their academic jobs than men—7% more likely as assistant professors, 21% more likely as associate professors, and 29% more likely as full professors—and less likely to be promoted than men—7% less likely as assistant professors and 12% less likely as associate professors (z-test, all p < 0.001). The persistence of gendered attrition and promotion patterns across career age contrasts claims that a lack of gender diversity among senior faculty is primarily due to slow demographic change and long career lengths [8]. The gendered attrition pattern we observe predicts that, for a hypothetical gender-parity cohort of new faculty, women’s representation would fall to 40.6% after 35 years, a loss of nearly 1 in every 5 women faculty. Together, these results show a systematic effect in which women faculty leave academia at significantly higher rates than men, in all years of a faculty career. However, this large-scale statistical analysis says little about the reasons that women and men faculty leave academia.

II. Survey Analysis To understand why women faculty leave academia at higher rates than men, we applied an exploratory factor analysis to responses to our broad survey, based on four categories of faculty stress, grounded in the literature: research pressures related to the job itself (i.e., obtaining funding, scholarly productivity), work-life balance stressors related to juggling work and life (i.e., caring responsibilities, hours worked), workplace culture stressors related to how an academic feels around their colleagues (i.e., dysfunctional departmental culture or leadership, harassment), and departmental support stressors related to the (lack of) external support by their departments (i.e., salary, lack of administrative support).

We find that women leave in response to different stressors than men. Women who left academia selected workplace culture reasons for leaving 1.3 times more often than research pressures, 1.6 times more often than work-life balance, and 1.9 times more often than departmental support. Across nearly all career ages, women identify workplace culture as the most prevalent reason for leaving academia, except in the very early career, when work-life balance briefly dominates. In contrast, men who left academia selected reasons from all four categories with roughly equal frequency throughout their careers. Both current and former women faculty are also more likely to feel pushed out of their jobs than pulled into better jobs, compared to men (t-test, p < 0.001). This was especially true for women of color. While there are some meaningful discipline-level differences, these findings hold true across academia, even in disciplines where the majority of faculty are women.

Via free response, we asked respondents to explain what needed to be different in their former or current jobs in order to reduce the stress they felt in their positions (current), or for them to stay in their positions (former). We characterized the responses, then we estimated the themes and their relative frequencies. This qualitative analysis further emphasized the differential impact of workplace culture on women’s retention, and also allowed us to derive additional insights about the causes of stress and leaving that are not fully captured by the survey.

Significance Individual faculty experience academia differently depending on their gender, race, and career stage. Our results broadly quantify the complex and structural nature of retention, and show that gendered retention patterns are ubiquitous, significant, and driven by differences in how different faculty experience academia and their disciplines. Specifically, while past studies focus on work-life balance reasons for women's differential attrition, our work shows that issues with workplace culture are the most common reasons, especially for older women. To achieve equity in academia, departments and institutions will need to address the underlying reasons, across all career stages, that drive these unequal outcomes.

References [1] Wapman K.H. et al. “Quantifying hierarchy and dynamics in U.S. faculty hiring and retention.” Nature, 610, 120-127, 2022. [2] Kaminski, D., & Geisler, C. “Survival analysis of faculty retention in science and engineering by gender.” Science, 335(6070), 864-866, 2012. [3] Gumpertz, M. et al. “Retention and promotion of women and underrepresented minority faculty in science and engineering at four large land grant institutions.” PloS One, 12(11), e0187285, 2017. [4] Ceci, S.J. et al. “Women in academic science: A changing landscape.” Psychological Science in the Public Interest, 15(3), 75-141, 2014. [5] Martinez, L.R. et al. “Fleeing the ivory tower: gender differences in the turnover experiences of women faculty.” Journal of Women's Health, 26(5), 580-586, 2017. [6] Gardner, S.K. “‘I couldn’t wait to leave the toxic environment’: A mixed methods study of women faculty satisfaction and departure from one research institution.” NASPA Journal About Women in Higher Education, 5(1), 71-95, 2012. [7] Kozlowski, D. et al. “Intersectional inequalities in science.” PNAS, 119(2), e2113067119, 2022. [8] Hargens, L.L. & Long, J.S. “Demographic inertia and women’s representation among faculty in higher education.” The Journal of Higher Education, 73(4), 494-517, 2002.

ABSTRACT. Institutional leaders have long championed interdisciplinary research; however, researchers have paid relatively little attention to the people responding to such calls and their subsequent career outcomes. With the benefit of two large datasets spanning from 1986 through 2016, we show that interdisciplinary dissertations have become consistently more common in recent years as institutional leaders have highlighted the value of boundary-spanning research for solving important and emergent problems. With the benefit of survey data from a near-complete population of all dissertators in the US starting in 2001 through 2016, we observe a consistent upward trend in interdisciplinary dissertations. Unfortunately, we show that these interdisciplinary dissertators have experienced a comparably persistent penalty when considering salaries for their first year after earning the PhD. We also show that among interdisciplinary dissertators, individuals in lower-paying fields tend to earn more when choosing distantly related topic-combinations whereas researchers in higher-paying fields tend to be most rewarded for staying within relatively narrow disciplinary silos.

ABSTRACT. Rationale No standard measures exist comparing science capacity across nations. Many proxies attempt to capture the underlying construct or dynamic of scientific capacity. The Organization for Cooperation and Development (OECD) compares subsets of wealthy nations . The National Science Board’s Science and Engineering Indicators Report compares a limited number of nations based on investment in R&D, performance indicators (output), an intensity indicator for eight nations (Table RD-7), and a comparison of country R&D performance for 17 countries (Table RD-8). UNESCO reports on national statistics and indicators in its Science Report . Data without a workable framework or theoretical construct inhibits comparisons for policy purposes, a gap which we seek to address with this study. Across the literature, we find authors choosing many different, distinct variables or clusters of variables to compare nations. These analyses are customized for the purpose of a single article, and they do not provide consistent measures. A descriptive theory of scientific capacity and a standard set of measures would help to compare nations. However, no research program has adequately examined nations conducting science to produce a useful set of indicators or a national science capacity index. This paper reviews the literature to identify and assess the variables used to measure national scientific capacity. A complex-systems heuristic is presented to hierarchically categorize indicators and provide a multi-level theoretical framework for the categorization of numerous indicators. Next, we present data regarding indicators of national scientific capacity and conduct an exploratory analysis to assess measurement coverage, clustering, and convergent /discriminant validity in relation to other well-established variables. Lastly, we seek to ultimately move towards a composite index of indicators which may be of use in both theory testing and policy decision making.

Literature Support This paper draws inspiration from earlier efforts to compare nations on science capacity (alternately termed ‘impact’ (King, 2004), ‘wealth’ (May, 1997), or ‘productivity’’ (Cole & Phelan, 1999). Each of these measures has merit in themselves, but they do not disambiguate the individual national measures to an internationally comparable set--which we acknowledge is a challenging task. We approach this task by, first, defining the construct of interest as one of ‘capacity’ rather than productivity or impact—since those latter features will include contributions from international collaborations, which confound measures of national strength. We view capacity as the explicit and latent resources, capabilities, and competencies available to a nation to absorb and exploit new knowledge from wherever in the world it emanates (Porter, 2011). Capacity will not always be evident in outputs such as number of papers published or number of citations, or as inputs such as government or private sector funding. Thus, additional measures related to scientific and technical human capital (Bozeman, Dietz, & Gaughan, 2001)., regarding number and quality of researchers in a nation are important. So too needed are measures that capture the position of the nation within the international network of scientists (Whetsell, 2022), as nations may leverage relationships and network position within the system to deploy domestic capacity. Measures of national infrastructure, rule of law, political climate/academic freedom, and regulatory quality also need exploration (Coppedge et al. 2011; Grimm and Saliba, 2017). By proposing a broader approach than taken by King or May, we combine measures to propose a composite index. The index will distinguish among inputs, capacity, and outputs, with a complex systems view of hierarchical but loosely coupled interdependence between inputs, capacity, and outputs (Simon, 1996). Further, complex systems theory suggests that measures across nations often manifest empirically in highly skewed distributions that suggest power law dynamics where we can test efficiency of national systems (Katz, 2006; Wagner, 2009).

Methods Science capacity can be defined as the resources, capabilities, and competencies needed to create and support knowledge creation, conduct research and development, and disseminate science. Nations invest in science capacity in accordance with policy to the extent that resources are available. In the 21st century, in imitation of the wealthy nations, many more nations have adopted a science policy that targets science capacity building, and with some success. Data are now available that were lacking just a few years ago. We suggest that a greater variety of indicators can be used to assess national capacity to increase the “likelihood of converging to an accurate understanding of knowledge produced by research” (Kostoff, 1995, p.8) where a collection of indicators is more likely to give a better evaluation of research strength than one indicator alone (Martin, 1996). We begin by exploring the usefulness and complementarity of more than 12 indicators, grouped into three theoretical domains of their tiered contribution to science capacity: 1) facilities and infrastructure are needed to support and build capacity, represented by national wealth, and an educated populous who contribute to and create demand for knowledge economy; and social order and political environment; 2) means by which science is conducted, including numbers of trained scientists and engineers; R&D spending; institutions and equipment, and openness to international engagement; and 3) output of scientific and technological knowledge and its diffusion to the larger world through coauthorships; scholarly articles; citations and their impact; teaching; and patents as a proxy for embedded knowledge and demand. Frame (2005), in attempting a similar project, noted that “quantitative indicators of scientific and technological activity are often of questionable validity and reliability. This is particularly true in lesser developed countries, where the lack of data gathering skills may frequently result in the development of misleading indicators….” Frame did, however, find correlations among a similar set of indicators to those used in the current study (science and technology manpower data, expenditure figures, student enrollment in higher education, and number of degrees conferred), suggesting that the approach can have some validity. Data have improved. The Frame approach has new validity.

Efficiency of Investment A composite index of scientific facilities, means, and output could provide a useful measure of national science capacity, which we explore. We note that, at the time when variables were collected, capacity may be underused, used efficiently, or a nation may be outproducing what would be expected based on inputs. Thus, capacity should not be viewed as a fixed state but rather a continuous dynamic state that has here been measured at just one point in time, so dynamics must be considered in future research. As with any complex system, change and interrelationships are non-linear and there are multiple sites of agency with intricate levels of causality that cannot be fully captured in an index. To go beyond our variables and give a sense of science efficiency among nations, we will consider a Triple Helix (Leydesdorff & Etzkowitz, 1996) context—and explore how to contributions of these characteristics can aid in efficient use of resources.

Conclusion In this paper we propose an approach to measuring national science capacity based on a set of indicators, tied to previous research, presented within a complex-systems theoretical framework. We attempt to apply these concepts to compare the scientific facilities, means, and output of nations. We hope that the approach suggests pathways forward for the community to create useful tools for assessing current science capacity and for supporting international collaboration, and we look forward to comments.

References

Bozeman, B., Dietz, J. S., & Gaughan, M. (2001). Scientific and technical human capital: an alternative model for research evaluation. International Journal of technology management, 22(7-8), 716-740. Cole, S., & Phelan, T. J. (1999). The scientific productivity of nations. Minerva 37, 1-23. Coppedge, M., Gerring, J., Altman, D., Bernhard, M., Fish, S., Hicken, A., ... & Teorell, J. (2011). Conceptualizing and measuring democracy: A new approach. Perspectives on Politics, 9(2), 247-267. Frame, J.D. (2005). Measuring scientific activity in lesser developed countries. Scientometrics, 2, 133-145. Katz, J. S. (2006). Indicators for complex innovation systems. Research policy, 35(7), 893-909. King, D. A. (2004). The scientific impact of nations. Nature, 430(6997), 311-316. Kostoff, R. (1995). The Handbook of Research Impact Assessment. US Government Printing Office, DTIC Report Number AOA296021. Leydesdorff, L., & Etzkowitz, H. (1996). Emergence of a Triple Helix of university—industry—government relations. Science and public policy, 23(5), 279-286. Martin, B.R. (1996). The use of multiple indicators in the assessment of basic research, Scientometrics 36 3, 343-362. May, R. M. (1997). The scientific wealth of nations. Science, 275(5301), 793-796. Porter, M. E. (2011). Competitive advantage of nations: creating and sustaining superior performance. Simon and Schuster. National Science Board (2019). Publications Output: U.S. Trends and International Comparisons. Science and Engineering Indicators. http://ncses.nsf.gov OECD (2021). Main Science and Technology Indicators Database. http://oe.cd/msti OECD (2015). Frascati Manual 2015: Guidelines for Collecting and Reporting Data on Research and Experimental Development, The Measurement of Scientific, Technological and Innovation Activities, OECD Publishing, Paris. doi: http://dx.doi.org/10.1787/9789264239012-en Simon, H.A. 1996. The Sciences of the Artificial, 3rd ed. MIT Press UNESCO (2020). The UNESCO Science Report: Towards 2030. Online access ISBN 978-92-3-100129-1. Wagner, C. S. (2009). The New Invisible College: Science for Development. Brookings Institution Press. Whetsell, T. A. (2022). Democratic Governance and International Research Collaboration: A Longitudinal Analysis of the Global Science Network. arXiv preprint arXiv:2203.01827.

ABSTRACT. The democracy-science relationship has traditionally been examined through philosophical conjecture and single country case studies. There remains limited global scale empirical research on the topic. This study explores country level factors related to the dynamics of the global scientific research collaboration network, focusing on structural associations between democratic governance and the formation, persistence, and strength of international research collaboration ties. This study combines longitudinal data between 2008 and 2017 from the Varieties of Democracy Institute, World Bank Indicators, Scopus, and Web of Science bibliometric data. Methods of analysis include temporal and weighted exponential random graph models (ERGM). The results suggest positive significant effects of both democratic governance on international research collaboration and homophily between countries with similar levels of democratic governance. The results also show the importance of exogenous economic, population, and geo-political factors, as well as endogenous network factors including preferential attachment and transitivity.

ABSTRACT. As stewards of public money, government funding agencies have the obligation and responsibility to uphold the integrity of funded research. Despite an increasing number of empirical research deals with misconducts, a majority of them focus on retracted publications. How agencies spot funding-relevant wrongdoing and what happens next to those responsible remains unexplored, particularly for emerging science powers. Following a chronology of China’s anti-misconduct policies, we retrieved and analyzed all publicized investigation results from China’s largest basic research funding agency over the period from 2005–2021. Our findings reveal that both the “police patrol” and “fire alarm” approaches are adopted in identifying misconduct and deterring fraud. The principal reasons for investigations are journal article retractions, whistleblowing, and plagiarism detection software. Among six funding-related misconduct types punished and publicized, the top three are fraudulent paper, information fabrication and falsification in the research proposal, and proposal plagiarism. The most frequent administrative sanctions are debarment and recouping of grants. This article argues that more systematic research and cooperation among stakeholders is needed to cultivate research integrity. Specific training and education should be provided for young scientists and researchers in less-developed regions, both of whom make up a large share of funding-relevant research offences.

ABSTRACT. Science is essential to innovation and economic prosperity. Although studies have shown that national scientific development is affected by geographic, historic, and economic factors, it remains unclear whether there are universal structures and trajectories of national scientific development that can inform forecasting and policymaking. Here, by examining countries’ scientific ‘exports’—publications that are indexed in international databases—we reveal a three-cluster structure in the relatedness network of disciplines that underpin national scientific development and the organization of global science. Tracing the evolution of national research portfolios reveals that while nations are proceeding to more diverse research profiles individually, scientific production is increasingly specialized in global science over the past decades. By uncovering the underlying structure of scientific development and connecting it with economic development, our results may offer a new perspective on the evolution of global science.

ABSTRACT. After five years of operating the Transformative Innovation Policy Consortium (TIPC), a global research network involving researchers, policymakers and funders from 12 countries, a second phase of TIPC is about to be launched in 2023. The second phase will draw on the research, experimentation, policy engagements and lessons learned over the past few years and the research agenda developed. A key lesson from the first phase of TIPC is the need to better understand what capabilities are needed to design, implement and evaluate Transformative Innovation Policies (TIP). In response to this lesson, several of the TIP research activities involved the development of new tools, such as transformative theories of change guided by a framemwork of transformative outcomes and set within a formative evaluation framework for TIP projects and programmes. The insights reveal that researchers and policymakers working on TIP need certain sets of capabilities (we refer to these as “TIP capabilities”, for clarity in this context). Although some capabilities may be shared, others may be specific to researchers (who are active facilitators of TIP policy experiments), or to policymakers or funders (who connect experiments to other political initiatives). Therefore, it is not entirely clear, yet, what the full range of these capabilities entails, hence the importance of this panel session to help unpack the issue with inputs from a wider audience. To this end, the session will feature researchers who have been involved in TIP experiments that cover Africa, Europe and Latin America, to reflect on their experience of working with policymakers and other researchers in TIP projects. Reviews of the relevant literature reveal a gap in the conceptualization and empirical research on the issue of the types of innovation capabilities required for transformative change and the specific nature of what could be understood as transformative innovation capabilities. Each of the speakers has taken up this issue specifically in their work, and through dialogue around the following questions, we will bring these ideas together in a synergistic way.

ABSTRACT. Background and research questions Against the background of a steadily rising rate of rapid urbanization and the mounting pressures imposed by climate change, pollution, austerity, aging populations, limited resources and inequality, cities are facing unprecedented challenges along with potential disruptive events and external shocks (Fastenrath et al., 2019). Cities are not only particularly vulnerable to these challenges, but also well-positioned to address and navigate these changes through the tools of urban innovation policies in conjunction with other local policies (Bulkeley et al., 2019). In fact, with the shift of political agendas to matters of sustainable developments, the center of gravitation in innovation policy is shifting from national to urban and international levels. Cities are knowledge centers in their national economies; multiple actors with resources and capabilities can be mobilized in local milieus to experiment, co-shape and co-design solutions in order to build resilient and sustainable systems (Bulkeley et al., 2016; Fuenfschilling et al., 2019; Geels, 2005;).

Our research thus starts from the following assumptions: - Pro-active innovation policies are emerging at the local level, accompanied by the novel tools and activities available at this level too. Formulating, experimenting with and implementing novel policies and solutions in cities differs significantly from other levels of government, but is not yet well understood. - Cities are democratic entities where futures are imagined and alternative directions of change are contested through much more direct citizen feedback than at higher policy levels. Cities formulate overarching visions and policies, yet initiatives tend to be placed within sectoral departments, usually characterised by strong fragmentation, which complicates the horizontal cooperation needed for experiments as well as collaboration with internal and external actors and stakeholders. - The most daunting step in transition policies is to move from individual experiments to broader and generalised transformations. In order to enable sustainable transitions, local contexts will need to address all stages of the policy cycle from envisioning and design to implementation and learning, in order to generalize innovative solutions and thereby contribute to a wider transformation process. Moreover, local policies are also required to practically address emerging constraints and mobilise the opportunities offered by national and transnational/European policies in order to accelerate transitions. Generalisation of innovations is driven by yet another important dimension in the context of urban innovation policies, namely the mutual learning processes across cities. Current European policy initiatives try to foster cross-city replication and adaptation processes.

The objectives of this paper are i) to propose a conceptual framework for Transformative Innovation Policy in Cities that takes into account the aforementioned changing features of urban policies for sustainable transitions, ii) identify common patterns, differences and their determinants across a range of cities in Europe, iii) point to some lessons learned from these experiences for urban innovation policy in general.

Approach and methods Underpinned by literature research, a series of internal workshops and a preliminary study on the case of Amsterdam (Bundgaard et al. 2022), we have developed as part of an abductive research process, a five-layer conceptual framework. The Framework seeks to better understand the processes of shaping and implementing both incremental and transformative innovation policies in the urban context. This framework covers the guiding visions behind transformations, the mental models underpinning innovation policy, the organisational and institutional structures and processes for – mainly incremental – innovation, the spaces and conditions for experimenting with radical novel solutions, and the processes of transformative generalisation to turn these experiments into widespread practices. Based on this framework, six case studies have been conducted in three European countries(Sweden: Lund and Malmö, in France: Pau and La Rochelle, in Austria: Linz and Klagenfurt). By assessing and comparing the preliminary results of these cases we aim to deepen our understanding of advanced practice in their pursuit. The empirical investigation is based on secondary sources on strategies and policies of the six selected cities and a programme of interviews with policymakers, partner organizations, CSO’s and other stakeholders, to complement and refine our theoretically founded framework and allow us to further explore the instruments and activities relied upon for the purpose of transformative change further. The framework is planned to be operationalized and made accessible to policymakers through a series of workshops over the course of 2023.

Expected results Through this work, we expect to identify what policies, instruments, activities and actors are central to the emergence of sustainability-oriented innovation and transformation policies in cities. Moreover, we want to develop more differentiated hypotheses on the influence of the respective national contexts on the scope and practices of urban innovation policy, by taking into account both the differences between the institutional conditions in the three countries, and how they determine what kinds of policies are actually promising and feasible. Building on this knowledge we expect to provide insights into “good practice” in terms of what can proactively be done by municipalities and other stakeholders to facilitate and steer cities towards sustainable futures. Through the advancement of our conceptual framework, we hope to devise novel inroads for how Transformative Innovation Policies in Cities could be better understood, designed, assessed and ultimately improved.

Significance of research approach and findings There is a growing demand for novel sources of inspiration and knowledge to help city authorities develop and implement truly transformation-oriented policies and thus to better cope with the challenges of sustainability in an increasingly turbulent global and societal environment. Solutions for complex challenges call for inter- and transdisciplinary approaches. Current policy and governance challenges are characterized by high uncertainties in transformation processes, multi-actor settings and often face a shortage of organizational capacities and domain/system knowledge to design, implement and manage complex change processes. Consequently, the framework developed is fed by contributions of scientists from different backgrounds and disciplines (such as sociology, urban planning, economics, geography, political science, innovation studies etc.), who apply the respective context-specific lenses on the case studies in the selected cities. Moreover, it is inspired by dialogues with practitioners from local authorities to European policy and various stakeholder organisations. With these ingredients, we hope to mobilise our insights for a more transformative governance of cities.

References Bulkeley, H., Coenen, L., Frantzeskaki, N., Hartmann, C., Kronsell, A., Mai, L., ... & Palgan, Y. V. (2016):Urban living labs: governing urban sustainability transitions, Current Opinion in Environmental Sustainability, 22, 13-17 Bulkeley, H., Marvin, S., Palgan, Y. V., McCormick, K., Breitfuss-Loidl, M., Mai, L., … & Frantzeskaki, N. (2019): Urban living laboratories: Conducting the experimental city?, European urban and regional studies, 26(4), 317-335 Bundgaard, L., et al. 2022): Transformative Innovation Policy in Cities: Illustrating a Five-layer Framework using the case of Amsterdam, Paper for the EU-SPRI 2022 Conference, Utrecht, 1-3 June 2022 (submitted for publication in Technological Forecasting and Social Change) Fastenrath, S., Coenen, L., & Davidson, K. (2019): Urban resilience in action: The Resilient Melbourne Strategy as transformative urban innovation policy?, Sustainability, 11(3), 693 Fuenfschilling, L., Frantzeskaki, N., & Coenen, L. (2019): Urban experimentation & sustainability transitions, European Planning Studies, 27(2), 219-228 Geels, F. W. (2005): Technological transitions and system innovations: a co-evolutionary and socio-technical analysis. Edward Elgar Publishing.

ABSTRACT. Based on the experience from the Community of Practice in Transformative Innovation Policy in South Africa (TIP-SA), this presentation highlights the importance of networking capabilities by key actors in TIP-SA. Networking capabilities have been essential in promoting knowledge exchange, collaboration, learning, influence and contextualisation of TIP ideas in South Africa. It is argued that dedicated efforts to building networking capabilities can help span the boundaries between academia and policymakers, as a way of supporting an ecosystem that encourages transformative innovation and accelerates positive policy impact on society.

ABSTRACT. The emergence of the transformative innovation policy (TIP) approach is disrupting the systems of innovation for development landscape of academic discourse and engagement with policy practice. It draws on multi-level systems theory to advance a theoretical and methodological framework, a ‘transformative’ theory of change (TToC) that hinges on the achievement of ‘transformative outcomes’. We address a gap in understanding relate to the capabilities – that is the ‘transformative innovation capabilities’ – required to design, implement, and evaluate TToC through experimental interactive practice. Although, there is no clear conceptualization of “transformative innovation capabilities” as such, in the TIP related literature (personal communication with Johan Schot, 2022, Carolina et al, 2022), we build off the work of Penna, Schot, Velasco and Molas-Gallart (2022) on institutional capacities and capabilities for ‘transformative mission-oriented’ policies. The object of our analysis is the emergence and evolution of the “transformative innovation capabilities” of a coalition of actors in South Africa, aimed at co-creating and strengthening transformative innovation policy initiatives. We aim to understand what transformative innovation capabilities are and how these are being strengthened. What do the coalitions of actors need to know and have the power to do to achieve transformative innovation outcomes? What are the principal structural, systemic, landscape and regime level forces that are directly enabling or conditioning the expression of their innovation capabilities in practice?

ABSTRACT. Background and rationale As policy issues have increasingly become scientific and technological in nature, research information pervades the work of legislatures. Yet, since the U.S. Congress has no requirement to cite its sources, it remains unclear where the evidence comes from, who is using it, how, and to what end. These represent core questions about how science impacts society on which new techniques in data analytics and machine learning may be able to shed light. We investigate the extent to which digital records from the U.S. Congress can reveal patterns in evidence citation and quotation through a case study of an organization mandated to provide government advice, the National Academies of Sciences, Engineering, and Medicine (NASEM). As a private, non-profit organization congressionally chartered during the Civil War, the National Academies is directed to “investigate, examine, experiment, and report on any subject of science or art” for any department of government. This presentation will review methodological challenges and findings from three types of analyses of congressional and NASEM data: 1) which entities in Congress cite the National Academies most frequently and in what contexts; 2) co-occurence of National Academies references with named entities; and 3) the development of techniques for identifying National Academies quotations in Congressional documents.

Data All available text from U.S. Congressional data—hearings, bill text, committee prints, and the Congressional Record and reports—were downloaded between June-October 2022 from govinfo (API), the Sunlight Foundation and GovTrack.us-maintained Congress github site (bulk) and Propublica (bulk) sources. National Academies consensus and workshop reports were downloaded from the NASEM site in HTML form in May 2022.

Approach overview Meta-data and processed information was deposited in a mongodb document store. The document data was analyzed using a python-based data mining approach. Our ETL (Extract-Transform-Load) process used minimal steps during the transformation phase. Data was loaded locally as it was available. Minimal metadata was associated with files to enable matching across various sources, i.e., subsequent versions of the same bill have different abbreviations in the file names. Only the latest version of a bill/document has been used. The content itself of the document was converted to text from original html or pdf format. For data analysis we used Python and its libraries sbert, sklearn, and faiss; mongodb for meta data storage and full text index; and Pdf2jpg and easyocr for image-based PDF-to-text conversion.

Analyses For the first two sets of analyses, we identified named entities and their co-ocurrences within all sentences in the congressional document texts, limiting the results to those related to the National Academies. In order to detect Congressional quotation of NASEM documents, we conducted a three-step analysis. First, all sentences from the NASEM reports were indexed using faiss with their vector embeddings using the sbert sentence transformers library and its pre-trained paraphrase-multilingual-MiniLM-L12-v2 model. All sentences from congressional documents were also embedded with the same sbert model. For each sentence within the congressional documents, the top 10 most similar sentences from the NASEM reports were selected for more detailed analysis in steps two and three. In the second step, common legal verbiage, such as the titles of acts and statutes, were removed using a SVC (support vector classifier) trained on sbert embeddings of selected human coded (labeled) sentences. Finally, unlike in the first step, which measured semantic similarity based on vectors, exact phrase-based quotations were identified, based on matching of k consecutive tokens. We experimented with k and (no)stemming.

Results Initial results illustrate the frequency with which the National Academies (sciences, engineering, and medicine, and the National Research Council) have been cited within congressional hearing and bill text between the 103rd and 117th sessions of Congress. Within all available bills and ~95% of downloaded hearings at the time of the analysis, 332 bills and 188 hearings referenced the Academies, which represents ~0.2% of total texts. Most of the citations are for the National Academies (216), National Academy of Sciences (127), or National Research Council (94), with fewer references to the National Academy of Medicine (26) and Engineering (6). Bill meta-data indicated that the following topics were areas in which the Academies were most likely to be cited: health, armed forces and national security, and science, technology, and communications. The committee hearings most likely to reference the Academies in bill texts include: House Committee on Science, Space, and Technology, House Committee on Appropriations, and House Committee on Veterans' Affairs. The committees associated with the bills that cite NASEM most often include House Energy and Commerce, House Transportation and Infrastructure, House Science, Space, and Technology, and House Armed Services.

Anticipated results A more comprehensive named entity recognition (NER) and co-occurrence analysis is still being processed and will be available with faceted analysis in relation to various government document properties, like the few mentioned above. In the third analysis we distinguish between potential directionality of the quotation based on the sequence of publication dates, i.e., whether the text was published in congressional or NASEM documentation first. We note that many ‘quotations’ actually refer to previous congressional documents, further, many of the semantically similar sentences refer to different concepts, documents, or events. To address the noise in the findings, the sentences have been stemmed and the analysis conducted on a longer sequence of k consecutive tokens, starting with 8 tokens. We will show examples of successful quotation detection as well as false positives.

ABSTRACT. Introduction

During the COVID-19 pandemic, policymakers in education have been tasked with creating policies to mitigate the impact on students' learning outcomes. The availability of new research related to COVID-19 has presented an opportunity for education policymakers to use the most up-to-date information when making decisions. However, it has been found that despite the large number of scholarly papers published during the pandemic, education policies tend to cite older pre-pandemic research. This case study aimed to investigate if education policymakers' preference for citing older research is due to a lack of relevant newer research or if older research is of higher quality.

Methodology

We began by identifying 2,452 COVID-19 education policies from 176 governmental sources worldwide on Overton. Next, we extracted the DOI of each cited scholarly paper in these policies, matching them with 8,818 records using Elsevier's International Centre for the Study of Research (ICSR) Lab database. An initial analysis found that over 75% of scholarly papers cited in COVID-19 education policies were published before 2020, despite the availability of over 450,000 papers on various aspects of the COVID-19 pandemic indexed on Scopus.

To test our hypothesis that policy makers cited older studies because new research published during the pandemic was not relevant or of poorer quality, we used a combination of citation analysis, natural language processing, and regressions. Specifically, we identified the Scival Topic ID for each cited paper published before 2020, and then filtered each topic for uncited papers published after 2020. This resulted in grouping 6,600 cited papers into 2,589 topics, and matching these to 5.2 million uncited publications published after 2020.

However, processing over 5.2 million publications is a resource-intensive task. To minimize computational demands, we reduced our database to topics where the ratio of cited to uncited papers was at least 1 in 500. This reduced the corpus to 695 topics covering 7,131 cited papers and 555,259 uncited papers. Applying this filter captured over 80% of the cited papers while reducing the computational requirement by at least 90%.

For the reduced corpus, we vectorized the abstracts of both the cited and uncited papers using term frequency-inverse document frequency (TF-IDF) and calculated the cosine similarity between the vectorized abstracts of cited papers and the uncited papers within each Topic ID. After a stepwise manual validation, we considered two abstracts similar if they had a cosine similarity score of at least 0.25.

To test the influence of research quality on policy impact, we ran two regressions, a logistic and Support Vector Regression, on the papers matched in the previous step. For both regressions, our dependent variable was whether a paper was cited in policy or not, while our independent variables comprised a mixture of quantitative proxies of research quality, including a paper’s field weighted citation index (FWCI), the publishing journal’s CiteScore, and the average h-index of a paper’s co-authors. We also added open access level as an independent variable to control for possible issues with access to research.

Results

Our results reveal a significant disparity between fields in terms of policymakers' ability to cite new research. Specifically, education policymakers were better at citing new medical research over social science research, including education research. Our initial analysis showed that most cited papers that were published after 2020 were medical, while the majority of cited social science papers, including education research, were published before the pandemic.

Furthermore, our analysis showed that policymakers' preference for citing older papers is not due to a lack of relevant modern research. In fact, we found that 51.2% (3,397 of 6,627) of the cited papers published before 2020 were matched with at least one similar paper published during the pandemic. This rate varied significantly between fields, with sociology research having the lowest matching rate of 33%, while cited paediatric papers had the highest matching rate of 77.8%. On average, there were over 70 similar papers published during the pandemic for each of the matched pre-pandemic papers.

We also found no observable relationship between research quality and policy impact. Both regression models failed at predicting policy impact, and a machine learning prediction model had a very low recall rate (<10%) for identifying papers that would be cited in policy based purely on the research excellence parameters used in the model. Only FWCI and average author h-index were statistically significant variables. However, the high coefficient for FWCI (0.23) can be explained by the fact that newer research papers have had less time to accrue citations. These results indicate that research quality does not necessarily translate into policy impact in the context of COVID-19 education policy-making, and that other, non-academic factors could be influencing which research was cited in education policy.

Discussion

The case study was able to empirically show that policy makers often rely on a narrow range of scholarly evidence, rather than drawing upon the full range of available research. Education policymakers have not taken full advantage of the vast volume of COVID-19 research published during the pandemic, instead relying primarily on older research. One interesting finding is that education policymakers were better at citing new medical research over social science research, pointing to structural issues in the diffusion of knowledge from academia to policy. This finding highlights the importance of not only conducting research but also disseminating and communicating it effectively to policymakers in different fields.

The matching algorithm also found that policymakers' preference for citing older papers is not due to a lack of modern research. Instead, the algorithm was able to identify similar papers published during the pandemic for over 50% of the cited papers published before 2020. Our analysis of the relationship between research quality and policy impact yielded surprising results. We found no observable relationship between research quality, as measured by various quantitative indicators, and policy impact. This implies that the impact of research on education policy is not solely determined by its quality, but other factors may also come into play, such as the perceived relevance of research to specific policy goals or the interests and values of policymakers.

Our study has important implications for education policymaking during crises. It highlights the importance of effective dissemination and communication of research to policymakers, particularly in social sciences. In the future, policymakers may need to adjust their approach to incorporate a wider range of scholarly evidence to ensure that policy decisions are based on the most up-to-date and relevant research.

ABSTRACT. Research on the conditions under which evidence use is most likely to occur in legislatures has historically been hindered by limitations in available data and relevant metrics. Without formal systems for the citation of evidence within legislatures, tracing the policy impact of engagement between the scientific and legislative communities can be challenging. However, new methodologies may offer the opportunity to address these deficits and, in turn, build scholarship on evidence-based policymaking processes and reveal ways to bolster institutional capacity for decision-making on complex scientific and technological issues. Four interdisciplinary studies from the United States, United Kingdom, Germany, and an international dataset illustrate the development of potentially transformative quantitative methodologies for the study of evidence use: machine learning, natural language processing (NLP), and behavioral modeling. K. L. Akerlof and colleagues employ data analytics and machine learning to explore institutional patterns in citation and quotation from the U.S. Congress and National Academies of Sciences, Engineering, and Medicine (NASEM). Similarly, Afagh Mulazadeh’s study identifies and analyses the explicit sources of scientific knowledge on AMR used in UK parliamentary scrutiny by applying NLP across two corpuses she is building using open UK parliamentary data and Elsevier’s International Center for the Study of Research (ICSR) lab database. Combining Elsevier’s ICSR Lab and Overton databases, Basil Mahfouz analysed over 15,000 education policies published by governments throughout the COVID-19 pandemic to evaluate if decision makers used the ‘best’ available scholarly knowledge. In contrast, Henriette Ruhrmann from the Technical University Berlin applied quantitative behavioural modelling based on psychometrically validated survey data from 1,115 researchers and 162 legislators in Germany to predict science-policy engagement. These techniques from the computational and quantitative social sciences offer the potential to shed new light on how legislatures use evidence, what types of evidence, and what drives researcher engagement and its use by legislators. Focusing on legislatures is a highly relevant extension of the existing body of research that predominantly focuses on the executive branch of governments. Further, the methods enable researchers to evaluate the effectiveness of legislatures at using scholarly research, at scale and with the potential to replicate the studies in different geographic contexts. The panelists will also describe current challenges in using bulk data compiled by legislatures, and the creation of new datasets, such as the Elsevier ICSR Lab database, Overton, and a large-scale psychometrically validated survey of German legislators and researchers.

ABSTRACT. Background and Relevance Techno-scientific innovation needs democratic guidance. For democracies to revisit societal values in light of technological transformation, science must rethink how to engage with legislators. Legislatures represent diverse constituent interests, act as a forum of debate on controversial issues, and scrutinize government action. Therefore, legislators are responsible for aggregating, articulating, and representing societal interests and values in light of scientific progress. Researchers, conversely, should contribute to the co-development of a shared knowledge base to inform parliamentary action and the direction of future research.

Knowledge Gap Previous behavioral science-based research addresses the behavioral determinants of researcher engagement with industry and the public, but evidence on science-policy interaction remains scarce. On the policy side, quantitative modeling remains focused on the executive. Therefore, evidence on the behavioral determinants of legislator research use is urgently needed to promote engagement effectively and efficiently.

Research Question Focusing on the individual engagement behavior of researchers and legislators, which personal and institutional factors drive engagement activities and research use? How can quantitative modeling inform change in the science system? The core hypothesis motivating the study is that the institutional support research organizations offer (e.g., incentives, resources, training, or recognition) promotes policy engagement.

Theoretical Framework The study adopts an interdisciplinary approach linking science-for-policy studies and behavioral science. The study leverages behavioral science theory to explain and model researcher-policymaker/legislator engagement behavior. The COM-B model, developed by Michie et al. (2011), structures the inquiry. The COM-B model posits that (C)apability, (O)pportunity, and (M)otivation determine (B)ehavior. Researchers can apply the COM-B model by operationalizing the domains for any target behavior. The unique advantage of the COM-B model among behavioral science models is that it systematically links the framework domains (C, O, M) to specific types of behavior change interventions. This linkage allows me to translate the research findings into actionable practitioner guidance for research organizations seeking to support engagement.

Data and Methods Situated in Germany, my empirical research design combines two methodological approaches: First, I conducted 22 qualitative interviews with highly-engaged researchers and legislators in Germany around a data-rich technology policy case study in early 2022 (A). Based on these semi-structured 30-60-minute interviews, I conducted a two-step qualitative content analysis. In the first step, I assigned primary deductive codes based on the COM-B model, and in the second step, I derived inductive sub-codes following Mayring (2000) and Hsieh and Shannon (2005).

Second, I developed and conducted two quantitative, psychometrically validated survey studies based on the COM-B model for knowledge ex-change behavior with 1,115 researchers and 162 legislators from federal and state parliaments in Germany in mid-2022 (B). The goal of the survey studies was to quantitatively test the external validity of hypotheses developed in the interview study (A) for larger samples of researchers and legislators using multiple regression analysis. The survey design operationalizes the COM-B model based on insights from the interview study and literature on science policy engagement. I leveraged or adapted validated item batteries wherever possible. To operationalize researcher policy engagement, the dependent variable, I adapted Tartari et al.'s (2014) Academic Engagement Index for policy engagement activities. I validated the selection of engagement activities in focus groups with 16 experts in the field of knowledge exchange.

Results Based on the interview study (A), I aggregated the core motivations of researchers for policy engagement, their institutional support, and engagement activities in a Sankey chart. The qualitative data supports the hypothesis that institutional support leads to more in-depth policy engagement. For legislators, the interview study findings demonstrate that they adapt their engagement strategies to their limited time, often prohibiting engagement with researchers whose communication does not meet their needs. Moreover, legislators tend to favor the field of their academic training. In general, they demand more researcher engagement and, in particular, more consideration for their needs as knowledge users.

The survey studies (B) compare the relative effects of variables in the domains of capability, opportunity, and motivation in driving science-policy engagement. I identified that for researchers, mission-driven motivation to solve societal problems and apply research findings is the strongest predictor of researcher policy engagement, followed by communication skills and institutional support. For the researcher survey, the multiple regression model explains 32.7% of the variation in policy engagement behavior (adjusted R2). For legislators, preliminary results suggest that motivation and capability (specifically, research literacy) are the strongest drivers of research use. In the area of opportunity, social influences (specifically, injunctive norms) promote evidence use. Different regression model specifications explained 40.9% to 44.4% of the variation in legislator research use.

Understanding the interplay of these three determinants (capability, opportunity, and motivation) can inform the design of evidence-based behavioral change interventions; for example, institutional support mechanisms for researchers engaging with policymakers/legislators. Ideally, such support structures should target the most critical determinants of engagement behavior to optimize their effectiveness.

Limitations Modeling human behavior is inherently challenging. Even methodologically rigorous survey research remains limited. This study has three limitations: First, survey respondents (and interviewees) self-selected to participate in the study creating the risk of non-response bias. Second, respondent errors or necessary parsimony may negatively affect their precision with which survey item scales capture the underlying constructs. Lastly, though the data may carry implications for science policy systems globally, I encourage caution in extending conclusions outside the study’s German context.

Conclusions and Policy Issues As research organizations acknowledge their new responsibility as interlocutors in societal transformation processes, they are obligated to build institutional capacity. Specifically, they should create new organizational structures to support their researchers in integrating increasing demands for policy engagement with expectations for excellence in research and teaching.

Based on the behavioral model, I propose an evidence-based approach for research organizations to support policy engagement in practice- novel to Germany. My interactive practitioner resource catalog presents support mechanisms currently in use and highlights how they compare in terms of efficiency. By translating the study findings into actionable guidance for research organizations, I hope to contribute to more strategic, evidence-based capacity-building of organizational structures to promote engagement between research and policymaking.

ABSTRACT. Background and rationale

Since the 1980s, competition has become a widespread mechanism for the allocation of public research funding in most advanced countries (Geuna 2001; Musselin 2018). Competition has been implemented at the organizational level through the introduction of performance-based funding (Hicks 2012; Krücken 2019; Teixeira, Biscaia and Rocha 2021), as well as at the individual level through various types of competitive grants managed by research funding organizations (Lepori and Reale 2019). This implementation has occurred in public sector contexts within which new policy rationales have supported the adoption of private sector practices, such as quasi-markets and efficiency-focused management models (Teixeira et al. 2004; Lepori 2011).

Whereas we know much about how competition is implemented in research systems, we know little about how it subsequently unfolds. The sociological literature suggests that competition should be understood as a socially constructed phenomenon (Arora-Jonsson, Brunsson and Hasse 2020), which is enacted in (strategic action) fields (Fligstein and McAdam 2011) and is largely dependent on social hierarchies (Krücken 2019) and networks (White 2001) in these fields. This means competition in research systems will unfold differently depending on how it is organized, as well as on how it relates to specific historical contingencies (Arora-Jonsson, Brunsson and Edlund 2023). The ways in which these processes unfold are likely to affect core dimensions in competition for research funding: self-selection among applicants (Viner, Powell and Green 2004), narratives to request resources (Velarde 2018), and strategies deployed by scientists to acquire funding (Laudel 2006). Such dimensions are, ultimately, likely to have an important impact on the outcomes of competition, including the novelty of funded applications (Boudreau, et al 2016), and, thus, the ability to support research with the potential to engender scientific breakthroughs (Laudel and Gläser 2014).

Set against this backdrop, the aim of our paper is to generate new knowledge about two elements that are central to constructing competition for funding: the extent to which grants are perceived as scarce and desirable (Arora-Jonsson, Brunsson and Hasse 2020). From previous research funding literature, we know that perceptions of scarcity and desirability vary depending on the type of grants and the position of scientists (Laudel 2006). There has, however, not been any systematic investigation of how grants are perceived by the actors involved in the competition, and, specifically, on the ways in which this may be affected by rules and discourses disseminated by funding agencies.

Methods

To address this aim, we build on theory from the sociology of quality markets literature (White 2001; Krücken 2019), as well as from the commodity and consumer behavior literature (Lynn 1991; Wu, et al 2012), which provides us with extensive insights into how scarcity perceptions are generated, how they affect perceptions of quality and desirability, and how they, in turn, impact the behavior of actors (Aggarwal, Jun and Huh 2011; Nichols 2012).

Empirically, we provide a case study consisting of two schemes housed under the European Union’s Framework Programs (EU-FP) for research and innovation: the European Research Council’s (ERC; Laudel and Gläser 2014) bottom-up individual grants and the European Commission’s (EC) top-down collaborative actions (Ulnicane 2015). These two schemes represent contrastive cases in terms of their goals, narratives, and implementation approaches.

Using a mixed methods approach involving official documents, a survey, and a series of interviews, we thus contrast perceptions of scarcity and desirability among applicants to ERC grants and collaborative projects, seeking to grasp how these perceptions are affected by the goals, narratives, and implementation approaches from public authorities and funding agencies. To avoid disciplinary bias, we focus on potential applicants in the area of immunology.

Scarcity and desirability of ERC and collaborative grants

The ERC’s budgets are, as with any other economic resource, limited, and this, by default, creates certain scarcity. Scarcity is, however, intertwined with desirability in constructing competition for the ERC’s project grants, because resources that are scarce and difficult to access typically become desirable and attractive to access as well (Zuckerman, 1977). The desirability of ERC grants has been reinforced through bottom-up applications – implying that scientists do not require any nominations from universities or research councils – and beneficial funding conditions – including large monetary amounts that scientists have at their autonomous disposal for lengthy duration periods at European host organizations (Schreck, 2007; Wolinsky, 2010). The desirability of these grants has, finally, been further constructed through Europe-level allocations, which are adjudicated by prominent panellists that are seen as elites and experts in their respective disciplines. This serves to envelop the ERC’s grants in a meritocratic aura that is reminiscent of Merton’s (1942) scientist norms.

Scarcity in collaborative calls varies according to the budget allocated and the broadness of the calls. For example, we can expect lower success rates in broad calls supporting the application of digital technologies in the health area than in calls targeting the development of a specific treatment for a particular disease. We can denote overall increasing levels of desirability and thus competition throughout the years, notably due to the shift from an EU-FP relying on the juste retour principle to today’s predominant excellence narrative. The juste retour principle implied that EU Member States could expect to receive EU-FP funding in proportion to their financial contributions to the EU budget. From the 2000s and onwards, notably with the Lisbon Strategy targets and the inception of the European Research Area, the policy narrative underlying EU-FPs saw increasing importance of New Public Management elements, with past results as a key criterion for the allocation of funding, and of the concept of excellence that encompasses all EU-FP funding instruments (Young, 2015; Hoenig, 2018). Further, collaborative schemes in EU-FPs provide resources to strengthen and expand collaboration and thus stimulate knowledge exchange. They provide networking opportunities and generate new collaborations that can result in co-publications, patents and new projects.

The foreseen survey and interviews can help us explore more in-depth how researchers perceive both grant schemes, notably in terms of career opportunities, but also identify mechanisms of self-selection or the potential influence of institutional settings on the desirability of such grants. At the moment of writing, we drafted a questionnaire that will be sent to hundreds of potential applicants to EU-FP grants in November/December 2022. We expect to be able to share advanced results by the time of the conference.

Significance

Our paper offers contributions to the research policy literature and the sociology of markets literature, by advancing new theoretical and empirical understandings of scarcity and desire as core elements in the construction of competition in public research funding.

From a policy perspective, and as suggested by the sociology of law literature (Edelman, Uggen and Erlanger 1999; Lascoumes and Le Galès 2007), we highlight how, beyond setting rules for competition, the design of grant schemes also sets norms and values that might deeply affect the perceptions of applicants concerning the desirability and scarcity of grants. And the perceptions of applicants can, in turn, impact their behavior because, as we have known for some time, “if (wo)men define situations as real, they are real in their consequences” (Thomas and Thomas 1928: 571-572). Taking this into account may help tailor policy measures to better connect the outcomes of competition to the achievement of policy goals.

ABSTRACT. HIV/AIDS has been a major focus for research funders. The US National Institutes of Health (NIH) alone has spent over $70bn on HIV/AIDS. Such investments ushered in antiviral drugs, helping to reverse a rapidly growing HIV/AIDS pandemic. However, the idea that research can deliver unexpected benefits beyond its targeted field, in fact, predates HIV/AIDS to at least Vannevar Bush’s influential 1945 report. Cross-disease spillovers – research investments that yield benefits beyond the target disease – remains unexplored, even though it could inform both priority-setting and calculations of returns on research investments. To this end, we took a sample of NIH’s HIV grants and examined their publications. We analyzed 118,493 publications and found that 62% of these were spillovers. We used Medical Subject Headings (MeSH) terms assigned to publications to explore the content of these spillovers, as well as to corroborate non-spillovers. We located spillovers on a network of MeSH co-occurrence, drawn from the broader universe of medical publications, for comparison. We found that HIV spillovers were unevenly distributed across disease-space, and often in close proximity to HIV, which, when discounted, reduced our spillovers estimate to 40%. We further reviewed 1,000 grant–publication pairs from a local sample and 1,000 pairs from a remote sample. For local spillovers, a quarter seemed to be unexpected, on the basis of on their grant description; for remote spillovers, that proportion increased to one third. We also found that the NIH funding institutes whose remits were most closely related to HIV/AIDS were less likely to produce spillovers than others. We discuss implications for theory and policy.

ABSTRACT. Background: With the aim of making it easier for researchers to produce policy-relevant research, the UK Government now requires all departments and arms-length bodies to publish annually-updated statements of their evidence needs, called ‘Areas of Research Interest’ (ARIs). We describe how ARIs are produced, and how they are used to support this aim.

Aims and objectives: In this paper we offer a description of ARIs and their development by UK governmental departments, and an assessment of how different stakeholders have responded to or otherwise used the ARIs.In the UK, the Government Office for Science (GOS) supports the Government Chief Scientific Adviser (GCSA), and holds a cross-government remit to support science capability across departments. It relies on 'soft power' networking and influencing rather than mandating. It also holds the policy for Areas of Research Interest. As part of this remit, Giulia Cuccato led a team of civil servants in the Government Office for Science (GOS) to develop guidance for departments in developing their ARIs, and supporting and tracking their ARI-related activities. Alongside this work, Kathryn Oliver and Annette Boaz have been embedded in GOS to explore and support better production of ARIs, and more effective engagement with them by funders, researchers and intermediaries. We have been involved in supporting the development of ARIs across the UK government, in helping departments use ARIs to access relevant evidence and expertise, and in researching how ARIs could be optimised to support the research-policy system.

Methods: We draw on 25 interviews, approx 50 hours of observations of meetings and roundtables, and our 3 years' embedded researcher experience to describe the ARIs, their different functions across settings, institutions and audiences. We describe how ARIs are produced, what ARI-related engagement occurs, and how they function as a systems level intervention.

Findings: ARIs were intended to identify strategic research priorities for departments, but in practice we have found that they have a much broader set of uses. We found that departments use them to improve internal working and relationships, to implement the agenda of the Chief Scientist, to support other governmental processes such as spending reviews, the Integrated Review, and the Science Capability review. For some they are a reflection of their policy priorities; for others an articulation of the activities and structures of their internal science system; a statement of likely research commissioning priorities; and/or a statement of research areas around which they would welcome collaboration or input. However, the end products appeared to be mostly appropriate for the departments in question. By and large, they were seen as useful internal tools to negotiate and communicate with policy colleagues around budgets and priorities, and useful external tools to solicit help.

Universities and academics find them useful to plan engagement activities such as workshops and fellowships, but often tend to view them as poorly-written research questions. ARIs can help the research community to understand what government departments want from them. This happens most effectively when there are opportunities for dialogue or a clear narrative about the policy history behind each ARI.

Identifying relevant expertise and research is a real challenge for government departments, particularly where resources are limited. Framing problems is an important step for departments, because it dictates what research and which experts are considered relevant and appropriate. We found that officials in government departments were committed to addressing the challenge of diversity and inclusion in academic-policy engagement, but we unsure how best to go about improving practice in this area.

Strengths of ARIs: ARIs work well as an external articulation of research and evidence needs for departments. They offer funders, intermediaries and researchers insights into what departmental research agendas. Universities and intermediaries in particular have used ARIs to develop their own strategic engagement plans (see, e.g Heckels, 2020). Most departmental ARI documents now contain contact details as well as ‘asks’ and ‘offers’ for each ARI. This makes it easier for funders, intermediaries and researchers to know how to respond (e.g. by getting in touch for a conversation, arranging a research collaboration or responding to a research tender).

ARIs as a systems intervention: The ARIs were proposed to encourage the production of more policy-relevant research. This has been describe as a ‘deficit model’, suggesting that if decision-makers had better evidence, their decisions would improve. This model has been widely criticised as being based on some fundamentally flawed assumptions about how decision-making works (Jones and Crow, 2017) and on how evidence informs that process (Locke, 2002). The ARIs may have been planned to address this illusory ‘deficit’, but in practice perform a far greater range of functions which help to connect the policy research system in complex ways.

The true value of ARIs may be in illuminating the ways in which the research-policy system is connected, and how we can intervene most effectively to support this system.

Weaknesses of the ARIs: systems challenges

Not everything can be or is articulated as an ARI: ARIs are not able to articulate the totality of departmental research needs. For some departments, this is due to political or operational sensitivity, and for others, they prefer to only publish ARIs on topics where they are currently seeking external input. It would be a mistake, therefore, to think of ARIs as a complete and exhaustive list of the topics on which government is seeking input.

ARIs are not research questions: Academics frequently describe ARIs as poorly written research questions. An alternative, more useful phrase might be “research needs”, which helps to give the impression that there is a process attached to them, that they are valued, and broader than research questions. They are more usefully thought of as topics for conversation.

ARIs are difficult to access and analyse: By 2018, most departments had published at least one version of their ARIs, which then sat on the government website in pdf or html formats. There is as yet no easy way to search for ARIs by topic, department or year, which makes it difficult for researchers to identify relevant topics or potential collaborators This also means that departments are not easily able to identify shared cross departmental areas of interest.

Finding relevant evidence and expertise takes time and work: While some departments had resources dedicated to engagement around the ARIs, others did not. While relevant research often exists (as bodies of primary research, in research and practice communities, or in ongoing funding investments), this knowledge is often inaccessible, being behind paywalls or requiring time and skill to find and absorb.

Key conclusions: The ARIs have great potential to enable funders, government, research organisations, researchers, and intermediaries to work together in a more effective way. We have observed that merely producing ARIs is not a sufficient intervention; instead, it requires skilled mobilisation work by people within all these organisations to be able to optimise their production and use.ARIs are a mechanism for organisations to share their research interests with external audiences in the form of a published document,. They also have a much broader set of uses, including connecting departments with each other and helping intermediaries shape engagement plans. All groups would benefit from more robust evidence to choose effective engagement mechanisms, and more can be done to make the ARIs discoverable and useable. Overall, the ARIs are a useful tool to illuminate, and begin to connect different parts of the research-policy system.

ABSTRACT. The rationale

Citizen science (CS) is a research approach with potential to build policy-society bridges. On the one hand, citizens can draw the attention of scientific research to understudied topics that could then been addressed by policy making. On the other hand, CS opens opportunities to democratise the policy processes (Cohen & Doubleday, 2021) which makes it particularly promising in the case of environmental policy.

The Rio Declaration developed in the context of 1992 United Nations "Conference on Environment and Development" established the basic principles for environmental democracy, which included access to information, public participation and access to justice. Similarly, the Escazú Regional Agreement (2018) on Access to Information, Public Participation and Justice in Environmental Matters in Latin America and the Caribbean, seeks to strengthen citizen rights to participate in environmental decision-making through specific forms of policy governance.

Although the potential exists, CS literature has highlighted several challenges faced by CS initiatives when engaging to the policy-making sphere. They range from issues regarding data quality to power imbalances and institutional resistance (e.g. Nascimento et. al. 2018; Haklay, 2021). The EU has created an inventory of selected practices from CS projects that attempted to link with environmental policy-making sphere (EC 2018). But there is no empirical research analysing the perspectives of policy makers regarding their connection to CS projects. This paper aims to fill this gap in the context of environmental justice in the Matanza-Riachuelo basin.

The context

The basin is a heavily polluted area in Buenos Aires, Argentina. There are almost 14,000 industries, many of them dumping their waste into the river, while 18% of the population is not connected to the drinking water network. The area is ruled by different government jurisdictions (Municipalities, Province, and Nation) normally governed by different political parties, which has made it very difficult to advance with comprehensive solutions.

We are involved since 2020 in a CS project, which co-designed along with 59 community actors from the basin, a digital platform for citizens to share experiences on key areas for environmental justice. These data open several opportunities for sanitation policy making (e.g. cost effective monitoring of water quality, democratic approaches to natural areas conservations, etc.), including the possibility of broadening participation, as it is legally requested. The early stage of the project (2020) through interviews and workshops we created a knowledge coalition of researchers (12), policy actors (13) and community actors (47). As a result of this process, we understood that policy makers believe in the potential of CS related to a better identification of community priorities and in trust-building for policy actions, but there are several challenges and reservations that need to be addressed (e.g. on data quality, the politics of participation, etc.).

Thus, in 2022 during the last stage of the project we aimed systematising the perspectives of policy actors regarding the potential uptake of CS in sanitation policy. These are the entry point to identify policy options that can contribute to strengthen the CS-policy link, because they shed light over different dimensions where CS initiatives could contribute to decision-making processes, from their standing points. This paper describes the result of this exercise. The method

We used Q-method to identify policy makers’ perspectives regarding the uptake of CS knowledge in policy formulation. It is a research technique to identify underlying patterns of meaning and discerning people’s perceptions of their world (Webler et. al., 2009, McKeown & Thomas, 2013). People participate in interviews where they are provide with a set of statements expressing ideas on a given topic to order them in a normally distributed grid. Participants are invited to explain the reasons justifying their way of organising statements in the grid. The statistical analysis then proceeds by correlating participants’ sorting of statements, which take the place of variables in factor analysis (McKeown and Thomas, 2013). Factors are interpreted with inputs from the interviews.

We presented 52 statements to 14 policy actors. Statements were developed through an iterative process by both co-authors using, primarily, inputs from the review of 36 international papers reporting evidence of CS projects that intended to establish some type of relationship with environmental policy spheres. In addition, some few statements were built using transcriptions of the public events in which policy actors in Argentina made affirmations regarding CS potential. We codified statements in three dimensions

• Linking mechanism: three forms of citizen intervention through CS initiatives that may connect to public policy: i) citizen-driven-data, ii) citizen participation in agenda-setting and iii) citizen participation in the governance of common goods or public services. • Policy-making phases: three phases of the policymaking processes to link with CS initiatives: i) problem identification, ii) policy implementation, iii) policy-change (e.g. creation of new programs). • Valorisation of CS-policy link: positive or negative.

Two factors were extracted using principal component analysis and rotated using varimax methods.

Results

We found two non-confrontational perspectives regarding the potential contribution of CS in policy-making processes. We confirmed our findings of the knowledge coalition building process that there is a generally optimistic opinion regarding the potential contribution of CS to socio-environmental issues. Stakeholders believe that CS promotes empowerment, inclusion, and more responsive policies since they benefit from situated knowledge. The highest valued mechanism explaining optimism in the CS-policy link is the participatory nature of CS -rather than the capacity to produce and make available citizen-driven data. This is not surprising in the context under analysis that has participation as a formal mandate for sanitation policy.

Although they have some common background in the way they value participation, the two perspectives differ regarding what specific contribution to the policy making process CS could enhance, or, in other words, in what aspects of policy making there are more prospective links with a CS approach. One of them (Perspective A) finds the potential mostly relate to the initial phase -problem identification- and in creating opportunities to change policy, while the other (Perspective B) believes that the highest contribution is in the implementation phase. Remarkably, policy actors more directly involved on environmental issues load more heavily on perspective B.

By combining this quantitative analysis with qualitative material from interview transcripts, we spell out the perspective as following

Perspective A: "CS produces ideas to create new policies or policy programs. These initiatives are particularly useful in the context where there is a lack of policy actions, to fill a vacant area, thus complementing the available policy knowledge. Citizen participation makes visible concrete socio-environmental problems".

Perspective B: “Synergies can be created between public policy agencies and CS initiatives, contributing to the improvement of existing policymaking processes. Participation increases trust in policy actions, which is necessary to enhance policy effectiveness”.

Conclusions

We contribute to the scarce empirical literature addressing the potential of linking CS with policy making using evidence from a highly complex context of Argentina.

Our approach is different to empirical papers providing information on the CS-policy link because while they share insights obtained from the experience of CS, we explore the nuances of the other side of the CS-policy link analysing the perspective of policy actors.

We confirmed that the potential contribution exists also in the minds of policy actors. However, depending on the specific policy context, the contribution could be more complementary or more synergetic with existing policy actions

ABSTRACT. Science, technology and innovation (STI) are called to play a fundamental role in the quest for overcoming the current global challenges, and deep tensions around the models of economic growth, and the social and planetary unbalances and the aggravated and reinforced dynamics among them (UNDP 2020). We are faced with an increasing inequality in wealth, global unsustainability, child labor, new ways of slavery, political polarization and challenges in human rights (Giuliani 2018). Thus, the call is for urgently rethinking the current development paradigm, to prioritize sustainability and social inclusion, and radical new relationships between humans and nature within the planetary boundaries (Rockström, Steffen et al. 2009).

This very complex scenario leaves open and uncertain questions that require new approaches, methodologies, and frameworks in the search for new practices, institutions, ways of life, guiding principles and ethical standards to define new ways of life within safe planetary boundaries which leave behind a growth paradigm based on the depletion of natural resources and the environment. Science, technology and innovation must be placed at the forefront of this process, as instruments of sustainable and inclusive development. In a different vein, for some decades STI policy have gained an explicit space within the policy concert, particularly in the developing world where this process is lagging behind. In Latin America, countries have defined STI policies and, in some cases, strategic plans, although with diverse emphases, scope, objectives, approaches and guiding principles (Dutrenit, Aguirre-Bastos et al. 2021). In spite of this well established and acknowledged role, its legitimacy is clear at the discursive level, but its intertwinement and embeddedness into crucial development aspects is still missing (Dutrenit and Sutz 2013, Bianchi, Bortagaray et al. 2021, Bortagaray and Aguirre-Bastos 2021). Furthermore, it is not evident what is best governance scheme to better advance in this direction, to foster the necessary changes across the policy spectrum placing STI as a fundamental driver of change. What type of policy space should STI have within the overall policy spectrum? Should it be organized as a discrete policy domain, or should it be designed in a way that cuts across others like agriculture, health, industry, social development? What scope, governance models and systems better serve such transformation, and with what specific policies, institutional arrays, and the extent to which they need to be changed or created? (Sachs, Schmidt-Traub et al. 2019). Current societal challenges are profoundly complex and novel, calling for flexibility, adaptation, experimentation and policy learning. STIP key (new) role is part of a “normative turn” (Daimer, Hufnagl et al. 2012), a paradigm shift in STI policy with growing importance of directionality and normativity in STI, and an instrumental role of STIP for solving societal challenges, and advancing sustainable human development.

This work attempts to analyze such changing scenario, its scope and scale in the context of two very different countries, Uruguay and South Africa, particularly taking into account how STI policy connects with climate change strategies. The work is in its initial stages, and draws on an ongoing collaborative effort in this regard.

Empirically it studies both the STI and the climate change policy spaces on their own, to then analyze (i) the extent to which STI permeates climate change strategy, (ii) how does climate change appears in the STI policy domain, (iii) and their linkages and intersections. It also investigates their connections to other policy domains, such as agriculture, industry, health. The explored hypothesis is that STI policy has evolved in relative isolation of other development arenas and in the particular climate change policy in the case of Uruguay. In South Africa, this trajectory has been different, and STI policy has been more tuned to development problems including climate change. Part of the explanation lays in the different ST policy governance systems these two countries have.

References Bianchi, C., I. Bortagaray, F. Liurner and E. Magallán (2021). Aportes de las políticas de ciencia, tecnología e innovación al desarrollo sustentable en el Uruguay del siglo XXI. Principales desafíos para una agenda de transformación. Serie Ideas para agendas emergentes. UNDP. Montevideo. Bortagaray, I. and C. Aguirre-Bastos (2021). Innovation Policies for Inclusive and Sustainable Development: Insights from the Central American Region. Policy and Governance of Science, Technology, and Innovation. Social Inclusion and Sustainable Development in Latin America. G. Ordóñez-Matamoros, L. A. Orozco, J. H. Sierra-González, I. Bortagaray and J. García-Estévez, Palgrave Macmillan Cham. Daimer, S., M. Hufnagl and P. Warnke (2012). Challege-oriented policy-making and innovation systems theory: reconsidering systemic instruments. . Innovation system revisited - Experiences from 40 years of Fraunhofer ISI research. F. ISI. Stuttgart: Fraunhofer Verlag: 217-234. Dutrenit, G., C. Aguirre-Bastos, M. Puchet and M. Salazar (2021). Latin America. UNESCO World Science Report 2020. Paris, UNESCO. . Dutrenit, G. and J. Sutz (2013). Introducción. Sistemas de innovación para un desarrollo inclusivo. La experiencia Latinoamericana. G. Dutrenit and J. Sutz. México D.F., Foro Consultivo Científico y Tecnológico, AC - LALICS. Rockström, J., W. Steffen, K. Noone, Å. Persson, F. S. l. Chapin, E. Lambin, T. M. Lenton, M. Scheffer, C. Folke, H. J. Schellnhuber, B. Nykvist, C. A. De Wit, T. Hughes, S. van der Leeuw, H. Rodhe, S. Sörlin, P. K. Snyder, R. Costanza, U. Svedin, M. Falkenmark, L. Karlberg, R. W. Corell, V. J. Fabry, J. Hansen, B. Walker, D. Liverman, K. Richardson, P. Crutzen and J. Foley (2009). "Planetary Boundaries: Exploring the Safe Operating Space for Humanity." Ecology and Society 14(2). Sachs, J. D., G. Schmidt-Traub, M. Mazzucato, D. Messner, N. Nakicenovic and J. Rockström (2019). "Six transformations to achieve the Sustainable Development Goals." Nature Sustainability 2: 805-814. UNDP (2020). The next frontier. Human development and the Anthropocene. Nueva York, UNDP.

ABSTRACT. Equitable Energy Transition Problem Framing: A Comprehensive Success Factor Analysis (CSFA) Approach

Background and Rationale

Problem framing is essential to address sociotechnical system challenges because it reduces ambiguity and facilitates a shared understanding of the problem. For large-scale sociotechnical systems challenges like developing an equitable energy transition plan, problem framing becomes more complex, as capturing the subjectivity and nuance of the many involved stakeholders becomes a daunting task. Current approaches to framing energy transitions embrace either a reductionist or holistic lens. Both framing approaches have their limitations in achieving an equitable plan. A reductionist approach fails to capture the complexity within the system, and the subjectivity needed to attain an equitable renewable energy solution. Holistic approaches, on the other hand, do not adequately capture cross-scale, cross-level interactions that occur at different abstraction levels within a system.

To address these limitations, this work employs a holistic approach designed to frame challenges spanning levels of abstraction, plurality, context, and scope, thereby facilitating the development of an equitable energy transition plan in complex sociotechnical systems.

Method

We employ Comprehensive Success Factor Analysis (CSFA), a method for grand challenge problem framing developed at the Institute for Innovation Science at Purdue University. CSFA is a holistic method that helps reduce unknown-unknowns, identify barriers, mitigate risks, and achieve high-impact through the organized exploration of various elements of a system that are necessary to yield desired outcomes. CSFA encourages systematic exploration of broad framing categories including ‘system organization’, ‘operations’, ‘users and actors’, and ‘dynamic properties.  In addition, CSFA incorporates perspectives from 9 overarching disciplinary lenses – psychology, physiology, politics, operations, education, environment, economics, technology, and sociology (P³OE³TS).

To carry out this analysis we pursue framework-informed systematic exploration of web-derived information on equitable energy system transitions. The energy transition system is analyzed at the national level. Renewable energy, in this work, encompasses solar, wind, bioenergy, hydro-electric, hydrogen, and geothermal energy. To perform CSFA and identify success factors needed for a low carbon, renewable energy transition, we reviewed nearly 400 documents and drew out perspectives on 16 categories of success factors that compose the CSFA framework, namely: infrastructure, equipment and supplies, workforce and talent, capital/finance, practices and mechanisms, awareness and acknowledgement of needs, motivation, enabling strategies, adoption and habit conversion, mechanism for evaluation, mechanisms for sustainability, resilience, security and safety, policies, government/ leadership support, and stakeholder interaction.

Result

Several hundred success factors were identified that are vital to achieving an equitable energy transition. These factors provide a conceptual framework that can guide energy transition policy deliberations towards achieving equity by ensuring that: 1.) individual and collective experiences, identities, backgrounds, and environments are taken into consideration during policy discourse, and, 2.) the dynamics of cross scale and cross level interactions within the sociotechnical system are accounted for. Collectively, these factors highlight that transition interventions must differ by level of abstraction, and are influenced by cross-scale (e.g., jurisdictional, management, and network scale) interactions, calling attention to the need for a portfolio of solutions to achieve widespread transition success.

Significance

This research identifies success factors that need to be addressed by policymakers and other key stakeholders to achieve equitable energy transitions. Our findings, which are location agnostic, provide a user-friendly means to support energy transition planning and discourse that helps reduce unknown unknowns and expert bias (identified in discourse theory) during problem framing, helping to foster more equitable outcomes.

ABSTRACT. We examine whether government sponsored R&D affects the development of clean technologies with a higher impact on subsequent technological development. The empirical analysis uses information on USPTO patents granted during the 2005-2015. Starting from patents acknowledged in government funding, we build a control group by matching firm level information on Patstat. We combine linear regression and propensity score methods to control for an eventual sorting of riskier projects into public funded projects by firms and for non-random (public) treatment at technology level. We also assess the distributional impact of government supported R&D in green technologies. Results show that green patents benefiting from public funding have a significantly larger impact that the other patents developed by the firms in the sample, and that this impact becomes larger along the distribution of patent citations. Our results complement the literature on market-pull policies to reduce greenhouse gas emissions, showing that technology-push policies represent a relevant option to determine the speed and direction of technical change in the field of clean technologies.

ABSTRACT. Background and rationale

In this paper, we analyze the objective of inclusion in the science, technology and innovation (STI) policy context. Inclusion is increasingly gaining importance as STI policy objective aiming at providing equal opportunities for different members of society to benefit from and to participate in innovation, and promoting closer interaction between science and society. Hence, it is valuable to scrutinize its use and symbolic role in the broader STI policy language. We begin the analysis by reviewing the extant STI policy literature. Secondly, we carry out a comparative analysis of the US Broader Impact Criterion (BIC) and the European Responsible Research and Innovation (RRI) frameworks.

The objectives of the research are twofold. Our STI literature analysis aims at synthesizing the scattered knowledge and multiple approaches to inclusion as a policy objective. Secondly, we analyze how inclusion as a science and innovation policy objective is both defined and operationalized in the BIC and RRI evaluation frameworks.

The rationale for the comparative approach arises from the different ways in which inclusion is conceptualized, understood, pursued and evaluated in different policy contexts. In other words, we are interested in the different contextual and systemic realities and value-bases in which inclusion is promoted as a science and innovation policy objective. We hypothesize that these realities have an effect not only on the development of the societal impact itself, but also the actual evaluation practices as focusing devices in understanding the impacts.

Methods

Our empirical research is based on comparative analysis that focuses on the contextual differences in policy formulation and implementation. In addition, it focuses on the policy instruments, the broader innovation system and its elements that influence innovation processes and outcomes, and sheds light on the conceptual ambiguity characterizing innovation concept and inclusive policies in general. The co-existence of multiple definitions in academic literature and among practitioners hampers the understanding of these terms and their further uptake in both the private and public sector. 

The comparison analyzes the concept of inclusion in the context of two different frameworks. Our theoretical literature based conceptual framework synthesizes the key dimensions of the broad science and innovation policy literature. This helps to compare the cases in terms of their approaches to inclusion. In practice, we analyze the extant research on BIC and RRI. Additionally, we analyze the key policy documents related to the frameworks and the evaluation criteria operationalizing objectives related to inclusion and evaluation practices. The document analysis is based on the principles of theory-driven content analysis, utilizing the conceptual framework built in the theoretical section of the paper.

Results

The findings present different approaches to conceptualize and operationalize inclusion as a science and innovation policy objective. RRI and BIC are importantly different criteria in terms of their approaches to inclusion, the former taking a more process-oriented and the latter more outcome-oriented approach to promoting inclusiveness. While RRI focuses on collaboration, partnerships and interactive processes as a part of the research, BIC aims primarily at benefiting e.g. science, technology, engineering, and mathematics by means of concrete outcomes of the research. In the RRI framework, inclusion refers primarily to public engagement and “inclusive participatory approaches”. Instead in the BIC, inclusion is approached primarily as the participation of different marginalized groups in research and education activities, disseminating research results to wider audiences, and collaboration and networking activity with external actors.

Significance

The paper contributes to the STI policy literature by presenting a conceptual framework of inclusion as an STI policy objective. Research on the different ways to understand and operationalize inclusion as part of the wider policy attempts to increase the societal impact of science and innovation can contribute to the ongoing academic debates on societal impact as well as its evaluation amidst increasing societal complexity. It sheds light on the embeddedness of policies in different systemic contexts and value-bases that affect not only the creation and use of the policy language but also the concrete evaluation practices attempting to capture the impacts of policymaking.

The analytical distinction between inclusion as means and inclusion as an objective is useful especially in light of STI policies emphasizing the societal impact of research and innovation. In the process-based view on inclusion, inclusive processes appear as a means to increase the societal impact and legitimacy of STI policies, whereas the output related view emphasizes inclusion as a goal in itself. These different meanings given for inclusion are adding to conceptual ambiguity.

Altogether, extant research has identified diverse disconnections not only between the policy and practice of increasingly complex innovation space but also between the agency and structures that direct attention e.g. to the lacking acknowledgement of the role of users and consumers (including minorities and disadvantaged groups) in transformative innovation policy discussions. These disconnections, combined with the growing complexity of policy concepts, are decreasing the legitimacy of STI policies. Inclusion can be a powerful policy concept in promoting societal impact of research, and research on its conceptualization and operationalization can increase its value as an integrative policy concept. However, in addition to the identified need for transformative policies to be more inclusive, future research needs to account for the growing complexity and ambiguity of policy language regarding inclusive policies.

ABSTRACT. Background and rationale

The growing application of artificial intelligence (AI) to manufacturing, particularly when integrated with smart factory robotics, automation, and logistics, promises to transform the sector. AI will generate impacts across all aspects of industrial systems and industrial innovation, including product development and design, manufacturing production, supply-chain management, sustainability, competitiveness, user interaction and workforce training. Impacts will also be generated for employment and job tasks, affecting both shop floor and white-collar occupations. The potential positive impacts of AI in manufacturing have been recognized by industrial and innovation policymakers, with strategies and programs initiated in the US and other leading economies to accelerate the development and diffusion of advanced manufacturing and AI.

Alongside business and policy expectations that AI-driven manufacturing will drive industrial performance, generate innovation, and revitalize national and regional industrial economies, there are also many worries, generating debates about responsible innovation in AI manufacturing. Among these concerns are apprehensions about the displacement of human agency and labor, the rise of new mundane (and less-well paid) tasks that support AI, and the ethical implications associated with bias and privacy, as well as issues related to data security and consequences for small and mid-size enterprises.

While the scale, scope, and directions of impacts associated with AI in manufacturing will vary across sectors, companies, and occupations, strategic choices made early about how AI-driven manufacturing will be designed, implemented, and governed could lead to material differences in avoiding or mitigating adverse effects and risks and ensuring a more equitable balance of benefits and costs. This is a key premise underwriting initiatives to embed attention to ethics and responsible innovation in AI manufacturing. This paper offers a formative assessment of this proposition by drawing on early experiences with a large-scale project to develop and promote AI manufacturing. We reflect on whether and how consideration to ethics and responsible innovation can be framed and operationalized, anticipate future developments, and derive insights for innovation management, university-industry-community partnerships, and innovation policy.

Methods

The case study focus is the Georgia Artificial Intelligence Manufacturing Technology Corridor (GA-AIM) which aims to accelerate research, scale-up, and transfer of AI and related digital technologies in industry in Georgia, USA. Through a university-industry-community partnership, led by the Georgia Institute of Technology, GA-AIM seeks to integrate the deployment of AI technologies in industry with related job training and outreach to underserved communities. The project has received federal funding ($65 million) under the US Economic Development Administration’s Build Back Better Regional Challenge.

GA-AIM has an embedded ethics and responsible innovation team (which includes the authors of this paper) which is focused on two key program elements: an AI-manufacturing test-bed facility and AI-manufacturing community engagement. The team is using action-research methods, including targeted studies, and engagement with designers, users, and other stakeholders to facilitate GA-AIM in addressing ethics and responsibility and to foster learning and adjustment that advances the responsible and ethical use of AI-manufacturing technologies.

Results and significance

GA-AIM started in 2022. For the paper, we anticipate findings from targeted initial studies on the use of augmented reality (AR) systems for training and upskilling manufacturing workers and AI technologies for monitoring the health of manufacturing systems. We will highlight key ethical issues that are implicated in the design, development, and use of these systems and how designers are addressing them. These include ethical issues that are commonly discussed in AI ethics – including bias and privacy – and ones that are less commonly discussed, including AI-human interaction. AI-human interaction is a particularly important issue in manufacturing contexts, although it is an issue that is relatively neglected in AI ethics codes and policies. Additional ethical issues we will consider are trust and trustworthiness. Many AI researchers place a lot of emphasis on trust – on the importance of developing AI systems that users trust. Less attention has been given to the question of the criteria that should be met for a system to be worthy of the trust of users, including users of different backgrounds and levels, as is likely in manufacturing environments. We will explore what is changed through attention to ethics and responsible innovation in the design and deployment of these systems, what are the impacts of those changes.

There is some prior work on the involvement of social scientists in responsible research and innovation (RRI) in emerging technology research centers and projects, including in nanotechnology and synthetic biology. There is a critique that RRI to date has focused more on responsible research than responsible innovation. The GA-AIM project not only targets the use of AI innovations but also integration with workforce training and community economic development. This is significant in that it adds new dimensions and challenges related to the framing and operationalization of ethical and responsible innovation for emerging technologies. In addressing this topic, and drawing on action-research findings, we aim to contribute to scholarly discussion about how responsibility and ethical approaches in AI can be framed, adding considerations related to manufacturing environments and community development which, to date, have been less explored. We will examine important (and often overlooked) issues of the operationalization of attention to ethics and responsible innovation, with the aim of deriving exploratory insights for innovation management, university partnerships, and innovation policy.

ABSTRACT. Background and rationale

Since the 17th century, learned societies - voluntary non-profit organizations involving academics - have existed for the advancement of scholarship, research, disciplines, publishing and public understanding of science. Publishing scientific journals and books has traditionally been an important part of the activities of learned societies but they also take other various activities including arranging conferences, supporting research, and popularizing knowledge.

The most well-known learned societies are the national science academies, in which membership is typically based on invitation and merit. However, there also exists a much broader archipelago of local, national and international societies, whose membership is open to all academics, and often also to interested professionals and citizens. In English-language literature, a learned society can also be referred to as a learned/scholarly/scientific association or scholarly society. Occasionally, the term professional society is also used. We have estimated in a working-paper that there are around 9000 learned societies in Europe (https://doi.org/10.5281/zenodo.5513560).

In historical perspective, learned societies constitute, along with universities and other kinds of research performing organizations, the foundation of contemporary academia. International Survey on Research Integrity (IRIS: https://doi.org/10.17605/OSF.IO/XB9RK) provides the broadest available indication of the continued relevance of learned societies. Over 82% of 60,885 active researchers from Europe, United States and other countries, who in 2021 answered the IRIS survey question concerning professional societies, identified at least “a little bit” with the societies, and 62% identified “moderately”, “a lot” or “a great deal”.

Yet, surprisingly little information and research exists on their current number, forms of organizations and operation, or contributions to scientific and societal impact of research in specific countries, regions or globally. While more attention has been paid to the obstacles and challenges for promoting open access in the landscape of learned society journals, next to nothing is known or said about the (potential) role of learned societies in the area of responsible research, including research integrity and research assessment. However, we argue that these societies create networks of scholars and professionals and have discipline specific expertise on scholarly activities that should be exploited more.

In this paper we address the following questions: 1. What role does open science and responsible research play in societies’ activities? 2. Do the members expect open science and responsible research to be part of the societies’ activities?

To answer these questions, we would need a global review of learned societies, their activities and membership. The scope of our present study is mainly focused on Finland, from which we have a comprehensive report in 2019 (https://doi.org/10.23847/isbn.9789525995190). Besides Finland, one survey study has covered learned societies in Portugal, and another one social science societies in the UK.

Methods and data

We provide data from Finland and from Finnish societies. Currently, almost 300 societies are members of the Federation of Finnish Learned Societies (TSV), and they have more than 250,000 individual members (Finland has a population of 5,5 million). Recently TSV collected two broad datasets from its member societies and the individual members of these societies.The first dataset (N=114 unique societies) concerns the actions related with responsible research of learned societies. The second dataset (N=1540 individual members of learned societies) concern the membership of the societies. By integrating these two datasets we are able to provide insight about the open science and responsible research activities from the viewpoint of the societies and their membership.

Results

Over 94% of 114 learned societies in Finland, who answered the survey in 2021, indicated that the promotion of scientific activities is a major part of their activities, followed by the promotion of general understanding of science and societal impact. Generally speaking, the promotion of responsible research and open science has not yet become a key part of learned societies’ operations. Yet around 70 per cent of the societies participate in developing open science, 60 per cent publish immediately openly available publications. However, activities linked to research integrity, open data, open education and citizen science are rarer.

Nevertheless, the majority of societies were interested in developing their activities in one or more areas of responsible research: identifying and proposing experts to working groups, committees or as evaluators (65 % of societies), promoting open science and organizing events (more than 50 %), evaluating the quality of research (more than 40 %), assessment of social impact, science education and research ethics (approx. 30 %).

The membership of the learned societies regards traditional forms of operation, such as networking, publications, events and the popularization of science as the most important parts of the societies’ operations. Over 90 per cent of the respondents agreed at least partially with the statement that the promotion of the openness of research-based knowledge is important, and more than 80 per cent agreed that this should be free of charge to the readers. Almost 90 percent of the respondents agreed at least partially with the statement that the promotion of research integrity should be an important task of societies, and almost 80% agreed also on the importance of responsible assessment of researchers and research quality. Almost 60% agreed on the promotion of citizen science, while 45% agreed that producing open learning materials is an important task of societies.

Significance

Our case study in Finland shows that societies are active in the most traditional forms of open science, such as open access publishing. Publication activities are indeed one of the learned societies’ key forms of operation, and a significant proportion of learned societies’ publications already complies with the principles of open science. The societies have been important enablers and promoters of open, non-profit and scientific publication activities without author fees (so called Diamond OA) in Finland. Other forms of open science and responsible research are not yet established but highly supported by the members of the societies. The perceptions from the membership should guide the future development of the societies.

Data on society memberships shows that learned societies represent the science community on a broad scale in terms of age, field of education, work organization and place of residence. The networks created by the societies are cross-organizational and often international and provide a unique resource for identifying experts of different domains and creating collaboration between the domains. Thus, societies have a great potential in implementing and developing field-specific open science practices through engagement of researchers at the grass-root level towards a more responsible direction. Policy makers, research performing and funding organizations could develop more strategic collaboration with learned societies as a valued intermediary between science and society.

ABSTRACT. Suggested topic area: Societal impact

Background and rationale: The literature on societal impact of research has been pointing to a diverse range of ways to identify, analyze, and measure how scientific knowledge flows from its production spaces to engage with and influence society (Viana-Lora & Nel-lo-Andreu, 2021; Smit & Hessels, 2020; Bornmann, 2013). Part of this debate is related to the impact of research on public policy design, implementation, and evaluation processes (Boaz et al., 2009; Newson et al., 2018). In general, such studies adopt a diversity of theoretical and methodological approaches that basically follow two paths, which are not mutually exclusive: forward tracking, starting from research to identify its impact on policy; and/or backward tracking, starting from policy to identify the use of scientific research. In both paths, one of the possible approaches refers to the use of bibliometrics and altmetrics techniques to capture and analyze the use of research in policies and measure its impacts (Tahamtan & Bornmann, 2020). In addition, the debate on the societal impact of research - in its various social dimensions and spaces, including policy - allows us to put into perspective and debate the role of stakeholders directly involved in the production of scientific knowledge, especially universities and funding agencies.

Objective: Considering this debate, our study aims to explore the potential of bibliometrics to identify and analyze, in an exploratory way, how policy documents have been created based on research funded and promoted by the São Paulo Research Foundation (FAPESP) in two main areas: society and environment. We assume herein the inseparability between the social and environmental spheres. Considering the environment and society as elements that mutually affect each other, it is common for research funding institutions to include the analysis of environmental impact within the social scope. The Higher Education Funding Council for England (2019), for instance, considers that health and environment are part of the definition of social impact by affirming that it means “[a]n effect on, change or benefit to the economy, society, culture, public policy or services, health, the environment or quality of life, beyond academia” (HEFCE, 2019: 118). Regarding FAPESP's financial support for projects, in the field of life sciences, which accounted for 45% of the disbursement made by the institution in 2020, the health area was responsible for more than 25% of this investment, totaling more than BRL 247 million that year. Thus, as for societal impact, we have decided to focus on health-related research funded by FAPESP. Likewise, amongst the several projects supported by FAPESP in the environmental field, climate change-related research has been receiving most of the grants, which reflects the global trend of counteracting the emission of greenhouse gas and warming of the earth’s surface. In other words, we will seek to understand how policy organizations have been using research funded by FAPESP in these two major socio-environmental problem areas, namely health and climate change. This study answers the following research question: to what extent is it possible to measure the impact of research funded by FAPESP in the areas of health and climate change by exploring bibliometric resources?

Methodology: Here we make use of Overton, considered the largest and most comprehensive database that tracks how research (papers) are cited in policy documents produced by government agencies, think tanks, nongovernmental organizations, and intergovernmental organizations in 182 countries (Fang et al., 2020). The research explores the following path: from the policy documents found under the umbrella of "health" and "climate change", we have identified the papers written by authors from Brazilian research institutions. After this initial screening, we have selected only the research funded by FAPESP. Such procedure allows bibliometric data to be obtained from both the funding-related articles and the policy documents that use them. Afterwards, an exploratory analysis was conducted, focusing on four aspects: (i) the characterization and type of political organization that cited research funded by FAPESP, including country of origin; (ii) the sub themes of impact within health and climate change most related to research funded by the agency; (iii) how the research is being used, considering the debate about political appropriation and mobilization of research; (iv) and basic bibliometric analysis of the identified papers. Although the study cannot be considered comparative in nature, the methodology enables the establishment of similarities and differences between the impacts considered (health and climate change).

Significance of the study: By using Overton, this study explores ways to connect funding to the impact of research on public policy, attempting to dialogue with the literature on the social impact of research, as well as empirically bringing about novel analyses on one of the main Brazilian research and innovation funding agencies. On the one hand, the focus on the funding agency is justified by its prominent and influential role in stimulating the societal impact of research within the scientific community; on the other, funding agencies are embedded in a context of political and social pressures to demonstrate the societal impact of funded research, also seeking for political and societal legitimization (Curry et al., 2020; Watermeyer, 2019). Moreover, FAPESP is considered one of the most important research and innovation funding institutions in Brazil. Regarding the chosen impact themes, it is known that health and environment are fruitful fields for the dialogue between research and public policies (Newson et al., 2018; Bornmann et al., 2016), as well as relevant themes to society, closely related to the Development Sustainable Goals (SDGs). Thus, we discuss herein, from the data and analyses carried out, the role of funding agencies in promoting studies capable of politically impacting health and the environment.

References Bornmann, L. (2013). What is societal impact of research and how can it be assessed? A literature survey. Journal of the American Society for information science and technology, 64(2), 217-233.

Viana-Lora, A., & Nel-lo-Andreu, M. G. (2021). Approaching the social impact of research through a literature review. International Journal of Qualitative Methods, 20, 16094069211052189.

Boaz, A., Fitzpatrick, S., & Shaw, B. (2009). Assessing the impact of research on policy: a literature review. Science and Public Policy, 36(4), 255-270.

Curry, S., De Rijcke, S., Hatch, A., Pillay, D. G., Van der Weijden, I., & Wilsdon, J. (2020). The changing role of funders in responsible research assessment: Progress, obstacles and the way ahead. RoRI Working Paper,

Newson, R., King, L., Rychetnik, L., Milat, A., & Bauman, A. (2018). Looking both ways: a review of methods for assessing research impacts on policy and the policy utilisation of research. Health research policy and systems, 16(1), 1-20.

Smit, J. P., & Hessels, L. K. (2021). The production of scientific and societal value in research evaluation: a review of societal impact assessment methods. Research Evaluation, 30(3), 323-335.

Tahamtan, I., & Bornmann, L. (2020). Altmetrics and societal impact measurements: Match or mismatch? A literature review. El profesional de la información (EPI), 29(1).

Bornmann, L., Haunschild, R., & Marx, W. (2016). Policy documents as sources for measuring societal impact: how often is climate change research mentioned in policy-related documents?. Scientometrics, 109(3), 1477-1495.

Fang, Z., Dudek, J., Noyons, E., & Costas, R. (2020). Science cited in policy documents: Evidence from the Overton database. In Altmetrics conference. http://altmetrics. org/wpcontent/uploads/2020/11/02_submission_Fang_Dudek_Noyons_Costasaltmetrics20. pdf.

ABSTRACT. Introduction Universities play a key role in shaping the social structure in which scientific activities are performed. Wapman et al. (2022) showed that graduates from the 20% most U.S. prestigious universities occupied 80% of the faculty positions in the country. This is due to the fact that authors from top universities benefit from an environment that gives them higher productivity and recognition (Way et al. 2019). Such inequalities at the individual level influence the research priorities of the country, as it allows those top institutions to have a fundamental role in setting the research priorities within the scientific community (Clauset et al. 2015). In tandem, research has shown a strong alignment between an author’s sociodemographic characteristics and the topics they study (Kozlowski et al. 2022a). However, little research has studied the intersection of these intersectional inequalities and how they might be mediated through institutional prestige. In this project, we focus on how prestige relates with the socio-demographic representation of authors, of research topics and research impact. Also, we will analyze the role of mission-driven institutions, such as Historically Black Colleges and Universities (HBCU) and Women’s Colleges (WC). Two main research questions drive the analysis. First, we want to assess whether those types of institutions have a specific topical profile, and how it varies as a function of authors’ intersectional identities. Second, while it has been widely studied that institutional prestige drives impact (Way et al. 2019), it is important to acknowledge how this relates with race and gender inequality. How, if at all, does the citation bias against marginalized groups—women, Black and Latinx authors— vary by the prestige of the institutions in which they work?

Methods To answer these questions, we use more than 4.5 million articles from more than 800 US universities published between 2008 and 2020 and indexed in the Web of Science (WOS). Following the method developed by Kozlowski, Murray et al. (2022b), authors of the selected papers were assigned a probability over each racial group based on the association between their family names and racial categories found in the 2010 US Census (USBC 2016). Gender was inferred using authors' given names, based on Larivière et al. (2013). We consider a researcher's identity as the combination of four racial categories —Black, Latinx, Asian, White—and gender, considered in a binary way. Given the limitations of the data and inference algorithms, it is not possible to automatically classify authors into Census categories “Native Americans” and “Two or more races”, and gender outside the women-men binary, which further highlights the importance of studied based on surveys and self-identification of authors as complements to large-scale secondary analytical studies. We used historical WOS data to compute the average number of citations of US universities between 1980 and 2019. We use three different classifications as complementary indicators of prestige, as proxies for perceived prestige (US News & World Report), research prestige (average citations) and selectivity prestige (Carnegie Selectivity Index). Each of these three rankings is divided into three groups: ‘high,’ ‘middle’ and ‘low’. We also consider mission-driven classifications: HBCU and WC. We used topic modeling (Blei et al. 2003) to infer the research topics of articles based on their titles, abstracts and keywords. We define the topical profile of a group as the proportion of papers this group contributes on each topic with respect to the total number of publications in the topic. Topical profile can be applied both to groups of researchers' identities and institutional categories. To compare two topical profiles, we use the Spearman rank correlation, as the relation between topical profiles is not linear. We focus on the correlation between the topical profiles of institutional categories and authors' identities. If the correlation between those groups is high, it means that they tend to publish on similar topics. We also build a linear model to predict the effect of author's identities on impact (citations and Impact Factor). By splitting articles by their institutional prestige groups and running the linear model for each, we show the differential effect of race and gender by institutional prestige.

Results and discussion There is an increasing debate around the role of elite institutions in the reproduction of systemic inequalities in science (Clauset et al. 2015; Wapman et al. 2022). These institutions do not only have a higher accumulated economic capital—as exemplified by their endowment—but also of symbolic capital. A handful of institutions produce a large proportion of research articles, and those attract an even larger impact (Way et al. 2019). This study provided an intersectional perspective on the role of institutions. It provided evidence that institutions are key to the stratification process of science and that, across different institutional tiers, the experiences of racialized and women authors is qualitatively different from that of White men. Aligned with their missions, HBCUs and WCs occupy a topical space that is closely related with that of Black and Latinx, and White women (respectively). In previous works we found that thesetopical profiles relate with the lived experiences of these populations, and have a larger focus on issues such as racial discrimination, migration, and gender-based violence (Kozlowski et al. 2022a). However, as we climb the ladder of institutional prestige, we observe a sharp decline of participation in those topics. Far from being a simple consequence of the composition of their faculty, this reflects on a more nuanced phenomenon: authors from marginalized groups in top institutions have a topical profile that differs from the one of their institutions, but also differs from that of their own identity in other institutions. Top institutions are failing to leverage topics that are relevant to marginalized populations. We have also found that the impact gap by race and gender is affected by institutions. Articles from top institutions gather more citations and are published in higher impact journals. But within institutions the gap persists, and the highest impact gap is found on top institutions, even after controlling by topic and other relevant covariables. These results demonstrate the strong role that institutions play in shaping the research profile of a nation. Initiatives that serve to diversify hiring may serve to expand the topical profile of an institution, but only if other mechanisms are in place to ensure that scholars are free to pursue the full landscape of research questions. Our work suggests a penalty for this deviation that is borne particularly by minoritized populations.

Cited References

Blei DM, Ng AY, Jordan MI. 2003. Latent dirichlet allocation. J. Mach. Learn. Res. 3(null):993–1022 Clauset A, Arbesman S, Larremore DB. 2015. Systematic inequality and hierarchy in faculty hiring networks. Sci. Adv. 1(1):e1400005 Kozlowski D, Larivière V, Sugimoto CR, Monroe-White T. 2022a. Intersectional inequalities in science. Proc. Natl. Acad. Sci. 119(2):e2113067119 Kozlowski D, Murray DS, Bell A, Hulsey W, Larivière V, et al. 2022b. Avoiding bias when inferring race using name-based approaches. PLOS ONE. 17(3):e0264270 Larivière V, Ni C, Gingras Y, Cronin B, Sugimoto CR. 2013. Bibliometrics: Global gender disparities in science. Nature. 504(7479):211–13 USBC UCB. 2016. Frequently Occurring Surnames from the 2010 Census. Census.gov. www.census.gov Wapman KH, Zhang S, Clauset A, Larremore DB. 2022. Quantifying hierarchy and dynamics in US faculty hiring and retention. Nature. 610(7930):120–27 Way SF, Morgan AC, Larremore DB, Clauset A. 2019. Productivity, prominence, and the effects of academic environment. Proc. Natl. Acad. Sci. 116(22):10729–33

ABSTRACT. Researchers at elite universities tend to dominate scientific discourse, with higher productivity and far greater prominence, and thus shapes the direction and pace of scientific discovery as a whole. Such dramatic disparities could be due to simple meritocratic sorting, reflecting genuine differences in individual scientific ability, but they could also reflect non-meritocratic factors like luck and structural advantages. Understanding which mechanisms drive this prestige-productivity pattern would shed critical new light on efforts to accelerate scientific discovery, and inform policies aimed at broadening participation.

Past work has shown that among early-career faculty researchers, placement at a more prestigious institution can cause greater productivity and prominence. That is, where you work is the greater determinant of your impact than where you trained. However, the particular mechanism through which prestige underlies this pattern remains unknown. Through multiple lines of evidence, we show that differences in available funded non-faculty scientific labor drive substantial prestige-productivity inequalities, and the scientific dominance of elite universities can be explained by their substantial labor advantage over researchers at less prestigious institutions, primarily in disciplines where faculty lead and collaborate with a research group. Our analysis leverages cross-disciplinary, longitudinal data on the education, employment, and publications of 78,802 tenured or tenure-track faculty spanning 4,492 departments across 25 disciplines in science, engineering, and the social sciences at 292 PhD-granting U.S.-based universities, over the period 2011--2017, which we combine with researcher-level productivity data encompassing 1.6 million publications from the Web of Science. We complement these data with institution-discipline-level counts of graduate and postgraduate (non-faculty) researchers, institutional covariates, and discipline-specific measures of prestige.

First, we show that faculty's annual productivity, measured crudely as their mean publications per year, increases substantially with environmental prestige, with elite researchers being roughly twice as productive as researchers at the least prestigious institutions. We isolate the component of total productivity that could be driven by differences in labor by partitioning each faculty's ``total'' productivity into two sources: (i)~group productivity (publications coauthored with non-faculty research group members), and (ii)~individual productivity (all other publications). In disciplines with group collaboration norms, a larger group will tend to drive greater group productivity, but not in disciplines without such norms. We show that in such disciplines, group productivity is both substantial and grows systematically with prestige, even as the individual group members are no more or less productive. Finally, we show that research labor is highly concentrated within the most prestigious environments, indicating that elite researchers tend to have larger research groups.

We then test this ``labor advantage'' hypothesis using a series of predictive models, showing that funded labor consistently plays a significant role in predicting productivity and group sizes in disciplines with research group collaboration norms, but not in disciplines that lack these norms. Next, using a matching experiment on mid-career moves, we show that faculty who move to an environment with more available funded labor tend to have groups that are significantly larger after the move than those who go to environments with less labor. Finally, we quantify a systematic relationship between larger faculty group sizes and greater group productivity that is independent of prestige.

Taken together, these results identify the environmental mechanism by which prestige drives greater scientific productivity, and show that it is the profound labor advantage of elite working environments that allows their scientists to dominate scientific discourse. More broadly, our findings suggest relatively simple interventions for both increasing scientific productivity and increasing the diversity of scientific advances. Accounting for the non-meritocratic effects of research environments will be an important component in developing predictive theories of knowledge production.

ABSTRACT. Background and Rationale Over the last forty years, many institutions in the U.S. higher education ecosystem have changed their institutional strategy and/or identity to place a greater emphasis on research and graduate education (O’Meara, 2007). This widespread phenomenon, often termed “striving,” has occurred for many reasons, most notably via strategic efforts by academic leaders to enhance institutional prestige and ensure long-term stability and prosperity (Jaquette, 2013). Although striving behavior can occur at any institution, this paper looks specifically at the case of institutions seeking to become or advance their competitiveness within the ranks of doctoral research universities – the most prestigious and research-oriented of American universities. Importantly, while the striving phenomenon has been examined by a variety of scholars, often overlooked are the experiences of tenure-track faculty, the very people who are expected to increase the research profile of their institution (Gonzales et al., 2014; Jackson et al., 2017; O’Meara & Bloomgarden, 2011). This study examines the differential experiences of faculty at striving research universities to better understand the patterns of workload and academic research productivity that emerge in these transitioning institutions. Faculty at striving institutions are typically expected to enhance their research programmes in furtherance of institutional goals. However, the degree of institutional support and relief from other job duties (teaching and service) they receive in furtherance of these goals varies considerably, sometimes creating intractable time conflicts (Sabagh et al., 2022). Striving institutions also broadly experience a transitional phase in their internal norms, expectations, and logics (Morphew et al., 2018; Morphew & Huisman, 2002), which can lead to ambiguity in evaluation (Jackson et al., 2017; Véliz & Gardner, 2019). Critically, support and reward systems for research are weaker at striving research universities than at established R1 research universities (Griffith & Altinay, 2020; O’Connor et al., 2011). This results in a complex system of institutional policies that both hinder and support faculty research productivity at striving research universities – one which is likely to have disparate effects on faculty at different career stages and from underrepresented groups. In the broader context of studies of academic research productivity in individual faculty, examining striving research universities helps address an important theoretical gap. Most or all of the major inputs into research productivity have been identified by previous studies (Bland et al., 2005, 2007; Toutkoushian & Bellas, 1999). These include individual characteristics (e.g. intrinsic motivation and research skill), characteristics of the employing institution of the researcher (e.g. rewards structures, clear goals and guidance, sufficient time allocation, lack of red tape, doctoral students, and support structures), and social characteristics (e.g. welcoming climate, peer support, peer expectations, and collaboration opportunities). Despite these many factors being well known, their interrelation is complex and remains ambiguous (Gläser et al., 2014). We do not know, for instance, how varying degrees of institutional support for research may disparately affect individuals. Another complication is that understanding the relationship of the factors which support research productivity is particularly challenging because they tend to largely co-occur, leading to multicollinearity, particularly for institutional factors (Bland et al., 2007). Academic work environments typically have either most or few of the characteristics supporting research productivity, with little in-between. In other words, different categories of institutions (e.g., liberal arts colleges) tend to have internally consistent bundles of research supporting and hindering policies, such as teaching loads, rewards structures, and graduate programs. Researchers have thus struggled to determine which institutional factors are most causally important for research productivity. This greatly complicates the interpretation of studies that compare colleges across institutional categories, models, and niches. That is, studies with doctoral research, comprehensive, and primarily undergraduate institutions in their sample. Such studies, unsurprisingly, find that the average productivity of faculty varies across institutional type, due to the strong correlation of institutional type and the presence research-supporting characteristics. It has been known for some time that the strongest predictive factor of a random academic researcher’s productivity is the type of institution at which they work (Pellino et al., 1981). In other words, the research productivity of individual faculty tends towards the average research productivity of their employing institution. This, however, does little to enhance our theoretical understanding of research productivity. In particular, cross-sectional comparison creates a risk of conflating the distinctive characteristics of these institutional categories that are cause variation in research productivity with those that are unrelated. That is the core motivation to study research productivity in the striving university setting. Because these universities are undergoing change, we can observe variation in institutional support for research within a broadly similar category of research institutions, particularly in teaching loads. Striving institutions have unusual combinations of characteristics that are both supportive and deleterious for research productivity, due their transitional nature. This will hopefully lead to a greater understanding of which factors are most important for supporting research productivity. To examine these complex issues, we ask two questions: 1.) In a striving research university environment, how do individual characteristics and institutional factors interact to enhance and inhibit faculty research productivity? 2.) In a striving research university environment, what is the precise relationship between faculty time allocation and research productivity?

Methods This study uses a subset of approximately 800 faculty working at 117 striving research universities from a 2011 national survey of tenure-track faculty in the U.S. The dataset provides a reasonably large sample size along with detailed individual characteristics, work histories, job duties and time allocations, work psychology measurements, and self-reported measures of research productivity. Striving research universities were identified as those that entered into the Research 1 or Research 2 Carnegie Classifications between 2010 and 2021, after having previously been in a lower classification. The survey data were analyzed using descriptive statistics and a combination of regression techniques, including OLS and count regression.

Anticipated Results Although there are many interrelated factors, preliminary findings suggest: First, faculty interest in research is varied and an important factor in research productivity. Next, teaching duties have a more negative impact on research productivity than comparable service loads. Additionally, there is variation in research productivity by race and sex not explained by other factors in the model. Finally, there are diminishing marginal returns to time allocation to research, particularly when research comprises the majority of a tenure-track faculty member’s work. More findings are likely to emerge as this study is refined further.

Significance This research helps advance our understanding of the relationship of the many factors that influence research productivity and may inform the decisions of academic leaders and policymakers. Better understanding of the connections between the mechanisms by which research productivity operates could help improve the effectiveness and equity of striving institutions’ strategic initiatives. This is particularly relevant as long-term downward trends in both domestic and international enrollment at U.S. colleges suggest that the financial pressures which have helped spur striving behavior are likely to continue. In addition, these insights could help identify ways to assist faculty build research capacity more broadly.

ABSTRACT. Researchers on innovation and human capital mobility have considered mobility as a voluntary event where employees choose to enter the job market. However, mobility can be involuntary (forced), and it is often the case. By exploiting the fall of Nortel Networks as a natural experiment, we distinguish voluntary mobility from forced mobility and examine the consequences on innovation performance at the individual level. We find that the two aspects of mobility have different consequences on employees' innovation performance when studied separately from the perspective of the source and destination firms. We contribute to the literature on human capital mobility as a driver for knowledge creation and diffusion, particularly forced mobility which has so far received very little attention. Also, we underline a better understanding of the mobility mechanism on knowledge creation and diffusion patterns by showing how forced mobility compares to voluntary and no mobility.

ABSTRACT. Background: Over the last several years, growing international interest in environmental protection has led to the promotion of low-carbon industries and pursuit of sustainable development on a global scale (Mensah et al., 2019). For research and development (R&D), which is essential for strengthening a firm’s innovation capability, the most important element is funding. This also applies to green firms, for whom financial investment into R&D is crucial (Noailly & Smeets, 2021; Yuan et al., 2020). Researchers using a resource-based view of the firm have four categories: physical, financial, organizational, and human capital (Barney, 1997). We analyze R&D funding based on resource-based view theory. Arrow (1962) argued that if R&D is funded from the outside, a moral hazard and adverse selection due to asymmetric information are possible. Gertler (1988) explained that external funding may be difficult and more expensive than internal funding if elements of market imperfection are present, such as transaction cost, agency cost, and asymmetric information. Some scholars have argued that firms tend to invest their internal R&D funds because of this risk. In other hands, Noailly and Smeets (2021) argued that the green firms, like clean energy technology sector, are capital-intensive and large investments are a prerequisite. We address the following research question: Does the innovation barrier of green firms affect innovation performance? and what are the effects of external and internal funding-based innovation strategies, respectively, on this relationship?

Purpose: This study examines whether green firms can overcome barriers to innovation and improve innovation performance through research and development(R&D) funding. We analyzed innovation performance with internal and external R&D funding for firms facing barriers to conducting green technology innovation activities.

Data: This research utilized the data on the Korean Green industry from the 2015 Green Industry Innovation Survey (GIIS), which was conducted by the Green Technology Center, Korea. As the data contains overall innovation, including the objectives of green innovation, resources used for green innovation (input of innovation), and green innovation performance, GIIS data is believed to be appropriate for this research.

Result: The analysis results indicated that internal R&D funding was more influential than external R&D funding. The results showed significant innovation performance for firms using internal, rather than external, R&D funding to overcome barriers; internal R&D funding mediated the relationship between innovation barrier and performance.

Contribution: First, This study demonstrates the need for green firms to recognize barriers they face and conduct R&D despite these barriers. Further, we add to the academic discussion on the importance of R&D investment in technological innovation when overcoming barriers to innovation. Second, the findings emphasize the importance of green firms using internal resources to fund R&D to improve innovation performance, rather than injecting external R&D resource, despite the recognized barriers. This study indicates that, despite recognized barriers, entrepreneurs need to invest in R&D for innovation, and it is more important to invest internal resources than to take on the many risks associated with external funding.

Limitation and Further study: First, it did not take time lags into account. This study was conducted by analyzing only the 2015 data. Innovation performance includes sales growth rate and patents, which may have time lags. Second, knowledge exploration and external cooperation can be used as strategies, in addition to R&D funding, to overcome barriers. Third, barriers to innovation may vary depending on a firm’s characteristics and industrial environment.

ABSTRACT. Background and context

In the last few years, innovation profiling and segmentation has become an essential method in academic and policy circles for analysing innovation behaviour of firms. When firms innovate, they are often faced with a wide variety of challenges or impediments. These are more pronounced in the agricultural sector, especially, since businesses in the agricultural sector have experienced a variety of issues that have hampered their innovation efforts. These obstacles include, among other factors, resource (financial and human), institutional and regulatory, as well as, environmental factors (climate change, droughts, floods, etc.).

Meanwhile, in an effort to address some of the challenges to innovation faced by agricultural businesses, policy-makers have often applied a blanket approach to design policies which, in some circumstances, have failed to meet the needs and requirements of agricultural businesses. Some of these policies have not produced the desired effects due to a lack of understanding of the characteristics and profiles of innovation in agricultural businesses and, in particular, the barriers to innovation in these firms. This is reflected in weak innovation performance in agricultural businesses, which has remained much lower when compared to other sectors of the economy, especially those in developing nations.

Innovation remains nonetheless a critical vehicle for addressing the sector's challenges, notably those of poor productivity, provided that a conducive framework for innovation is developed which addresses the impediments to innovation in the sector. This can help ensure the agricultural sector’s long-term viability and sustainability (Ulvenblad et al., 2018).

Unfortunately, simple aggregate indices of the number of innovation-active enterprises that consider a specific barrier to innovation as very important provide little relevant information for policymakers. To gain a better understanding of the structure and complexity of barriers to innovation in the agricultural sector, it is necessary to profile businesses based on the barriers they face, given that alleviating or at least minimising barriers to innovation may stimulate more innovations in the sector, thereby increasing the number of businesses that are actively engaged in innovation.

Objectives Although existing literature on innovation barriers has, to a limited extent, explored the different types of barriers to innovation in other sectors of the economy, the primary goal of this study is to investigate clusters of agricultural businesses based on barriers to innovation they experience.

Methods

This study used data from the South African baseline Agricultural Business Innovation Survey covering the period 2016-2018 (Agri-BIS 2016–2018). The Agri-BIS 2016-2018 was based on the guidelines of the Organisation for Economic Co-operation and Development’s (OECD) Oslo Manual (OECD/Eurostat, 2005). The survey used the methodological recommendations for the Community Innovation Survey (CIS) of the European Union (EU) countries, as provided by Eurostat, the Statistical Office of the European Commission. The survey focused on ascertaining how agricultural businesses innovate. The core questions asked about the businesses’ product, process, organisational and marketing innovations. The survey also asked questions about the different innovation activities and outcomes.

To determine the barriers to innovation, the survey incorporated additional questions on the factors that impede agricultural innovation. These questions asked businesses about the different factors that they considered highly important during the reference period 2016–2018.

This study applied a K-modes machine learning clustering algorithm to analyse the clusters of agricultural businesses that have experienced a wide range of innovation barriers. Python programming language was used to implement the k-modes machine learning algorithm on the dataset using the Scikit-Learn and pandas library.

Significance and contribution

This study makes an important contribution and bridges a key gap in literature in that, it is one of the first of its kind to apply a machine learning algorithm to cluster agricultural businesses in terms of the barriers that affect them. Secondly, this study allows for a more detailed examination of the implications in terms of prospective innovation policy instruments targeted at alleviating impediments to innovations on weak innovation performance in agricultural businesses as a result of the different types of barriers. The results may be used to inform policy by better targeting instruments and tailored policies to that address barriers to firm innovation in specific clusters in the agricultural sector.

ABSTRACT. Exploratory innovation relies heavily on a constant stream of new external knowledge that can be combined and recombined with the knowledge one already holds. As such, scientific knowledge flows, which result from searching for knowledge in scientific research articles, provide the necessary external knowledge elements for exploratory innovation. However, few studies explore the role that scientific knowledge flows play in exploratory innovation. Hence, in this study, we test how several characteristics of knowledge flows impact exploratory innovation in the private sector using firms in the field of biomedicine and the life sciences as our sample. The characteristics include the intensity, breadth, and novelty of scientific knowledge flows. In addition, we examine the moderating role diversity might play in these relationships. Our results indicate that intensive scientific knowledge flows characterized by novel knowledge elements tend to generate greater levels of exploratory innovation, while broad scientific knowledge flows typically have a negative influence on this type of innovation. Additionally, the diversity of the firm’s internal knowledge base also has a negative moderating effect on the relationship between scientific knowledge flows and exploratory innovation, no matter how the knowledge flows are characterized.

ABSTRACT. An increasing amount of works have studied the geographic dimension of research using publication data. Most of them traditionally rely on metadata associated with authors' affiliations, as they usually assign publications to the geographic location of their authors' institutions. Although this approach is useful for identifying the places where scientific knowledge is produced, it tells little about the places that get studied. The latter dimension is crucial, although often disregarded, as there is a potential mismatch between the places where authors work and the places they write about; the places that fund research and the places that get direct benefits from it; and, more generally, between the social demand for research where it is more needed and researchers' supply where it gets higher individual rewards.

We advance the understanding of the places that get studied, based on the geographic content embedded in research documents. In particular, we extract mentions of geographic places from titles and abstracts of scientific publications (geographic content) to identify broad empirical regularities worth further explanation (stylized facts). Our analysis concentrates on three main categories of stylized facts: what places get studied, who studies them, and who uses them. Besides listing general empirical patterns, we link our findings with ongoing debates on knowledge inequalities and knowledge diffusion to provide avenues for further research. Moreover, we discuss the potential implications for science policy of moving from a production-assigned to a content-based conception of geography. 

We rely on Natural Language Processing (NLP) models for Name Entity Recognition (NER) to identify references to geographic places in the whole universe of documents in Microsoft Academic Graph (MAG). Specifically, we extract words that refer to geographic places from titles and abstracts in  13 languages, based on their contextual and grammatical role. In this way, we overcome some of the main constraints of related efforts, which usually rely on small samples of documents written in English, focus primarily on social and environmental sciences, and consider a bounded set of geographic locations.

We find that the places where knowledge is produced differ from the places that get studied. On average, 40% of all journal articles written in a country study foreign geographies. Likewise, 73% of all documents written about a country come from foreign-affiliated authors. By discipline, studying foreign geographies is more common in philosophy, physics, and arts, while it is less frequent in medicine, psychology, and business. 

Most geographic attention concentrates disproportionately on a handful of countries. In absolute terms, geographic attention is concentrated in the US and China, as measured by the number of papers about these countries. Regarding their relative share in the world's population, the US, Australia, and European countries receive disproportionate attention. Relative to their share in the global GDP, the US and Nordic countries still received higher-than-average attention, although results vary according to the studied academic field. In all cases, African countries are the most neglected areas. Overall, geographic attention aligns with a country's income more than its population's share, and dollars of income tend to deserve more attention than individuals.

Our work has significant implications for research on the geography of knowledge. Until now, the geographic dimension of research has been reduced to the places where knowledge is produced. However, in an era of improved access to communication technologies, rise in collaborations, and increased high-skilled migration, we should expect a wide mismatch between where authors work and where they write about. Hence, understanding the empirical regularities of the places that get studied creates a new lens to raise new questions and develop alternative explanations on the geographic distribution of research.

For policymakers, understanding geographic content could reveal how research efforts translate into direct knowledge for the places they govern. In addition, geographic content analyses can help monitor how actors perceive specific geographies and how places benefit from the research produced (and funded) by others. Although increasing the number of documents about a place should not become the main objective of science policy, unveiling what and why places get studied expands the analytical toolbox to assess the social impacts of research.

ABSTRACT. The Costa Rican economy has long been regarded as one of the region’s most stable and socially developed economies. In this paper we made an analysis of the evolution of Costa Rica’s strategy of economic diversification and capabilities building traying to analyse the rationality of the policies in different stages.

Most of the literature on the phenomena known as the “middle income trap” (MIT) suggests the relevant roles of capabilities, structural change and technological progress as the key factors able to better positioning a middle income country moving to better conditions (Vivarelli, 2014). A general argument, as suggested by Perez-Sebastian, 2007; Agenor and Canuto, 2012, is that once some countries reach the middle-income level, the pool of unemployed and underemployed rural workers drain out, wages start to rise, but an significant problem is that benefits from imitation and importing foreign mature technologies decrease in importance. In this condition, changing the structure of the economy (diversification from low productivity sectors into high-productivity ones) and on the types of product exported are the most important drivers in the strategies to overcome MIT (Gill and Bhattasali, 2007). It is important to consider that capability building and catch-up by domestic firms depend greatly upon the nature and features of innovation systems and it is necessary to consider the heterogeneous nature of the knowledge base, the specificity of the national, sectoral, and regional contexts, and the role of institutions in which innovative activities occur (Malerba and Lee, 2021). In the innovation system approach, the set of interrelated components is understood as a system working towards a common objective. Therefore, the set of parts and aspects of the economic structure and institutional set-up that affect learning and research are understood as the Innovation System (Lundvall, 1992). Chaminade and Edquist (2010) understand innovation policies as those public actions that drive innovation processes, both for development and dissemination processes. The rationality of innovation policies is based on trying to solve system problems. There are some failures in the system that hinder the possibility to reach these objectives. System problems to meet those objectives can usually be understood as problems of: infrastructure provision and investment problems; of transition; of "lock-in"; institutional; network problems; skills and learning; exploration-exploitation imbalance; complementarity problems (Chaminade and Edquist, 2010). Borrás and Edquist (2013), suggest a possible way to group policies and instruments to foster innovation, considering: regulations (intellectual property, regulations), economic transfers (competitive funds, exemptions), soft instruments (alliances, agreements). Orozco (2017) argues that to increase the efficiency of such policies and instruments, it is important to consider the geographical, sectoral or firm size specificities. The rationality of policies in GVC approaches is aimed mainly to facilitate trade. According to OECD (2012) and Catteneo et al (2013), in the approach of GVC there are two main objectives for policies: suppressing/reducing obstacles to trade at the border, including trade facilitation and increasing the accessibility and connectivity of the domestic market, and the security, predictability, reliability and efficiency of transports/logistics, telecommunications and ICT. More specific to innovation and building capacity, the approach recognizes that GVCs facilitate capacity constraints, since a country does not need to develop a fully integrated industry to participate in international trade. But capacities and productivity remain key factors for foreign investors and lead firms. In the new context, lead firms have to define strategies where innovation centres are decentralised. Lead firms need to innovate in developing countries, and solve their specific needs and eventually, they can bring the results back home, thereby contributing to boosting the developing countries’ exports (Govindarajan and Trimble, 2012). This requires the host developing country to develop innovation capacities, based on education and skills. There is a strong connection between the policy approaches of innovation systems and GVC. It is clearer for the approach of sectoral systems of innovation. But it is also clear that for more geographical approaches of innovation systems, it is also necessary to consider both the challenges and the opportunities from the conditions of international trade. Most of the challenges for economic and social upgrading in GVC require strong innovation systems. Considering the rationality of both approaches, it is clear that Costa Rica's export promotion and foreigner investment attraction policies have been mainly based on the GVC approach. Indeed, the strategy has focused on lowering barriers to international trade and consolidating an institutional framework to attract investment and promote new export products. From the point of view of the innovation system approach, there are still many gaps in the country's policies, as will be discussed in the paper. Our argument is that these gaps have contributed to the MIT in the country. We develop an historical analysis of the policies in the country and present some data to show the main changes in the structure of the economy and the exports.

ABSTRACT. Innovation studies traditionally employ bibliometrics and patent analyses to model ecosystems – methods that are less suited to measuring innovation in industry sectors where patents and publications are not used frequently by innovators. Some such sectors may be quite innovative despite a lack of patents or publications, for example the agriculture and agri-food sector, which is currently undergoing deep transformations relating to digitalization, climate changes and consumer preferences. Studying innovation in these sectors can be challenging due to a lack of accessible data. Official statistical sources often do not provide adequate coverage, and traditional surveying techniques such as personal interviews, focus groups, paper-and-pen or electronic questionnaires are increasingly problematic due to declining response rates or insufficient quality of data. It is crucial to explore alternate data sources and methods in sectors such as agri-food (Gök et al. 2015). Experimenting with webscraping techniques provides a non-intrusive method of collecting the public expressions of organizations with regards to their innovation activities. Previous studies have shown the potential of using text analysis in innovation studies (Antons et al. 2020; Youtie et al. 2021) and some underline the potential of content analysis of organizations’ public communication (Daas et van der Doef 2020). In this context, this study investigates whether innovation activities in sectors such as agri-food can be identified and captured; and what aspects of innovation activities are disclosed using public communications. The case we explore is the Canadian food and beverage processing sector and value chain due to its characteristics (Finco 2018; Vlachopoulou et al. 2021) and we develop a method to detect innovation-related themes conveyed by the sector’s organizations. In brief, the model is able to recognize paragraphs on websites that pertain to "innovation" and classify those, producing a coding scheme that can be used to analyze the agri-food sector. Data We base the analysis on a list of 2,506 organizations belonging to the Canadian agri-food ecosystem for which websites (URLs) have been identified. This agri-food ecosystem contains food and beverage processors, private sector suppliers of services and goods, private and public research and development institutions, government and non-for-profit organizations, and higher education institutions. The list was developed using manual and automated keyword web searches (food and beverage manufacturers by type such as products produced, etc.), and augmented by membership lists from associations of types of food and beverage processors, directories of organizations, and other public sources. Using a proprietary web analyzer tool created for this research, the data employed in this study were retrieved in May 2022. For this project, 47,531 English-only web pages are analyzed. Method We develop the method in four (4) main steps: 1) pre-processing of text data; 2) selection of paragraphs containing words related to innovation as a topic and vectorization; 3) topic analysis of the contexts in which innovation is used; 4) supervised classification process of select paragraphs related to the topic of innovation in new websites. This fourth step comprises two (2) sub-steps: 4.a) expanding the training corpus with a positive and unlabeled (PU) data classification; and 4.b) testing the model with a test data set. In step 1) pre-processing of text data, we apply classical Natural Language Processing (NLP) steps to prepare unstructured textual data for further analysis. Specifically, we execute morphological analysis and a lemmatization procedure for each word, then filter out stopwords so that only tokens corresponding to nouns, adjectives, and verbs that respect a document frequency threshold of 30 are kept. Next, we identify the most frequent bigrams of words to create a bag-of-words model. In step 2, we select text segments containing at least one of the words from a curated list representing innovation concepts. This results in a Document-Term Matrix with 69,826 segments and 3,879 unique unigrams and bigrams. In the third step, we use the Latent Dirichlet Allocation (LDA) approach to detect key topics across all segments containing on the innovation-related words. Next, a series of parameter tests occur that result in a model with eight (8) topics selected with a topic coherence score of 0.54. The LDA output consists of two matrices that allow us to analyze and interpret each topic and to create latent features for documents that feed the fourth step of our method. Step 4) consists of expanding the training set using supervised classification. The expanding method is based on the contextual modulation phenomenon involving conceptual expressions. This phenomenon describes the semantic process of the text segments where a concept is evoked, but without any standard lexical anchorage. Specifically, we formalized this task as a positive and unlabeled data (PU) classification problem. This type of method aims to expand the set of positive data from unlabeled data. Thus, we label those segments containing innovation-related words as the P dataset (positive dataset), and then we mark the unlabeled dataset U. Then we retrieve an equal number of labels that do not contain the identified words. Next, we apply the previously developed LDA model to retrieve feature vectors for the U dataset. Finally, we train models and select the best model using the F1-measure. This final step leads using the supervised training method on unseen websites to select organizations having more intensive innovation branding. With the help of agri-food experts, we then evaluate the results. Expected results We have implemented all but the final step in the method. We developed a significant topic model that encompasses the main subjects of our corpus well. In our first experiment, the model obtained an F1 score over 0.8. We anticipate that further iterations of the model will lead to improvement and that the model, programming, or methodology will be able to classify unseen websites to identify those organizations communicating about the identified topics, as well as those with higher intensity of such communications. Conclusions This study experiments with machine-learning and using unobtrusive methods to expand knowledge of innovation ecosystems. It provides a model to identify firms with higher innovation intensity in their communications. As next step, research into validation of the technique, for example conducting collaborative research with industry associations representing the main stakeholders of the Canadian agri-food ecosystem would be recommended.

Bibliography Antons, D., Grünwald, E., Cichy, P. et Salge, T. O. (2020). The application of text mining methods in innovation research: current state, evolution patterns, and development priorities. R&D Management, 50(3), 329‑351. Daas, P. J. H. et van der Doef, S. (2020). Detecting innovative companies via their website. Statistical Journal of the IAOS, 36(4), 1239‑1251. Gök, A., Waterworth, A. et Shapira, P. (2015). Use of web mining in studying innovation. Scientometrics, 102(1), 653‑671. Finco, A. (2018). Lessons of Innovation in the Agrifood Sector : Drivers of Innovativeness Performances. Lessons of Innovation in the Agrifood Sector : Drivers of Innovativeness Performances, 181‑192. Vlachopoulou, M., Ziakis, C., Vergidis, K. et Madas, M. (2021). Analyzing AgriFood-Tech e-Business Models. Sustainability, 13(10), 5516. Youtie, J., Ward, R.*, Shapira, P., Schillo, R. S., Earl, E.L. (2021). Exploring New approaches to understanding innovation ecosystems. Technology Analysis & Strategic Management, 7(Sept):1-5.

ABSTRACT. 1 Introduction

Artificial Intelligence (AI) technologies promise to revolutionize the knowledge production process. At the core of this recent AI revolution are machine learning (ML) algorithms: computer programs that improve performance as they are exposed to an increasing amount of data. An example of disruptive technology based on ML is AlphaFold – an AI algorithm developed by Google’s offshoot DeepMind, which solved one of the most challenging problems in the field of biology: the prediction of protein’s structures based on amino-acid sequences (Jumper et al., 2021; Callaway, 2020). This breakthrough and many others underpinned by developments in deep learning (DL), a subset of machine learning (ML) algorithms that relies on neural networks and manually classified data to solving problems (LeCun et al., 2015). Due to the extremely promising results in wide areas of application, DL has been regarded as a new method of invention and potentially a general-purpose technology in which the next industrial revolution maybe based (Crafts, 2021). Although a growing literature in Science and Innovation Studies has studied the impact of DL in the knowledge production process (Bianchini et al., 2022; Klinger et al., 2021), little attention has been given to its inception and the institutional context in which DL has emerged as a field of study.

It is widely known that the recent developments of DL are intimately linked to the Canadian Institute of Advanced Research (CIFAR), a research organization based in Toronto that finances basic research with a high-risk, high-reward philosophy. Since its foundation in the 1980s, CIFAR has been consistently interested in the advancement of AI and was at the forefront of the recent upsurge of ML technologies. A testament of CIFAR’s centrality in the DL field is the fact that two of the most used annotated datasets in DL, CIFAR-10 and CIFAR-100, are named after the Canadian institution. However, the exact nature of the role these datasets and the institution behind them in the blooming of DL as field of research has yet to be explored.

In this paper we analyse the emergence of deep learning as a technoscientific field, that is, a domain in the middle of scientific enquiry and technical problem-solving (Kastenhofer and MolyneuxHodgson, 2021). More specifically, we examine the role played by labelled datasets and the funding institute that supported the birth and growth of deep learning. Within this perspective, we regard annotated datasets as technological artifacts that allow the development of the field. Then we draw on the literature discussing the emergence of new scientific disciplines to provide a picture of the development of DL as the dominant approach in ML & AI, and the role of labelled datasets in that process.

2 Data and Methods

To answer these questions, we perform an analysis that includes both qualitative and quantitative elements. We start with a thorough review of the literature that encompasses the general conceptual framework and some specific historical accounts of the development of CIFAR. Several sources have been consulted both online and physically to better understand the nature and development of CIFAR as an important institution in the Canadian innovation system, and in particular the evolution of the work on AI through different stages.

The literature review has been complemented with semi-structured interviews with relevant actors, including prominent academics working on the field of AI and deep learning, as well as CIFAR personnel linked directly or indirectly to the creation of CIFAR-10/CIFAR-100. Two kinds of interviews were conducted: “general” interviews with academics working on AI, not necessarily related to CIFAR datasets, with the aim of getting an understanding of the field and some general features that practitioners might look for in a training dataset; and more “specific” interviews with strategic individuals that were directly or indirectly related to the development of the CIFAR datasets.

Finally, to better understand the reasons behind the continuous use of CIFAR-10/CIFAR-100, a survey was developed using Qualtrics and distributed to academics and practitioners who have used those datasets in their work. To find those who have used CIFAR datasets, a query was conducted using the Scopus database by Elsevier to find papers that contained CIFAR-10 or CIFAR-100 in their titles, abstracts or keywords. A total of 5,267 papers were identified in that search. Then, the authors of each paper were identified and individualized, to finally retrieve the email of the corresponding author. A total of 2,535 different emails were collected. The collected papers are then used in a bibliometric analysis focused on citations.

3 Findings

We find that CIFAR datasets were fundamental for the developments which lead to the DL revolution and still shape the trajectory of the field. Specifically, we identify CIFAR-10 as the most important technological artifact used to develop DL architectures. We trace the creation of these dataset to the CIFAR NCAP Summer School in 2008, where the labelling of the datasets was conducted mostly by graduate students over the supervision of George Hinton, a prominent scholar in the field. We also learned through our interviews that CIFAR-10 became a benchmark due to its technical specificities, namely the nature of the images, their size, the number of samples and categories. Moreover, the timing of its release was deemed crucial to the popularity of the datasets, considering that no other similar dataset was available at the period.

The survey confirms the insights taken from the interview and reveals an additional role CIFAR datasets played in the diffusion of deep learning methods. We reveal that CIFAR-10 and CIFAR-100 are used extensively in the training of computer scientists working with ML. Many researchers not only teach courses using CIFAR databases, but also were themselves exposed to the datasets while following graduate programs. This finding highlight teaching as an important channel through which CIFAR-10/CIFAR-100 impacts the field of DL. Finally, we learn from our bibliometric analysis that the CIFAR datasets are still relevant in the scientific production of developing countries, given its easy accesibility and low computational requirements. This suggests that the same technical characteristics that lead to the initial success of datasets are still driving research in DL to this day.

ABSTRACT. This study explores the use of keyword frequency from corporate websites to build innovation indicators of firms. We used the online data from 2,413 companies that participated in 29 questionnaire-based investigations between 2010 to 2016. We extracted the content of the corresponding websites via snapshots hosted on The Wayback machine and analyzed them based on keywords related to innovation concepts (innovation, collaboration, open innovation, R&D and IP). We built a nominal scale from the questionnaire-based data that categorize firms based on the level of evidence of innovation and on their level of time to market. Then, we produced multinomial logit regressions with the nominal scale indicator as dependent variable to test the contribution of our Web-based indicators to increase the relative risk ratios of being in a group of firms that shares similar evidence of innovation and time to market pattern versus other groups. Our preliminary results show that when one of the web-based indicators is high, our control group (no evidence of innovation while being in the bottom 20% in terms of time to market), does not see any significant positive increase in relative risk ratios when we compare it to any other groups. Other results are consistent with what one would expect from groups with greater evidence of innovation, which show a significant positive increase in relative risk ratios when we compare them to the other lower groups. However, some other significant results seem at first glance to be inconsistent with what would be expected. Indeed, this raises the question of whether we are capturing patterns of market signaling from the mid-lower tier while observing the absence of the need to perform any form of market signaling from the companies in the upper tier. After all, actions still speak louder than words it seems.

ABSTRACT. Background and rationale Firms engage with a variety of practices to disclose knowledge resulting from their R&D efforts. Although these disclosures can lead to unintended knowledge spillovers and hinder a firm from fully capturing the benefits of R&D, they also enable the firm to accrue a range of benefits – e.g. reputation building, gaining access to external knowledge and financial resources, establishing intellectual property rights – that contribute to the firm’s innovative performance. Scholars have extensively examined patenting and publishing as forms of knowledge disclosure and their impact on innovation (e.g. Alexy et al., 2013; Arora et al., 2018; Hicks, 1995; Rotolo et al., 2022), and more recently, the extent to which these forms of disclosures are substitutes (Blind et al., 2022). Yet, our understanding of how firms disclose knowledge through their websites and how this channel of disclosure relate to other channels of disclosure remains scant.

In line with recent efforts aimed at expanding our understanding of the innovation process on the basis of novel text-based indicators (Bellstam e tal., 2021; Gatchev et al., 2022), in the paper, we examine firms’ behaviour in disclosing and signalling their R&D activities and strategies on their websites, and compare these disclosures with disclosures firms make in patents and publications. To do so, we build a large-scale textual dataset derived from the website pages of a sample of firms. We employ Natural Language Processing (NLP) and topic modelling to map firms’ knowledge activities in term of topics as reported on their websites.

Methods We piloted the study on a dataset that combines publication and patents data related to a sample of 9000 UK firms with textual data extracted from the websites of these firms in the year 2020. We employed a text mining-based approach that uses firms’ textual data at scale to delineate topics from website data and to match these with the well-established Microsoft Academic Graph (MAG) Field of Study (FOS) model. The MAG is the largest publicly available dataset of scholarly publications and the largest dataset of open citation data (Shen, Ma, & Wang, 2018). MAG data models scholarly communication activities which consist of six types of entities – publications, authors, institutions (affiliations), venues (journals and conferences), fields of study and events (specific conference instances) ¬– and the relations between these entities – e.g. citations, co-authorship. The relations between the entities are described in more detail in (Sinha et al., 2015). The FOS classification can be conceived as one of the broadest classification systems for knowledge transparently available for use. FOS are the results of a hierarchical topic model run on the entire MAG data corpus. More precisely, the hierarchical topic model produces unique FOS identifiers (ids) by creating a hierarchy of five levels (about 700,000 topics).

Extant research has shown the potential of website data in creating valuable information on innovative activity at a firm-level (Bellstam e tal., 2021; Gatchev et al., 2022). For example, website data can be used to better understand innovation outcomes, strategies, and relationships (Gök et al., 2015). Running the analysis at scale also allows drawing industry-wide structures from the data. Ashouri et al. (2021) used website data from 96,921 medium-high and high-technology firms to create a model of industry structure Our study leverages the framework presented in Ashouri et al. (2021) to devise a transparent approach to creating a classification: we infer a classification to firms’ website data using NLP and the hierarchical topic model-based MAG FOS categories.

Utilizing the firms’ URLs with an automated website scrapping system, the textual content of firms’ websites was retrieved – the approach is described in detail in Ashouri et al. (2021). The data platform uses the capacities built in BIGPROD1 project [www.bigprod.eu]. For retrieving and hosting the raw data, a ‘hybrid’ design was adopted, with part of the infrastructure being located on-site premises and the other part in the cloud (MS Azure Cloud Platform). Raw text collected from firms’ websites, web scrapping task and MAG publications’ text required cleaning and harmonization. Therefore, pre-processing steps involved cleaning procedures (e.g. removal of stop words and non-alphanumeric characters, stemming and lowercase transformation) applied to harmonize and increase the consistency of the text. For NLP to work, the natural language (text) needs to be transformed into a numerical vector form. Text vectorization techniques, namely tf-idf, Bag of Words and vectorization, are very popular choices for machine learning algorithms, can help convert text to numeric feature vectors. Therefore, to quantify and convert text into numerical representation in documents, we compute a weight to each phrase that signifies the importance of the phrase in the document and corpus. The tf-idf approach is a transformation applied to texts to get vector representation of vocabularies. Then it is possible to obtain the similarity of any pair of vectors to a quantified measurement.

Assigning FOS is now possible to text representations coming from firms’ websites. The delineation of these categories on firms’ website data and the descriptions of firms’ products is very granular given the large range of FOS categories, the values are standardized descriptions of what a company is doing.  Because these FOS categories are hierarchically linked, it is possible to easily assess how similar any pair of FOS categories are to each other. This can be used to go from low-level specific descriptors to high-level categories. In this study, we used the 100 most popular FOS ids. For each of the firms in the sample, their website was scraped and indexed using a pre-determined vocabulary. A weighted vector of vocabulary codes was then used to assign a vector of FOS ids to each of the websites. This process resulted in a weighted vector for each firm in the sample: a vector represents a firm’s website content using FOS ids and their associated weights.

Expected results and implications We expect that the study will shed light on the extent to which firms engage in knowledge disclosures on their websites as revealed by the emergence of topics from website textual data. This will enable us to delineate different types of firms on the basis of their disclosure behaviour as well as to increase our understanding of how website disclosures may relate to other forms of disclosures such as patenting and publishing in terms content (e.g. what topics are disclosed on websites that are/are not disclosed in patents and publications). In this regard, by comparing the R&D topical signals received from firms’ website content with the ones collected from patent and publication data, we can isolate firms’ behaviour in communicating their activities to their stakeholders and examine the dynamics associated with this phenomenon (e.g. what topics are disclosed on websites before being disclosed in patents and publications). We expect the study to contribute to two main streams of research in innovation studies literature, i.e. research examining firms’ knowledge disclosures and the role that these have in the innovation process, and research developing novel measures and indicators of innovation on the basis of the increasing availability of text data.

ABSTRACT. Background and rationale Basic research is the catalyst for promoting the growth of high-tech firms that are the main driving force of technological innovation. As society progresses, the link between science and technology gets closer. The development of high-tech industries has an increasing demand for original knowledge and innovationse, making basic research even more important nowadays. Therefore, understanding how basic research influences the development of firms becomes a significant issue. Identifying and measuring basic research accurately is fundamental to investigating its influencing mechanism. However, it seems that a standard measurement method has not yet been determined, because of the abstraction of the concept basic research. For now, there still remains limitations in existing methods in current empirical research. For example, most methods are either indirect to basic research or too coarse-grained to measure the basic research intensity. Such measures include overall R&D expenses and the number of scientific papers, which would lead to the inaccuracy and even incorrectness of findings and policy suggestions. High-tech firms, as one of the main bodies in research investment, assume important responsibilities in R&D activities. Encouraging firms to invest in basic research is a crucial means for a country to enhance the competitiveness of science and technology. Therefore, it is particularly important to study firm’s basic research, especially whne the empirical research on this topic is still lacking. Biomedicine is a research-intensive industry in both China and abroad. Its progress and development is closely related to basic research, especially the high-level basic research. In this study, we measure the basic research in biomedical firms by applying an indicator, Level Score, that can quantitatively reflect the basicness of an individual paper.

Method “Level Score”, a quantitative index that Ke (2019) proposed, can well measure the basic level of a research paper. By establishing the occurrence network of MeSH terms and using the LINE network representation learning algorithm, we can obtain the translational axis in a vector space, pointing from the center of basic type MeSH terms to the center of applied type MeSH terms. Then we can calculate the appliedness score of each MeSH term, which is the cosine similarity between the translational axis and the MeSH term. The Level Score of a research paper is the average appliedness score of all its MeSH terms, ranging from -1 to 1. The smaller the score is, the more basic the research paper is. Based on this indicator, we then quantify the basic research of biomedical firms in the biomedicine industry. We derive a list of all biomedical firms from 2015 to 2017 according to the Standard Industrial Classification (SIC) from Compustat database. Using the method of keyword matching, more than 20,000 firm papers published from 2015 to 2017 are successfully identified from MEDLINE database. Their reference papers are obtained from Microsoft Academic Graph (MAG) using doi as an intermediary. We then calculate the Level Score of all firm papers and their reference papers. We observe the distribution of the basic level of firm papers and their references, investigate the dependence between firm’s basic input and output, and come to some valuable conclusions.

Results The preliminary results of our study show that: (1) In the biomedical industry, there is an obvious binary distribution of "basic research-applied research" in firm papers. That is, firms clearly publish scientific papers of both basic research and applied research. In the meanwhile, papers’ basic level increases along the path of “MEDLINE papers - firm papers - firm reference papers”, revealing a hierarchical supporting mechanism of basic research. (2) Biomedical firms adopt diversified R&D strategies. There is also a binary division of "basic-applied" in firm’s research preference. But this reference deflects more to basic research, as basic type papers accounts for 60%-80% in most biomedical firms. Further, the R&D strategy adopted by biomedical firms is related with firm size. Our results show that, small and medium-sized firms are more likely to choose only one of the research type, a relatively extreme strategy. Yet large firms appear to be more conservative, carrying out both basic and applied research at the same time. (3) Basic research is important for biomedical firms in terms of the relationship between research input and output. We find that biomedical firms cite more papers of basic research more than those of applied attributes, and that some firms even only cite basic papers. Regarding references as a kind of knowledge input and papers as an output, we find that the basic type knowledge input can produce both basic and applied output. But it is more difficult to obtain basic type output from applied input than to get applied output from basic input.

Significance First, by applying the Level Score index in biomedicine industry, we further verify its feasibility in measuring basic research in this study. According to the results and the conclusions, we believe that this index has the potential to be further promoted in both theoretical and empirical research of science and technology assessment. In this way, our study can provide new ideas for the relevant empirical research in the future in terms of measuring basic research. Second, we find some interesting preliminary results regarding the status and rules of basic research in biomedical firms, through measuring the basic level of more than 20,000 firm papers and more than 1 million reference papers. The results of our study reflect the important role that basic research plays in the development of biomedical firms. The conclusions will help improve the measurement methods of basic research and provide more empirical evidence for the field of biomedicine. Third, based on the results of our study, we tentatively propose some suggestions for firm basic research. Theoretical support will be provided for government to formulate relevant policies. An implications will also be provided for firms to strengthen the efforts in basic research to improve R&D and innovation capabilities, which is of great significance to achieve the long-term development of firms.

ABSTRACT. One of the roles of federally-funded R&D, beyond its primary goal of advancing technologies critical to US security, is to fuel economic growth and US competitiveness. Thus, a recent cause for alarm (e.g., Arora, Belenzon and Patacconi 2018, Gruber and Johnson 2019) is the 67% decline in federal R&D. Not surprisingly, this decline corresponds to declines in US R&D productivity and nominal GDP growth. One policy response is the Chips and Science Act of 2022, which authorizes approximately $174 billion through FY 2027 to support the nation’s science and technology base, 75% of which is for Research (R). Thus the Act responds to a common misperception that the decline in federal R&D has come from R. In fact, federally-funded R has maintained a constant share of GDP. Essentially the entire 67% decline in federal R&D has come from development (D). Because R goes principally to universities, while D goes principally to firms, universities’ share of federal R&D has grown sevenfold (from 5% to 35%), while firms’ share has declined 70% (from 71% to 22%). This shift is curious, since at least one study found that federal R&D contracted to industry stimulated more private R&D investment than other federal R&D (Levy and Terleckyj 1983). Moreover, the same study found that federal R&D contracted to industry increased labor productivity, while other federal R&D decreased labor productivity. Such dramatic shifts and their coincidence with declining R&D productivity suggest the former may be responsible for the latter. If so, a likely explanation for declining R&D productivity is that federal R&D enhances firm R&D productivity. Such a finding might come as a surprise, since a common conception of federal contractors is that they are less innovative than commercial firms (Gansler 2013, Srinivasta 2019, Fischetti 2020). One reason federal contractors might be less innovative is that they evolve to thrive in the highly idiosyncratic and bureaucratic federal procurement system (e.g., Josephson et al. 2019). A second reason is that they may substitute lobbying and influence expertise (necessary to secure government contracts), for innovative expertise (necessary to succeed in the market). While it is obvious that government contracting is cumbersome, it is less obvious the impact this has on innovation. As a shining counter-example, Operation Warp Speed (OWS) represents federally-funded innovation at its best. In nine short months, beginning late March 2020, the government put out a call for vaccines, evaluated 100 responses, funded development and manufacturing expansion for six candidates, and administered the first vaccine December 13. Nor is OWS an isolated example. It is well known, for example, that the Internet originated with ARPAnet, but federal R&D also led to other important general-purpose technologies such as lasers, MRI and GPS (Singer 2014). Accordingly, the net impact of federal R&D contracting on firm innovation is an empirical question. While prior research has examined this question, and concluded that the productivity of federal R&D was a small fraction of that for firms own internal R&D (Griliches 1986), the data was cross-sectional, and pre-dated the decline in federal R&D funding. Accordingly, it may have captured a period when the federal government was overinvesting in development. We re-examine the question of federal R&D contracts on innovation by comparing federal contractors with commercial firms. Our measure of innovation is firms’ Research Quotient (RQ), defined as the firm-specific output elasticity of R&D. Accordingly, RQ captures the percentage increase in firm revenues associated with a 1% increase in R&D. In aggregate, RQ captures the contribution of industrial R&D to economic growth (one of the goals of federal R&D). To conduct our test, we first characterize RQ separately for firms with and without federal R&D contracts. While the RQs of both type firms have been decreasing, those with federal R&D contracts have higher RQ throughout. This is true after adjusting firms’ financials to treat R&D contract dollars as an R&D input, rather than as revenues. We next examine whether the higher RQ of federal R&D contractors reflects a selection effect or a treatment effect through a difference in differences (DiD) estimation relative to the time of first federal R&D contract. Our results reveal that the higher RQ of federal R&D contractors reflects both treatment and selection. Federal R&D contractors have significantly higher RQ prior to first contract, and RQ increases significantly following the first contract. The question then turns to why federal R&D contracts increase firms’ RQ. One plausible explanation is that the government issues R&D contracts to firms from whom it expects to procure the ultimate good/service. If so, then RQ is mechanically higher for these firms—increases in R&D precede planned increases in revenue from procurement. We test this by exploiting a shock to federal funding—the Budget Control Act of 2011 (BCA), also known as sequestration, which dramatically decreased federal spending. Using dynamic DiD estimation, we find that while the RQ of federal R&D contractors decreases following sequestration, the decrease is not-significant. Thus, the higher RQ of federal contractors is not merely an artifact of the R&D being an early stage of a longer-run contract. We therefore explore whether federal contracts change the character of firms’ R&D in ways that enhance its productivity. We look at two dimensions of R&D character, its expansiveness and its impact. Both measures are based on patents, thus we can’t test the 50% of firms who choose not to patent their innovations. Nevertheless, for the subset of firms who do file patents, we find that the R&D of federal contractors is more expansive and impactful than that of commercial firms. Taken together our results indicate that federal R&D contractors are at least as innovative as their commercial counterparts. Thus, the benefits of federal R&D contracting appear to outweigh the bureaucratic costs they impose. These benefits reflect both selection and treatment effects: government contractors have higher RQ prior to their first federal R&D contract, but in addition their RQ increases after receiving these contracts. The higher RQ of federal contractors cannot be explained by the fact these contracts are pre-cursors to subsequent procurement contracts. Rather it appears that federal R&D contracts enhance the character of firms’ R&D—making it more expansive and impactful. One reason this may be true is that the funding agencies provide a conduit from research at universities and government labs to firms who commercialize those innovations. In doing so, these contracts help solve the “valley of death” problem that few university and lab inventions are commercialized. Our results have implications for policy. First, we find no evidence that greater use of commercial firms would increase the impact of federal R&D. In fact, we find that federal R&D contractors are at least as innovative as their commercial counterparts. More importantly, our results suggest the decline in federally-funded R&D to companies likely contributed to the decline in US R&D productivity. By extension, the rise in funding to universities appears not to have generated any benefit. Perhaps this is because the shift in from D at firms to R at universities has produced an imbalance, such that research generated by universities now exceeds industrial capacity to commercialize it.

ABSTRACT. Introduction:

Innovation clusters are often known as innovation ecosystems and have a significant impact on the German economy and regional development. Different federal states already have placed various cluster policies to support the cluster initiative. These policies are designed to congregate diverse actors (e.g., SMEs, large companies, research institutes, and regional institutions) with similar interests in particular demographic proximity [1]. Moreover, clusters with excellence in energy topics also lead territorial energy-transition and support multiple actors to develop rapid and scalable innovations. Seen in this light it became a natural interest for policymakers, investors, and firms to analyze the ongoing activities of leading energy clusters. Such analysis could give a broader overview of current trends and the mutual interest of a diverse set of stakeholders. In this context, the present work considers analyzing energy clusters with different perspectives such as geographical scope, members companies, and focus topics. A total of 44 energy clusters along with 4524 members are taken into account in this study.

Background and Context:

In the literature, the theory of innovation clusters has been widely studied by various scholars. There are different approaches have been developed to analyze an innovation cluster. These methods include namely qualitative analysis, interviews, patents, etc. Nevertheless, the majority of approaches rely on either controlled input (e.g., cluster manager interview) or a lack of updated activities (e.g., patent analysis) [2]. Along these dimensions, previous studies were unable to provide ground activities of energy clusters and lacked providing the real impact of various energy cluster policies. Following the discussion above the present work has tried to answer the following research questions.

Research questions:

1-Who are the major actors involved in setting up an energy innovation cluster and how these clusters are organized to meet the goal of various participants? 2-How unstructured data from the cluster and their member's websites can be used to gain insight into recent innovative activities in the energy sector?

From the theoretical point of the view, the work has provided a comprehensive overview of major cluster policies in different federal states of Germany.

Methodology:

Over time website data is thriving and is considered as an essential source to measure emerging innovative actions. In this context, the energy-cluster website contains the most updated about cluster activities such as ongoing activities, events, projects, and publications. Therefore, the proposed methodology primarily banks on the website data. As an initial step, a list of energy clusters is obtained from the Cluster platform Germany [3]. A total of 44 energy clusters along with 4524 members are taken into account in this study. Furthermore, an internally developed web-scraping tool is used to crawl the cluster and member's data. The tool performs systematic and guided web-scraping for searching a keyword presence on the particular web-page. Two distinct sources namely European Structural and Investment Funds (ESIF) and Scival platform are used to mine energy-innovation related keywords. Different categories and sub-categories are assigned to cover a wide range of topics and actions related to major energy fields. Moreover, a commercially available company database is used for matching firms in the member's list. Python based natural language toolkit (NLTK) is used to clean the data and different analytics such as co-relation analysis of topics, decile ranking, and frequency analysis is done by utilizing text data.

Main outcomes: The main result highlights are summarized below:

• The work has analyzed the major actors driving cleantech innovation clusters in Germany. To this end, the composition of different clusters has been analyzed. Similar to innovation clusters active in other fields, the key members in an energy cluster are firms (Large, Medium, Small, and Startups), research institutes, universities, public organizations, and financing actors. However, cluster composition in the energy sector is also populated with several other actors (e.g., Training centers, Innovation matchmakers, co-working facilities, innovation consulting, etc.) and individuals.

• The overall results have indicated that the majority of energy clusters are very specialized in certain topics such as hydrogen, carbon, and bioenergy and are getting notable attention from various stakeholders. In addition, various cross-sectoral topics (e.g., bio-hydrogen, Organic solar cells) are also emerging due to the growing interaction between different sectors.

• Energy clusters and their member count is unevenly distributed in Germany. Therefore, new policy measures are required to create a supportive framework for the regions with relatively lower participation.

References: 1- G. M. Z. Koecker, Clusters in Germany - an empirical based insight view on emergence, financing, management and competitiveness of the most innovative clusters in Germany, Report (04 2008). 2- M. Tvaronaviciene, K. Razminiene˙, L. Piccinetti, Aproaches towards cluster analysis, ECONOMICS & SOCIOLOGY 8 (2015) 19–27. doi:10.14254/2071-789X.2015/8-1/2. 3-Cluster platform Germany. https://www.clusterplattform.de/CLUSTER/Navigation/EN/Service/Glossary/glossary.html (accessed in November, 2021).

ABSTRACT. Background and Rationale: Accelerating technological innovation is essential for meeting long-term decarbonization goals and for supporting economic development and employment by expanding emerging green industries. Currently mature technologies will meet only a quarter of long-term sustainable energy goals according to the International Energy Agency. Addressing this challenge therefore requires ensuring that new climate technologies scale up rapidly and can be deployed cost effectively. Energy innovation systems research suggests that various actors—especially governments, start-ups, large firms, and private investors—and their interactions will be involved in scaling up innovation to a commercially ready product. However, currently, there is no systematic approach for connecting empirical data on commercially viable, early-stage technologies and the various actors involved, with systems models that investigate large-scale deployment and inform energy and climate policy decisions.

Standard methods of developing learning curves require a track record of deployment that new technologies lack, and expert elicitations are resource-intensive and may be less amenable to analyzing commercial breakthroughs. Linking realistic, bottom-up empirical evidence on technologies such as start-up investment data, project databases, and historical technology analogues with macro-level energy systems models can help overcome these difficulties to inform scaling trajectories for emerging technologies. Here, we propose a framework for integrating such granular real-world data into an open-source integrated assessment model, i.e., the Global Change Assessment Model (GCAM), and use direct air capture and aviation as examples of how this framework can support analysis of technology development and just energy transitions.

Methods: The technologies selected are based on the following criteria: (a) technologies in early development stages that will be important for deep decarbonization pathways (b) technologies developed by start-ups with substantial investment records, particularly from private investors (especially strategically and financially motivated corporations), (c) technologies with new model developments in GCAM (often related to previously understudied areas, indicating a strong potential for high-impact results).

We examine two case studies. Our assessment of direct air capture (DAC) technologies draws on publicly available investment data on commercial purchase agreements for carbon dioxide removal, as reported by Stripe and www.cdr.fyi, and plant capture rates from IEA project data and announcements from company websites. Our examination of aviation utilizes ICAO records of sustainable aviation fuel (SAF) offtake agreements and novel aircraft design timelines to investigate decarbonization trajectories in the US and EU that would limit the need for offsetting emissions in net-zero pathways.

We use these two cases to estimate variations in technology diffusion through projected changes in the S-curve. An S-curve has three degrees of freedom: the inflection point, the growth rate, and the maximum value. The inflection point and growth rate are derived from historical technology analogues by fitting an S-curve to historical capacity data for those technologies. The maximum value is calculated using investment data and publicly available purchase agreements or plant operation announcements.

We implement the changes in S-curves in GCAM. GCAM is a technology rich model with detailed regional representation of energy production, transformation, distribution, and end use. GCAM projects systems outcomes through 2100 at five-year intervals, including energy supply and demand by technology and vintage, regional prices for goods and services (e.g., electricity, agricultural commodities, etc.), emissions of greenhouse gasses and short-lived species, and climate outcomes. For each five-year time step, GCAM solves for a set of market prices such that supply equals demand for all markets and sectors and across all regions (32 geopolitical regions, 384 land-water regions).

In the DAC case study, we use purchase agreements and plant announcements for 16 DAC start-ups which report the amount of carbon a company plans to capture in the near future. Assuming a start year of 2010, the first year of operation for a DAC plant according to IEA, we project the next ninety years of DAC assuming a growth rate and inflection year identical to different historical analogues. We specify the exact capacity of DAC for which GCAM should solve at every five-year time step, using the S-curve parametrization calculated with historical technology analogues and near-term DAC projections.

For aviation, we use global data on SAF offtake agreements through 2035 to project scaling pathways for SAF using a similar approach with historical analogues. We also use granular data on novel aircraft deployment timelines to determine the potential speed of a transition to electric and hydrogen fueled planes, accounting for fleet turnover rates. We specify these pathways in GCAM to determine how these technology innovations can contribute to 2050 net-zero goals.

Anticipated Results: Preliminary results suggest that, if DAC adoption follows historical analogues, the timing, growth, and market saturation of DAC technologies differ significantly from what is typically observed in models that currently inform global, national, and sub-national climate policy. The median logistic growth of technologies in our dataset (measured in cumulative capacity) is 17%, but with a wide range of growth (2% - 200%). This result suggests that technologies grow at vastly different rates, impacting the cumulative capacity of the technology as well as the time it takes for the technology to scale up to its eventual steady state.

Preliminary analysis of aviation in the US and EU suggest that the sector can deliver emissions reductions that limit the need for offsetting in net-zero pathways. Full decarbonization of the aviation sector will likely place additional burdens on biofuel production and scaling of clean electricity generation uncless coupled with demand reduction policies such as mode shifting.

Significance To reach a net-zero world, rapid development and deployment of pre-commercial technologies is essential. In order for researchers and policymakers to accurately assess the potential contributions of these technologies to decarbonization pathways, new methods of analysis are needed to determine realistic scaling pathways. Here, we demonstrate two case studies of how such analysis can be developed, and the resulting implications for ambition for carbon dioxide removal, negative emissions, and decarbonization of the aviation sector.

ABSTRACT. *Background and Rationale* Spillovers of knowledge across technology domains are one of the key drivers of technological innovation, as knowledge initially developed in one technology area can be applied in another area to enable discovery and invention, reduce technology costs, and improve technology performance. Contributions of technology spillovers to clean energy innovation are expected to play a major role in the global push towards cost-effective decarbonization of the energy system. Despite the importance of technology spillovers, we have limited understanding of the micro-level processes of how knowledge spillovers across technology domains occur, what factors enable or affect spillovers, or how spillovers can be most effectively spurred and leveraged by public policies targeting the decarbonization of the energy sector.

*Methods* We have undertaken a multi-year mixed-methods investigation of technology spillovers in three important clean energy technologies: solar photovoltaics (PV), lithium-ion batteries (LIB), and solid-state lighting (SSL) (specifically, white light-emitting diodes (LED)). We use process tracing to combine and integrate quantitative and qualitative empirical evidence from a broad survey of the scholarly literature and primary documents, expert interviews, analysis of citations in patents and scientific publications, and a machine learning-based method of patent text analytics to understand the mechanisms and enablers of technology spillovers. Using these methods, we reconstruct the historical contributions of spillovers to individual innovations in the three technology areas, and then develop an inductive generalized typology of technology spillover mechanisms and enabling factors.

We find evidence of crucial knowledge contributions made by technology spillovers to 15 innovations in solar PV, 12 innovations in LIB, and 9 innovations in white LED-based SSL. In all three domains, identified spillovers were particularly important at the early stages of innovation, as they enabled key components and manufacturing processes that eventually became integral parts of the first commercial LIB and white LED products and the solar cell design that dominated the PV market for several decades. However, the impact of spillovers is not limited only to the stages of early research and development (R&D). We also find evidence of spillovers occurring at later stages of technology demonstration and market formation. Across the three areas, spillovers contributed to innovations in technology components, materials, system architecture, and manufacturing processes. In addition, spillovers drove a majority of improvements in consumer experience characteristics of white LED-based lighting products.

*Results* Based on the analysis of identified spillover processes in three technologies, we identify four types of “spillover mechanisms,” different ways by which a spillover can occur: (1) learning and researching, (2) communication and collaboration, (3) human mobility (both physical and across disciplines or fields), (4) exchange of physical objects that embed knowledge, such as manufacturing equipment. Importantly, these four mechanisms are not mutually exclusive, as a spillover can operate through several simultaneously occurring and supporting mechanisms. Observed spillovers also differ in how intentional they were. In many cases, spillovers occurred as a result of a targeted search for external knowledge needed to solve a local problem or targeted application of available knowledge in a new area or for a new purpose. However, a few notable spillover cases also occurred serendipitously in the process of undirected “blue-sky” research.

We identify many broad categories of enabling factors for technology spillovers, including, but not limited to, external sectoral and market shocks; booms of R&D in external but related technology domains; freedom of search in the laboratory setting, both in academia and industry; multidisciplinary education and training; cross-disciplinary hiring and team composition; multi-sectoral firms; and academic-government-industry partnerships and knowledge exchange events.

Finally, we highlight four categories of public policies that enable spillovers: (1) public R&D funding that stimulates cross-disciplinary knowledge search and collaboration; (2) public funding for technology demonstration projects; (3) deployment policies that create incentives for strategic entrepreneurial knowledge search by incumbent firms in other industries; and (4) cross-cutting policies that support cross-sectoral and systemic coordination, such as government-industry round tables and roadmapping exercises. We also find that “stop-go” funding cycles negatively affect spillovers into clean energy technologies, potentially delaying resulting innovation and technology deployment.

*Significance for Policy* Based on our findings, we propose a set of five principles that can guide the design of energy and innovation policies and management practices that leverage technology spillovers to accelerate clean energy innovation.

First, for any organization, firm, or funding agency, there is a need to recognize and acknowledge the trade-offs that exist in supporting innovation activities in a focal knowledge domain against supporting a broader multi-disciplinary knowledge base involving multiple knowledge domains. Access to a broad pool of knowledge can accelerate innovation in the focal domain through technology spillovers, but it may not work on its own without a deep understanding of the focal domain and the ways by which external spillover knowledge can be applied in it. These trade-offs suggest that there is a balance to be found between the depth and breadth of knowledge search in an organization, and that this balance can be proactively pursued and managed.

Second, policies that aim to leverage technology spillovers for innovation should be flexible, both in the choice of particular policy and funding instruments and in policy design, to allow for knowledge search in unexpected directions. Some of the most notable spillovers in our study occurred when researchers were able to pursue “blue-sky” research or worked in public- or industry-funded mission-oriented R&D while still being allowed a certain freedom of search.

Third, we find that continuous knowledge exchange between science, engineering and manufacturing is important for the generation of spillovers, which often requires deliberate management of boundaries within organizations and disciplines to alternatively nurture development of knowledge within an organization or discipline and cross-pollinate with external knowledge across the stages of innovation.

Fourth, cross-disciplinary, cross-sectoral exchange should be supported at all organizational levels: from individuals (e.g., through temporary or extended placements of public sector and university researchers in industry) and events (e.g., conferences mixing scientists, engineers, and industry representatives) to teams (e.g., multidisciplinary team composition), organizations (e.g., through academia-industry collaborations and alliances) and platforms for broad cross-sectoral and cross-disciplinary collaboration (e.g., industry roundtables, roadmapping, and visioning or foresight visioning exercises).

Finally, all instruments and activities supporting technology spillovers should not be restricted only to R&D policy. They should also be included in the policy mix for the support of innovation at all stages, including technology demonstration and market deployment. This principle is particularly relevant for innovation in clean energy technologies, in which deployment policies are known to have played a crucial role in stimulating dramatic cost reductions and technology deployment over time and are expected to continue playing this role in the future.

ABSTRACT. Background and rationale

For Germany, the transformation of the energy system has become more urgent than ever. The necessary acceleration of climate protection and the gas supply shortage since the Russian war in Ukraine have initiated a reorientation of the German energy transition as a mission-oriented innovation policy strategy. Its objectives consist of a triangular between environmental objectives such as emission reduction and renewable energy diffusion, affordability and security of supply. However, recent developments have increased the importance of technological sovereignty and strategic industrial policy as additional policy objectives. In this situation, the German Ministry of Economic Affairs and Climate Protection aims at reconceptualizing its Energy Research Program until 2023. In a research project funded by the ministry, our team receives the opportunity to accompany this reconceptualization. The ministry's objective is to make use of a better operationalization of the overall socio-technical objectives of the mission-oriented energy transition. Moreover, the Energy Research Program shall be better coordinated with other stakeholders in the innovation policy system and other policies. These interfaces to different stakeholders and policy instruments need to be identified and explicitly addressed in a new governance concept of the program. For example, missing interfaces need to be discussed from both a theoretical and a practicability viewpoint for better coordination of how successful projects from the program can receive further support via regulatory or demand-sided instruments. In that way, the new Energy Research Program shall be embedded more narrowly to the mission - and guarantee better coordination with other existing innovation policy and energy policy instruments.

Research questions and objectives

The recently initiated research project has the opportunity to accompany innovation policy design processes in real-time to dismantle the processes of designing and coordinating policy instruments. The insights into the reorientation process help explore innovation policy from a practitioner's perspective, which needs to be included in conceptualizing policy recommendations. The project pursues two objectives. First, it aims to translate the existing knowledge from the literature and our previous research on an agile, mission-oriented science and innovation policy (Weber et al., 2021) to the German Energy Research Program's case. Thus, suitable and practicable policy recommendations for a new governance concept and possible instruments shall be developed and simultaneously discussed with the relevant policy actors. Second, the project aims to accompany the debate and the implementation process for the new Energy Research Program to explore what challenges practitioners need to deal with during the policy design and implementation process. Therefore, the project addresses the following research questions: • Which policy instruments and governance processes are part of the German Energy Research Program? • How can new research on an agile, mission-oriented innovation policy be conceptualized into practicable policy recommendations for the reconceptualization of the Energy Research Program? • What organizational capabilities are required for the concerned policy actors (ministries or project operators) to support the design and implementation processes and learning cycles within the Energy Research Program? The first research question uses previous research on the analysis of innovation policy instruments and their design principles to structure existing activities within the Energy Research Program. Such a structured view is missing so far. Based on the literature, the mapping of policy instruments and governance processes helps to find starting points for proposing policy recommendations. With the term governance processes, our project refers to the processes how instruments such as calls for projects address specific socio-technical needs of the energy transition and how they are designed in detail. Governance processes also include strategic adjustments of instruments along specific socio-technical needs. Moreover, these processes target the capability to select successful projects for further support or other policy instruments aiming at increasing the production or diffusion of the successful technology. Governance processes include the monitoring of key performance indicators and evaluation activities to adjust instruments and governance processes. The second question focuses on the translation of existing literature to the case of the Energy Research Program. The third question examines the implementation challenges for policy actors' mission-oriented instruments and governance processes.

Methods

The research project is based on a qualitative case study design using different data sources. It consists of two phases. First, the mapping of instruments and the governance processes is conducted based on a structured review of public documents, the existing research literature and participating observations during meetings and workshops in the ministry. Simultaneously, a structured literature review on an agile, mission-oriented and transformative innovation policy, with a particular focus on the energy transition, is conducted. As a result of the mapping and the literature review, a proposal of possible mission-oriented governance processes will be elaborated and discussed in workshops with the relevant actors and stakeholders in the ministry. This includes the challenge to actively address the interfaces of the Energy Research Program with other ministries and instruments within the energy transition innovation policy mix. In the second phase, we accompany the implementation of the new Energy Research Program via semi-structured expert interviews in the relevant departments of the ministry, the project operators and other relevant stakeholders. Thus, we may refine the conceptualization of mission orientation in its daily policy implementation processes. The interviews, the participating observations and the public documents on the Energy Research Program represent the empirical base of the qualitative case study, which is analyzed in an iterative coding process using MAXQDA. As the second phase is supposed to begin in March 2023, we will present the proposal for an agile, mission-oriented energy research program and preliminary empirical findings on its implementation.

Preliminary results and significance

Our previous research on agile innovation policies suggests that mission-serving programs based on several instruments and governance processes require the connection between strategic objectives and the operative selection and implementation of policy instruments and their design features (Weber et al., 2021). The proposed project explores the connection between the strategic and operative levels of policy-making by considering how strategic objectives are operationalized in the Energy Research Program, for example, in the form of calls for projects. In addition, the identified interfaces to the concerned policy actors and to other instruments and programs of the German energy transition policy mix help to understand how coordination at the strategic and operative level of innovation policy-making looks like. Moreover, previous research has highlighted the problem that often new instruments are implemented without considering if existing instruments can cope with new challenges. As a result, programs and policy mixes become far more complicated than necessary (Flanagan et al., 2011). By explicitly identifying the instruments and design features of the program, we can analyze unnecessary policy path dependencies and discuss the option to replace or abolish instruments. Furthermore, implementing mission-oriented policy concepts on former purely supply-sided instruments such as direct project funding programs represents a new field because mission-oriented policy research often focuses on demand-sided measures for accelerating the diffusion of innovations. The proposed project fills this gap by using the long experience of this supply-sided program and by detecting the interfaces to other instruments within the mission-oriented innovation policy mix.

References

Flanagan et al. (2011): Reconceptualising the 'policy mix' for innovation. Research Policy 40(5), 702-713. Weber et al. (2021): Agilität in der F&I-Politik. Konzept, Definition, Operationalisierung. Study on the German Innovation System 8-21 for the German Expert Commission for Research and Innovation.

ABSTRACT. 1. Background and Rationale

Climate change is a grand challenge of our era. From an economic point of view, government intervention is justified to address two market failures: one environmental and one technological. The former results from an overprovision of electricity from fossil fuels, while the latter from an underprovision of research to develop greener technologies. Environmental technological policy must jointly address both market failures due to their interconnections. On the one hand, the standard approach to climate policy has been to limit the emission of greenhouse gases (GHG). On the other hand, the adoption of greener technologies can reduce or eliminate GHG emissions.

This paper focuses on the development of one of such greener technologies: nuclear fusion, which is the reaction that occurs within the stars, including the Sun, to generate their own energy. A technological breakthrough in nuclear fusion energy can help to decarbonize the power system because it can provide clean, safe, and affordable energy without the amount of nuclear waste resulting from fission reactions, which currently occur in fission power plants around the world. However, fusion energy research is still on the edge between science and technology because it still requires large investment in basic research before developing technological applications.

Despite the infantry stage of fusion energy research, this industry is experiencing a transition from publicly to privately funded research. The influx of private funds is changing the objectives of fusion energy research from an approach based on open science, where the scientific results are shared within the global fusion community, to an approach based on proprietary technologies, where each company protects its technological applications with Intellectual Property Rights (IPRs).

Whereas the enforcement of IPRs can create economic incentives for private companies to generate even more technological applications in the field of nuclear fusion and will eventually lead to the commercialization of fusion power plants, stricter IPRs are often associated with high social costs that result from restricting the access to the knowledge that was previously shared within the global fusion community and thus limit the effectiveness of nuclear fusion in climate-change mitigation policies.

2. Methodology

Building on the new Economics of Science, this paper develops a systemic approach to science and innovation policy in fusion energy research as it transitions from public to private funding. The overall objective of this systemic approach to policymaking is to strike a productive balance between the publicly funded community of scientists, who tend to favor full disclosure of their results, and the community of privately funded technologists, who tend to favor disclosure only after receiving a patent. This productive balance between the two communities can be achieved when policies fund open science and check against excessive incursions of claims to IPRs. To achieve this objective, the methodology employed by this paper is to interview key players in both communities to gather information on how they view open science in fusion energy research. Interviews allow to gather rich information on this topic and they are a qualitative research method that can be used as a foundation for future quantitative research on this topic. To the best of our knowledge, no other authors have outlined a systemic approach to policymaking in this industry. Despite its limitations, the qualitative method employed in this paper has the ability to assist policymakers who seek a regulatory framework to guide the future of this industry.

3. Anticipated Results

Some of these interviews have already been collected by Michel Claessens, who was the spokesperson from 2011 to 2016 for the International Thermonuclear Experimental Reactor (ITER) project, which is currently the largest publicly funded project in fusion energy research. These interviews provide preliminary evidence that scientists who are working in the ITER project follow the Mertonian norms of Communalism, Universalism, Disinterestedness, Originality, and Skepticism (CUDOS).

These preliminary results support a systemic approach to policymaking in fusion energy research that stresses the benefits of openness in science because all the intellectual property developed in the project is shared equally by all the global members of ITER.

Claessens also interviewed some of the private companies that are supplying equipment to ITER. These managers also confirmed that openness in science allows the communal, universal, and original approach required to supply hi-tech components that meet the stringent specifications of a first-of-a-kind project, such as ITER, whose consortia of suppliers span across the world.

4. Significance

Climate change is a complex problem that requires multiple solutions from multiple disciplines. Science and innovation policy in greener technologies can help to mitigate this complex problem by fostering the development and adoption of technologies that can reduce GHG emissions. Although not a panacea, nuclear fusion can provide clean, safe, and affordable energy in the future.

However, fusion energy research is undergoing two paradigm shifts that will jointly shape its future. First, it is transitioning from science to technology as more and more technological applications are providing different approaches to the future commercialization of electricity generated from fusion power. Second, it is transitioning from public to private funding as more and more private companies are entering the newborn fusion industry. This large influx of private funding is shifting the focus of fusion energy research from full disclosure of scientific results within the global fusion community to stricter enforcement of intellectual property rights that could have a negative impact on future research.

Now more than ever fusion energy research needs a regulatory framework to guide this transition. This paper plans to build a systemic approach to policymaking in fusion energy research to show that science and innovation policies that favor openness in science can provide greater economic and social utility to both the publicly and privately funded research communities in nuclear fusion. The main policy recommendation derived from this paper urges the governments that are currently sponsoring private endeavors in fusion energy to keep science open in exchange of public funding.

The systemic approach to policymaking derived from this paper can also provide a policy toolkit for managing complex and dynamic technologies in several other sectors, such as aerospace. Moreover, the implications for sustainability derived from this paper can assist the policymaking system with a multicriteria approach to assist the formulation and implementation of climate-change mitigation policies in other technologies of public interest, such as transportation.

ABSTRACT. In August 2022, the Whitehouse Office of Science and Technology Policy announced new requirements from federal agencies “Ensuring Free, Immediate, and Equitable Access to Federally Funded Research”. Namely, this requires executive agencies and departments to:

Update their public access policies as soon as possible, and no later than December 31st, 2025, to make publications and their supporting data resulting from federally funded research publicly accessible without an embargo on their free and public release; Establish transparent procedures that ensure scientific and research integrity is maintained in public access policies; and, Coordinate with OSTP to ensure equitable delivery of federally funded research results and data.

In 2019, we saw the number of open access academic papers published globally, passing 50% of all papers published. There are also many new business models and routes to support open access as the push has steadily gained moment since it was first conceived in the 1980s. Authors and funders have a plethora of green and gold open access publication routes, although managing the cost implications of the latter is an ongoing discussion.

Open academic data is largely a new concept to researchers and funders alike. Whilst several fields have been benefiting from making data available for some time, for example, genomics and astrophysics - many communities are struggling with the concept of where and how to make their research data accessible. Several are struggling even with the concept of what ‘data’ means to their field.

The State of Open Data report from Digital Science surveys researchers on their attitudes and concerns with regards to open academic data. Since 2016, we have monitored levels of data sharing and usage. Over the years, we have had 25,000 responses from researchers worldwide providing unparalleled insight into their motivations, challenges, perceptions, and behaviours toward open data. The State of Open Data is a critical piece of information that enables us to identify the barriers to open data from a researcher perspective, laying the foundation for future action. This year’s report also includes guest articles from open data experts at the National Institutes of Health (NIH), the White House Office of Science and Technology Policy (OSTP), the Computer Network Information Center, Chinese Academy of Sciences (CNIC, CAS), publishers and universities.

A separate Digital Science report led by Ripeta is titled The State of Trust & Integrity in Research has insights into how researchers are actually acting in practice when it comes to open research data. It demonstrates that improved data sharing policies will boost public trust in research

In addition to its detailed analysis of the five major funders, Ripeta found that with regard to 62 key funders worldwide: 71% of funders required data management plans 68% of funders covered expenses of data management and sharing 66% specifically mentioned data sharing repositories as a mechanism of making data publicly accessible 26% of policies mentioned how long data should be retained.

Locating funding agency policies proved to be particularly difficult, with many not surfacing through web searches. There was also significant policy variability between funding agencies, with many policies having differing requirements for implementation. Utilizing Ripeta’s services, funders and government agencies have the ability to analyze and monitor compliance within established open science or data management and sharing policies, while also identifying areas of good institutional research practice.

Exclusive analysis by Ripeta in one section of the report compares the policies and practices of five major world funders: the Bill and Melinda Gates Foundation, the European Commission (EC), the National Institutes of Health (NIH) in the United States, the National Natural Science Foundation of China (NSFC), and the German Federal Ministry of Education and Research (BMBF). The analysis shows that although each of the funders have their own policies in place to support data sharing, there are discrepancies between those policies and how they translate into practice. For example, the number of research papers published in open access (OA) journals varied greatly depending on the funder. The NIH (95%) and the Bill & Melinda Gates Foundation (93%) far outstripped the representation of OA publications of other funders, especially the NSFC (33%).

This session will discuss what we know about how to encourage researchers to make their data available and how we can ensure compliance with the OSTP memo going forward.

ABSTRACT. Since Google entered mainland China in 2006, its share of the total search engine market of mainland China rapidly increased to 40.08% by the end of 2009. Together with the Chinese firm Baidu, which offers a similar service portfolio and held a market share of 58.47%, Google effectively became part of a duopoly (Kong et al., 2022). Google was, hence, a main source of information in China, especially of information from foreign countries (Kong et al., 2022; Wang et al. 2020). Like any search engine provider operating in China, Google was obliged to follow the strict censorship guidelines imposed by the Chinese government, but, in January 2010, Google decided to discontinue the censoring of search results on its China search page. This decision rapidly escalated in a sudden and unannounced withdrawal of all Google services from China, leaving millions of users without access to the world’s top search engine overnight. From the 30th of March 2010 onwards, users in China could not access Google services anymore (The Guardian, 2012; The Official Google Search Blog, 2012; Bloomberg, 2014; Xu et al., 2021; Kong et al., 2022). In this paper, we investigate the effect of Google’s sudden exit from China on the scientific research output of Chinese scholars. Access to information in the form of books and research material has been shown to be crucial for the generation of new knowledge (Furman and Stern, 2011; McCabe and Snyder, 2015; Waldinger, 2016; Berkes and Nencka, 2019; Mueller-Langer et al., 2020; Furman et al., 2012; Biasi and Moser, 2021; European Commission, 2012). A lack of access or high accessibility costs can, hence, be a key barrier to new discoveries and knowledge creation. Not surprisingly, information and communication technologies have been shown to enhance science production by increasing the availability of information and, hence, reducing search costs (Agrawal and Goldfarb, 2008; Ding et al., 2010; Winkler et al., 2010; Kim et al., 2009). Google’s sudden exit from China, therefore, bears the risk that Chinese researchers lose touch with the research frontier and persistently lag behind their foreign peers. Using Google’s exit from China to assess the effect of barriers to information accessibility has several advantages which address common challenges for causal estimation. First, Google’s exit was exogenous to science production and unexpected as it was the result of a rapid escalation of political tensions between the Chinese leadership and Google (Zheng and Wang, 2020; Xu et al., 2021; Kong et al., 2022). Second, Google was, at the time of the sudden withdrawal of its services, one of the main sources of knowledge for China (Kong et al., 2022; Wang et al. 2020) and its scientists (Qiu, 2010). Our empirical analysis focuses on the field of economics following prior studies such as Kim et al. (2009), McCabe and Snyder (2015), Liang et al. (2022), and Piracha et al. (2022). Economics is a research field with a simple knowledge production function as it does not rely on material and expensive equipment (Stephan and Levin, 1992). New insights are published almost exclusively in scientific journals rather than in books and conference proceedings which are often not well covered in bibliometric databases (e.g. Michels and Fu, 2014). Hence, an estimated effect of the sudden decrease of information accessibility on scientific output is less likely to be confounded by other effects resulting from the knowledge-generating process or publication strategy of the field. To derive causal results, we use a Difference-in-Difference (DiD) approach employing a control group of researchers located in Taiwan and Hong Kong following Zheng and Wang (2020) who argue for a control group that is culturally, economically, and geographically closely related to China. Our results show that researchers affiliated with Chinese institutions experience a significant decline in both their research output quantity and impact as measured by citations received by the future literature. The magnitude is about 28% for co-author-weighted publications and 30% for co-author-weighted citations. We explore the proposed underlying mechanism of information accessibility further and show that the productivity and impact of those Chinese scholars that work with foreign co-authors are less affected by Google’s exit. These scholars can use their interpersonal networks as a channel for knowledge access (Singh, 2005; Mohnen, 2022). The publication output and impact of these scholars decreases by smaller shares of 20% and 22%, respectively, supporting that the mechanism of knowledge accessibility is responsible for the decline in publication output after Google’s withdrawal. In further analysis, we find that the effect in terms of quantity and impact is stronger for those scholars with the highest impact as measured by their citation stock over publication stock before Google’s exit. The publication output and impact of the top 25% scholars decrease by 39.5% and 37.5%, respectively, while the publication output of the scholars at the bottom of the impact distribution decreases by 20%. There is no significant effect for the scholars at the bottom of the impact distribution in terms of impact. The large effects on the top scholars raise concerns about the ability of China to stay in touch with the research frontier in the medium and long run with potentially harmful implications for economic growth (Griliches, 1992; Jaffe, 1989). We make several contributions to the literature. First, our work adds to our understanding of the determinants of knowledge creation (Stephan and Levin, 1992; Stephan, 1996, for an overview) and more specifically of the role of information and communication technology in knowledge creation (Agrawal and Goldfarb, 2008; Ding et al., 2010). Prior studies have shown that access to network technology (Agrawal and Goldfarb, 2008; Ding et al., 2010, for the case of BITNET) eases information accessibility and facilitates the knowledge production of scientists. Here, we focus on Google as a general search engine and complement prior findings for different technologies. Second, we contribute to recent literature that focuses on positive information shocks such as the availability of access to libraries (Berkes and Nencka, 2019; Furman et al., 2012; Biasi and Moser, 2021), of research resources (Furman and Stern, 2011) and of online access to scientific journals (McCabe and Snyder, 2015; Mueller-Langer et al., 2020) and their impact on knowledge creation. We differ from these studies in two ways. First, these studies focus on the access to prior scientific knowledge available in form of books, journals, and research resources while we focus on the access to a search engine that covers a much broader scope of information. Second, we explore a negative shock of information availability to assess the effects on science production while prior studies focus on positive shocks of information availability. Third, we add to the developing literature that focuses on the implications of Google’s China exit. These include a higher stock crash risk for firms (Xu et al., 2021) and a decrease in corporate innovation (Kong et al., 2022; Zheng and Wang, 2020). Other than these prior studies, our focus is on the scientific rather than on the corporate sector.

ABSTRACT. Background and rationale Over the past years, research funding agencies have demanded studies on the impact of the research financially supported by them to verify how the creation of knowledge relates to improvements in economic, social, and policy-related areas (Aiello et al., 2021; Sandes-Guimarães, Velho & Plonski, 2022; Milat, Bauman & Redman, 2015). More importantly, research funding agencies are concerned about demonstrating the relevance of scientific research and studies to address societal problems, i.e., about producing evidence of the impact of the funded research. This movement is important because it promotes higher accountability and advocacy directed to different types of stakeholders and it demonstrates the effects of investing in research. This is also part of a larger discussion that investigates the science of science, or research on research, that aims to identify how scientific research can be made more open, inclusive, and impactful. Considering the above, through a brief literature review, we have come across multiple impacts deriving from research. In order to conduct our study, we decided to group them into three clusters, namely, economic, policy, and societal. This division allows us to identify similarities and differences between the many impacts of investing in research according to their specific contexts. In addition to understanding aspects related to the literature on research impact, thinking of the societal impact of research is increasingly connected with discussions on Equity, Diversity and Inclusion (EDI) in academia. In this sense, we consider that incorporating EDI indicators in the literature review together with traditional bibliometric indicators is relevant as it sheds light on aspects that are often neglected when discussing the state of the art of a certain topic. In other words, this research approach allows us to identify the status quo and marginalized discussions of relevant topics by including indicators of genre, race, location, and language.

Objective This study aims to show an overview of the economic, policy and societal impact of scientific research from 2015 to 2022 through the elaboration of a scientific mapping, as well as critically discussing it. This scientific mapping considers four main areas: a) thematic evolution, b) methodological approach, c) performance based on traditional bibliometric indicators; and d) performance based on new bibliometric indicators (EDI). It is important to note that this study is part of a larger project to investigate and develop research impact indicators for several initiatives carried out by the São Paulo Research Foundation (a.k.a. Fapesp) in Brazil.

Methodology Through a bibliometric analysis carried out in the Web of Science and Scopus databases, we provide the scientific mapping concerning the three broader impacts mentioned herein, namely economic, policy, and societal. It is important to clarify that to all impacts considered, basic bibliometric and literature review techniques will be applied. Common to all impacts is the inclusion of EDI metrics, as well as the identification of articles that address the role of funding or funding agencies in impact evaluation. However, considering the vastness of sub-themes implicit in each of these impacts, we resorted to a few methods to delve deeper and focus on each more effectively. The economic impacts will be evaluated based on three pillars, namely: university-industry collaboration, focusing specifically on an output indicator of university-industry co-authorship in scientific articles; the relevance of funding for the development of innovation and entrepreneurship ecosystems; and the importance of funding for the creation and development of small business programs, such as the SBIR. Regarding policy, we intend to identify how the literature on policy impact of research addresses the flows of knowledge from academia to policy (forward tracing method) and the use of scientific knowledge by policymakers (backward tracing method). Taking into account the research objectives, the following aspects will be emphasized: methodologies for evaluating the different flows and use of knowledge between science and policy-making; the debate on the role of funding and funding agencies in promoting the impact of research on policy and ways to assess and evaluate this, including the incorporation of new bibliometric and altimetric methods to measure impact; and the thematic dimension (public policy area reached) of the articles identified, looking for trends in knowledge areas that present greater predominance on the impact of policy research. As for the societal impact, we had to make a more delimited cut, as the literature informs that the social impacts encompass several fields, like health, the environment, public policies, and education. However, it is important to state that the social and environmental areas are relevant and frequently cited when it comes to the societal impact of research. Thus, by assessing the main thematic projects supported by Fapesp, we found that the health area is the social field that has received the highest financial support in recent years, which emphasizes its relevance as a field of research and project development. On the other hand, considering the relevance of environmental issues, we verified that the environmental-related area that receives the most funding from Fapesp is ​​climate change, which reflects the general concern with global warming, expressed and emphasized by reports like those by the Intergovernmental Panel on Climate Change (IPCC). For these reasons, we chose health and climate change as the societal impacts to be further assessed in this study.

Significance of the study The analytical exercise of identifying and developing a scientific mapping of the economic, policy, and societal impact of research allow us to identify similarities and differences between recently developed impact assessments. The intercambiation of methodological approaches is a rich contribution of this study as new ways of assessing a certain impact could be limited to a field of knowledge and have not yet overcome the disciplinary barrier; this may be useful when applied to other areas of knowledge. Another relevant contribution of this study is that the scientific mapping provides us with an outlook on research opportunities and points out where contributions can be made to the field. In addition, evaluating the state of the art in each dimension based on traditional performance indicators gives us an overview of the status quo of a topic of academic interest which is a well established way of providing information and evidence of how impacts are being assessed in recent literature. However, as we previously outlined, funding agencies are concerned about ways to make research practices more open, inclusive and impactful and an effective way of addressing and investigating this topic is to consider alternative indicators while performing a literature review. In that sense, we provide a relevant contribution by stimulating the incorporation of EDI indicators when conceptualizing a study, and also by developing new bibliometric indicators based on EDI.

References Aiello, E., Donovan, C., Duque, E., Fabrizio, S., Flecha, R., Holm, P., Molina, S., Oliver, E., & Reale, E. (2021). Effective strategies that enhance the social impact of social sciences and humanities research. Evidence & Policy, 17(1), 131-146. Milat., A. J., Bauman, A. E., & Redman, S. (2015). A narrative review of research impact assessment models and methods. Health Research Policy and Systems, 13(18). Sandes-Guimarães, L. V., Velho, R., & Plonski, G. A. (2022). Interdisciplinary research and policy impacts: Assessing the significance of knowledge coproduction. Research Evaluation, 31(3), 344-354.

ABSTRACT. Background The Open Science movement continues to grow across the globe with support from the European Commission (EC), the OECD (OECD) , the United States National Academies (National Academies of Sciences, 2018), amongst others. The Center for Open Science (COS), a non-profit organization, was founded in 2013 to “increase the openness, integrity, and reproducibility of scientific research” (CoS). In 2015, CoS, with several universities, funders, and publishers, published the Transparency and Openness Promotion (TOP) Guidelines, eight standards scholarly journals can implement to further openness and transparency (Nosek et al., 2015). In 2020, CoS launched the TOP Factor to assess the degree to which journal policies are promoting transparency and reproducibility. The TOP Factor is based on the TOP Guidelines, e.g. transparency of data/code/materials/research design and pre-registration. The TOP Factor is calculated by summing the level of implementation of the 8 original standards plus 2 additional standards (CoS).

The Web of Science Core Collection is a global, multidisciplinary citation index widely used for scientometric analysis. The list of journals in Core Collection is available on the Web of Science Master Journal List (Clarivate). The Core Collection includes 21,879 active journals as of January 2022, covering sciences, social sciences, and arts & humanities. These journals pass a rigorous editorial selection process (Clarivate).

In 2020, Clarivate collaborated with the Center for Open Science to incorporate TOP Factors on the Master Journal List (Clarivate). As of February 2022, 1,232 of the Core Collection’s journals (5.6%) have a TOP Factor. Previous studies have analyzed TOP Factor for journals in specific disciplines, including sleep research and chronobiology (Spitschan et al., 2020), herpetology (Marshall & Strine, 2021), and sport science (Hansford et al.), (Hansford et al., 2022), (Hansford et al., 2021). Other studies have found a positive correlation between open data sharing, one aspect of open science, and citation impact (Piwowar et al., 2007), (Piwowar & Vision, 2013). This study explores the relationship between TOP Factor and journal impact measures across all research disciplines.

Methods The TOP Factors were drawn from the Web of Science Master Journal List in February 2022. To calculate TOP Factors, Center for Open Science uses 10 standards scored individually on a scale of 0-3 (disclose=1, require=2, verify=3). These 10 scores are summed to create the overall TOP Factor with a maximum of 30 (CoS). The citation metrics are from InCites Benchmarking & Analytics, a research analytics database based on Web of Science Core Collection data. Institutional Profiles journal categories were used. This category scheme uses 6 broad academic disciplines.

The citation indicator used is Journal Citation Indicator (JCI). The JCI is the mean category normalized citation impact (CNCI) for the journal. CNCIs are calculated at the document level for articles and reviews from the previous three years, counting citations for four years. The JCI is normalized for document type, publication year, and category. The average JCI for any category is 1. A JCI of 2 indicates that a journal is receiving twice the expected number of citations than the average journal in the category.

Results The majority of the 1,232 Web of Science Core Collection journals with a TOP Factor are in social sciences (786), and the fewest are in physical sciences (69). Social sciences also had the highest percentage (12%) of its titles having a TOP Factor, and physical sciences had the lowest (2%). The median TOP Factor was 1. Very few approached the maximum TOP Factor of 30. Physical sciences has the highest average TOP Factor (4.7) despite having the lowest number of journals and lowest percentage of its category having a TOP Factor. The 2020 JCI for these journals ranges from 0.11 to 11.25 with a median of 1.26. We found no clear correlation between the journals’ JCI and TOP Factor, despite previous studies that found a citation advantage for papers with open data. Since open data is only one of the attributes that TOP Factor reflects, further research is needed to explore the relationship between openness and impact.

Significance This is one of the first studies to analyze TOP Factors across a multidisciplinary set of journals. There are some disciplinary trends, with social sciences having the highest number and percentage of titles with a TOP Factor, and physical sciences having the lowest. Overall, only 5.6% of the Web of Science Core Collection journals have a TOP Factor, and the median TOP Factor is 1 out of a potential 30. This suggests that adoption of the TOP guidelines is not widespread in the publishing community, and those who have pledged to adopt them are not fully embracing them. We found no correlation between the TOP Factor of a journal and its normalized citation impact despite the perception that greater openness leads to greater impact. Further research could explore disciplinary trends, as well as changes over time.

ABSTRACT. “I think the article is a parody thing, or a joke, or maybe just [obscenity] insane. Fair warning, you will not be any smarter or better informed after reading it, but you might get a good belly laugh or two [retracted publication].” -Twitter User

Background: Scholarly literature can be retracted for multiple reasons, including faulty analysis, plagiarism, or falsification. Retracted scholarly literature raises questions about the validity and rigor of academic work. In some cases, retracted articles can hinder an academic field’s growth by inserting misleading or wrong information into a topic’s body of knowledge.

Especially in cases where articles are retracted due to flawed reporting or methods, it is useful to examine the nature of retracted papers. Conversely, it is also important to understand retracted papers because it is arguably worse for a flawed paper (i.e., one with faulty analysis, not reproducible, or another reason that is related to its results) to not be retracted, or take extended periods of time to do so.

Methods: Basic analysis The publicly available RetractionWatch (RW) dataset comprises our base set of data. We only include articles with a DOI in our analysis - this restriction appears to mainly exclude articles affiliated with Russian institutions. To inform our basic analysis, we perform summary statistics on the original dataset, including assessing retractions by year, country, reason for retraction, and subject matter. We also examine these measures on the subject of papers that are technology-related - we measure this by the RW subject (either “computer science” or “technology”), or if the paper is assigned an AI-related label using a SciBERT classifier.

Citations analysis Using a corpus of scholarly literature (containing merged publication data from Web of Science, Digital Science, arXiv, Microsoft Academic Graph, and the Chinese National Knowledge Infrastructure) we assess the number of times each retracted paper was cited, comparing the retracted papers’ number of citations to a stratified sample of our corpus. To create the stratified sample, we controlled for paper publication year, publication country, and a binary variable indicating whether or not the paper is technology-related based on the criteria described earlier. For this comparison, we additionally employ “citation percentiles,” which indicate the percentage of papers published in the same year and field as a given scholarly article that have fewer citations (e.g., a biology paper published in 2010 in the 95th citation percentile is cited more than 95% of 2010 biology papers). Finally, to examine the practice of citing retracted papers, we measure retracted papers’ citations before and after their retraction years, again subsetting technology-related articles against all articles.

Social media impact analysis To gauge social media impact of retracted papers, we connected our papers to Dimensions’ Altmetrics database. Altmetrics provides data on tweets, public Facebook posts, and other social sharing data on papers in the Dimensions database. It does not cover the full Retraction Watch database, but does cover most English-language publications. We compare social media metrics on retracted articles to the same stratified sample of scholarly literature that we use for our citation analysis.

Pilot results/expected results: Basic analysis: China has triple the number of total retracted articles than the second-highest country with retracted articles, the United States, with India, the United Kingdom, and Japan in 3rd, 4th, and 5th places respectively. Iran has the 6th highest global retractions despite not placing in the top 20 countries for scholarly paper publication globally. Regarding technology papers, China has ten times as many retracted papers as the United States, with the United States, India, Iran, and Malaysia following with slim margins respectively. Biology and medical papers were by far the most common subject in the general corpus, with triple the number of retractions as the second most common field, technology-related papers (over 12000 versus 4000), followed by business and engineering respectively. More papers over the past decade have been retracted than in previous decades, as other analyses have reported (insert source). On average, it took two calendar years for a technology-related paper to be retracted; in contrast, it took four calendar years for a non-technology related paper to be retracted. This discrepancy drops to 2 years to 3 years when we only consider papers originally published after 2009.

Citations analysis: Retracted papers continue to be cited after retraction. On average, 53% of citations for retracted papers occurred after the retraction year for non-technology-related papers, while 57% of citations occurred after the retraction years for technology-related papers.

The Kolmogorov-Smirnov test shows that retracted and non-retracted papers have different citation-percentile distributions for both technology-related and general research papers. Specifically, retracted publications follow a uniform distribution (i.e., the median percentile is 0.5) more closely, and the sample papers have higher percentiles.

Altmetrics social media analysis: To understand how the public discusses and interacts with retracted research, we used the Altmetrics API to retrieve tweet IDs that mentioned retracted technology-related papers. We retrieved the tweet texts and applied the Valence aware dictionary for sentiment reasoning (VADER) sentiment analyzer on the text. Repeating this process with our sample of non-retracted technology-related papers, we compare the difference in tweet activity of retracted and non-retracted publications.

VADER detects the polarity of text and provides a sentiment score on scale of -1 (negative) to 1 (positive). Retracted papers have a low average sentiment score of 0.03, compared to the non-retracted papers with an average sentiment score of 0.16. For retracted papers, positive tweets reflected sarcasm and negative tweets reflected frustration and disappointment.

For example:

Positive sentiment: “I'm no chemist, but pretty sure this is the greatest paper Big-E has published.“

Negative sentiment: “Sadly, misinformation is not only plaguing social media but the scientific community. Here are two bizarre examples of poor peer-review standards and the flourishing of low-standard “scientific” journals.”

For non-retracted papers, positive tweets reflected praise of and genuine interest in the research mentioned and negative tweets generally received a low score due to the research topic (e.g., depression or cancer).

For example:

Positive sentiment: “Oooh! Shiny! This was certainly worth waiting for. Also great to see coverage of leading edge research in high profile non-academic press”

Negative sentiment: “New research on breast cancer risk assessment”

Significance/discussion: China, the United States, and India lead the world in the number of retracted papers, both for technology and non-technology-related papers. The top 10 countries with the highest number of retracted papers differ when disaggregated by topic, and differ from the top 10 countries ranked by general scholarly output. Russia, Iran, and other countries with less sophisticated scholarly research infrastructures are overrepresented in the top countries list; this phenomenon may be due to less rigorous research methods, peer review, or research standards on the front-end.

Papers are both more likely to be retracted over the past decade and papers retracted in more recent years have a shorter retraction period (as to be expected, if they exist in this dataset). The higher retraction rate may be due to more rigorous review of papers post-publications and reduced stigma regarding research retraction. Still, papers tend to be cited more often post-retraction than before, which has significant implications for advancing scholarly research in all fields, but particularly nascent ones like technology-related fields.

ABSTRACT. The scientific elite wield considerable power in shaping the evolution of scientific research (Azoulay et al., 2019; Chu & Evans, 2021) and reap sizeable rewards for their contributions (Allison et al., 1982; Merton, 1968; Xie, 2014). Opportunities to enter the scientific elite, however, are limited and are distributed with clear disparities across any number of social cleavages. This presentation reports on research that tackles the issue of publishing in elite journals, perhaps the most salient mechanism for entering the scientific elite (Heckman & Moktan, 2020), and in particular, the effect that journal editors have on that process.

Research shows that editors and reviewers are more likely to support papers that are closer in topic to their own research areas (Krieger et al., 2021), and which are written by academics who are nearby in the collaboration network (Dondio et al., 2019; Ductor & Visser, 2022; Teplitskiy et al., 2018), who have won notable awards (Huber et al., 2022), and are members of elite professional networks (Crane, 1967; Laband & Piette, 1994). Current estimates of editorial gatekeeping’s effect size are small, but it is difficult to know how much confidence to place in these results, since much of this work is focused on contexts where an editor is unlikely to be an active gatekeeper on the one hand, or an effective gatekeeper on the other. We have a growing number of studies set in non-elite and in multidisciplinary journals (Dondio et al., 2019; Teplitskiy et al., 2018), while those studies that address elite journals have virtually all been in economics (e.g. Colussi, 2018; Ductor & Visser, 2022; Laband & Piette, 1994).

Our project will provide the most substantial evidence to date relating to an editor’s effect on the publication process. We have collected the most comprehensive longitudinal dataset of journal editors in the social sciences, with over 3000 editors at over 1000 journals. This allows us to make examine the level of editorial influence in elite journals, specialist journals, and the broader mass of journals in each social science field. The breadth of the data mean that we can also provide some of the first looks at editorial gatekeeping outside economics in the social sciences.

To make full use of this data we are leveraging the Web of Science with state-of-the-art author disambiguation to assemble multiple measures of the distance between authors and editors. The Web of Science itself is used to assemble the collaboration network in science, the canonical approach to judging editorial gatekeeping. We also assemble the institutional affiliation network to provide an alternative view on the same principle of relational closeness. To further control for the topic similarity between authors’ research and an editor’s research we link papers to their SPECTRE document embeddings, high-dimensional text embeddings provided by Semantic Scholar. All of these measures of similarity are allowed to vary over time, not only as editors change at journals but also as researchers’ (and editors’) research interests evolve over time.

Using a series of field-specific relational event models (de Nooy, 2011; Quintane et al., 2014; Schecter & Quintane, 2021) we will report on the likelihood that someone gets published in elite journals, conditional on their distance from the current editor(s). We expect effect sizes to be small, but also for them to vary substantially across fields and across the journal hierarchy.

The most significant limitation to our research is that we do not have access to reviewers, nor can we pair them to the papers they evaluated. Editors of course can and do still wield power over the publication process, but it is typically less direct than that of reviewers. Another area where our design falls short of what would be ideal is that we cannot disentangle self-selection by authors themselves from editorial decision-making. Researchers at least in part—though this almost certainly varies dramatically across (sub)fields—make their choice of which journals to submit to on the basis of the identity of the current editor. This means that while it is entirely reasonable to speak of the effect that editors have on a person’s likelihood of publishing in a given journal, it is insufficient to prove that owes to editorial gatekeeping, specifically.

Still, we are optimistic about our paper’s potential. There are three main contributions we expect to make with this research. It will (1) provide a rare look at elite publishing practices in social sciences other than economics; (2) highlight variation in how editors affect elite publishing across several social science fields; and (3) document variation in terms of how editors affect publication across the hierarchy of journals within a field.

ABSTRACT. Background

We examine the ‘Villum Experiment’ (VE), a funding program instigated by the Villum Foundation, a Danish private funder who supports blue-sky research in the natural and technical sciences in Denmark. The VE instrument is supposed to support unorthodox ideas in their early phase. More specifically, the instrument was created for those research projects out of the ordinary that challenge the norm and have the potential to change fundamentally the way we approach important topics. The VE should therefore promote ‘risky ideas’ and ‘new paths’ in the natural and technical sciences. Interestingly, to mitigate well-known biases in peer review, applicants are kept anonymous to the reviewers. Assessments are solely based on short structured applications where the focus is upon the ‘idea’, no CVs or names are disclosed. Non-blinded peer reviews are seen as conservative and supposedly incentivise ‘safe applications’. Likewise, exposure of names, affiliations and past performances through CVs is assumed to causes biases resulting in for example, Matthew effects, and gender and ethnicity disparities. A double-blind mechanism is assumed to alleviate such challenges. The questions is to what extent?

Methods

We use all application data (e.g. names, ids, full text) and review scores from five rounds of applications (2017 to 2021) - corresponding to 2000+ applications with around 250 funded projects. We examine different aspects of the VE programme, such as its mechanisms and potential impacts. Here, we focus on potential gender bias and disparities in who is funded. The current overall application and success rates, somewhat surprisingly, given the double-blinded mechanism, suggest a consistent gender difference where men are seemingly more successful than expected given their share of applications. Despite a double-blinded review process, women are 4.3 percentage points less likely to have their proposals funded than men (9.18 \% vs. 13.5 \%). To make sense of this persistent gender disparity, we use a series of bayesian multilevel models and post-stratification to investigate potential explanations.

Results

Deeper analyses reveal two important self-selection/sorting mechanisms and potential gender disparities related to the demographics of the overall researcher population. First, we show that gender differences in success rates are partly explained by women self-selecting into one evaluation panel ('Life Science') to a much higher degree. Where men are more evenly distributed across the four evaluation panels ('Earth and Space Science', 'Life Science', 'Physical Sciences and Mathematics', 'IT and Engineering'), 46 \% of women applicants apply within life science. Because success rates differ across panels, and the 'Life Science' panel has consistently lower overall funding rates, women tend to self-select into stronger competition. The overall success rate differences are then a case of Simpson's Paradox, where results almost reverse when dis-aggregated across evaluation panels and application years.

Second, more experienced applicants tend to be more successful in having their proposals funded, with professors more successful than associate professors, and associate professors more successful than assistant professors and postdocs. However, women comprise a much smaller fraction of the more experienced applicants. Gender differences in funding rates are therefore also a function of the demographic disparity in academic ranks more broadly.

Lastly, women comprise around 25 \% of professorships in Denmark and 15 \% within the natural sciences, However, among the applicants with the rank of professor, only 11 \% are women. Using a Multilevel Post-Stratification (MrP) model, we calculate weights and simulate success rates for each academic rank and gender combination among applicants as if they were proportional to the entire population of Danish researchers.The model shows that the remaining gender disparity in funding rates are likely influenced by a lower self-selection into the program by the highly experienced women, and differences would largely disappear if more women applied.

In summary, gender disparity can still arise even when doubled-blinded review standards are employed. Larger structural imbalances in field composition and career advancement in the science system contribute to differential success in funding competitions.

Significance

The results shed light on two important discussions within science policy. Echoing recent calls for attention \parencite{traag_causal_2022}, we exemplify the importance of distinguishing between gender bias (discrimination) or gender disparity (differences) when assessing fairness in the allocation of merit in science. Gender differences in funding rates can still emerge when bias (conscious or not) in review processes are extremely unlikely, due to indirect causal effects of imbalances in the scientific system. Assessing the mechanisms contributing to gender differences in funding rates are important as changes to peer-review systems cannot eliminate such disparity by itself. Unequal access to research funds are also a product of inbuilt differences in career advancement (i.e. a leaky pipeline), and less representation of women in some fields of science. Solutions to such disparities necessitates a better understanding of the driving factors, because gender bias and structural disparities are not easily solved using the same policies.

ABSTRACT. Background This paper aims to contribute to, and advance, the understanding and empirical study of the effects of research governance on scientific fields by exploring the governance effects on a scientific field in the context of two different universities. That is achieved by building on a recent framework for the study of governance effects on research fields (Nedeva et al., 2022) and extending it, conceptually and empirically, by conducting a comparative analysis of the reported behaviour of members of the same research field, within the same governance regime and two rather different universities. This paper is the third within a research line on studying governance effects on scientific fields (Nedeva et al. 2022, Morales-Tirado, in progress).

Rationale Debates regarding the study of research governance effects on global scientific fields in the literature unfold along several lines. Some studies, quantitative as well as qualitative, investigate governance effects on context-specific research organisations, namely universities and research institutes (Lorenz 2012; Luukkonen and Thomas 2016; Vinkenburg 2017; Glaser 2019; Luo, Ordóñez-Matamoros & Kuhlmann, 2019; Thomas et al. 2020; Strinzel et al. 2021; Kozlowksi et al. 2022; Ramos-Vielba, Thomas and Aagaard 2022; Feenstra, & López-Cózar, 2022,). Other, more nuanced studies extend their research interest to include studying the effects of governance on the epistemic choices of members of local knowledge communities (Glaser, 2019). Methodologically studies of governance effects usually aim to measure change using opinion-based survey techniques, case study/interview approaches, bibliometrics or lines of investigation seeking to unpack the (soft) causality mechanisms that may or may not affect organisational, personal and group selections.

While contributing to the understanding of governance effects on science, these approaches share a significant shortcoming in that they generally fail to extend beyond the local conditions for knowledge creation and hence fail to capture the aggregate governance effects at the level of transnational, global research fields.

This was conceptually addressed in a recent paper (Nedeva et al., 2022) by proposing a novel heuristic for linking the characteristics of performance-based evaluation arrangements (PREAs) and the properties of research fields. Next, a comparative analysis of the governance effects of different governance arrangements of the same research field extended this framework empirically. This paper explores the (potential) difference in responses of the local members of the same research field, within the same governance arrangements and two different universities.

Methods We used a novel framework to study governance effects on scientific fields, one that recognises three contexts where different effects may occur, e.g., the research space context where performance based evaluation arrangements (PREAs) are embedded, the research field context where knowledge claims are assessed to award reputation (publications, grant capture etc.), and the context of research organisations where individual and collective performance are evaluated for organisational career purposes (Thomas et al., 2020).

We kept the PREA and research field contexts constant and allowed variance in terms of the organisational context. Hence, we interviewed members of research groups in two universities. To use a terminology from Paradeise and Thoenig (2015) one of the universities is a ‘top-of-the-pile’ and the other one is a ‘wannabe’.

Our questions were designed to capture the interactions between university leaders (administrators) and the members of the local knowledge network, or research group. We conducted a total of twenty interviews (13 for a top-of-the-pile university and 7 for a wannabe university). We explored interactions (and power play) in the context of nine selection points (Nedeva et al., 2022).

Preliminary results We analyzed the interviews around selection dimensions concerning organizational authority, namely (1) organizational career, (2) knowledge production, and (3) knowledge dissemination. Results indicate that university governance arrangements matter but do not change the actions of this type of field members.

We contrast the responses of both universities around the selection of new group members, promotion and probation, and we find that for the wannabe university members, recognition of the university authorities is more important than for those from a top-of-a-pile organization. Yet, field considerations (e.g., field recognition) override organizational pressures. In regards to selection for access to research infrastructure, methods and skills, behaviour is unaffected by local influences in both instances. Similarly, our results suggest that selections for access to knowledge networks, decisions over publication outlets and submissions for PREA assessments are dominated by F-type notions.

Significance This paper is an important empirical test of the framework for the study of research governance on scientific fields. We also believe that it contributes to the understanding and methodology for tracing governance effect on global scientific fields.

References Feenstra, R., López-Cózar, E.D. (2022) The footprint of a metrics-based research evaluation system on Spain's philosophical scholarship: An analysis of researchers' perceptions, Research Evaluation, 1-15. https://doi.org/10.1093/reseval/rvac020 Glaser, J. (2019) ‘How Can Governance Change Research Content? Linking Science Policy Studies to the Sociology of Science’, in Simon D., Kuhlmann S., Stamm J. and Canzler W. (eds) Handbook on Science and Public Policy, 419–47. Cheltenham, UK/Northampton, MA, USA: Edward Elgar. Kozlowksi, D., Larivie're, V., Sugimoto, C. R., and Monroe-White, T. (2022) 'Intersectional Inequalities in Science', Proceedings of the National Academy of Sciences of the United States of America, 119, e2113067119 Lorenz, C. (2012) ‘If You’re So Smart, Why Are You under Surveillance? Universities, Neoliberalism, and New Public Management’, Critical Inquiry, 38, 599–629. Luo, J., Ordóñez-Matamoros, G., Kuhlmann, S. (2019) The balancing role of evaluation mechanisms in organisational governance—The case of publicly funded research institutions, Research Evaluation, 28 (4), 344–354, https://doi.org/10.1093/reseval/rvz022 Luukkonen, T., and Thomas, D. A. (2016) ‘The ‘Negotiated Space’ of University Researchers’ Pursuit of a Research Agenda’, Minerva, 54, 99–127. Nedeva, M., Tirado, M.M., and Thomas, D.A (2022) Research governance and the dynamics of science: A framework for the study of governance effects on research fields, Research Evaluation, 1-12. https://doi.org/10.1093/reseval/rvac028 Paradeise, C., and Thoenig, J.-C. (2015) In Search of Academic Quality. Houndmills, Basingstoke: Palgrave Macmillan. Ramos-Vielba, I., Thomas, D. A., and Aagaard, K. (2022) Societal Targeting in Researcher Funding: An Exploratory Approach, Research Evaluation, 31, 202–13. Strinzel, M., Brown, J., Kaltenbrunner, W., de Rijcke, S., and Hill, M. (2021) Ten Ways to Improve Academic CVs for Fairer Research Assessment, Humanities and Social Science Communications, 8, 251. https://doi.org/10. 1057/s41599-021-00929-0. Thomas, D. A., Nedeva, M., Tirado, M. M., and Jacob, M. (2020) Changing Research on Research Evaluation: A Critical Literature Review to Revisit the Agenda, Research Evaluation, 29, 275–88. Vinkenburg, C. J. (2017) Engaging Gatekeepers, Optimizing Decision Making, and Mitigating Bias: Design Specifications for Systemic Diversity Interventions, The Journal of Applied Behavioral Science, 53, 212–34.

ABSTRACT. 1. Question

How do scientific laboratories vary in how they operate, and do they change over time? A legacy of scholarship in the study of scientific productivity has established the demographic, career, and resource factors that explain scientific production. A notable pattern in this work is a preeminent focus on individual investigators.

Whereas some scholars underscore that this focus on individual investigators is an artifact of the availability of data, an emphasis on individuals is especially problematic in the context of scientific productivity because labs are known to exhibit considerable heterogeneity. Science and technology studies demonstrates that how labs assemble people, technologies, and resources varies substantially and meaningfully in ways that are consequential to productivity and innovation. Furthermore, organizational studies further attests to the fact that organizational heterogeneity is significant for explaining differences in performance, innovation, and turnover. Organizations embrace ideologies, logics, “blueprints,” or models for organizing which is instantiated in how they operate and can guide subsequent organizational evolution.

Therefore, we ask, how do lab models vary among scientific labs across organizations and over time?

2. Theoretical Framework

From the literature on organizational emergence, persistence, and change, we establish two tenets on how organizations evolve. First, organizations are reflections of their time and are imprinted with the characteristics at the time of their founding. In this view, the features adopted by organizations during founding are highly stable and persist over long periods of time. Second, organizations can learn, adapting to new experiences or developing new forms of knowledge through sensemaking and the enactment of new patterns of cognitive associations. In this view, organizations may change with the experience of success and failure, changes in networks, the incorporation of new information, ambiguity, or by random chance.

Drawing on and comparing the literatures on organizational imprinting and organizational learning, we compare two competing sets of propositions, notably:

1. Imprinting: Scientific labs express an organizational model established at time of founding that persists over time. 2.Learning: Scientific labs express an organizational model that changes and adapts over time that increasingly differs from its time of founding.

We evaluate these competing propositions in the field of academic scientific labs across over 30 American universities using a combination of statistical models and machine learning.

3. Data

The data we use to examine the organization of scientific labs comes from the Institute for Research on Innovation & Science (IRIS) UMETRICS (Universities: Measuring the Impacts of Research on Innovation, Competitiveness and Science) research dataset. The 2020 UMETRICS release consists of data provided by 33 universities over a time frame of 1 to 19 years. UMETRICS captures expenditure data, including data on awards, wage payments to individuals, and vendor transactions resolved at the month level.

To identify labs, we associate each faculty member to his/her/their set of awards. For each faculty member, we generate nonoverlapping, 1-year windows that represent “snapshots” derived from monthly expenditures. For each faculty-window, we construct manifest variables derived from expenditure activity. These measures provide a window into the variety of ways labs search for support, spend their money, and assemble personnel. As such, we use these measures to identify lab models.

4. Methods

To characterize the heterogeneity of lab models, we exploit two independent clustering methods: a model-based method, latent class analysis (LCA), and an unsupervised machine learning method, k-medoid clustering based on Gower’s distance. A fundamental analytical decision involves specifying the appropriate number of clusters to represent the data. Thus, we utilize both clustering strategies to assess the extent to which clusters derived from both methods overlap. To assess the extent to which lab models persist or change over time, we sort lab windows into clusters and track the trajectory of a lab across its windows over time.

5. Results

5-1. Lab Models

Our interpretation of the latent classes and a k-medoid clustering analysis of a sample of the UMETRICS data suggest a 6-class solution represents the most parsimonious solution.

Most labs employ what we term a small venture lab model: these labs are unexceptional when it comes to expenditures, number of federally funded projects, and personnel, yet nonetheless make use of competitive federal funds to conduct research.

In contrast, a subset of labs employs models that are characterized by relatively big spending and the lion’s share of federal awards. Two of these have a strong association with medical schools and include the Medical Research Facility and the Specialized Medical Boutique: the former is notable for its employment of postgraduate research staff, while the latter is notable for being helmed by an English PI. The Productive Powerhouse, while associated with a medical school, holds multiple administrative affiliations across the university. It employs relatively more graduate and postgraduate employees, and these employees are constituted by relatively more women, ethnically English individuals, and individuals on multiple awards (i.e., their funding is split across different award accounts).

The two remaining lab models are unexceptional when it comes to expenditures and awards yet pull their weight in the employment of graduate students. The Engineering Factory is a leaner version of the Productive Powerhouse and is strongly associated with a school of engineering. The Doctoral Factory is associated with a greater number of graduate employees and graduate employees with split funding.

5-2. Lab Models Over Time

To characterize the persistence and change of lab models over time, we estimate the posterior probability that a particular faculty-year “belongs” to a particular latent class and assign each to one and only one class. Next, we assemble each faculty-year for each faculty member (N = 64,733) to construct a chronological account of each faculty’s lab model.

We find that a slim majority of faculty employ the same lab model throughout the course of their career on record (27,565) compared to those who make at least one switch (25,114). However, the number of switches a faculty member incurs appears to correlate with the length of their recorded career. Furthermore, faculty who begin their careers with certain lab models are less likely to make a switch later in their career.

Thus, we find strong evidence for organizational imprinting, i.e., that scientific labs express a model established at time of founding that persists over time. However, we also find evidence for organizational learning, i.e., that scientific labs express a model that changes and adapts over time that increasingly differs from its time of founding.

6. Significance

Our analysis advances our understanding of scientific production and the organization of scientific laboratories in three ways. First, our study addresses the persistent call to study the evolution of organizational populations beyond the rarified set of Fortune 500 companies. Second, our study offers an operationalization for the organization of scientific labs that enhances the fit between our understanding of science as a social system with our actual measurement. While data platforms like UMETRICS improve our capacity to analyze the value of American research and development, our conceptual tools for utilizing such platforms in service of theorizing scientific production require attention. Finally, we shift the spotlight away from individual investigators to laboratories that propel the frontier of scientific knowledge, participate across multiple networks of activity, and produce graduates who go on to populate the scientific and engineering workforce.

ABSTRACT. Background Government investment in basic research produces benefits that counteract the forces that lead the private sector to underinvest in such research (Rosenberg, 2009). Governments differ in how they organize publicly funded scientific research. In general, this spending is divided between funding distributed to performers outside of the government, such as universities and private firms (i.e., extramural R&D), and funding to government research facilities (intramural R&D). Intramural research facilities, often termed “public research institutes” or PRIs, are assumed to undertake research directly relevant to governmental needs that might not be pursued by extramural performers. However, few studies of PRIs exist in the published literature. The OECD launched a study in 1989 on government research laboratories, with a follow-up study conducted in 2011 (Sanz-Menéndez et al., 2011). In the United States, Bozeman and Crow (1998) published a series of examinations of federal government research laboratories. Cruz-Castro et al. (2020) and Zacharewicz et al. (2017) produced a series of scholarly works looking at organizational and managerial issues in public research organizations, and Hallonsten (2017) recently conducted a review of what he terms the “third sector of R&D” (encompassing government research laboratories but including non-governmental research institutes).

This study focuses on a specific form of PRI in the United States: nationally owned research laboratories operated as federally funded research and development centers (FFRDCs). FFRDCs evolved from their origin in World War II to constitute a significant part of the U.S. public R&D enterprise, accounting for approximately 36 percent of the research performed at federal facilities in 2018. The FFRDC is an organizational form different from most PRIs in other nations. These institutions are typically formed and owned by the U.S. federal government, but operated by academic, industrial, and special-purpose entities under bespoke agreements with specific agencies (known as the FFRDC’s “sponsor”). As a result, these PRIs straddle the boundary between intramural and extramural government laboratories. They retain the status of a governmental entity, but with many of the attributes of a non-governmental organization. The current study is limited to a specific subset of FFRDCs, the Research and Development Laboratories (RDLs). Compared to other types of FFRDCs (Study and Analysis Centers and Systems Engineering and Integration Centers), RDLs are major producers of multi-disciplinary scientific knowledge (Hruby et al., 2011).

  Key Research Questions Our paper presents descriptive statistics about scientific output from 1981 through 2020. We analyze the publications for all Research and Development Laboratories (RDLs) and for specific subsets of the RDLs (such as laboratories functioning primarily as scientific user facilities) by field of science. We characterize the scientific impact of that output using measures such as the average of relative citations. We also analyze patterns of collaborative research at RDLs (based on co-authorship networks across all scientific papers). Specifically, we examine the following sets of research questions:

RQ1 How has the production of scientific knowledge at RDLs (measured as the number of scholarly articles) varied relative to overall federal basic research funding to FFRDCs? What differences can we observe in scientific productivity and impact at specific RDLs, especially between user facilities and multidisciplinary research centers?

RQ2 As US federal entities, RDLs are expected to focus on national priorities, and their R&D activities are expected to benefit the US science and technology enterprise. We can characterize the “consumption” of RDL input by analyzing the institutional affiliations of authors who cite RDL publications. How does the citation of RDL publications vary across the nations and types of institutions citing those works?

RQ3 US federal R&D funding is concentrated in mission-oriented agencies, such as the Department of Defense, Department of Health and Human Services, and NASA. How has the disciplinary focus of RDLs shifted in relation to the R&D budget priorities of their sponsoring agencies?

Data and Methods This research-in-progress analyzes the place of FFRDCs in the US basic research landscape, primarily through bibliometric analysis of the publication output of the 29 RDLs as designated by the National Center for Science and Engineering Statistics (NCSES). Data for this paper was obtained from the Web of Science (WOS) database, covering the Science Citation Index Expanded, the Social Sciences Citation Index, and the Arts and Humanities Citation Index, over the 1973-2021 period. The name variations of each FFRDC analyzed were retrieved from the institution and department fields of the WOS. Each paper was assigned the subfield of the journal in which it is published according to the NSF field and subfield classification of journals (Hamilton, 2003), and this classification scheme was used for the field-normalization of their number of citations (Waltman et al., 2011). Full counting of papers was used—each FFRDC contributing to a paper obtained a whole unit. Networks were created with the UCINET and Netdraw softwares (Borgatti, 2002; Borgatti, Everett & Freeman, 2002).

We also obtained public-use data files and other outputs from two NCSES surveys: the annual FFRDC R&D Survey (the FFRDC Survey), and the Survey of Federal Funds for Research and Development (the Federal Funds Survey). The FFRDC Survey captures measures of R&D spending, and some data on the sources of funding for those R&D expenditures. The Federal Funds survey provides data on overall R&D obligations by specific federal agencies by field of science and engineering, and also R&D obligations to individual FFRDCs.

We use the FFRDC and Federal Funds Surveys to provide information on the resources (funding obligations) and activities (expenditures) for R&D at each RDL, and for their primarily federal government research sponsors. We then set up various models to derive findings on how the FFRDCs’ scientific output changes over time in response to shifts in federal agency R&D priorities and in the nature of research conducted at these facilities.

Significance Link and Scott (2021) conducted an analysis of RDL publications for eight laboratories, and used their data to estimate the elasticity in the production of scientific output for those laboratories (the number of additional publications generated from incremental budget increases). Our work goes beyond this foundation by providing a more comprehensive dataset of publications over a longer period of time, so we can detect long-term trends. We also are able to investigate more thoroughly how publication volume and patterns vary based on the nature of the laboratory and the distribution of R&D funding received by agency and field. We will present our findings showing that the RDLs, in general, produce publications with greater impact than overall U.S.-authored publications (using citation measures). We also show that RDLs are heavily engaged in international collaboration, and that their knowledge outputs are also consumed significantly by nations other than the United States. We also estimate how RDLs, consistent with the literature on PRIs, change their scientific output in response to changes in the R&D priorities of their sponsors and of the U.S. national government.

ABSTRACT. Data is increasingly considered to be a key component in stimulating innovation. Numerous promising possibilities have been opened up by rapidly emerging, data-intensive technologies, including the Internet of Things (IoT) and artificial intelligence (AI). The analysis and interpretation of big data are critical in the growth of technology firms in terms of AI training and computing capabilities. Small and medium-sized enterprises (SMEs), with their limited resources internally, particularly face a serious challenge of implementing innovation, which increasingly requires the effective processing and use of various kinds of data. The smart city provides an important opportunity for creating data-driven innovation. Significant amounts of data are increasingly available from various sources through sophisticated devices and equipment scattered in smart cities. Many smart city projects across the globe provide rich opportunities for SMEs to explore data-driven innovation. China, in particular, has recently been active in collecting and utilizing various kinds of data in smart cities. The availability of and access to data help to improve the software development of private enterprises in China, where massive amounts of data resources are collected and maintained by the public sector. In the process of smart city development, there are also many tasks that are complementary to each other, including connecting databases, building online platforms that connect different data coming from various data sources, operating and maintaining these online platforms, and providing products and services to citizens. These diverse kinds of tasks involved in smart city projects initiated by local governments have brought about new business opportunities for innovative SMEs in China. To implement the policies of encouraging the development of SMEs by the central government, municipal governments have introduced policies that give priority to SMEs in participating in smart city projects. Those companies that have access to the data collected in smart cities and held by government agencies are expected to benefit from utilizing the rich data for creating innovative products and services. There were few empirical studies conducted, however, to examine how data are actually managed and provided in smart cities and how they affect companies’ innovative activities. It remains unclear how public agencies and private enterprises collaborate on data and how that influences the innovation performance of SMEs in China. In smart cities, different types of public-private collaboration are involved, including hardware purchase, platform building, platform operation, and data analysis. It is not yet well understood how these different types of collaboration influence the availability and accessibility of data and consequently the innovative performance of SMEs. In mainland China, smart city development has been promoted actively to tackle severe urban issues, including air pollution, traffic congestion, and public safety. The Ministry of Industry and Information Technology (MIIT) established the China Smart City Industry Alliance in 2013 to implement smart city projects, and a policy paper, Guidance on Promoting the Healthy Development of Smart City, was published in 2014 by eight government departments. In 2015, the development of smart cities was promoted by the Prime Minister as the future development direction for cities in China. Encouraged by these policy measures, nearly 300 smart city projects have been approved so far. Smart city development in China has particularly emphasized the linkage and synergy with big data through advanced information and communication technologies, including IoT and cloud computing. Very few empirical studies have been conducted so far to investigate what kinds of data are collected in smart cities, how these data are available, who has access to the data, how these data are managed, what incentives are provided to encourage data sharing, and what impacts are made on stimulating innovation. In this study, we examine how data are managed through collaboration between the government and companies in smart cities and how the mode of data collaboration influences firms’ performance on innovation in China. By focusing on the case of SMEs in the Guangdong province, this research aims to shed light on what kinds of data are available and used in smart cities and how the government and enterprises collaborate on data to facilitate innovation. The analysis of this study utilizes data on more than eight million contracts extracted from the Government Procurement Database managed by the Ministry of Finance. The database contains rich information on government procurements, including the goods and services procured, the date of the contracts, and the monetary size of the contracts. Data on companies are assembled with regard to the registered capital, industry, software products, and patents in 1990-2021 from the database of Tianyancha. The applications of patents are also analyzed by using the patent database maintained by the State Intellectual Property Office (SIPO) of China. The SIPO patent database provides complete information on all patent applications and grants in China, including the application and publication number of the patent, the application and grant years, the classification number, the type of patent, and the assignee of the patent. The knowledge and technological domains are identified by analyzing the patent classification numbers and the content of the descriptions of the inventions. By using these data sources, panel data is established with key characteristics of SMEs, software and patents outputs, and their record on government contracts. The contracts are divided into three categories, namely, data analysis, platform building, and hardware purchase, based on keyword identifications. To deal with the unbalance between the treatment group (the companies that obtained government contracts) and the control group, we use propensity score matching (one-to-one nearest neighbor matching) to narrow down the sample size of the control group to that of the treatment group. Then we apply the event study methodology to examine whether there are significant differences in innovative outputs of software products and patents before and after the companies receive government contracts. We also compare how the innovation performance of companies differs based on the types of contracts these companies obtain. That makes it possible to identify what kinds of data collaboration would be effective in improving the innovative performance of SMEs. Our preliminary analysis suggests that many of the government contracts obtained by SMEs that have innovation outputs would be concerning platform building. After receiving government contracts, firms that conduct data analysis for smart city projects would tend to have more patent outputs compared with companies with similar characteristics. On the other hand, firms that implement platform building and platform operation for smart cities would be likely to have more software products. That difference in innovative performance would be considered to be influenced by the quantity and quality of data available and accessible to companies. While government contracts for buying hardware devices and equipment for smart city development would encourage innovation in technological products and services in SMEs, they would not have a significant impact on the innovative performance of these firms. Government purchases of hardware that contains data-intensive technologies would promote the use of these products in smart cities. That, however, would not involve any substantive exchange or transfer of data possessed by the government and would not significantly contribute to stimulating innovation at firms. Policy implications will be explored for various modes of data governance, including government-led, industry-led, and public-private partnership approaches, with their impacts on facilitating innovation and addressing societal concerns about data security and privacy.

ABSTRACT. Title: Are Digital Innovation Policies Effective in Promoting the Development of Digital Economy in China?

How latecomers of emerging countries catch up with the incumbents on technological innovation is always an important topic (Guo et al., 2021). China is also facing challenges in upgrading its manufacturing value-added in the global market and transferring to a more sustainable economic growth model. Digital technology has become the driving force of Chinese economic growth and transition (Li, Hsu, Mao, & Zhang, 2022). In 2013, the size of the digital economy was about 9.5 trillion, accounting for 20.3% of China's GDP. In 2020, the size already increased to around 39.2 trillion, taking a share of 38.6% of GDP. China is also a leading digital economy in the global market. According to CGTN ( Empowering digital transformation for china's economy. 2022), the size of China's digital economy is ranking second in the world in 2020.

Three elements shape the development of China's digital innovation: a large consumer market, a well-developed supply network of digital products, and a unique societal and regulatory structure (Li et al., 2022). On the one hand, in the past ten years, the rise of the middle classes has stimulated consumer demand for high-end products. It has brought a large group of users to digital innovation enterprises in China. On the other hand, the "deep supplier networks, a large skilled workforce, and a developed logistics infrastructure" have enabled China to develop a completed digital innovation ecosystem. As the Chinese government plays a predominant role in economic development, China's institutional system has significantly affected demand and digital supply-side innovation. China has pointed out to develop a national strategy for the digital economy in its Medium to Long-term Planning for Scientific and Technological Development plans. For example, in the Thirteenth Five-Year Plan Outline of National Economic and Social Development of the People's Republic of China, the Chinese government proposed implementing a national big data strategy and promoting the open sharing of data resources (Xinhuanet, 2016). Furthermore, in Fourteenth Five-Year Plan Outline, the Chinese government has proposed accelerating digital development and building a Digital China (Xinhuanet, 2022). In response to the government's national strategy, Chinese governments at all levels have introduced policies for digital economy development and digital transformation. The data from the Chinese policy database shows that in 2013, the Chinese government introduced 342 digital innovation policies, and in 2021, the Chinese government introduced 1242 digital innovation policies. The figures have tripled in the past eight years. Although the Chinese government has implemented a series of innovation policies to stimulate digital innovation in different regions of China, existing digital innovation studies may not systematically evaluate and compare the effectiveness of these policies in fostering digitalization in China. The government may waste many valuable resources on policies that are not very effective. Therefore, it is incredibly intriguing and helpful to find out how the Chinese government uses policy instrument mix to promote the development of the digital economy and comprehensively evaluate the policy effectiveness of digital innovation. Thus, the research questions of this paper are as follows:

Research Question 1: "What kind of policy instruments are frequently used by the Chinese government to promote the digital economy?"

Research Question 2: "Is every type of policy instrument effective in boosting the digital economy?"

Research Question 3: "Among all applied policy instruments, which policy instruments are more effective in facilitating the digital economy?"

The unit of analysis is a province (excluding Hong Kong, Macaw, and Taiwan). This study will collect two kinds of data. The first is the 2013-2021 digital innovation policy document at the province level, collected from Bailu Zhiku (http://www.bailuzhiku.com/) by a web's crawler. The second is the 2020 digital economy index by provinces, collected from the microdatas.cn database. The approach of Natural Language Processing will be used to clean and process the collected policy documents. In this process, some Chinese keywords (“税收减免Tax deduction,” “孵化器 Incubators,” “政府采购 Procurement” and “补贴 Subsidy” etc.) will be used to identify the policy instruments in the policy text. Based on the innovation policy instrument framework proposed by Elder et al. (2016), we define thirteen types of instruments in this study, including "R&D tax credits," "Direct R&D support," "Training and skills," "Entrepreneurship," "Technical services and advice," "Cluster," "Collaboration," "Innovation network," "Procurement," "Innovation prizes," "Standard," "Regulations," "Technology foresight." This study will also construct a multivariate regression model to test if a certain innovation policy instrument significantly impacts the digital economy development of provinces in China and to compare the effectiveness of different types of innovation policies.

The preliminary results show a huge regional heterogeneity of digital innovation development in China. Coastal regions like "Guangdong" (N=302), "Zhejiang" (N=193), "Jiangsu" (N=188), "Fujian" (N=165), "Shandong" (N=128), "Beijing" (N=100), "Tianjin" (N=85) and "Shanghai" (N=73) have launched more digital innovation policies. Southern provinces pay more attention to digital innovation than northern provinces (Guangdong, Zhejiang, Jiangsu, and Fujian compares to Shandong, Beijing, and Tianjin). Besides, Chinese governments frequently use direct support for R&D and innovation, training and skills, and cluster policy when developing a digital economy. Lastly, from the multivariate regression results, we observe that both demand (=12.57, p<0.05) and supply policies (=2.62, p<0.05) can significantly improve the development of the digital economy development and the demand-side policy is more effective than the supply-side policy.

  References Elder, J., Cunningham, P., Gok, A., & Shapira, P. (2016). Handbook of innovation policy impact. Beaverton: Ringgold, Inc. CGTN (Producer), & . (2022). Empowering digital transformation for china's economy. [Video/DVD] Retrieved from https://www.youtube.com/watch?v=2ZlBAnKUVoA Guo, B., Ding, P., Greidanus, F. J., & Li, W. H. (2021). What makes a successful industry-level catch-up? general framework and case study of china’s LED industry. Frontiers of Engineering Management, 8(2), 284-309. Li, L., Hsu, C., Mao, J., & Zhang, W. (2022). Contextualising digital innovation in today's china: Local practices and global contributions. Information Systems Journal (Oxford, England), 32(3), 623-629. doi:10.1111/isj.12379 Xinhuanet. (2016). 中华人民共和国国民经济和社会发展第十三个五年规划纲要. Retrieved from http://www.xinhuanet.com//politics/2016lh/2016-03/17/c_1118366322.htm xinhuanet. (2022). 中华人民共和国国民经济和发展第十四个五年规划和2035年远景目标纲要. Retrieved from http://www.gov.cn/xinwen/2021-03/13/content_5592681.htm

ABSTRACT. Studying the dynamics of data-driven innovations (DDI) is becoming increasingly important to map the drivers for developing data-intensive solutions by SMEs. Such an analysis provides substance to an analysis of barriers to the adoption of DDI as well as the impact of policies and legislative acts, like the Data Act, and public-funded data-sharing platforms, on fostering innovation and promoting growth and social well-being. To that end, a combined collection of structured and unstructured data will be used to analyze the adoption and diffusion of DDI in the German industry.

ABSTRACT. This paper is submitted to the panel entitled "The Dynamics of Science and Technology Governance in Emerging Sectors" (submission 6247)

Abstract This paper aims to study the emergence of emerging innovation ecosystem with focus on three selective intelligent technologies, including artificial intelligence (AI), Internet of things (IOT), and intelligent manufacture. The key research question this talk try to answer is how to form and develop the innovation ecosystem of emerging technology industries? How does the institutional factor influence the evolution of innovation ecosystem during the development process of science and technology? As emerging technology is constantly developing rapidly, its definition, scope, application, and technology governance approaches align with the emergence of emerging technologies. This paper considers the current development status of various industries and the recent policy content of emerging technology industries mostly aims to promote smart technologies such as artificial intelligence, the Internet of Things and smart manufacturing. Though, establishing innovation policy framework to enhance the governance of the emerging technology are important, but the empirical studies focusing on the convergence of various emerging sectors and social impacts on the technology emergence process are still less clear. This paper will therefore explore the formation of the innovation ecosystem base on the multilevel prospective, taking academic research, technological development, commercialization process, and social impact on the three emerging intelligent technologies. In addition, we will focus on the interplay between multilevel prospective on emerging sectors. Finally, the main focus of this panel will move onto macro level with focus on the interactions between emerging sectors and the institutions. Combining social network analysis and multilevel analysis, this research will analyze the innovation ecosystems of various emerging sectors. Ultimately, this project will propose tailor-made policy recommendations to further promote the developments of emerging sectors.

ABSTRACT. Background

Studies of science and science policy often rely upon analyzing the ‘structure’ of research fields. This is used to explore how scientific fields evolve, whether they are affected by policy and funding, and performance comparisons across universities and research systems (Porter & Rafols, 2009; Braam & van den Besselaar, 2014; Langfeldt et al. 2020). Traditionally, field structure has been investigated via citation-based relationships, touching upon fields’ knowledge pools (Van Raan & Tijssen 1993; Creswell 2009; Porter & Rafols 2009; Boyack & Klavans 2014). Elsewhere, as in classical sociology, structure is explored through a mix of social organisation and intellectual aspects, including the norms, intellectual conventions, governance rules and authority relations of ‘scientific communities’ (Merton 1968, Crane 1971, Whitley 2011).

In this paper, we instead explore co-existing ‘structures’ of a research field, comprising an ‘interlaced’, ‘multilayered’ or multiplex network phenomenon (Heimeriks et al. 2003, Teurtscher et al. 2014).

Rationale

Traditional approaches to investigating and mapping the structures of research fields, we posit, while in part useful to the study of science dynamics, have some unfortunate shortcomings. First, by assuming a unitary structure of science/scientific fields these approaches limit the investigation of complex, inherent dynamics. And second, by often inferring structures and relationships rather than mapping these directly, the explanatory power of traditional approaches is somewhat limited.

Still, unpacking the sources of inherent science dynamics, or the dynamics of scientific fields, is vital for understanding the evolution of science, the ways in which exogenous factors interlace with inherent sources to shape the workings of the sciences.

Methods

Building on an understanding of structure originating with ‘new institutionalism’ (Powell & DiMaggio, 1991) we assume that relationships can be empirically accessed through (social) exchange or distribution.

Using co-nomination to trace three different kinds of exchange occurring within scientific networks, we mapped three different structures in a physics field – intellectual, collaboration and technical. (For more on the use of co-nomination to map research fields please see Karaulova et al. 2020)

Social network analysis techniques were used to investigate further the links between the different networks/structures. The network maps were generated using VantagePoint and Gephi.

Results

To test this ‘structures’ perspective, we used multiple rounds of co-nomination analysis, a reputation-based approach combining snowball sampling and social network analysis (Karaulova et al. 2020). We mapped three exchange networks for a particle physics field.

We then highlight actors simultaneously active in one, two or three networks. This shows researchers performing multiple roles, and engaging multiple exchange relationships, perhaps not captured by citation-based approaches or study of single networks. It foregrounds the importance of other roles beyond intellectual influence, such as functions that technicians and equipment developers undertake, and social organizer roles that may help crystallize the field – a task more often associated with women, according to our in-progress analysis.

Significance

This ‘structures’ approach adds multidimensional character to whether scientists appear central or peripheral in a field. It has potential implications for how researcher performance is evaluated by universities, for organized mitigation of intersectional biases in science, and for how funders tackle coordination dysfunctions in fields developing over time (Tuertscher et al. 2014; Kozlowski et al. 2022).

Overall, it promises to transform how we study research fields and science dynamics, by shifting our perspective from one-dimensional ‘structure’ to key multiplex ‘structures’. Using it to study additional cases could lead to better understanding of how research governance can support fields. It could also drive policy and funding to become tailored to bespoke dynamics of key field ‘types’, as revealed by their multiplex structures.

References

Braam, R., van den Besselaar, P. (2014) Indicators for the dynamics of research organizations: a biomedical case study. Scientometrics 99, 949–971. https://doi.org/10.1007/s11192-014-1235-5 Boyack, K. W., & Klavans, R. (2014) Creation of a highly detailed, dynamic, global model and map of science. Journal of the Association for Information Science and Technology 65: 670–685. Scopus. https://doi.org/10.1002/asi.22990. Crane, D. (1971). Transnational networks in basic science. International Organization,25, 585–601. https://doi.org/10.1017/S0020818300026333 Creswell, J. W. (2009). Editorial: Mapping the field of mixed methods research. Journal of Mixed Methods Research,3, 95–108. https://doi.org/10.1177/1558689808330883 Heimeriks, G., Hörlesberger, M., & Van Den, P. Besselaar (2003). Mapping communication and collaboration in heterogeneous research networks. Scientometrics, 58, 391–413. https://doi.org/10.1023/A:1026296812830 Karaulova, M., Nedeva, M. & Thomas, D.A. (2020) Mapping research fields using co-nomination: the case of hyper-authorship heavy flavour physics. Scientometrics 124, 2229–2249. https://doi.org/10.1007/s11192-020-03538-x Kozlowski D, Lariviere V, Sugimoto CR, Monroe-White T.(2022) Intersectional inequalities in science. PNAS 119, 1-8. https://doi.org/10.1073/pnas.2113067119 Langfeldt, L., Nedeva, M., Sorlin, S., and Thomas, D. A. (2020) ‘Co-Existing Notions of Research Quality: A Framework to Study Context-Specific Understandings of Good Research’, Minerva, 58, 115–37. Merton, R. K. (1968). Social theory and social structure. New York: Simon and Schuster. Porter, A.L., and Rafols, I. (2009) Is science becoming more interdisciplinary? Measuring and mapping six research fields over time. Scientometrics 81, 719. https://doi.org/10.1007/s11192-008-2197-2 Powell, W. W., & DiMaggio, P. (Eds.) (1991). The New Institutionalism in Organizational Analysis. University of Chicago Press. http://catdir.loc.gov/catdir/toc/uchi051/91009999.html Tuertscher, P., Garud, R., and Kumaraswamy, A. (2014) Justification and Interlaced Knowledge at ATLAS, CERN. Organization Science 25(6):1579-1608. https://doi.org/10.1287/orsc.2013.0894 Van Raan, A.F.J., Tijssen, R.J.W. (1993) The neural net of neural network research. Scientometrics 26, 169–192 . https://doi.org/10.1007/BF02016799 Whitley, R. (2011) ‘Changing Governance and Authority Relationships in the Public Sciences’, Minerva, 49, 359–85.

ABSTRACT. Background and Rationale

Synthetic biology is a recently established field in which practitioners re-design and engineer organisms or parts of organisms with the intention of changing their behaviors or characteristics. The hope is that such genetically modified organisms or components can help address pressing societal challenges, such as pandemics, global warming, pollution, and food security. Synthetic biology, for example, was integral in the development of mRNA vaccines to combat COVID-19.

Synthetic biology, however, is not only practically consequential; it is also theoretically interesting. Theoretical frameworks, such as the Triple Helix model and Pierre Bourdieu’s field theory, treat social life as composed of different domains with varying levels of dependence between them, for example the government, academia, and industry in the Triple Helix model. Some researchers might be more focused on producing research to be consumed primarily by individuals in the same field, whereas researchers in other fields often target their research at solving problems in areas unrelated to their field of study, e.g., how to make more pest resistant crops or how to inform policy decisions about the allocation of funds to social welfare programs. Synthetic biology is positioned at the nexus of many fields: seeking to solve myriad societal problems identified by policy makers and other researchers and often relying upon the private sector to support and implement research products.

Social media studies of science encourages researchers to focus on more than scientists and scholarly objects on social media and suggests that this would be important for synthetic biology, where non-academic communication is particularly important in bringing together a variety of stakeholders. Such communication occurring between fields and sectors frequently occurs on social media, and synthetic biology is no different. This study poses the following question: in a social media environment where diverse stakeholders discuss synthetic biology as well as its applications and implications, what specifically is discussed? Answering this question is essential not only to developing a more encompassing study of synthetic biology, beyond traditional bibliometric and even altmetric data, but also to understanding more generally how the interdependence of certain scientific fields with other fields, businesses, and even sectors impacts the approaches needed to study them.

Data and Methods

Data used in the following analyses come from Twitter, a common platform for synthetic biologists and other actors who communicate about synthetic biology. The Twitter Application Programming Interface (API) allows users to programmatically query the full history of tweets. Because research indicates that synthetic biology stabilized as a research field around 2010, the data in this paper consist of tweets from 1 January 2010 through 30 September 2022.

The following query was used to obtain relevant tweets during this period: #synbio OR #syntheticbiology OR "synthetic biology" OR synbio OR (iGEM -igemcity) OR (#iGEM -(#igemcity)). Hashtags are a feature of social media platforms that allow users to connect their posts with ongoing conversations without having to respond to specific individuals. Querying phrases that are not hashtags identifies relevant posts, even if they are unconnected to broader conversations identified through hashtags. Synbio is a common label used as shorthand for synthetic biology. iGEM stands for the International Genetically Engineered Machines competition, which brings together teams of high school students, college students, postgraduates, and community lab members from across the globe. Manual review of synthetic biology Twitter content and discussions with subject matter experts indicated that iGEM constitutes a significant part of the synthetic biology discourse on Twitter. Tweets were preprocessed by removing emojis, numbers, URLs, hashtags, account mentions, retweets, and non-English tweets and by making all text lowercase. There are 396,154 tweets in this corpus.

Hierarchical stochastic blockmodels (hSBMs) were used to analyze the content of this tweet corpus. hSBMs may be considered an unsupervised method because they help users understand—without specifying any prior knowledge or thematic structure—the different topics discussed in a set of texts. hSBMs represent tweets as a bipartite network in which tweets and the words used within them are connected and then uses statistical approaches to community detection to uncover topics. Topics are conceived as communities in which words are densely connected to each other because they frequently co-occur in tweets. hSBMs do not require the user to specify beforehand a given number of topics, which is usually required for methods like topic models despite being a quantity of interest, and can represent topics as hierarchically nested.

Results

The hSBM identifies four “levels” of topics. The most abstract level contains 25 topics, and the most concrete level contains 924 topics. The second and third most abstract levels contain 84 and 230 topics, respectively. The most abstract level has topics about the iGEM competition, biotech startups, global markets, genomics research, major news stories, molecular engineering research, and popular science writing, among others. Topics at the most concrete level are very specific. One example topic is specifically about sustainability and research containing specific organisms (e.g., pseudomonas), and another is about sustainability specifically regarding biofuels. A final example of a very focused topic mentions the market performance of certain synthetic biology companies. The intermediary levels unsurprisingly present topics that are less focused or concrete but still not as abstract as at the highest level. One finding common to all topic levels is that non-academic content is much more common than academic content.

Conclusion

The fact that many topics do not pertain to academic research and/or research objects (e.g., article DOIs) reinforces the call of social media studies of science to expand the scope of studies on social media beyond scientists and their immediate activities. Synthetic biologists, for example, rely upon tooling and equipment from industry, and companies often work closely with professors to launch new products. Crucially, social media discourse surrounding synthetic biology also includes attention to ethical, legal, and societal issues, like sustainability and dual-use research. Theoretically, this means that domains in which scientists’ activities are oriented toward other types of actors (e.g., businesses, policy makers) or solving applied problems cannot be fully understood by studying only scientific output. On a more practical level, this means that studies often need more than extant compiled data sources, e.g., altmetric statistics, because such sources do not contain data necessary to understand science-society dynamics.

ABSTRACT. Introduction The fifth generation (5G) of mobile communication network (which succeeds 4G) has been accompanied by an increase in the number of connected devices and thus an increasing consumption of mobile data worldwide. In addition to increasing throughput and bandwidth, the introduction of this technology will support future applications that will impose additional requirements, including low connection latency. While the literature abounds with scientific articles addressing 5G, there is a lack of studies exposing a more detailed view of scientific collaboration in this area. To fill this gap, this study aims to answer the following questions: 1. What is the trend in terms of scientific collaboration in 5G? 2. What are the structural properties of the co-authorship network and is there an evolution of this structure over time? 3. What are the marked trends in international and inter-university collaboration? 4. Which countries and universities are most central to the network? Question 1 aims to determine the importance of scientific collaboration in the 5G field by assessing the evolution of links between researchers and comparing the proportion of collaborative versus single-author papers (intensity of collaboration). In other words, this first step will determine who works with whom on 5G. In answering Question 2, the co-authorship network will be analyzed at the structural level to identify the existence of a small-world structure (deemed more efficient in terms of knowledge transmission) within the network and to measure the degree of cohesion over time. Question 3 first aims at determining the intensity of international (between countries) and inter-university collaboration. Second, it will measure the evolution of the countries’ collaboration with their main partners. The strong rivalry between Huawei and the other players in the game may not have the same impact in the scientific networks as researchers hedge their bets and play both sides. Question 4 will allow us to identify the most central countries and universities in the network, in other words who have the most influence. In sum, answering all of these questions will brush a broad picture of the evolution of the international network of scientific collaboration in 5G. Literature review Throughout the bibliometric literature on 5G, it is interesting to observe two strong trends. First, several researchers use bibliometrics to paint an international picture of 5G research in general. The idea is to collect all scientific papers on 5G and to focus mainly on the number of papers published, the authors, the countries as well as the most productive and most cited universities (Aslam et al., 2020; Mao, 2021; Semwal et Pande, 2021). The second category of papers focuses solely on a particular aspect of 5G. For example, Farouqi, Arshad, and Khan (2021) review the state of research on security and privacy related to 5G networks. Dixit et al. (2020) focus on papers dealing with the antennas needed to transmit and receive 5G signals. Thus, these studies have paid little attention to the collaboration in this field. To fill this gap, this paper aims a clearer picture of the evolution of the scientific collaboration network related to 5G Data and Methodology All papers were extracted from the Thomson Reuters Web of Science (WoS) database using the term “5G” in the title, abstract or keywords. The choice to restrict ourselves to the term “5G” is due to our desire to eliminate false positives, due for instance to some keywords related to 5G technology components (e.g., milimeter wave, edge-computing, or massive-MIMO) being not exclusively related to 5G. This first search yielded 43 155 scientific articles, from 2000 to 2021 which were then manually examined to eliminate those that did not correspond to the fifth generation of mobile communication. The final database contains 13,561 articles. A series of three-year staggered co-authorship networks was then characterized using SNA packages from R, an open-source statistical software, and Gephi, one of widely used Social Network Analysis software. The methodology therefore comprises two steps: 1. Analyse and understand the evolution of the collaborative structure of 5G co-authorship network the intensity of scientific collaboration, the existence of a small-world structure and the degree of cohesion); 2. Calculate the centrality of countries and institutions in the co-authorship network (degree centrality, betweenness centrality and eigenvector centrality). Preliminary findings Three main findings emerge from this analysis. First, and not surprisingly, 5G research is mostly performed in collaboration (very few articles are published by single authors). For each of the periods under study, more than 96% of the papers are the result of collaboration with an average of four authors per paper. In addition, the network is not highly fragmented since the main component represents more than half of the researchers during the first period (2014 to 2016) until reaching more than 60% in the last period (2019 to 2021). Second, the results indeed show that the co-authorship network possesses the properties of a small world in all periods. Thus, the structure of the 5G co-authorship network indicates an optimal structure for knowledge diffusion and information sharing. This contrasts with the cohesion measures obtained: The results show low density in each of the time periods and a negative trend over time for centralization and centrality by proximity. In other words, there is less and less cohesion around central researchers and it is increasingly difficult for researchers to “reach” others indirectly through the network. Turning now to collaboration, the results show that national collaboration is favored over international collaboration in the 5G domain and that this trend is increasing over the years. For example, it is noticeable that from the 2017-2019 period, China favors domestic collaboration over international collaboration, where it has been losing ground over the last three periods. This is clearly observable by the relative decrease in collaboration with its main collaborators and in particular with the United States. There are several reasons for this trend. First, the weakening of collaboration during the last periods of this study coincides with the arrival of the COVID-19 pandemic, this cannot be ruled out as a possible root cause. Indeed, entry restrictions imposed by the United States impacted collaboration as most research collaborations usually begin with face-to-face meetings, to establish the necessary trust relationship, rather than by video conferencing. Second, exacerbated political tensions between China and the United States from 2018 onwards may also contribute to explaining our observations. Chinese scientists are reluctant to travel to the United States because of foreign interference investigations. For example, the U.S. Department of Justice under the Trump administration launched the “China Initiative” program in 2018 to investigate alleged Chinese spying in research and industry. However, despite the set of measures instituted by the U.S. government, the trend of collaboration with China from the U.S. perspective is still growing: the results show an increase in U.S. collaboration with China that has stabilized in the last two periods (2018-2020 and 2019-2021). Our results also show an increase in collaboration with China from the points of view of Italy, Spain, the United-Kingdom, and Sweden. However, the political tension between the U.S. and China over Huawei’s role in 5G network deployment eventually reached Europe and the rest of the world. This may explain the decrease or stabilization, especially in the last period (2019-2021), of the collaboration between European countries, Japan, South Korea and Canada with China. Finally, with respect to the positioning of countries within the collaborative network, the results show that China, the United States, Great Britain, and France stand out from other countries in all three measures of centrality. Implications This article provides a description of the 5G co-authorship network. The results may be the basis for future work. For example, other studies could investigate in more detail the impact of public policies on collaborative trends at the international level.

ABSTRACT. Scientific research has contributed to increasing human prosperity 1, as well as to creating new human challenges, including environmental sustainability. However, these benefits and challenges are not evenly distributed across the population 2. This is due to how research priorities are directed. For instance, in health, R&D (public and private) focuses mainly on the diseases that mostly affect people from the richest parts of the world, even if these are not the diseases with the highest burden on human life 3,4. In agriculture, ‘revealed’ research priorities in agriculture only partially relate to ‘revealed’ demands for new science7.

There are several reason why research directions do not align well with the societal demands, including: path dependence8, difficulty in navigating the sheer complexity of research and societal demands, as priorities are influenced by several, competing and related factors and actors9–15, with different interests and incentives 16,17, which are unequally represented in research and decision making organisations 18–21.

Behind these systemic reasons is also how researchers themselves respond to incentives that shape their research trajectories. The evaluation of research based on the scientific excellence of its outputs, reduces funder’s decision making to only one of the potential objectives of investment in research. And in evaluating this objective, it misses out the multiple, combinatorial ways, in which research may advance the frontier and, equally importantly, its multiple directions22 and their potential impacts on societies.

In this paper we compare the research that is related to the Sustainable Development Goals (SDGs) with the rest of the published research. Alongside scientific excellence metrics, we use indicators that measure potentially broader contributions of scientific research to societies. We ask the following questions. Are there specific characteristics that make some research more likely to be SDG-related and that funders should target? Are there characteristics of SDG-related research that makes it more likely to be used in industry or policy reports? Does SDG-related research also appear in "excellent” publications using standard bibliometric measures?

To identify research related to the SDGs we devised a method to assign research areas/clusters to specific SDGs. This approach reduces the uncertainty and ambiguity of assigning individual publications to an SDG, and allows us to include publications that contribute to SDG-related research even if they do not use SDG-specific language in the title or abstract. First, we built a query with a set of terms that are strongly associated with each SDG, using policy reports, grey literature, scientific publications and web forums, alongside United Nations sources. We extracted relevant fragments from these texts, then selected keywords within them, first using text-mining techniques and then a manual selection. We then used those SDG-related queries to search 4,013 clusters of publications in the Web of Science (WoS) published between 2015 and 2019. A cluster comprises a number of published documents which are related to each other because of their citation pattern. Each cluster, then, represents a research area covering broadly similar topics. Based on the results of the search, we connected each research area to one or more SDGs, depending on the proportion of publications that included our SDG query terms in their title and abstracts. For example, 22% of publications in the ‘multidimensional poverty’ research area used terms relating to SDG1. The proportion of publications for each SDG was determined by manually reviewing the topic of the research area, and the title and abstract of the most cited publications.

To better understand how different types of research 23, are related to different SDGs, we used publications meta-data and citations to characterize publications according to four measurable features associated with the potential societal impact of research 24,25: • Collaborations 26–28 o extent of international collaborations (co-authors from more than one country) o extent of collaborations between high income countries and the rest of the world (co-authors from more than one country) o funding (extracted from acknowledgments) o industry authorship (co-authors from industry) • Public use 29,30 o industry (citations in patents, Patstat) o policy documents (citation in policy documents, Overtone) o news stories and Twitter posts (citations in news and twitters) • Open access 31,32 (open access paper); and • Multidisciplinarity 33 (Rao-Stirling diversity measure across subject categories 34)

We then compared these measures to standard measures of academic reputation 30,35.

Next, we clustered SDGs based on the similarity in each of the above indicators of all publications related to that SDG. This resulted in three clusters: a cluster of social challenges related SDGs (red): SDG 1 (no poverty), SDG 4 (quality education), SDG5 (gender equality), SDG10 (reduced inequalities) and SDG16 (peace, justice and strong institutions), also including growth (SDG 8) and innovation (SDG 9); a cluster of social functions and technical solutions (yellow): SDG 6 (clean water and sanitation), SDG 7 (affordable and clean energy), SDG11 (sustainable cities and communities), SDG 12 (responsible consumption and production), also including SDG 2 (zero hunger); a cluster of natural environment and health (green): SDG 13 (climate action), SDG 14 (life below water) and SDG 15 (life on land), which shares features with health publications (SDG 3).

Here we briefly summarise the main findings. Research related to social challenge SDGs, is more used in policy, potentially more impactful in society, and is the most multidisciplinary. Despite this, and despite being of at least as high quality as the average publication in WoS, it attracts less funding than average, and does not benefit from the same level of collaborations across countries. Research related to social functions and technical solutions SDGs is the most focused on basic sciences and technology applications, and the closest to industry. However, it does not attract much public or policy interest. Research related to natural environment and health SDGs is highly used in policy and society, attracts the most funding, and is most likely to be co-authored internationally and to be open access. SDG-related research, on average, is published in top cited journals as the WoS average

Taken together, the results indicate a need for greater public funding for research that focuses on the complex social determinants of sustainability, to complement, rather than follow, private funding.

As with all studies that map research published in academic journals, the methods are subject to certain limitations. In particular, the WoS does not cover most non-English language journals or those with high local relevance, and much research, especially in low-income contexts, is not published in academic journals. However, our findings are still crucial in mapping and characterizing the contribution to the SDGs of academic research, which accounts for a large proportion of research funding and is widely used in policy and industry. In Rafols et al (2021), we suggest a tool and method that allows users to review the results in this paper using different interpretations of SDG-related research.

ABSTRACT. Background and Rationale Applying an ethnographic approach to the research and application of bibliometrics, this paper synthesizes the scope and relevance of scientometric work with actor-network centered approaches from qualitative social sciences. In doing so, I draw on data collected within the Competence Network for Bibliometrics (Kompetenznetzwerk Bibliometrie, KB) in Germany. The KB is funded by the federal ministry for research and education and is the leader for the development of bibliometric tools in Germany. As a network with 22 high-profile member organizations including Fraunhofer and Max-Planck institutes, KB follows the research output from every field of study, reveals its impact, provides metadata, and makes research visible across disciplinary boundaries. It links partner research institutions and stipulates new disciplinary dynamics in the field of bibliometrics and scientometrics. Recently, the center has initiated an organizational transformation process from a center structure towards an agile network. Reasons for this are manifold, but predominantly rooted in the need for pooling bibliometric skills and making them available to the wider public including universities, government agencies and the industry. Design, control, and implementation of this transformation process is a complex organizational challenge given that the center now comprises of 22 consortium partners. This paper critically examines 1) this transformation process and elaborates on the scientific benefits it has produced so far and 2) the potential this restructuring hold for the fields of skills transfer and science communication with the goal of strengthening exchange between science and government agencies as well as industry. The study and application of bibliometrics is by definition concerned with quantitative measures. This paper promotes an innovative mix of applying ethnographic methods to a field that is shaped by quantitative approaches in order to uncover, and later utilize interpretations, analyzes and attitudes from network partners. Such feedback will be instrumental in structuring later stages of the transformation process.

Methods The aim of the on-going research presented here is to shed light on the benefits and functioning of the KB transformation process towards a network. Methods applied draw from ethnographic and related qualitative methods in order to gain a deep understanding on the 1) transformation process towards an agile network structure, 2) how the implementation procedures applied are viewed by network members and 3) how network members evaluate the changes made so for a) their work, b) their own productivity, c) the overall benefits for the whole network. Applied methods are participant observation in the virtual network meetings, different forms of interviews (online: open, theme-focused, structured) as well as in situ research in selected partner institutes in Germany.

Anticipated Results Anticipated results provide insights into how the transformation process is perceived by selected members of the network. Collected data consists of internal voices from the network and will inform the future implementation process of the organizational transformation of the KB. It also acts as quality assurance as it helps to evaluate and, if necessary, adjust strategy. Furthermore, data will also shed light onto how an agile network structure for a science consortium shapes and improves communication and general interaction between the sciences as well as government bodies and industry.

ABSTRACT. While governments seek to spur innovation and technology commercialization through funding programs, we have little insight into other outcomes that these programs may enable. This study examines the relationship between the U.S. government's SBIR program and the long-term success of participating firms.

Introduction Many governments support innovation and technology commercialization in firms through funding programs that target promising technologies. Two such programs in the US are the Small Business Innovation Research (SBIR) and Small Business Technology Transfer (STTR) program created to support R&D in small firms. The secondary objective of these programs is to help organizations cross the “valley of death” in which innovation is promising, but too expensive to pursue for private investors (Bonvillian, 2011). Research exploring the long term effects of such programs provide mixed results (e.g., Kapsali, 2011; Qian and Haynes, 2014; Lanahan et al., 2022; Dutta et al., 2022). Work has shown the positive impact of these programs such as increased commercialization of innovations (Audretsch, Link, & Scott, 2002), and increased commercial success of these innovations (Archibald, & Finifter, 2003), but long-term studies are rare. Given that the US government spends over a billion dollars each year on these programs, more insight is warranted. For example, we do not know the extent to which resources from SBIR and STTR programs influence the success of ventures in terms of VC, IPO and being acquired.

Theoretical Background - Summary The SBIR and STTR programs are two of the latest and biggest programs that the U.S. government has enacted to support small business development (Bonvillian, 2011). Through 2021, SBIR and STTR have cumulatively provided almost $60 billion in funding (Small Business Administration, 2022). While both programs fund R&D, the SBIR focuses on innovation in small businesses while the STTR focuses on innovation in public/private collaborations, particularly between small businesses and nonprofit research institutions. Both programs are coordinated by the U.S. Small Business Administration. The SBIR program supports innovation activities in small, often nascent organizations. Implemented in 1982, the SBIR program seeks to promote high-technology innovation in small businesses by funding R&D projects. Federal agencies in the U.S. government with an R&D budget over $100 million must allocate 3.2% of their budget to SBIR grants. The agencies then solicit proposals from small businesses related to the agencies’ needs. In 1982, the program provided over $38 million for R&D in small businesses through 789 awards. Through 2021, over 170,000 awards have been made totaling more than $53 billion (SBIR.gov, 2022). The STTR program was established in 1992 to support innovation collaborations between firms and public organizations such as universities and government labs and facilitate the transfer of intellectual property from research institutions to the market. Federal agencies with R&D budgets over $1 billion must allocate 0.45% to funding STTR grants. Five federal agencies participate in the STTR program. Through 2021, the program has grown to provide over $5 billion with 16000 grants. The overarching objective of these programs is to support R&D activity that is often expensive, at times beyond the means of small, nascent ventures (Link & Scott, 2010; Bonvillian, 2011). Thus the programs enable these firms to better compete with larger firms with more resources, encourage technology commercialization and support economic growth (Small Business Administration, 2013). While basic science lays the foundation for this technology, government projects tend to build applications of basic science work, usually having a proof of concept to get government funding (Rogers et al. 2001). Thus, these projects are vetted by an experienced audience and are closer to commercialization than academic projects based on foundational scientific research. This can help firms build social capital in addition to helping them commercialize innovations (Gray et al., 2022). Additionally, the SBIR awarding agencies choose to fund firms that they believe are viable and potentially successful bringing their technology to market (Lanahan et al., 2022). Giga et al. (2022) found that these grants correlated to higher patenting rates for small businesses. Hypothesis 1: Technology firms supported by SBIR and SBIR grants have a higher likelihood of success than other technology firms.

Obtaining VC funding is an important milestone for founders wishing to complete an IPO. Raising VC funding is the sale of a firm’s equity to external investors, providing financing to help startups grow. In 2021, VC fundraising reached new highs with 17,000 firms raising over $300 billion (Pitchbook-NVCA, 2022). VC funding acts as a signal to external stakeholders on the value of the firm and its likelihood of growth. VC backing can lead to high growth and IPO (Chemmanur et al. 2021). Obtaining both government grants and VC will support a firm’s ability to succeed in the long term. Hypothesis 2: Technology firms supported by SBIR grants and VC have a higher likelihood of success than other firms.

Methods We selected the genomics sector of biotechnology firms in the U.S. Genomics is the study of a complete set of genetic information or an organism. 1983 is a starting point since it is the year of earliest relevant startup activity. This ensures that the data are not left-censored (Blossfeld & Rohwer, 2002; Yamaguchi, 1991). A database of all genomics companies started in the U.S. before 2019 was compiled by triangulating many sources including patents, government grants, industry lists, scientific publications, university websites, VC listings, directories, press releases, and articles. To be included, over 50% of its activity such as products, patents, R&D, or sales, must be related to genomics. The firm must also be for-profit and independent. This classification process parallels previous works that identified new technology firms (e.g., Schoonhoven, et al., 1990; Woolley, 2017). The database consists of 619 firms and 36 firms were dropped due to a lack of reliable data for the founding teams resulting in a final sample of 583 firms. Using press releases and deal listings such as Crunchbase, we determine the date of the first VC funding, IPO, and acquisition for each firm as appropriate. For each, the following variables were collected or constructed: a dichotomous variable of event attainment, year of event, and time between firm founding and event. Several firm and macro level controls were included. We used the event history analysis technique using STATA with maximum likelihood estimation, robust standard errors and Weibull distribution.

Findings In total, 47% of the firms obtained SBIR or STTR funding, 58% obtained VC, 18% had an IPO, and 30% were acquired. The models show that firms with government funding were less likely to obtain VC, go IPO or be acquired. However, when firms with both government funding and VC were almost three times more likely to go IPO than other firms.

Significance This study develops our understanding of the relationship between government support and other measures of firm success: VC, IPO and acquisition. The results indicate that if the government is seeking to support high-growth firms that are acquired or go public, agencies should work with other ecosystem institutions such as other investors to improve the firm’s likelihood of success.

ABSTRACT. Technology Transfer and Commercialisation in Nigerian Universities: Motivation, Barriers and Policy Options Background and Rationale Transitioning from resource-based to knowledge economy has placed premium responsibilities on the knowledge institutions to support innovation (Adelowo, Olaopa and Akinwale, 2017; Adelowo, 2018; Adelowo and Surujlal, 2020). New knowledge creation and dissemination are fundamental roles of higher educational institutions (HEIs) and research organisations (ROs) (knowledge institutions), although some large corporations sometimes establish research base. The core mandates of these institutions have consistently undergone changes over time to reflect different levels of economic and technological development. For instance, educational institutions are set up to supply critical human capital needs at all levels to provide leadership and appropriate governance to both administrative and technical systems in the society. Later, research and development (R&D) activities was added given the substantial knowledge pool existing in the institutions. Translating R&D outputs to useful products, processes and services in the industry has become parts of the mandates of knowledge institutions in the last few decades, and for this to happen, these institutions have to be entrepreneurial in their approach. The arguments for knowledge institutions to become entrepreneurial in their engagements was clearly articulated in the knowledge spill-over theory of entrepreneurship (KSTE) (Acs et al. (2013) and the need for universities to be responsive to societal challenges (Etzkowitz (1998). This implies that knowledge institutions possess talents and tangible research results that are capable of transforming the society or an economy. This argument becomes more obvious when economies with most innovative firms and regions, including global leaders in high-tech are considered. For instance, most of the top twenty leading innovative companies globally emerged from the developed and newly industrialising economies that have not only invested heavily in knowledge creation but also created a channels through which those knowledge outputs are transferred for economic renewal and societal benefits (BCG, 2019). These economies are also home to top ranked universities and research institutions, supplying critical skills and knowledge inputs to the industries in a coordinated manner. Therefore, for developing countries in Africa, particularly Nigeria to harness knowledge outputs for development, there is the need for the knowledge institutions to embrace entrepreneurship and innovation as parts of coherent policy to achieve both institutional and developmental goals. Parts of the process to embrace innovation and entrepreneurship include the creation of and, or strengthening existing mechanisms for knowledge transfer in these institutions, including technology transfer offices, technology incubators, knowledge parks, innovation hubs and other innovation infrastructure. Nigeria has since 2006 started establishing technology transfer offices in its knowledge institutions to serve as a collation hub for R&D outputs and to foster linkages with industry for the purpose of technology commercialisation among others. To date, about forty-three intellectual property and technology transfer offices (IPTTOs) have been created/established in different institutions across the country. Despite its existence for more than a decade, the performance of few of them have been recently assessed, suggesting improvement in the patenting activities among the scientists and researchers (NOTAP, 2019). Also, institutions without the IPTTOs have also demonstrated great potentials in creating commercialisable research results but without any documented (known) mechanisms for its management. Data and Methodology Therefore, this article x-rays the strategic approach(es) adopted to sensitize stakeholders and manage research outputs in Nigerian universities and research institutes using primary data collected through cross sectional survey conducted among thirty-one (31) intellectual property and technology transfer offices and one hundred and sixty-seven patentees. For the institutions without the IPTTOs, we also established the mechanisms adopted to manage tangible research outputs within them. The motivation and barriers to patenting and commercialisation were examined among scientists and researchers in the selected universities. Preliminary Results The results show weak management capabilities in the TTOs, as most of the managers and staff do not have adequate knowledge and training in IP management. Most of the staff (67%) of the TTOs, including the directors regarded themselves as permanent staff, however many of them have primary department or faculty they are affiliated to in the same university. The basic office infrastructure in the TTOs are fairly adequate while only few of them have automated their IP database. On major activities of the TTOs, IP sensitisation and training for faculty members, industry collaboration, facilitation of inter/intra departmental linkages and collation of R&D outputs are reported as the dominant ones. The TTOs are majorly financed by the host institutions and sometimes by the internally generated revenue. Financing TTOs from the institutions’ budgetary allocations is perceived as additional financial burden, given that universities and research institutes are not adequately funded. The Offices hardly receive funding from the National office for Technology acquisition and Promotion, an Agency of government that is responsible for establishing and funding IPTTOs in the country. The major challenges confronting the offices are inadequate funding, personnel, training opportunities and inability to sufficiently foster industry collaboration for research uptakes. Few of the TTOs have assisted their institution to develop robust IP policy at the time of data collection. From the inventors’ perspectives, key motivations reported for filling patents are promotion, prior knowledge of IP, expected monetary returns, career advancement and recognition/award/fame that is associated with it. Only a handful of inventors have commercialised their inventions, and that was done through private arrangements. Major barriers to inventions commercialisation are lack of industry collaboration, poor funding, inadequate innovation infrastructure, and lack of IP policy or incentives. The paper concludes with policy recommendations to improve innovation ecosystems in the universities.

References Acs, Z. J., Audretsch, D. B., and Lehmann, E. E. (2013). The Knowledge Spillover Theory of Entrepreneurship. Small Business Economics, 41 (4): 757–774. Adelowo C. M., Akinwale O. Y. and Olaopa O. R. (2017). Innovation and Knowledge Transfer in Nigeria. International Journal of Research Innovation and Commercialisation, 1 (1): 57-73. Adelowo, C. M. (2018). Factors influencing Academic entrepreneurship in Nigerian universities. International Journal of Innovation and Regional Development, 8(4): 337-357. Adelowo, C. M. and Surujlal, J. (2020). Academic entrepreneurship and traditional academic performance in universities: evidence from a developing country. Polish Journal of Management Studies Vol. 22, No. 1, pp. 11-25. Etzkowitz, H. (1998). The Norms of Entrepreneurial Science: Cognitive Effects of the new University-industry Linkages. Research Policy, 27 (8): 823-833.

ABSTRACT. Industrialisation is an important engine of growth and ‘catch up’, but also associated with harmful carbon dioxide emissions and hence with climate change. This poses a challenge for sustainable industrial development, particularly for late industrialisers on how to industrialise while also mitigating carbon dioxide emissions. This paper investigates the effect of technology intensity across manufacturing industries on carbon dioxide emissions: are medium- and high-technology manufacturing industries less emissions-intensive than low-technology manufacturing industries in developing countries? The paper analyses this relationship for a panel of 68 developing and emerging economies over the period 1990-2016, by adapting the environmental Kuznets curve and the stochastic effect by regression on population, affluence and technology approaches. Using two alternative measures of emissions and estimating generalised method of moments model, the results show that medium- and high-technology manufacturing industries are associated with lower carbon dioxide emissions than low-technology manufacturing industries. The results also show that these differences vary by the income levels of countries. These findings have important policy implications, suggesting that a shift towards more technology-intensive manufacturing production processes may be a more environmentally sustainable industrialisation path for developing countries.

ABSTRACT. Literature on Transformative Innovation Policy (TIP) is growing fast (Haddad, et al., 2022; Diercks, et al., 2019) and, although it has received some attention in Latin America, the role of the State in TIP in the context of emerging economies can be further explored (Ordóñez-Matamoros, et al., 2021). The context of Latin American emerging economies represents differentiated implications for the role of the State in TIP. For instance, gaps in governmental capacity for policy design and implementation, the differentiated types of challenges and priorities for Latin American countries in contrast to more developed economies in the Global North (e.g. socioeconomic inequality, violence, high levels of poverty, biodiversity loss, among others), the central role of politics that shape innovation policies in line with contested democracies, among other factors that define the role of the State in this context.

In order to better understand the implications of TIP in emerging economies we need to further inquire on how these specific features redefine the conceptualizations of the State in shaping innovation policies. We argue that a ‘performative’ perspective on the role of the State is needed, instead of the prevailing ‘monolithic’ approach that overlooks the interplay between different agents at multiple levels of TIP, which might lead to variegated outcomes in terms of transformative change.

In this research we inquire on what are the multilevel roles of States in TIP in Latin American emerging economies. In line with Borrás & Edler (2020), here we argue that TIP is a complex process in which the State plays diverse roles -even contradicting ones- depending on de facto governance settings. Therefore, we unpack these roles at three governance levels in the case of energy policy in Uruguay and the policy for social appropriation of knowledge in Colombia. We aim at disentangling the way in which the strategies of different actors shape the ultimate capacity of the State to design and implement (or fail) innovation policies with transformative potential at different governance levels.

Here we define transformative innovation policy (TIP) as “a set of public actions and instruments, through which governments mediate and mobilize resources towards more sustainable and inclusive sociotechnical systems via the promotion of knowledge and innovation production, diffusion and use with a long-term perspective” (Ordóñez-Matamoros, et al., 2021, p. 119). Considering the comprehensive ambitions of this policy approach, governments deploy differentiated forms of action and roles. While it can often be an active promoter of meta-governance for transitions (Kuhlmann & Rip, 2018), literature usually portrays this role as a reaction to failures in markets (Nelson, 1959), systems (Woolthuis, et al., 2005), or transformation processes (Weber & Rohracher, 2012). Despite its explanatory value, the failures approaches assume the role of the State as linear, reactive and limited to problem-solving (Turnbull, 2018).

As a response to that, we draw on Borrás & Edler’s (2020) typology of roles of the State, according to the drivers of sociotechnical change (State or non-State actors) and the modes of governance involved (hierarchical or heterarchical). Furthermore, we use these initial typologies in a multilevel heuristic, in order to grasp the complex interactions that take place in TIP implementation processes. We ground our reflection on the analysis of two cases: Uruguay and Colombia. First, we analyse Uruguay’s ‘Energy Policy 2005-2030’ approved between 2008 and 2010. This is generally regarded as a successful State-level policy for energy sovereignty, with successful results in a short period of time in terms of reducing energy costs and strengthening national energy industry. This is related to the scope of this transformation, the time span in which it took place, and the political consensus on which it was built. We explore this transformative process by focusing on the underlying institutional arrangements that shaped the State’s capacity in this case and the role of innovation policy therein.

Second, we analyse the Colombian Social Appropriation of Science, Technology and Innovation (SASTI) Policy, in place since the early 1990s. Here we observe an emerging direct approach to development that challenges the path-dependence dynamics that drive innovation policy exclusively towards increasing productivity and economic growth. In contrast to the Uruguayan experience, this case shows how governments do not necessarily have a purposeful approach on transformative innovation, but their role is rather shaped by the bottom-up agency efforts of institutional entrepreneurs, who nest transformative initiatives in spite of overarching business-as-usual policy environments.

These cases follow the case study research method (Yin, 2018). The data studied considered legal regulations, policy documents, academic and technical documents, and interviews with actors involved in the cases. It is worth mentioning that the two cases took a different scope and focusing unit of analysis. Thus, while the Uruguay case considered the transformative policy, the Colombia case was focused on the institutional entrepreneurs. Such differences do not affect the case analysis. Instead, it contributes to discussing from different viewpoints the state’s roles in the Global South, enriching the study.

Early findings suggest at least three possible sets of roles of the State. The first set depicts an optimistic or supportive role of the state. This category is constituted by the roles described so far by literature, as illustrated in section two. The second set is shaped by those roles that describe a neutral position of the State towards transformative processes. This neutrality means an indifferent position of the State towards transformative challenges. In other words, the State does not support nor block transformative processes if there are any in development. Finally, the State can perform a negative set of roles to interrupt or erode the transformative processes. At this point, the question of what are the conditions that explain the deployment of any of these three sets of roles by the State emerges.

This research contributes to a more critical understanding the governance challenges that the Latin American context represents for policies aimed at sustainability transitions and social inclusion, for in such settings it is not possible to assume that the State plays a specific and distinguishable role in addressing transformative governance demands.

ABSTRACT. Background and rationale: Recent literature has focused on the importance and many potentials of innovation or mission-led policies to respond to social, environmental, and economic challenges (Mazzucato, 2018). This framework recently gained attention by defending the role of governments in setting the direction of technical change, promoting innovation, and the diffusion of new technologies. Thus, mission policies help create and shape new markets via targeted innovation-led oriented policies and can be considered an opportunity for countries with less mature R&I systems to accelerate the development of their capacities (Fisher et al., 2018). The main differences between mission-oriented policies (MOPs) and other typical innovation and industrial policies may be in terms of “scale (bigger), scope (broader) and target (more specific)” (Alves, Vonortas & Zawislak, 2021, p.81). MOPs often use a mix of policy instruments going beyond the mere realm of R&I policies and require horizontal policies cutting across governance levels (Fisher et al., 2018). Ultimately, mission-led policies are seen as a helpful way to employ a policy direction that is smart, inclusive, and green (Mazzucato & Perez, 2014). In fact, scholars do consider that “green growth” is the ultimate goal and the next big technological and market opportunity that can stimulate private and public investments (Mazzucato & Perez, 2014). A promising technology that falls under the umbrella of “green growth” is the production and adoption of low-carbon hydrogen. Producing hydrogen from renewable sources is seen as a global alternative for decarbonizing energy production and economic activities by 2050. In an attempt to close the gap between rhetoric and action, the International Energy Agency (IEA) has set a roadmap that establishes pathways to reach net zero emissions by 2050 that highlights the importance of hydrogen as an energy vector (2021). Similarly, governments from advanced and developing countries launched ambitious hydrogen strategies and directed economic stimulus funds to this area. Based on this background, we can consider that producing low-carbon hydrogen is one goal of a set of global policies that target green growth. Although MOPs have a major part to play in delivering better quality growth while addressing grand sustainable challenges, they can also face their own challenges in their design, implementation, and assessments. Alves, Vonortas and Zawislak (2021), emphasize that the mismatch between the expected goal and what is actually feasible (based on the available technological capabilities) creates what the authors call “a fuzzy boundary” that often leads to the unsuccessful implementation of missions. Another important aspect to consider is that, while setting goals and determining pathways, missions will hardly be reduced to a single development path or by a single technology. The establishment of a goal or a mission should consider the available innovation capabilities in order to promote a successful regime change or shape and create markets. We consider that MOPs may create an institutional incentive for regime change, catching-up, capability building, and eventually for market creation, but we must also consider that a MOP approach should acknowledge the many stages of development and diffusion of still immature technologies. Geels’ (2002) approach is helpful to understand the dynamics of regime change, the emergence of new technologies and their diffusion processes, and how they interact with extant sociotechnical regimes that are crucial for the societal sustainable transformation. One can consider that MOPs cannot be successful without the understanding and mastering of innovation capabilities and the characteristics of their technological path. A full understanding of these dynamics can directly impact the success of the implementation of the MOP framework for sustainable innovation.

Objective: In this article, we aim to analyze and discuss the effectiveness of applying the MOP framework in the context of innovation and diffusion of new technologies applied to low-carbon hydrogen and present a historical approach to the development of this sustainable innovation.

Methodology: We conducted an in-depth literature review of the recent history of the technological development of low-carbon hydrogen as an energy vector. A detailed mapping of global policies that are part of the mission approach applied to this topic was developed. This review covered papers, international reports, recommendations, and cooperation agreements related to low-hydrogen technologies. Significance of the study: This study contributes to academic research and literature by providing a detailed study that enhances the potential and limitations of the MOPs framework applied to the technological development of sustainable technologies. Our analysis demonstrates that MOPs can be helpful tools for addressing important societal problems and can also stimulate government and private actions that could not have happened otherwise. However, the detailed comprehension of the technological regimes and capabilities of a technology, which is the central discussion of a mission, is essential as this influences the implementation and the success of a MOP.

REFERENCES

Alves, A. C.; Vonortas, N. S.; Zawislak, P. A. (2021). Mission-Oriented Policy for Innovation and the Fuzzy Boundary of Market Creation: the Brazilian Shipbuilding Case. Science and Public Policy, 48(1). doi: 10.1093/scipol/scaa059 Fisher, R., Chicot, J., Domini, A. et al. (2018) Mission-Orientated Research and Innovation: Assessing the Impact of a Mission-Oriented Research and Innovation Approach. Final Report

Geels, F. W. (2002) ‘Technological Transitions as Evolutionary Reconfiguration Processes: A Multi-Level Perspective and a Case-Study’, Research Policy, 31/8–9: 1257–74. IEA – INTERNATIONAL ENERGY AGENCY. Net Zero by 2050. A Roadmap for the Global Energy Sector. IEA, Paris, 2021. https://www.iea.org/reports/net-zero-by-2050

Mazzucato, M. (2018). ‘Mission-Oriented Innovation Policies: Challenges and Opportunities’, Industrial and Corporate Change, 27/5: 803–15. Mazzucato, M.; Perez, C. (2014). Innovation as Growth Policy: the challenge for Europe. Working paper series, SWPS.

ABSTRACT. Background and Rationale

The development of many existing clean energy technologies and products has relied on the integration of knowledge and components that originated in different technological fields and industrial sectors. Therefore, to be able to design better public policies that support and accelerate clean energy innovation required for the decarbonization of the global economy, it is important to understand how such external knowledge contributes to the emergence and subsequent evolution of new technologies and industries.

Existing literature on industry evolution has emphasized the role of technological breakthroughs or discontinuities in the evolution of different industrial sectors, which is typically associated with undermining the competence of industry incumbents or enhancing the position of entrepreneurial new entrants. In many cases, these discontinuities are the result of technology spillovers that contribute external knowledge from other fields to innovation in the industry.

In this literature, industry disruption associated with technological breakthroughs usually leads to the emergence, growth, and eventual dominance of new sub-markets—specialized markets within a larger market. However, the emergence of sub-markets does not necessarily require technological discontinuity. Another stream of the literature suggests that sub-markets may also emerge in response to heterogeneous user needs, especially those unmet ones, offering an alternative to a supply-side view of the industry life cycle. It is not yet clear what role, if any, external knowledge plays in the emergence of sub-markets that serve new or previously unmet user needs.

In this study, we address this gap by investigating how external knowledge has contributed to the co-evolution of lithium-ion battery (LIB) technology and industry composition (i.e., actors and sub-markets) since the early 1970s. LIB is widely considered to be one of the key clean energy technologies contributing to global decarbonization efforts. The original LIB technology architecture emerged from a series of major scientific and technological breakthroughs which were awarded the Nobel Prize in Chemistry in 2019. More recently, the LIB industry has been dominated by the demand for batteries for electric vehicles (EV). These industry characteristics offer an opportunity to investigate both the supply- and demand-side perspective on the role of external knowledge in industry evolution.

Methods

We track how external knowledge has contributed to the co-evolution of LIB technology and industry composition over time by process-tracing external knowledge sources, main innovating actors, and the subsequent impact on technology and industry development for a set of 84 important LIB-related inventions assembled through a systematic literature review on the LIB history. In addition, for more recent inventions with a yet-unclear impact on future technology and industry development, we focused on inventions associated with 18 prominent companies and start-ups currently identified as promising by LIB industry experts, using Crunchbase to track the disciplinary and industry background of their founders and investors.

Results

Based on our findings, we divide the history of the LIB industry into four periods. The first stage, “incubation”, started in the 1970s with Stanley Whittingham’s discovery of electrochemical intercalation and concluded in the early 1990s with the commercialization of the first-generation LIB by Sony. In this period, the knowledge base of what we know today as the lithium-ion battery was formed by many scientific discoveries and technological breakthroughs, with most of them bringing external knowledge from other fields to solve critical scientific and technical problems emerging on a trajectory toward the first working LIB. Innovation during this period was mostly driven by researchers and inventors in large corporations, universities, and public research institutions.

A “stabilization” period followed the incubation stage during the 1990s when LIBs demonstrated superior performance compared to other rechargeable power sources and quickly started to dominate the market for consumer electronic devices. During this period, the LIB technology design stabilized, with the majority of LIB innovations being incremental improvements of different components or manufacturing processes developed by corporations to improve LIB safety, performance, and manufacturing cost. Notably, very few of these innovations involved external knowledge, indicating a decline in its importance for the LIB industry during this period.

The third stage of “expansion” started around the early 2000s when the market success of LIBs for consumer electronics attracted the attention of users and producers seeking alternative applications for LIB technology, such as power tools or electric vehicles (EV). Different and often unique user needs in these applications required adapting LIB technology to those needs, often relying on external knowledge about these applications. Furthermore, universities, also attracted by these opportunities and often supported by public R&D funding, leveraged their knowledge from other fields beyond LIB, such as nanotechnology or materials science, to experiment with alternative battery concepts or components that were previously abandoned or had not yet been commercialized but were now considered promising for new applications.

The fourth stage, “diversification”, started around the early 2010s with the onset of demand-pull public policies that supported rapid growth in the EV market. The resulting dramatic increase in the demand for LIBs created a new rapidly expanding sub-market of LIB for EVs, which is characterized by multiple emerging and competing LIB and post-LIB technological trajectories rather than a single dominant technology design. Two new types of actors entered the LIB industry in this period. One is incumbent car manufacturers as actively engaged downstream users who bring a wealth of external knowledge from the automotive industry. Another is a booming number of start-ups, often created as university spin-offs, that seek to commercialize LIB innovations and alternative battery concepts developed in the previous period.

Significance for Policy

By analyzing the history of the LIB industry and technology, we find that external knowledge has been instrumental in the emergence and evolution of this industry. However, its relative importance varied over time. Initially, external knowledge provided a crucial contribution to the formation of the knowledge base of the emerging LIB technology. It was significantly less important during the stabilization stage. In the following two stages of expansion and diversification, which were characterized by the emergence and rapid growth of a new LIB sub-market for EVs, external knowledge, again, became important for the adaptation of the existing LIB knowledge base for new applications driven by user needs and demands. Particularly in the most recent period of diversification, demand-pull public policies provided crucial support for the integration of external knowledge into the LIB industry knowledge base by incentivizing the entry of new types of innovating actors.

Our findings provide important implications for policies supporting clean energy innovation. Policy mixes should combine supply-side and demand-pull policy instruments to facilitate the generation, identification, transfer, and integration of external knowledge into clean energy technologies. For example, R&D funding policies, particularly at the incubation stage, should be designed in a way that makes relevant external knowledge easily identifiable and accessible to researchers and inventors, e.g. by encouraging multi-disciplinary team composition and cross-sectoral learning. Demand-pull market incentives, particularly during the expansion and diversification phases, can attract new entrants to the industry, e.g., incumbents from other industries that provide external knowledge about promising new applications and user environments, or university spin-offs and start-ups that ensure technology lock-in is avoided. Finally, policies that support industry coordination efforts, e.g., through public-private partnerships or road-mapping exercises, can help identify and close gaps in the industry knowledge base where external knowledge can be productively applied.

ABSTRACT. The current context in African countries is characterized by increasing environmental constraints, unemployment, and poverty, whence the need to achieve sustainable development. Indeed, it will ensure the satisfaction of the needs of the present generation without compromising those of future generations. Thus, countries must achieve sustainable growth, in particular by fostering infrastructure development; promoting research, science, innovation, and technological development; and scaling up global and regional partnerships for development (UN/ECA, 2016a). The recognition of the importance of technical progress and research and development (R&D) in fostering growth has led to the development of neoclassical, evolutionary, Neokeynesian, and institutional theories. Indeed, while the old neoclassical theory of growth integrates technical progress in exogenous form for the improvement of long-term growth (Solow, 1956), new theories do it in the endogenous form with an emphasis on knowledge accumulation (Romer, 1986), human capital (Lucas, 1988; Romer, 1990), public spending on physical capital and social infrastructure (Barro, 1990), and technological innovation related to R&D introduced in different forms (Aghion et Howitt, 1992; Grossman et Helpman, 1989; Romer, 1990). Evolutionary theories focus on supply through the central role of technological change in firms and the generation of novelty (Nelson et Winter, 1982; Silverberg et Verspagen, 1994). Neokeynesians consider the role of demand (the degree of consumer orientation) in technical progress and growth (Kaldor, 1966; Pasinetti, 1994). Finally, the institutional current is based on the economic organizational efficiency in terms of taxation and legislation on property rights, to name just a few, as crucial factors for improved economic growth (North, 1994). In their view, institutions must play the leading role in terms of technological policy to foster innovation behaviors of the most important economic actors through their mode of interaction. Thus, several systemic approaches to innovation have been developed after the national innovation system (NIS). Approaches that provide information on countries' capacity to generate and benefit from innovation (Freeman, 1987; Lundvall, 1992; Nelson, 1993). Indeed, the NIS advocates the interaction between different economic actors such as industries, universities, research centers, public institutions, financial institutions, and end users. Among these actors, industries and governments are considered respectively as the center and the main actor of the innovation process regulation. However, in the context of a knowledge-based economy, the Triple Helix (TH) innovation system views university institutions as leaders when it comes to the innovation process (Etzkowitz et Leydesdorff, 1995 & 1997; Leydesdorff et Etzkowitz, 2000). This approach advocates interactions between university, industry, and government as the key to improving the necessary conditions for technological innovation generation. The extension of the Triple Helix to Quadruple Helix model (Carayannis et Campbell, 2009) by taking into account social contribution has led to the development of the endogenous growth model of the national innovation system (Afonso et Monteiro, 2012; Monteiro, 2013). However, the environment, according to its current constraints in addition to the need to consider additional factors for the explanation of the technological innovation process has led to the extension of the Quadruple Helix model to Quintuple Helix (Carayannis et Campbell, 2010) and Ntuple Helix (Leydesdorff, 2000) that serves as a framework for sustainable improvement of country performance. However, the analysis of African countries’ performance reveals the underdevelopment of their national innovation systems as compared to those of developed countries which remain characterized by weak interactions between different spheres (Casadella et Benlahcen-Tlemcani, 2006; Gu, 1999). The studies on innovation systems in Africa also show an important synergistic contribution at the foreign level as compared to domestic and global ones (Mêgnigbêto, 2015; Mezouaghi, 2002). In addition, African countries, particularly those in Sub-Saharan Africa have recorded average growth rates of 5% over the past 15 years. But this growth is mainly due to commodity prices instead of technological innovation. However, the latter occupies the top position in Asia, emerging countries, and other developed countries (NU/CEA, 2016b; WEF, 2016). In terms of sustainability, there is a very poor performance of African countries as compared to those of other parts of the world despite the high natural resources endowment that abounds the continent. In fact, of the 180 countries included in the 2017 global sustainable competitiveness index report, the top four in Africa are Ethiopia (64th), Ghana (68th), Côte d'Ivoire (77th), and Kenya (80th). There are also 8 African countries in the 20 and occupying the last positions in such ranking (SolAbility, 2017). Notwithstanding, studies show that the development of the innovation system in addition to absorptive capacities (i.e. the different factors that affect the ability of countries to take advantage of foreign technologies) favor the convergence of developing countries towards developed ones (Abramovitz, 1986; Albuquerque, 1999; Archibugi et Coco, 2004; Edquist, 2001; Fagerberg et Srholec, 2008; Fagerberg et Verspagen, 2007; Filippetti et Peyrache, 2011; Godinho et al., 2004; Lee et Lee, 2019; Viotti, 2002). Also appears that the opening of the innovation system with trade, foreign direct investment, and integration determines the improvement of developing countries’ competitive performance (Lundvall, 1992 & 2015). Thus, how can the development of the open innovation system and absorptive capacities contribute to sustainable growth improvement in Africa? Therefore, this study aims at contributing to sustainable economic growth policy reinforcement in Africa by analyzing the link between the opening of the innovation system, absorption capacities, and sustainable growth. Thus, the generalized moment’s method in the autoregressive vector panel has been used. The data was used to cover a total of 27 countries from 2007 to 2016. The results show that the Open Quintuple Helix’s innovation system and the absorptive capacities of firms and universities have a negative impact on the sustainable growth rate in Africa. Moreover, only the increase in firms' absorptive capacity really coevolves with the Open Quintuple Helix innovation system and the sustainable growth rate. Finally, an infrastructure public policy shock leads to an improvement in investments and a sustainable growth rate. However, this shock leads to a decrease in the availability of scientists and engineers, in the firm’s absorptive capacity, and the efficiency of the Open quintuple Helix innovation system in the long term. Thus in terms of sustainable growth policy, African countries must strengthen investment and especially technological innovation infrastructure development. They must also ensure that the training of scientists and engineers and the firm’s absorptive capacity can be turned towards the acquisition and development of clean or green technological innovation.

ABSTRACT. Reuse of water has the potential to alleviate water shortages, reduce wastewater treatment costs, reduce water system energy consumption, and recycle nutrients. Many states have their own guidelines for water reuse. These guidelines include but are not limited to: guidelines for treating and reusing water, design and operation of reuse facilities, and water rights. States make their regulations within nationwide guidelines. Some states allow surface irrigation of food crops where there is no contact between edible portions and reused water, and some states do not allow food crops irrigation. Wastewater reuse has great potential to contribute to sustainably managing water to combat water scarcity and even help support a transition to a circular economy. Studies to date have not investigated the characteristics that influence the type of guidelines adopted and reinvented. In this study, we address this gap. We hypothesize (1) that physical characteristics such as drought in the region, population density, and the importance of the sector the reused water is being applied in have a large impact on water reuse policy adoption. We also hypothesize that (2) political and social characteristics such as governance ideology, and geographical neighbor status have a significant impact on the adoption of the guidelines. And, (3), we hypothesize that circular economy initiatives increase the adoption of wastewater reuse policies. We test these hypotheses in the US based on US EPA data. The findings explain the diffusion of wastewater policy and the relations between states. Moreover, this study enriches our understanding of environmental policy diffusion.

ABSTRACT. The socioeconomic advancement of any country is largely dependent on the science, technology, and innovation ecosystem of the country. Scientific and technological advancements are crucial in bringing technological and economic progress. This knowledge from the universities should flow to the industry for knowledge translation to address current challenges faced by society and also for bringing societal upgradation. The universities are the knowledge generators, and the industry is the knowledge consumers. There is a need to build strong university-industry linkages to enhance the STI ecosystem of the country. Although India has shown a significant rise in innovation capabilities by attaining the 40th global innovation ranking out of 136 economies worldwide, top innovation ranks amongst the low-middle-income countries and central and south Asian countries (Global Innovation Index Report, 2021-22). Despite that, India seriously faces challenges in university-industry linkages (at present, India stands at 41st global positioning in this parameter of GII, 2020-2021). Therefore there is a need to bridge gaps between universities and industry.

One of the key impediments in university-industry linkages is the lack of a facilitation platform and channels for translating university knowledge to the industry as industrial expectations are not met, and the innovation dies in the valley of death as per the innovation life cycle. To cater to the problem mentioned above, a detailed analysis of the university research in terms of Technology Readiness Levels (TRLs) has to be carried out to bridge the gap between TRLs levels. University research projects generally fall from TRL 1-4, and industry to take up the project required TRL 7-9; hence there is an immediate need to develop partnership mechanisms between industry and university to bridge this TRL gap from university to industry. The current paper focuses on collating the challenges faced by academia and the industry in terms of Science, Technology, and Innovation (STI) diffusion from university to industry and the necessary facilitation and enabling mechanisms required to bridge the same. The paper aims to bring out the university-industry linkages framework that can be explored for stimulating the university-industry linkage culture in India.

This guiding framework is developed after undertaking secondary research, and interview sessions with experts from academia (Panjab University, Chandigarh; Chitkara University, Baddi; Indian Institute of Sciences, Bangalore; Indian Institute of Technology (IIT), Delhi and Indian Institute of Chemical Technology (ICT), Mumbai ) and industry, open questionnaire-based feedback from industry and academia and undertaking various academic case studies. This framework can be practiced by universities in India to set up strong interconnects with Industry and explore university-industry linkages for innovation and technology transfer. Interview questionnaires and case studies were used as primary instruments, along with desk research for collating the information to understand the challenges faced by universities and industries in collaborating. Further, the best practice framework outline is proposed that will play a significant role in connecting the university and industry.

A Guiding framework to develop University-Industry Linkages is developed by 6 step process that comprises of following steps: • Step 1 Mapping of potential academic institutes and industry needs and challenges that will open the scope for collaboration between university and industry: Potential academic institutes in India were mapped for Expertise available (Faculty/scientific staff, trained researchers); Research Infrastructure (Facilities and instruments available); Intellectual Property (filed and granted); Technologies/products developed; Technologies available for licensing/commercialization and R&D status (R&D projects with TRL levels, etc.). On the other hand, the industry perspective was mapped by sessions of interviews with select Industrial associations (regional and national); Industrial clusters (regional and national; sector-wise); government recognized Industrial R&D units, and Industry (big corporations, MSMEs, start-ups, etc.). • Step 2 Develop Communicating Channels between Industry and Academia (I-A Enablers) Individually: To develop the communicating channels following source points can be explored by the university as Alumni association; Dean/Departmental head; Research and Consultancy Cell; Technology Transfer/Commercialization/Licensing Cell, with Legal/IP Cell; Corporate Laboratories; Centers of Excellence (as per industrial needs); I-A Clubs; Industry involvement in academic activities such as curriculum design and review, teaching and training, research advisory, etc.; Industrial Chairs and Industrial Fellowships; Entrepreneurship Cells/Clubs, Research Parks, and Technology Business Incubators (TBIs). • Step 3 Match Making (Industrial Needs/problems & Academic expertise/solutions): Dedicated team of I-A managers/officers have to be administered for the required job. Scouting of potential academic partners as per industrial challenge and requirements This can be done through Open Innovation Challenges, Industry hackathons, Networking by a Liaison officer/manager at I-A cell, One to one meetings with industry counterparts, Industrial surveys (open/closed), and Communicating science. • Step 4: Drafting MoUs and Agreements: Once the potential Industry-academia linkage is established, exclusive agreements defining the role and responsibilities of each partner, along with timelines and deliverables expected, will be drafted. A team comprising the legal adviser, IP advisers, Tech Transfer advisers, and financial advisers will be formulated to assist the I-A cell in drafting agreements. The agreement should address: Project aims and detailed scope, Risks and responsibilities to manage, Rights and remedies, Project management, IP management, Regulatory norms, Expenses and payments, Supervision, Schedules, Exit, and termination. • Step 5: Collaborative Action: The collaborative research will be undertaken with defined goals and objectives, resource allocation, and deliverables sought as per the timelines. Regular monitoring of the progress of the research work will be carried out, and evaluation in terms of the timely delivery of the project objectives by an external committee will be undertaken. • Step 6: Technology Transfer: Deliverables from the collaborative research undertaken in technology transfer or licensing will be explored. The culmination of the project with technology commercialization from academic entity to industrial entity will be laid out, and future long-term engagement plans for the same will also be discussed.

The study recommends the establishment of dedicated entities for facilitating the translational research ecosystem. The dedicated team will undertake the task of Liasoning between the university and industry. This will require and will cater to human resource and capacity-building initiatives. Each university should establish such translational research facilitation bodies. They can be named Technology Transfer Offices, Industry-academia cells, etc. Specific training in leadership and management human resources should be carried out to facilitate university-industry linkages and technology transfer agreements between the university and the industry. A dedicated platform (preferably virtual) should be set up where academic expertise can be highlighted, and industry problems/challenges can be posted. Specific mechanisms for connecting industry with academia as per their needs and priorities should also be developed and managed by a dedicated office/team. Regular display of academic knowledge through exhibitions, fairs, tech displays, and repositories should be maintained by the university portfolio. This guiding framework is developed after undertaking secondary research, interview sessions with experts from academia and industry, open questionnaire-based feedback from industry and academia, and undertaking various academic case studies. This framework can be practiced by universities in India to set up strong interconnects with Industry and explore university-industry linkages for innovation and technology transfer.

ABSTRACT. Background and rationale

This work aims to analyze the situation of the Anglophone Caribbean Region (ACR) scientific system. Despite being close to main research hubs such as the US and having cultural ties with others such as the UK, Netherlands, and other European countries, the ACR falls behind in research productivity compared to several regions of the world, which are closing the gap with the developed countries in specific research areas.

The ACR has specific barriers that make it a worth investigating region in terms of developing a well-suited scientific system for its development path. The ACR comprises CARICOM countries, which face similar obstacles. After the 2012 world economic recession, ACR experienced low economic growth rates. Also, an extraordinary economic downturn during the pandemic took effect, especially for those more tourism intense ACR economies. An increase in the frequency of natural disasters and historically high public debt rates are also causes of endemic fiscal issues in the region, which restrict countries' investment in areas such as science and technology. With a few exceptions in the ACR region, unemployment is pervasive throughout these countries, especially youth unemployment, which unfortunately ranks comparatively among the highest in the world.

However, the ACR region has similar advantages for developing a scientific system that fits its unique characteristics. Many initiatives were applied to foster research areas throughout several ACR countries, such as the green and blue economy, climate-resistant infrastructure, big data and digital development, geothermal energy, and several medical research investments focused on tropical viruses that can spread faster internationally. Also, despite the obvious negative consequences, the COVID-19 pandemic opened a window of opportunity for the region with the explosion of remote work. Some ACR countries (especially Barbados) took advantage of the last by attracting specialists in IT areas who work remotely for the US or Europe to their unique landscapes. While the consequences of the previous strategy are unknown for their STI system, other countries could have followed a similar approach.

These shared barriers and opportunities in the ACR open up the need to investigate which research disciplines will require more effort for capacity building and those in which the region excels. However, despite the similarities in the region, ACR countries and even specific islands within countries have their specificities. In this regard, it is essential to understand where the research capacities are located and where policy initiatives are needed for their development.

Descriptively, this research aims to answer in which research areas the ACR demonstrated an increase in its capacities during the last decades. Also, it turns necessary to understand in which countries this development occurred: this capacity building in specific research areas occurred homogeneously in ACR, or is the process concentrated in some countries? Also, it is intended to answer why capacities developed in specific research areas, and others do not.

The development of research capacities is an evolutionary process; thus, time is a key variable for this research. It would be a worthless effort to understand ACR research capacities nowadays without understanding how did evolve and interacted with the local ecosystem. For these reasons, our approach embraces quantitative and qualitative information analysis, explained in the next section.

Methods

This research describes and explains the evolution of the ACR’s scientific output in the last 25 years. First, built on bibliometric information from the Scimago database for the 1996-2020 period, this study illustrates the ACR research output compared –in relative terms- to the world. For this goal, based on the well-known revealed competitive advantage approach, the inquiry establishes the research domains where a region or a country shows a revealed comparative advantage (RCA). Nevertheless, more importantly, it shows how the regional and country-specific RCA evolved during the last 25 years. The last exercise makes it possible to identify descriptively how the research capacities raised or declined during the period of interest.

The empirical tool helps identify an RCA in a research discipline if it has a greater participation in the national scientific production than the participation of that area in the world's scientific production. Published articles and citation measurements of the scientific output are presented for 27 research areas in this research. Specifically, the RCA is defined as the quotient between the discipline's participation in the scientific output of a country and the participation of this same discipline in world scientific production. Formally,

RCA = ((P_is/P_i)/(P_s/P))

Where P_is are publications in a research area s in country i, P_i is defined as the total publications in country i, P_s is the publications in subject area s worldwide, and P is defined as the publications worldwide. This indicator was also developed for citations, focusing on both cases comparing RCA for ACR and the specific countries of the region throughout the entire 1996-2020 period. In terms of its interpretation, a revealed comparative advantage in a country's discipline is found if the RCA's value is greater than 1 in a given year. Otherwise, it does not show a competitive advantage for that specific year. Secondly, using the same database, we identify the specific research institutions that excel in research disciplines previously identified as the more productive in ACR. To go a step further in explaining the reasons for that success in such a deprived context for developing research capacities, the third goal of this research is interviewing leaders of the ACR's prominent research institutions and groups to understand their evolution. The last exercise aims to discern if the experience of prominent research institutions or groups is replicable in other ACR contexts.

Results and limitations

In our analysis, the ACR has revealed at least competitive advantages in six of 27 research areas for 1996-2020. These areas include agricultural and biological sciences, business, management, accounting, environmental science, medicine, nursing, and social sciences. However, enormous differences can be found in specific ACR countries. Heterogeneity in the ACR is the rule in terms of research capacities. Nevertheless, in some areas, it is possible to find no research capacities in the region and virtually in all ACR countries: Mathematics and Engineering and a few related disciplines.

These results raise questions for the debate regarding policy decision-making: Should the region enhance deprived research areas in all the countries? Or should they cooperate in interchanging the knowledge and professionals in the disciplines that each country revealed a competitive advantage? The experiences of successful research institutions and groups are replicable in other ACR countries? How could the ACR address its un-competitiveness in such important areas as mathematics and engineering?

The main goal of this research is to have the first long-term diagnosis of research capacities in the ACR. However, our analysis is not excent of limitations. First, the RCA as an analytical tool has its weaknesses mainly because our source of information comes from scimago database: meaning indexed databases, generating biases that minimize local non-indexed publications. Secondly, the interviews focus on successful research institutions or groups, which biases our results towards successful experiences. It is difficult or almost impossible to identify unsuccessful groups and the reasons for their decline since most of them are difficult to locate (few publications) or have already stopped operating.

ABSTRACT. Background and rationale: A novel theoretical framework named “Intersectional Antiracist Technology Framework” is developed by Dr. Danielle Wood and Dr. Katlyn Turner. This framework uses Systems Architecture to explain, evaluate and design approaches to incorporate Intersectional Antiracism within the Definition, Design and Distribution lifecycle phases of technology. The framework defines technology across four levels of scale including Concept, Artifact, Complex Product System and Complex Sociotechnical System. The framework seeks to demonstrate methods at each phase in the lifecycle of technology to employ an intersectional antiracist mindset and act towards promoting equity.

In this work, Zero Robotics (ZR) is used as a study case in STEM outreach activities to implement the Intersectional Antiracist Technology Framework. The framework provides a method to evaluate how the Zero Robotics program applies antiracist principles to the program design and execution. New insights and visions to the program design and distribution phases are being explored using the framework. Zero Robotics, an education outreach program led by Prof Wood at MIT, is the first U.S. space robotics competition since 2009. The program aims to engage young students from secondary schools in computer science and space technologies and prepare them for the future STEM workforce. Over 20,000 students and 4,500 educators across 15 countries have participated in the program.

Every year, there is a game challenge being designed for students to gain hands-on experience in engineering and coding. During the program, students learn about the fundamentals of science and robotics and practice their coding skills in the online simulation called the ZR Integrated Development Environment (IDE). The final competition is livestreamed from the International Space Station (ISS) and students interact with the astronauts and watch their code running in space. The program uses the space free flyer robots aboard the ISS called Astrobee, and it seeks a social impact in promoting STEM education and education equality. The ZR program lies at the intersection of Complex Product System and Complex Sociotechnical System scales in the framework which means that both technical and social factors should be considered when asking how the program can be designed with sensitivity to the identity of the program beneficiaries. Continuous development of the program is being made to broaden the participation of underrepresented groups and overcome the technical challenges from the transition to a new robot system.

Methods: New collaborations with Minority Serving Institutions, as defined by the U.S. Department of Education, Navajo Technical University (NTU) and California State University Long Beach (CSULB), are established to promote the participation of Hispanic and Indigenous communities. MIT, the founder of the ZR program, is committed to provide training to college students at NTU and CSULB on the IDE and ZR online simulation tools. NTU and CSULB serve as regional hubs for communities nearby and are responsible for outreach activities and local community support. Based on the principles in the Antiracist Technology Framework, this approach seeks to apply culturally sensitive implementation to ZR customized to the concerns of urban Hispanic students and rural Indigenous students. MIT learns from the experts at NTU and CSULB who are familiar with each local context.

In summer 2022, the first ZR Middle School (MS) competition using the new Astrobee system was held successfully after the transition to a new generation of space robots. It is also the first time for NTU and CSULB students to mentor and support middle school students in the space robotics program. The game challenge is an imagined story that motivates the code activities called “The Great Astro-Spelling-Bee”.

This tournament is a 5-week program and was conducted from June 27th to July 29th, 2022, with a final event live streamed from the ISS on August 3rd. Field days and local workshops are conducted in CSULB and MIT with campus tours and laboratory visits. The program experienced the design and distribution lifecycle phases as defined in the Intersectional Antiracist Technology Framework. Pre-program and post-program surveys from students and educators in the program are collected to evaluate the participation, design, and operation of the program. The method with respect to Indigenous data sovereignty to collect data from Native American students is in development.

Results: The presentation presents a graphical summary of the Zero Robotics program using the Systems Architecture Framework with an emphasis on Antiracist Technology Design. The graphical summary highlights the Context, Stakeholders, Objectives, Functions and Forms of the Zero Robotics Programs and shows how these system elements interact with identity-based features. By using the Systems Architecture framework, the MIT team leading Zero Robotics can identify gaps that need to be improved to increase the cultural relevance of Zero Robotics for the Hispanic and Native American communities. This includes providing language-relevant materials and asking survey questions in a manner that fits community culture. Responses to these findings are ongoing.

20 teams with 178 middle school students participated in the 2022 MS program, and 6 US states (California, Massachusetts, Illinois, Minnesota, Arizona, and New Jersey) and 3 Tribal Nations (Navajo, Hopi and Zuni) are involved. 40 of the middle school students are from Long Beach, Los Angeles, and Paramount Unified Districts. This improvement in diversity and inclusion is made from community outreach conducted by CSULB. Sixty-seven adults have supported the program operations in the summer, including 37 educators and 30 college students.

The pre-program survey with responses from 124 students shows that 60.5% of them in the program are male and 32.3.% are female. Moreover, 52.9% are Asian, 14.7% are Hispanic or Latino, 10% are Black or African American, 9.4% are White or Caucasian (using identity labels aligned with US Census conventions). The fact that females are underrepresented in the program is consistent with observation from the U.S. Census Bureau 2019 American Community Survey (ACS). In the survey, women only constituted 34% of the STEM workforce while 52% of the non-STEM workforce. Moreover, 91% of students expressed that they plan to go to college and concentrate in STEM majors, and 70.2% of students said they participate in the ZR program because of the programming aspect it offers. This implies that the majority of the students who come to the program already have some background in STEM fields.

The post-program survey to students has 31 responses. 77.4% of the students think the 2022 game is interesting, 71% expressed their desire to return next year and 77% would recommend ZR to their friends. The post-program survey to educators has 13 responses. On average, educators ranked ZR 8.7 out of 10 in terms of the impacts on students’ summer learning. 100% of educators thought the game was interesting and fun. 61.5% of educators would run ZR again and 76.9% would recommend it to their colleagues. These results give positive signals on the program impacts on students’ learning experience and the game challenge designs.

Significance: The “Intersectional Antiracist Technology Framework” incorporates aspects from science and technology studies, critical theory, design research, and systems engineering. Zero Robotics, as an impactful STEM education outreach program, is used as a study case to define and evaluate the framework. This framework is also used to explore solutions in the design and distribution lifecycle phases to improve social justice in education. Historical discriminations and inequalities in socioeconomic resources create barriers for certain groups to be engaged in STEM activities. These barriers shall be minimized to involve minority groups and equal opportunities should be provided to promote future workforce diversity. The outcome of this work is an evaluation of the current phase of the ZR program and an analysis of defining and applying the Intersectional Antiracist Technology Framework.

ABSTRACT. The emergence of affordable smart meters, solar panels, electric vehicle charging, and stationary storage systems, has spawned a new generation of policy innovations. These breakthrough technologies have motivated the creation of new policies, such as electricity tariffs that blend energy, demand and capacity in novel ways; new rules and regulations about who can sell electricity to retail customers; buy-back rates for excess solar generation; and minimum bills and interconnection fees to help ensure coverage of fixed costs.

The challenge now is to design retail electricity rates that reflect "cost causation" and prevent inequitable cost shifts—two Bonbright principles of utility ratemaking. As solar penetration advances, for instance, there may be a shift of costs from technology participants to nonparticipants, which has raised concerns about the equity principle.

The "classic" net energy metering tariff (where excess solar generation is bought back at a retail rate) has spawned an array of alternative tariff configurations. There are now buy-all, sell-all rates, value of solar tariffs, and net billing, as well as instantaneous versus monthly netting. Each of these options reflects stakeholder aspirations for these evolving technologies and markets.

In this study, we explore the reciprocal relationship between technological advances and policy innovation using a combination of qualitative and quantitative analysis.

First, we review the new wave of state net energy metering (NEM) policies and identify the extent that retail tariffs are evolving in response to technological advances and the market penetration of rooftop solar systems. The case studies show that policy innovations have been spawned by technological advances, but that their design is also a reflection of the structure of regional electricity markets.

Second, we conduct a statistical analysis of the continental U.S. states and the District of Columbia to explore how policies with various configurations correlate with technological adoption. A review of the new wave of state net energy metering (NEM) policies and a statistical analysis of correlates of their adoption are conducted. These allow us to identify the extent that retail tariffs are evolving in response to technology advances and the market penetration of rooftop solar systems.

Our research finds that net energy metering policy encourages the adoption of residential solar photovoltaics, with varying effects depending on the policy design. Further, we document the impact of increasing levels of adoption of distributed solar on the design of a new wave of variants to the design of net energy metering policy.

ABSTRACT. Though first-generation status has been studied in undergraduate student populations, little attention has been paid to first-generation faculty even though 16% of individuals who earned STEM doctorates in 2020 were first-generation (NCSES, 2020). Despite facing barriers typically associated with first-generation status (e.g., less material resources, lower cultural capital), a significant number of first-generation students are successful in becoming faculty members themselves. Conversely, recent studies have found high proportions of faculty with PhD parents suggesting that parental education provides significant advantages to accessing academia. Because parental education is often used as a proxy variable for socioeconomic status, this suggests that social background variables play a large role in shaping faculty careers. Using a nationwide sample of academic scientists in the United States from four STEM disciplines, we examine the extent to which socioeconomic status offers advantage (or disadvantage) to faculty as they advance their careers. We find that faculty with PhD parents are one and a half times more likely to work at higher-ranked and more competitive universities, while first-generation faculty are three times more likely to work in lower-ranked and non-doctoral-serving institutions. We find that first-generation scholars are especially disadvantaged in navigating the competitive work environment of research-intensive institutions.

ABSTRACT. Confronting climate change implies substantial and rapid acceleration into new innovative technologies that would promotes clean energy systems, zero-emission in energy systems, transport and mobility and industrial emission reduction that can reduce global emissions. This paper asked which countries have funded the most energy, transport, climate change, and industrial decarbonisation research? And why (i.e., what could have influenced the high/lower funding rate)? What are the disciplinary politics of funding? Which disciplines have received the most funding, and which countries have funded the most social science research? To what extent are funded projects interdisciplinary or trans-disciplinary? To answer these questions, this study catalogue and examines comprehensive global public research programs across some of the world's largest funders (and countries), including the most substantial funders of public research or countries responsible for the highest carbon emissions. It synthesizes data from 1990-2021 for 69 countries involving 163 research councils to examine and understand the global funding patterns on energy, transport, climate change, and industrial decarbonisation research. The study reveals the mean external cost for R&D funding is $2.3 billion, and the cumulative total amount of research funding from 16 countries that are the largest funders in the data was $2.268 billion from 97 research councils. Using percentile estimations of the cost of funded projects, the data reveals 69.4% from the United Kingdom, 8.11% European Union Commission, and 4.16% United States and others 18.9%. When we put the global context into the funding patterns, the data provides that the United Kingdom contributed about 40%, European Union Commission 27%, the United States 21%, and others 5% of the cost of the funded project. The analysis of global research funding across general areas reveals that Engineering and technology attract significant funds, with 28%, followed by social sciences and economics 27%, the Arts and humanities receive 18%, 16% was for Natural and physical sciences, and the least as shown in the data was 11% which is for Life sciences and medicine. The study discusses the results of the review, which are based on R&D funding as a game changer of deep transitions, before discussing how countries, research councils and academic institutions are being funded and how research results are creating a socio-technical change.

ABSTRACT. Introduction In South Africa, and many other African countries, the informal sector accounts for a significant proportion of economic activity. Informal businesses also localize goods and services for the communities within which they operate, providing accessibility to essential products and amenities that would otherwise be accessible only outside informal areas. Their understanding of local customer need and community dynamics often informs their pricing and payment terms. However, access resources such as funding, skills development and insurance that enable businesses to evolve and grow into sustainable micro-enterprises remains a challenge. For these reasons, informal businesses are particularly vulnerable to risk, which not only affects the owner, but employees and customers in the local community. One way in which owners respond to risk to stabilise, sustain, and even grow their businesses is through innovation. Innovation is crucial for informal businesses to compete and continue to provide cheap, flexible options for low-income households. A concern is that informal sector public policy and support programmes tends to be directed at formalisation, through business registration (Jorgenson, 2010). Our research shows that business registration is not an accurate indicator of graduation from informal to becoming a sustainable forma business. Business evolution is more accurately represented along a continuum of intermediate states rather than depicted as binary opposites of formal and informal. The argument presented in this paper is that, by supporting innovation in informal enterprises, progression along this continuum can be promoted towards building sustainable micro-enterprises. The emphasis should be on building capabilities to formalise.

Methods This study draws on a survey of innovation in informal enterprises, using methodologies adopted and adapted from informal sector research and the standard Oslo Manual, together with qualitative interviews and digital life stories of business owners in the informal sector. A total of 996 businesses were used to determine the varying degrees of informality in the informal sector, complimented by 48 qualitative interviews and thirteen digital stories. The research was conducted in a peri-rural area of the KwaZulu-Natal province in South Africa during 2019-2021. We present an analysis of events that prompt a response of innovation for informal business owners, which enable their businesses to survive or grow, and how these responses manifests in varying levels of informality over time. Importantly, the bottom-up approach adopted through the research design, which identified informal businesses by local community interviews and self-identification allowed for the inclusion of businesses that had at some point been registered as formal enterprises. The analysis adapts criteria developed by Mbaye and Gueye (2020) to determine the level of informality of informal businesses in the study. The criteria applied include registration status, business premises, number of employees, financial records, as well as a having separate bank account, and access to financial resources from formal financing institutions. The criteria were inversed for the analysis, meaning that if a business was not registered, it would be assigned as having met the criterion. Similarly, if the business did not operate from a separate premises, had no employees or financial records, etc., the criterion would be met, and the corresponding level of informality was assigned. The number of criteria met by each informal business corresponded to a level of informality ranging from 0 (formal) to 6 (totally informal). Findings and Discussion Our results show that every business in the sample met at least one criterion, meaning that none of businesses in the sample were considered completely formal. Most businesses in the sample met five criteria and were thus considered mostly informal. Using qualitative data, the study explored the relationship between innovation and levels of informality. The study found that negative events, which places the informal business under risk, acts as a catalyst for innovation as a response to mitigate or remove the risk to the business caused by the event. The nature of these events, which the study refers to as innovation events, include personal or financial difficulties such as crime, debt, illness, or the loss of a close relative. The data showed that these responses often led to business survival or growth, which causes a change in the level of informality of the business, either by increasing thereby becoming more informal, or decreasing by becoming less informal.

Approaches to support informal businesses are often directed at formalization through business registration, and support in the form of skills development or financing are often reserved for registered micro enterprises (CeSTII, 2021). In South Africa, registration of an enterprise requires an application to be submitted to the Companies and Intellectual Property Commission (CIPC), which can be submitted online or via participating banks. The cost of registration is R125,00 and registers the business with the South African tax authority, South African Revenue Service (SARS). It also requires financial reports to be submitted annually to the Commission. The cost and requirements for business registration are often unattainable for informal business owners, and programmes to facilitate registration do not consider the realities of businesses in the sector, such as those provided by the Small Enterprise Development Agency (SEDA). Policy support for informal businesses to move from informal to formal is underpinned by a binary and linear understanding of formality and informal business growth, referred to as the Vuvuzela Graduation Model. The model depicts formality in direct relation to business size. In our analysis, business size and registration status are merely two of the attributes considered in the evolution of informal microbusinesses. And the end-state we consider is whether businesses become sustainable economic entities.

An investigation of the qualitative data suggested that the nature of innovation events can be classified into the categories of financial or personal. Innovation responses ranged from actions taken to start the business, seeking additional avenues to market the business, to adding new goods or services for customers. We explored the determinants of the innovation response and found that competition, opportunity identification, social capital, having the existing skills, as well as family responsibility motivated owners to respond innovatively to crisis as enablers of business growth or to sustain its income stream. Thirteen of the seventeen innovation responses led to a change in the level of informality in the business, all of them resulting in decreased formality. The four innovation responses that did not result in a formality shift showed that business owners took action that had formality-shifting potential, such as learning new skills or adding new products or services to grow their business.

Nine of the seventeen business owners reported more than one innovation event. Of these, five business owners had innovation responses that led to a change in informality. The remaining four demonstrated potentially formality-shifting actions. The relationship between the change of informality after innovation event one and after innovation event two are not cumulative, meaning that the business does not necessarily become more formal after each innovation event. Our findings also show that potential formality-shifting actions do not necessarily shift formality after the second innovation event. The business may decrease, increase, or depict no change in formality at multiple points along the evolutionary trajectory towards complete formality. Even in the event of a business becoming totally formal, formality can shift, and decrease, depending on the outcome of the innovation response. Box 2 Mandla’s Flowers and Vegetables Conclusion Business size or registration is not an accurate indicator of growth or formality. Instead, the focus on formality as an outcome may very well be displaced. Many business owners use formal registration as a tactic for achieving contracts from public service organisations especially. An increase in the number of employees does indicate greater sustainability of the business in many cases. But this is very much sector-dependent, and the nature of employees, whether family or not, needs to be looked at to assess sustainability. Should not the long-term sustainable functioning of enterprises be the true goal?

Crisis events catalyse innovation in informal businesses. Informal sector researchers have in the past documented innovations as they perceived it. By and large these tended to focus on artifacts that were significantly enhanced or even transformed for use in informal businesses. It is only by using the concept of innovation systems that we are able to make the connection between innovation activities and the trigger events for business growth.

ABSTRACT. Since its establishment in 1999 according to the Framework Act on Science and Technology, the Korea Institute of Science and Technology Evaluation and Planning (KISTEP) has devoted itself to improving the quality of life of the people through the development of science and technology, innovative growth, and resolution of social problems. From science and technology policy planning and future forecasting, Research & Development (R&D) budget allocation and adjustment, national R&D project research, analysis, evaluation, and performance diffusion, preliminary feasibility study in the R&D sector, and international cooperation in science and technology, it provides in-depth research results. Today, in the face of domestic and international challenges such as competition for technological supremacy, new infectious disease, and digital transformation, the role and mission of science and technology are becoming more important. As a think-tank of science and technology innovation policy, KISTEP is providing detailed support for science and technology-oriented government operation based on data.

The innovative and challenging project is being promoted from 2020 to lay the foundation for creating innovative results by shifting away from the past “Fast Follower” method to a bold and challenging “First Mover” research project. In order to preemptively solve the challenges of science and technology related to people's happiness, quality of life, and creation of future innovation-leading industries, we are discovering and promoting innovative and challenging cross-ministerial R&D projects centered on clear missions. It covers social issues directly related to people's happiness and quality of life, such as environment, safety (disaster, accident, public safety, security), aging, health care, and food, and the creation of future innovation-leading industries. Themes that can be achieved through step-by-step improvement of existing technologies are excluded as much as possible, and research topics that will solve problems challengingly and will have a ripple effect if successful are mainly explored.

For the successful operation of the project, a promotion team has been established and is being operated. The name of the promotion team is KARPA (Korea Advanced Research Program Accelerator).

There are a total of five detailed criteria for selecting a research topic. “Clarity of Objectives” examines whether the problem to be solved and the objectives of the project are clear and specific. "Challenge" examines whether the business goal is to be world-first or world-class. “Innovativeness” examines whether disruptive innovation can be derived through R&D. "Differentiation" is reviewed to see if it does not overlap with the government's programs that have already been promoted or are currently being promoted. "Ripple Effect" determines whether a large scientific, technological, industrial, and social ripple effect is expected if R&D succeeds. Among the five detailed criteria, "Ripple Effect" is evaluated as the most important.

Research themes are being discovered through various methods. First, in the case of demand-based research, we are conducting a demand survey for new projects for related ministries and submission of proposals for innovative and challenging research themes for researchers from industry, academia, and research institutes. In particular, for industry-university-research researchers, the CIA (Crazy Idea Accelerator) online forum has been established, where researchers freely present ideas and experts evaluate and develop them, constantly discovering research themes. In the case of top-down, top-down research planning is being carried out based on key scientific and technological issues that need to be resolved nationally. We are pushing ahead with top-down planning to avoid step-by-step improvement of technology and to discover high-impact research themes candidates that can solve national problems and create future innovation-leading industries. By comprehensively analyzing reports related to domestic and foreign future strategies and promising technologies, promising research themes are derived, and blank areas that are missing from existing R&D projects are derived. In the case of the excellent researcher base, candidates for research themes are discovered by recruiting researchers with excellent ideas and insights. In addition, we are jointly discovering new research themes through collaboration with major research management organizations that are leading the planning of new projects.

When a topic is selected, a planning report is prepared through a detailed planning process over several months. The best industry-academic-research experts in Korea participate in the planning process. Detailed planning of the project is carried out so that the R&D project can proceed in the most efficient and effective way.

When the project implementation is confirmed through the government budget deliberation process, a project management team is launched for efficient project management, planning and evaluation are carried out centering on the head of the project management team (= Project Manager, PM). Flexible research methods are applied as much as possible to the selection of the research team necessary to achieve the project goals, the execution of the research, the execution of the research funds, and the evaluation.

After the inauguration of the project management team, the progress of detailed tasks are monitored, and feedback and follow-up measures are implemented so that the project can be operated as originally intended. A PM sharing meeting was held with the participation of the PM of each project management team to share the research direction, themes, progress, and to discuss and to adjust differentiation and related matters.

Institutionally, we are preparing a R&D support system that can maximize the creativity and autonomy of researchers by applying a research environment that does not blame failure and a flexible research method through this project. We plan to explore a total of 20 projects over 4 years, reflecting the government's R&D budget and implementing it. Topics discovered so far include about 10 convergence and original research, such as stratospheric drones for constant disaster monitoring, hypertube technology development for high-speed transportation, and CAR-T source technology development for solid cancer treatment. The topics discovered in 2020 and 2021 have been officially launched after planning and budget deliberation, and R&D is being actively carried out.

The current innovative and challenging project is being operated as a pilot project, and the promotion system needs to be improved in order to maximize operational efficiency and performance. It is necessary to prepare an operational plan for the second innovative and challenging project, such as securing separate financial resources for innovation challenge-type R&D and establishing a separate institution for efficient operation and management.

In this presentation, the purpose, progress, and significance of the innovative and challenging project carried out to create a more challenging R&D culture in South Korea will be analyzed. We would like to introduce the contents of the R&D program of the innovative and challenging project discovered over the past three years, and suggest the direction of future operation, including policy implications and system improvement measures revealed in the process.

ABSTRACT. Science and technology have played a leading role in preparing for a new era at each stage of Korea's economic and social development for the past half century. The government-led strong technology drive policy has become a driving force for Korea's major industries such as automobiles and semiconductors.

In this study, we will look at changes through science and technology, which is a sure preparation for the huge social changes caused by COVID-19 and an uncertain future, and analyze how the direction of innovation policies across social and industrial fields is changing due to COVID-19. .

Due to COVID-19, non-face-to-face and remote culture has spread, competitiveness in the bio field has become more important, nationalism has been strengthened, and risks have become routine.

Therefore, this study analyzes how these changes have affected Korea's science and technology innovation policy and suggests a direction of policy change.

The Korean government has presented the following major policy directions to strengthen the role of science and technology in a national crisis. In order to turn a crisis into an opportunity, the R&D model centered on the private sector was spread, and the industry's ability to respond to digital transformation and strengthen its self-reliance. In addition, a crisis response system in which industry, academia and research institutes cooperate based on science and technology has been established for crisis response.

First, in order to spread the R&D model centered on the private sector, a new R&D model was introduced and spread by delegating the full power of planning and management to private experts and supporting innovative system improvement such as competitive methods and internarional evaluation.

In order to accelerate the digital transformation, infrastructure investment through the Digital New Deal and the spread of various convergence services supported the smartization of industries (automation and intelligence) and enhancement of regional innovation capabilities. It is promoting digital transformation by supporting R&D for next-generation Data-Network-AI (DNA) core technology and spreading convergence to all industries. Expanding for the application of DNA technology to overcoming limitations in each industry and solving social problems, and apply digital innovative technologies such as cloud, IoT, digital twin, and block chain to various fields such as finance, medical care, manufacturing, and automobiles to create new industries and services.

To revitalize the non-face-to-face economy, we are strengthening support for core technology development and commercialization, and strengthening support for core technology development such as XR and 5G-based immersive content technology (high-quality voice and image quality technology, personal media technology, etc.) necessary for digitization of social and economic activities. . The government is also expanding support for the creation of non-face-to-face innovative services, focusing on areas with high non-face-to-face demand, such as education, distribution, and medical care. In addition, in order to improve efficiency and stability through digitalization of public infrastructure, it is preparing to improve the productivity of public infrastructure by securing the basic technology for smartization of aviation, road, railway, and city management and port and construction.

In preparation for periodic national crises that affect national life and national security, the crisis response system was strengthened in which the science and technology circles rapidly mobilize their capabilities to predict and respond in advance, and to suggest solutions to crisis situations. In each major crisis situation, government research institutes are designated as dedicated research institutes, and based on technical monitoring method setting and information sharing, proactive monitoring and crisis response R&D functions are strengthened and rapid emergency response research promotion system was established.

We are responding to global issues such as carbon neutrality in order to strengthen R&D for solving social problems based on social problems and to contribute to improving the quality of life. In addition, in order to promote structural transformation of society and economy, it is focusing on strengthening the industrial ecosystem and supporting technological innovation.

Responding to changes in the future social structure such as a super-aged society, strengthen R&D to improve the health and enjoy a pleasant life of the elderly population, systematically predict and manage disasters, and invest in the prevention, response, and recovery of accidents at construction and industrial sites It is also expanding R&D investment to secure public safety.

Before the corona pandemic, basic research, research for industrial development, and challenging research were emphasized by era, but recently, mission-oriented research to solve problems tends to be emphasized. As economic growth is slowing and investment in R&D is stagnant, it can be seen as a tendency to more clearly emphasize the direction of R&D.

As a concept that emphasizes the problem-solving function of R&D, mission-oriented research is emerging. It is a method of deriving a specific mission from a problem, identifying the science and technology necessary to achieve the mission, and conducting necessary research. Mission-oriented research is research carried out with the resolution of very difficult problems as its mission. It tends to be pursued with the goal of solving regional or national and social problems.

ABSTRACT. Introduction Nanotechnology is considered to be one of the most promising technologies of the 21st century. With applications in nearly every scientific discipline, scientists can use nanotechnology to measure, manipulate, and manufacture at the atomic, molecular, and supramolecular levels. Over the past 20 years, there has been major investment in global R&D funding for nanotechnologies directed towards the improvement of health diagnostics, drug delivery, nano-biopharmaceuticals, and vaccine development. In a race to handle the COVID-19 pandemic, many scientists turned to nanomedicine innovations to develop novel drugs. Thus, it is critical to understand the evolving role that nanomedical innovations have played in combating COVID-19 and other coronaviruses over the past 2 decades.

Rationale In our study, we conduct a bibliometric analysis of nanotechnology and coronavirus research in order to understand how these technologies have evolved over since 2000. Bibliometric analysis is a research method for exploring and analyzing large volumes of text scientific data, enabling scholars to decipher and map connections in the data. Given the volume of scientific literature produced over the past two decades in the fields of coronaviruses and nanotechnologies, we believe that a bibliometric analysis of coronaviruses and nanotechnology offers a comprehensive and efficient approach to analyzing data.

In the winter/spring of 2020, COVID-19, swept across the world, killing millions of people and shuttering almost every institution, business, school, and organization. Scientists immediately began finding cures and vaccines for this deadly virus. Soon scholars of science policy began researching trends in COVID-19 research. Over the past year, there have been many bibliometric analyses of COVID-19 and over the past 20 years, there are even more bibliometric studies of nanotechnology. However, there is limited bibliometric research on the applications of nanotechnology for combating COVID-19 and there are no bibliometric analyses of nanotechnology for coronaviruses in general, including the Severe Acute Respiratory Syndrome (SARS) and the Middle-East Respiratory Syndrome (MERS). The pool of publications we examine is a larger dataset than most bibliometric studies have examined with respect to nanotechnology and COVID-19. This study fills that gap by showing trends in nano-coronavirus research since 2000.

We have three hypotheses: 1) From 2000-2018, there would be a low volume of scientific publications that discuss nanotechnology and coronaviruses (nano-coronavirus research). 2) In 2020, we expect the amount of nanotechnology research on coronaviruses to spiked after the COVID-19 pandemic. 3) In response to the SARS and MERS outbreaks, we expect the global scientific production of nano-coronavirus publications to temporarily increase.

Methods This study uses PubMed as the source for nano-coronavirus articles. PubMed is a publicly accessible database of medical-related research. PubMed is hosted by the National Institutes of Health and contains over 34 million citations and abstracts. PubMed is a common database for bibliometric analyses related to medical fields.

Our first step was developing a search query to find nanotechnology and coronavirus articles. The team conducted an extensive literature review of nanotechnology and coronavirus bibliometrics articles to find keywords. After collecting the most relevant keywords, we tested the accuracy of each keyword to ensure it had the necessary recall and precision. Once we developed the keywords, the team downloaded the articles from Pubmed on July 18, 2022. We found 3,446 nano-coronavirus articles from January 2000- July 2022. The raw data was then imported to Biblioshiny Software to perform bibliometric analysis on publication trends. We analyzed the annual scientific production of nano-coronavirus research, the corresponding author’s country of origin, the multiple-country publication ratio, and the analysis of keywords within publication abstracts. The raw data generated by Biblioshiny was exported to Microsoft Excel through which figures were generated.

Results/Discussion Upon analysis of the bibliometric data, we found our first two hypotheses to be validated. Between 2000-2018 there was a low volume of nano-coronavirus scientific publications and by 2020 the volume of scientific production skyrocketed. However, we could not find evidence to support the third hypothesis. There was no rise in nano-coronavirus scientific production after the SARS outbreak (2002-2004) nor the MERS outbreak (2012-2016). Additionally, analysis of the corresponding author’s country of origin reveals that the US, China, and India are the top three leading countries in total nano-coronavirus scientific production. Unexpected findings in total scientific production between 2001 and 2022 include the United Kingdom ranked low at 20th and Iran ranking high at 4th.

Future Research Over the next few months, we are going to compare our keyword data for nano-coronavirus literature with nano-influenza scientific literature produced over the same time interval. Our initial hypothesis is that the two domains will have different patterns in research topics and keywords; however, in contrast to nano-coronavirus research that spiked in recent years, we expect nano-influenza scientific literature to have a steady rise in production. Moreover, we expect nano-influenza research to have been produced in countries that are traditionally regarded as powerhouses in general scientific research as opposed to countries such as Iran and Saudi Arabia. The findings of our complete study may be used to help policymakers allocate funding for emerging nanotechnologies that are designed to combat coronaviruses as well as other infectious diseases. Our ultimate goal is to ensure that countries are better prepared to handle scientific R&D in response to future pandemics and public health emergencies.

ABSTRACT. This study explores the unique case on how sovereign wealth funds (SWF) impact defense policies. Lately, SWFs have emerged as a critical actor in defense policy, as they are breaking the strong links between the government and the defense industry. SWFs directly affect defense companies, whose industries have mainly been determined by the requirements of their own military. As a result, SWFs are changing the business ecosystem and thereby influencing future domestic defense policies. This study examines the decision of Hanwha, the largest defense industry of South Korea to separate the unit producing cluster munition due to the ESG considerations of European SWFs, which provides an interesting case for the research on increasing influence of non-state actors on defense innovation policy.

“Cluster munition” is considered inhumane because it can attack large area targets and have a high rate of failure, which may cause civilian damage. Nevertheless, it is difficult to ignore their technical effects in terms of war capability. Under the security situation of South Korea, which is to confront North Korea, cluster munition is an important asset in South Korea's defense policy and a high priority of their weapon system acquisition program.

In this situation, the European SWFs have excluded investments in companies that violate the criteria of responsible investment. The criteria include environmental destruction, opaque management governance, and inhumane weapons manufacturers. Korea’s largest defense industry Hanwha, which is a Korean cluster munition weapon system manufacturer, has also violated this criterion. Hanwha accounts for 46 percent of Korea's defense industry's revenue. As SWF's investment in Hanwha as a whole continued to be excluded because Hanwha manufactures inhumane weapons, Hanwha stopped manufacturing cluster munitions. This is because the defense sector, which accounts for only 13% of the corporation's total revenue, could not cause the group as a whole to be stigmatized. Instead, the defense sector established a separate company that could manufacture the already developed cluster munition weapons and supply them to the Korean military.

This study is based on the signaling theory of third-party intermediaries. This theory argues that a signal presented by a third party can more easily gain the trust of other actors than the signals from the stakeholders. This is because the third party has a low possibility of information distortion. The third-party actor SWF invests according to its own standard, but this behavior can impact the investment standards of other investment funds.

This study will be conducted through interviews and archive surveys. We plan to interview government and government-funded research institutes such as the Ministry of Defense, Defense Acquisition Program Administration, Agency for Defense Development, and Defense Agency for Technology and Quality. In addition, military organizations such as the Joint Chiefs of Staff and Army Headquarters, and defense industries such as Hanwha, KDI (which is separate from Hanwha), the Korea Defense Industry Association, and SWF affiliates in Korea. Hanwha's corporate disclosure and investment data will also be analyzed in time series, and the government and public military data will be examined.

Weapon systems exhibit capabilities by integrating several subsystems, and system integration is easier for companies that have large internal capabilities. Hanwha is the representative SI corporation in Korea. The withdrawal of Hanwha from Korea's cluster munition means that the point centrality of the domestic cluster munition will be dismantled in the future, making it difficult to continue research and development and mass production.

This study is expected to provide insights into how foreign actors influence domestic defense innovation policies. Existing defense policies were decided in a top-down manner among limited internal actors. In particular, it is not common to be cross-bordered by actors other than internal actors in the subsequent mass production stage of the weapon system already deployed in the field after domestic R&D was completed. We believe the results of this study can be applied as a reference point when establishing defense policies for various countries in the future.

ABSTRACT. Block grants and project funding have been the two major forms of government funding for the science and technology field in the worldwide level. (Wang et al., 2018) Similar cases occur in China.(Aruhan et al., 2019) Ever since the science and technology reform initiated in 1985, the competitive project funding mechanism based on peer review has been the chief funding model in China. This has contributed to a huge breakthrough in national innovation capacity and development in various research fields and disciplines, yet it has also triggered a series of problematic issues, particularly for the individual researcher, i.e. over-competing causing less time in research, insufficient stable funding causing heavy workload from project application, etc. To solve these problems, the Chinese government issued a series of stable funding policies, starting with Fundamental Research Funds (FRD) to national research institutes in 2006. As one key type of block grant in China, FRD is granted to two types of research institutions. national research institutes and research universities. Recently studies have examined the efficiency and performance of block grant funding policy in the national research institutes and universities in China(Chen et al., 2018; Zhang, 2019;Fu et al., 2016; Liang et al., 2017), yet an in-depth investigation based on individual perspective, namely how individual researcher feels about the policy implementation has not been taken into consideration. Based on the two-factor theory（also known as Herzberg's motivation-hygiene theory），this study first combines the FRD policy intentions and expert opinions and establishes an FRD block grant policy implementation satisfaction evaluation indicator system. The evaluation indicator system is semi-structured, containing one part of closed questionnaire-based survey and one part of open-ended question. The closed questions consist of eight questions, three of which belong to hygienic factors and the rest five belong to motivation factors. The open-ended part consists of one question investigating the researcher individual suggestions to improve FRD policy implementation satisfaction. Next, this study hands out the questionnaire-based survey to various national research institutes in China. Based on the sample of 4,016 questionnaires received, this study then divides the respondents into two groups, namely the high-satisfaction group and the low-satisfaction groups. By adopting research methods of descriptive statistics, multiple linear regression and text analysis, this study shows the following conclusions: a. The satisfaction of FRD block grant policy implementation is positively and significantly associated with the amount of FRD funding the researchers receive. b. For the low-satisfaction group, the satisfaction of FRD block grant policy implementation is negatively and significantly associated with the pressure of receiving the FRD funding. c. The satisfaction of FRD block grant policy implementation is negatively and significantly associated with the age and professional titles of individual researchers. Further, the paper conducts the full-sample and grouped analyses on the individual suggestions for FRF policy implementation, revealing that to improve block grant policy satisfaction, researchers anticipate a larger amount of investment and an improved allocation scheme based on demands and strategic plans. Last, the study puts forward relevant policy implications based on these research findings.

ABSTRACT. The rapid development of AI technologies has propelled various countries to increase their research and development capacities in this domain as part of “the AI arms race.” At the same time, the widespread utilization of AI highlights the need for regulatory interventions. Despite the difficulty of the regulatory task and uncertainty associated with AI’s impacts, several countries have started “the race to AI regulation” and have come up with unique and innovative approaches to regulating this technology. The spectrum of regulatory proposals spans from hard laws and the prohibition of certain systems to industry self-regulation based on AI ethics. The most detailed hard law on AI is currently undergoing public discussion in the EU, and regulation for recommendation algorithms is already implemented in China. Meanwhile, the governance of this technology elsewhere is mostly conducted through soft law mechanisms, which include governmental strategies and frameworks, alongside private and non-governmental sector guidelines and codes of conduct, often realized in the form of ethics-based industry self-regulation. This spurs the ongoing debate about which of the two approaches better promotes consumer welfare. While strict regulatory requirements may better protect society against the risks of AI technologies, they also tend to hinder the pace of innovation. It is unclear to policymakers and researchers which approach (strict command and control or ethical industry self-regulation) maximizes consumer welfare, and under what conditions. The conceptual difficulty in addressing this dichotomy partly stems from the lack of a common framework that incorporates both sides of the argument. In response to this gap in the literature, this paper has developed a model to address the following interrelated questions: (1) What are the advantages and disadvantages of the two regulatory approaches? (2) What institutional factors influence the outcomes of the two approaches? (3) How should governments optimally balance the tradeoff between AI innovation and consumer protection in general? To empirically ground our conception of different levels of regulatory stringency, we first examine the regulatory proposals from the EU, the UK, the US, Russia, and China. Our document analysis shows that a more stringent approach to AI regulation is taken by China, the EU, and potentially the US (if the Algorithmic Accountability Act is adopted), whereas a more relaxed approach is taken in Russia and the UK. The proposed level of regulatory stringency depends on how much they prioritize stimulating AI innovation in the private sector. Having understood the trade-offs from the policy documents, we zero in on the regulation of AI systems that are developed by the private sector for commercial purposes. Unlike those developed by the state for national security or military purposes, the former

presents a more challenging case for regulators since they do not have direct control over the innovation, exploitation, and usage of such AI systems. We also set aside systems that are outright prohibited since the issue they present is one of legal enforcement rather than economic trade-offs. Thus, our primary interest lies in the grey area – the types of commercial exploitation that are within the legal boundaries, yet may be considered unethical once revealed to the consumers. Examples of such exploitation include the case of Cambridge Analytica, the usage of large language models for clickbait fake news generation, deepfake technologies for generating pornography, or the boosting of Amazon’s own products on its website. However, the logic behind regulating consumer-facing AI systems is intricate, not least because decisions regarding innovation, consumer protection, and frequency of usage are all decentralized among various stakeholders who pursue their own objectives. The understanding of the optimality of various regulatory approaches, therefore, calls for a systematic framework capable of analyzing the strategic interaction between various stakeholder groups. As such, we answer the proposed research questions by constructing a game-theoretical model which examines the complex incentive dynamics between innovation and consumer protection. It is important to acknowledge that our model is not designed to be explanatory for the differences in regulatory approaches chosen by different countries, because the countries’ choices could be irrational, affected by path dependence, or be the derivative of the political regimes in power. Instead, our model intends to contribute to the normative discussion on the optimal approach in regulating AI, and clarify the current academic and policy debates by pointing out how the optimal regulatory stringency is conditional on the institutional environments. The regulatory stringency chosen by the government is modelled by the probability that the exploitative practices of local AI companies are revealed to the consumers. This modeling choice is motivated by a unique challenge facing AI regulators. One key aspect of regulating AI is the difficulty of interpreting the workings of the black box systems, particularly what kinds of data are collected and what types of algorithms are used to extract valuable information by the companies. This fundamentally differs from industrial sectors where their social cost of production such as environmental pollution is relatively easily monitored and detected. In that sense, it is important that our model incorporates the possibility of revealing information to consumers, which will affect the behavior of consumers and their welfare in the end. After all, for unethical but lawful exploitation, it is consumers’ knowledge of such practices rather than top-down prohibition that acts as a disciplining device. Based on our game-theoretic analysis, we have developed an economic theory of how the welfare-maximizing level of regulatory stringency for AI depends on various institutional

parameters. Under high foreign competition, domestic innovation plays a relatively small role in serving consumers. On the other hand, consumers benefit most when they are not misled to underuse the highly competitive foreign AI systems. As a result, the prioritization of consumer protection should motivate a government to choose a high level of regulatory stringency under high foreign competition. Meanwhile, under low foreign competition (for instance, due to strong protectionist policies), the domestic AI industry can effortlessly win over local consumers from their foreign competitors. This means domestic firms can derive high marginal benefits in terms of market share from improving their algorithms. As a result, the robustness of domestic firms’ innovation incentives should motivate a government to also choose a high level of regulatory stringency under low foreign competition. Interestingly, under intermediate foreign competition, the government faces a delicate trade-off between consumer protection and innovation. Too stringent regulation stifles the innovation incentive of the domestic AI industry, whereas minimal regulation subjects the consumers to excessive exploitation. To maximize the actual consumer welfare, the government may strategically lower its regulatory stringency and turn a blind eye on some occasions. Across all institutional environments, however, minimal regulations are never compatible with maximizing actual consumer welfare. As such, the objectives of such regulatory design may be either rationalized by the prioritization of innovation, domestic producer surplus, or the perceived welfare of the consumers. In the latter, the government is primarily concerned with the image that this regulatory intervention produces without worrying too much about the actual protection of consumers – essentially using a loosely designed regulation as a PR tool. This suggests that further empirical studies should pay close attention to cases where governments are proposing very loosely defined regulations for AI.

ABSTRACT. The massive growth of global research activity in recent years has spurred an exponential increase in the amount of related scientific literature. Successful research in the face of the increasing complexity of modern scientific knowledge together with the diversity and depth of the studied problems requires an understanding of the related scientific landscape - the structure and interrelation of trends in science. The situation gets even more pronounced for the granting organizations on their way to developing efficient granting policies since they have to deal with many areas of science at the same time. Professional expertise remains the unique option to address the mentioned problem but it is often subject to personal opinion which, as consequence, requires a pool of experts engaged to perform single expertise. Seeking for appropriate experts willing to participate in the expertise becomes a challenging problem considering applied time limits. Therefore future scientific information management should belong to systems that will play the role of “virtual experts”, helping researchers and scientific foundations cope with the massive amount of research data and the high speed of scientific progress by facilitating manipulation, filtering, and combination of information. One of the features of such systems should be the ability to generalize scientific knowledge to identify scientific research trends and possibly even predict ones.

To satisfy mentioned requirements it is necessary to develop methods of data manipulation and processing that will accurately analyze the current state of scientific knowledge and have the potential to predict its future states as an intrinsic feature of such an analytic system. Available digital records open wide possibilities for statistical analysis of scientific documents and related metadata for topic modeling and evolution, knowledge mapping, citation indexing, etc. We have a lot of statistical tools to evaluate and measure the scientific outcome by calculating different scientometric indices. But while such measuring is efficient for evaluating the current scientific progress it is an open question of how to model future scientific discovery.

Recent studies by Wang at. al. use historical data on paper citations to predict later citations based on early patterns. It was shown that citations accrue to articles published in scientific journals over time according to a well-behaved log-normal distribution, with a rise in citations at the point of publication followed by gradual decay. The proposed model can be used to predict an article’s success and its development may lead to the methods of effective detection of promising research directions. But even in this approach selected articles needed a human expert to analyze their content to extract promising new hypotheses.

We propose to combine mentioned approach with a full-text analysis of scientific documents to identify promising research directions. The central element in our approach is a field-specific ontology based on a conceptualization of scientific knowledge. We use data collected by Science-WISE (SW) platform (http://sciencewise.info/) about frequencies of scientific concepts - the unique phrases that comprise one or more words and reflect certain basic notions of the scientific ontology. Investigating the dynamics of the entropic proximity measure of concepts we plan to find common patterns in the corresponding ontology graph at the moments preceding the appearance of new concepts as indicators for possible discovery.

The SW platform appeared as a result of a collaboration between physicists and computer scientists from EPFL and CERN. The SW uses modern methods of information retrieval, text analysis, and statistical data analysis to process large corpora of research publications and offers semantic recommendation systems, semantic bookmarking capabilities, and paper annotation. It is essential that the SW system excludes subjective statements about scientific ontology and provides evaluation and reflection made by the scientific community. In this study, we use an SW collection of concepts of high energy physics, derived with unique algorithms by processing and parsing the texts of scientific documents from ArXiv (https://arxiv.org/) preprint server.

Technically, scientific concepts are the keywords and a few of them are usually added to the article metadata to describe the main directions of the study. The collection of concepts commonly related to a particular scientific area forms the specific scientific ontology used to encode scientific knowledge in the texts of the documents. Information about the occurrence of the concepts in a particular text may identify the topic of the article. Statistics on the usage of concepts in various documents can be used to calculate the proximity between concepts. To evaluate such proximity we propose to calculate the amount of mutual information between concepts. For the collection of concepts, their mutual proximity is stored as a symmetric matrix of normalized distances between each pair of concepts. Scaling of the matrix on 2D or 3D plane can produce the so-called entropic map of concepts. By calculating the matrix for a particular time interval and then gradually increasing it with a certain time step we can investigate the dynamics of the underlying complex network of concepts. We explain the algorithm in more details in our previous study, where we use information about the dynamics of a such network of concepts to identify the hot topic trends.

The collection of concepts that can be related to a certain topic is changed over time. New concepts may be introduced by the scientific society to identify a new method, solution, or discovery. If the idea behind the concept is successful, later it can be used in many scientific documents on the corresponding topic so the amount of its associated Shannon entropy will grow. This also will lead to stronger connections with other concepts and eventually, this new concept may become a center of a new topic.

Detecting the patterns in the concepts network before and after the moment the new concept was introduced we plan to identify the necessary conditions preceding the creation of the concept. As a possible features to identify a successful concept we use the evolution of its Shannon entropy and citation history of the document(s) where it was introduced first.

We anticipate that detected patterns in the concepts network associated with successful new concepts can be used to predict future discovery. The detailed analysis of the parameters of the concepts network at a time preceding the moment of discovery may serve as a source of information for future prediction mechanisms that would augment scientific ability, increase productivity, and multiply returns from science for society.

ABSTRACT. This study focuses on the impacts of multiplex social networks on successful science productivity measured as funded research grants. While previous studies have demonstrated the diverse impacts of collaboration networks on science productivity by looking at the size, diversity, closeness etc. (e.g., Gaughan et al., 2018; Melkers & Kiopa, 2010; Newman, 2001; Siciliano et al., 2018; van Holm et al., 2021), few have looked at the impacts of social ties beyond publications. A social network tie becomes multiplex when multiple functions of interactions are shared within a single relationship (Ferriani et al., 2013; Ibarra, 1995). We look at how ties become multiplex across research collaboration, data and material exchange, and socialization outside of work. Network multiplexity helps individuals take advantage of the diverse types of resources available in a single relationship as such ties are connected through difference exchanges and high level of trust (Coleman, 1988; Methot, 2010). At the same time, network multiplexity can harm one’s productivity as the costs of maintaining such a rich tie can be costly and time-consuming (Mayhew & Levinger, 1976; Methot et al., 2016).

In our study, we ask two questions: (1) How does network multiplexity influence successful grants of academic scientists? and (2) Does the relationship vary by gender? Understanding differential impacts of network multiplexity by gender is critical given that literature substantially demonstrates marginalization of women in receiving social network benefits.

We use a 2017 national survey on US academic scientists in three STEM fields (biology, entomology, and ecology) to explore the interactive effects of network multiplexity and gender on successful science production. Drawing from social network literature, we develop hypotheses on the reverse U-shaped relationship between network multiplexity and productivity and how the relationship varies by gender. Using linear regression model, we test theories controlling for individual characteristics, work characteristics and organizational characteristics. The results will demonstrate that 1) faculty who have more multiplex network within their social networks will be more productive, 2) the positive impact of multiplex network will be reduced after reaching a tipping point, and 3) the effects of multiplex network will be different between male and female scientists.

Our study has both theoretical and empirical contributions. First, we contribute to understanding the impact of social networks on scientific productivity by integrating literature from the fields of social network, gender equity and personnel management. Second, this study can provide implications for workplace socialization in higher education institutions. We expect our results to provide venues for STEM fields in academic institutions where they can find ways to connect faculty’s socialization and their productivity. The study will conclude with policy implications to improve faculty’s productivity by emphasizing the importance of social networks and the gender difference which connects to faculty retention.

Reference

Coleman, J. S. (1988). Social capital in the creation of human capital. American Journal of Sociology, 94. https://www.crcresearch.org/files-crcresearch/File/coleman_88.pdf

Ferriani, S., Fonti, F., & Corrado, R. (2013). The social and economic bases of network multiplexity: Exploring the emergence of multiplex ties. Strategic Organization, 11(1), Article 1. https://doi.org/10.1177/1476127012461576

Gaughan, M., Melkers, J., & Welch, E. (2018). Differential Social Network Effects on Scholarly Productivity: An Intersectional Analysis. Science, Technology, & Human Values, 43(3), Article 3. https://doi.org/10.1177/0162243917735900

Ibarra, H. (1995). Race, Opportunity, and Diversity of Social Circles in Managerial Networks. Academy of Management Journal, 38(3), Article 3.

Mayhew, B. H., & Levinger, R. L. (1976). Size and the Density of Interaction in Human Aggregates. American Journal of Sociology, 82(1), Article 1. https://doi.org/10.1086/226271

Melkers, J., & Kiopa, A. (2010). The Social Capital of Global Ties in Science: The Added Value of International Collaboration. Review of Policy Research, 27(4), 389–414. https://doi.org/10.1111/j.1541-1338.2010.00448.x

Methot, J. R. (2010). The effects of instrumental, friendship, and multiplex network ties on job performance: A model of coworker relationships. University of Florida.

Methot, J. R., LePine, J. A., Podsakoff, N. P., & Christian, J. S. (2016). Are Workplace Friendships a Mixed Blessing? Exploring Tradeoffs of Multiplex Relationships and their Associations with Job Performance. Personnel Psychology, 69(2), Article 2. https://doi.org/10.1111/peps.12109

Newman, M. E. J. (2001). The structure of scientific collaboration networks. Proceedings of the National Academy of Sciences, 98(2), 404–409. https://doi.org/10.1073/pnas.98.2.404

Siciliano, M. D., Welch, E. W., & Feeney, M. K. (2018). Network exploration and exploitation: Professional network churn and scientific production. Social Networks, 52, 167–179. https://doi.org/10.1016/j.socnet.2017.07.003

van Holm, E. J., Jung, H., & Welch, E. W. (2021). The impacts of foreignness and cultural distance on commercialization of patents. The Journal of Technology Transfer, 46(1), Article 1. https://doi.org/10.1007/s10961-020-09775-9

ABSTRACT. Background and rationale: Funder research impact assessments (RIA) are exercises which measure the research outputs, outcomes, and impacts of research projects, portfolios, or programs. They include three key components: (1) activities which funders undertake to assess impact of their investment such as reviewing publications stemming from funded research and asking researchers about their perceptions of project impacts; (2) methods used within these activities such as bibliometric analysis to capture citation data and surveys or interviews to capture data about research impacts from investigators or stakeholders; and (3) frameworks used to guide these activities such as the Payback Framework or the Canadian Academy of Health Sciences (CAHS) Framework. The field of RIA has seen remarkable growth over the past 30 years. Several RIA frameworks have been developed and applied by research funders around the world, new technologies exist to better capture research impacts automatically such as Dimensions.ai or Researchfish, and best practice guidelines have been established to provide researchers and funders with high-level advice about conducting RIAs. Recent literature reviews have found that while RIA is a valid and essential tool to measure the wider benefits of biomedical research, there are clear methodological limitations that may limit the utility of RIA exercises and their potential conclusions.

To usher in the next era of RIA and mature the field, future RIA methodologies need to become more transparent and easily implementable. Current advice for RIA best practices such as those developed by the International School of Research Impact Assessment (ISRIA) provides an important foundation of the fundamentals of impact assessment, and several literature reviews provide an overview of available RIA frameworks for evaluation. However, the current literature falls short of offering practical guidance funders can use to design a RIA, including how to select an appropriate RIA framework and related impact indicators, collect and analyze data, and reflect on post-assessment steps that can be taken to improve RIA culture within their organizations. Our aim in this poster is to understand what data funders are currently collecting when performing their own assessments, how they collect this data, and how they use it to assess their own research portfolios. We draw on a subset of literature from our larger systematic review (Abudu et al. 2022) to understand the steps that funders have taken to implement an indicator-based RIA of a project portfolio.

Methods: Within our systematic review, we identified all published studies (n=21) which funders explicitly identified an assessment or evaluation-based RIA framework to guide their RIA activities. We reviewed these studies to examine: 1) the operational steps funders took to perform their RIA; and 2) the variation in how funders implemented the same RIA frameworks. This process allowed us to explore the varying indicator-based “operational pathways” that funders employed in their analyses and spurred us to organize the methods and approaches funders listed (such as literature reviews, identifying a sample of projects, documentary review, bibliometrics, surveys, and interviews) into a logical series of steps needed to complete a successful RIA. We then applied this series of steps to the papers in our review to understand the areas where funders are succeeding in RIA implementation and where challenges and opportunities for future development lie ahead.

For each paper included in our review, we coded data on the aims of the framework used by the funder and why it was chosen for use; the data collected by the funder to assess the research portfolio; and information the funder provided about staffing needs or requirements, resources used, and lessons learned during the assessment. In addition, we extracted data on seven steps for assessment implementation: Step 1: Setting the Stage for Analysis; Step 2: Framework Selection; Step 3: Metric/Indicator Identification; Step 4: Primary Data Collection; Step 5: Data Synthesis & Analysis; Step 6: Communicating Results; and Step 7: Reflecting on Best Practices/Capacity Building. These steps drew upon existing principles from the 2018 ISRIA Statement, the Reed et al. 2021 Methodological Framework for Evaluating Research Impact, the World Bank’s Roadmap for Implementing an Impact Evaluation, and informal conversations with A. Kamenetzky, an experienced RIA implementor at the National Institute of Health Research.

Results: We found that funders are using a wide range of frameworks (n=12) to implement RIAs, and that even when the same framework is used to analyze a research portfolio, funders are using different data sources, methods, and resources to support their analysis. Approximately half (n=10) of studies in the review identified metrics and impact categories that they planned to measure during their assessments within the methods sections of their papers. Primary data collection activities were the most widely reported activities across papers included in the review, however only thirteen studies reported activities to synthesize or analyze the data they had collected. No studies reported on activities to communicate results of their RIAs beyond the included review paper or included reflections on lessons learned after preforming the RIA.

Building on the analysis from this review, seven essential steps for RIA implementation – from setting up the RIA to data collection to an end-of-assessment reflection of best practices and opportunities for capacity building – were developed to assist funders. These steps may help funders to implement their RIA more practically and rigorously. We encourage funders to: discuss the preparatory activities that they undertook to set up their analyses in greater detail within papers’ methods sections (step 1); carefully select an assessment-based RIA framework for their analysis, considering if a legacy framework is applicable to their portfolio (step 2); pre-select indicators for impact measurement and specify them within the methods (step 3); detail all data collection activities undertaken to gather new and existing data for the assessment (step 4), as well as activities undertaken to analyze and synthesize data (step 5); share planned or completed dissemination activities to share results within the papers’ discussion section (step 6); and report lessons learned during the assessment process more formally to support knowledge-exchange of best practices across funders (step 7).

Significance: RIA for biomedical research can take many forms. In order to increase transparency and improve methodological quality and reporting of funder RIAs, we recommend that funders follow seven key steps for implementing a RIA. Incorporating the seven implementation steps will help to standardize the RIA evidence base and allow for much-needed future empirical research to determine optimal data sources and methods to best evaluate funders’ research impact. We hope that our work, in tandem with future research can inspire funders to feel more confident that their impact assessments are methodologically sound, meaningful to end users of their research, and providing valuable insights for their organizations.

References:

Abudu, Rachel, Oliver, Kathryn, and Boaz, Annette (2022), 'What funders are doing to assess the impact of their investments in health and biomedical research', Health Research Policy and Systems, 20 (1).

Adam, Paula, et al. (2018), 'ISRIA statement: ten-point guidelines for an effective process of research impact assessment', Health Research Policy and Systems, 16 (1).

Gertler, Paul J., et al. (2016) Impact Evaluation in Practice, Second Edition [online text], Inter-American Development Bank and World Bank <https://openknowledge.worldbank.org/handle/10986/25030>

Reed, M. S., et al. (2021), 'Evaluating impact from research: A methodological framework', Research Policy, 50 (4), 104147.

ABSTRACT. BACKGROUND There is a wide body of research on the gender gap in academia, but the gender differences in the content of citations remain underexplored. Additionally, we do not know if social cues like gender would remain relevant to the reception of scientific papers in the midst of a global public health crisis. In the pandemic, there is a higher tradeoff between speed and quality of information. Due to the high-stakes nature of findings, scientists should be cautious to embrace new information. But there is also a high pressure to form a consensus rapidly, so they must make decisions in a short period of time. Thus, this provides a rare opportunity to observe how a new scientific finding is acknowledged and legitimized in the scientific community from its very start to its end. This also provides better data coverage. Old papers are sparsely covered in bibliometric data and more recent findings take a long time to be legitimized. In contrast, the covid-19 started recently, so the data coverage is better, and its legitimization process is expedited due to the urgent nature of relevant research.

HYPOTHESES Past research suggests that stereotypes and bias will be even more impactful under highly uncertain and urgent conditions like the pandemic. The evidence has been shown in research about decision-making in the Great Recessions. Furthermore, in Status Characteristic Theory, gender forms a hierarchy whereby men have higher status than women. Gender is a diffuse status characteristic that influences evaluations of the most task. Men are advantaged as higher-status actors and women are disadvantaged as lower-status actors. There are also empirical studies that support such predictions. As female workers are perceived as less competent and less committed to their work, the legitimacy of female professionals is often questioned. In other words, female professionals are likely to experience pressure to prove their qualifications repeatedly. Gender disparities in the patterns of citations are also prevalent, and female-authored articles are systematically less central than male-authored articles in network positions. This suggests that female-authored articles tend to gain less attention in academia. For these reasons, I hypothesize that (1) female-authored articles will be received with more skepticism. I also theorize the relationship between novelty and the extent of skepticism. Because novel works are likely to be atypical, audiences will be more skeptical of novel offerings. Therefore, (2) when scientists claim that their article is novel, the article will be received with more skepticism. Additionally, as female accomplishments are often received with more skepticism, the effects will be stronger for female-authored articles than for male-authored articles. In other words, (3) the positive effect of claims of novelty on skepticism will be stronger for female-authored articles than for male-authored articles.

METHOD I use citation context data (sentences surrounding in-text citations) in Microsoft Academic Graph (MAG), an open-access database that covers bibliometric data of 225 million publications. I quantify the extent of skepticism in the reception of scientific papers and analyze the gender differences. In doing so, I rely on a text-based measure of uncertainty cues in academic writing developed by Chen et al. (2018). They manually compiled 61 uncertainty cues from academic writings and expanded this set of vocabulary into 2200 uncertainty cues using the word embedding approach. I count the number of uncertainty cues in the citation context to measure the extent of skepticism and then take a mean across the citation context within a cited paper. I construct a sample of covid-19 relevant articles. Articles should be published between December 2019 – December 2021. Articles have relevant keywords in their titles like covid or coronavirus. About 10% of papers are covered by citation context data. The final sample is 9,259 covid-19 papers. The main independent variable of this study is the gender of the author. Gender is coded by using the first names of authors. I rely on Lariviere et al. (2013) and use their gender dictionary. Another independent variable is claims of novelty. I use a novelty dictionary I developed with my collaborators in Leahey et al. (2022). It consists of 16 words that are synonyms of new and novel. This dictionary is applied to 9,259 abstracts of covid-19 papers. I control for a count of uncertainty cues used by authors because it can affect the extent of skepticism by the audience. Other control variables are the number of authors, months since 2019 December, citation count of an article, citation count of a journal, citation count of a first author, paper count of a first author, citation count of the last author, and paper count of the last author. I also generated a field variable. Life sciences, natural sciences, engineering, social sciences, and humanities. Finally, my method of analysis is Poisson regression because my dependent variable is a count variable and approximately follows the Poisson distribution.

RESULT The results suggest that female last-authored papers are received with more skepticism. Additionally, the number of claims of novelty is inversely related to the extent of skepticism in the reception of the paper. Results also indicate the moderating effect of gender and claims of novelty. As the number of claims of novelty increases, the extent of skepticism decreases, but this effect holds only for male last-authored papers and not for female last-authored papers. In other words, male last-authored articles claiming novelty enjoy the benefits of the lower extent of skepticism.

I conclude with the implications for gender inequality in academia. This study sheds light on the understudied mechanism of the gender gap in academia. In general, the results show that senior women academics who tend to be the last author in the authorship order are disadvantaged.

ABSTRACT. Background and Rationale

In the era of globalization, U.S. higher education institutions have become increasingly diversified with the growing presence of foreign-born faculty (NSF, 2022). As of 2021, foreign-born faculty account for around 33% of science and engineering faculty members employed in U.S. universities, increasing from 12% in 1973 (NCSES, 2021). Recruiting foreign-born faculty is also a crucial institutional strategy of U.S. universities to enhance their competitiveness in the global production of knowledge (Lawrence et al., 2014). 

As the scholarly community grows more diverse, the importance of foreign-born faculty in U.S. science enterprise has been recognized more than ever before. However, a paradox is emerging. Studies have consistently found foreign-born faculty to be more productive in publications, grants, and patents than their U.S.-born counterparts (Corley & Sabharwal, 2007; Kim et al., 2011; Mamiseishvili & Rosser, 2010). Yet, foreign-born faculty have also reported lower work satisfaction (Sabharwal, 2011a) and fewer career advancement opportunities (Kim et al., 2020; Sabharwal, 2011b) than U.S.-born faculty. These disparities require further examination: despite high productivity, what other factors diminish the work experience and satisfaction of foreign-born faculty?

Like many immigrants, foreign-born faculty face multiple challenges in integrating professionally and personally in a new cultural setting (Feeney et al., 2023; Kim et al., 2020). Specific to faculty experiences, how one balances home and work life, and the challenges in each setting, matters for a range of personal and professional experiences (Fox et al., 2011). As foreign-born faculty establish their professional lives in the U.S., they also establish their personal lives with the support of U.S. immigration policy, which often sponsors their families to come to the U.S. (Batalova, 2012). Like any working professional, academic scientists may experience work-family conflicts due to the interplay and interference between their work and family roles (Greenhaus & Beutell, 1985). While work-family conflict can negatively affect the work satisfaction of faculty members (Sarwar et al., 2021), particularly women (Carr et al., 1998), how this functions for foreign-born faculty has not been explored. Therefore, drawing upon demand-resource theory and social identity theory, this exploratory study addresses the following research questions:

1) How and why does work-family conflict function differently for foreign-born faculty and U.S.-born faculty?

2) Given the known gender disparities of work-family conflict in general, are foreign-born women more likely to experience work-family conflicts than their male counterparts? 

Data and Methods

This study uses data from the 2011 NETWISE II project, an NSF-funded national survey of academic scientists and engineers in four STEM fields: biology, biochemistry, mathematics, and civil engineering. The survey was designed to focus on female and underrepresented minorities in STEM fields and explore their careers, work environments, and professional networks. The stratified sample includes 9,925 academic faculty members from 521 academic institutions: all Carnegie-classified research-extensive and research-intensive universities, and selective teaching institutions (NETWISE, 2012). The survey had a total of 4,196 valid responses (40.4% response rate). The final sample used for this study includes 1,917 responses from 259 doctoral-serving institutions, of which 29% are foreign-born. 

As we are interested in the factors predicting the work-family conflict of foreign-born and U.S.-born faculty, this study uses logistic regression models with interaction terms of foreign-born status and key variables. The binary dependent variable indicates reported work-family conflict, which is generated from a scale developed by Netemeyer et al. (1996) (Cronbach’s alpha = .93). The two main variables of interest for this study are foreign-born status and gender. Moreover, based on the demand-resource theory, the key explanatory variables include those related to academics’ job demands (workload, work stress), job resources (supportive work climate, faculty perceptions of reward and tenure), and family demands (marital status, the presence of children). Additionally, several variables that may affect work-family conflict are controlled, including race/ethnicity, academic rank, discipline, and institutional types. 

Preliminary Findings

Our findings suggest that foreign-born identity interacts with work and family demands to influence work-family conflict among academic scientists. Controlling for other factors, being foreign-born predicts a higher likelihood of work-family conflict, though no significant difference is found between female and male foreign-born. Moreover, work demands are more likely to cause work-family conflict issues for foreign-born faculty than U.S.-born faculty, but not for family demands. This can be attributed to the strong work commitment of foreign-born faculty as well as the different family culture between U.S.-born and foreign-born faculty (i.e., individualist culture vs. collectivist culture) (Sallee & Hart, 2015).

Surprisingly, although a supportive work climate is negatively related to work-family conflict, foreign-born faculty who work in a more supportive work climate are more likely to experience work-family conflict compared to their U.S.-born counterparts. How might we account for this pattern? A possible explanation is that foreign-born faculty, in this case, are more willing to stay in the workplace rather than spend more time at home and caring for family, thus leading to work-family conflict. Further, while faculty’s perception of rewards is not significantly linked to work-family conflict, foreign-born faculty who are more satisfied with rewards and tenure are less likely to experience work-family conflict, as compared with their U.S.-born counterparts. This suggests that the perception of fairness in the workplace matters for academics’ well-being, especially for foreign-born faculty who often have fewer career advancement opportunities.  

Significance

Work-family balance is known to matter for a range of work-related experiences and the overall well-being of academic scientists, including job dissatisfaction, burnout, and turnover intentions (Post et al., 2009; Sarwar et al., 2021). In particular, foreign-born faculty often face considerable challenges in relocating and adapting to what is often a very different social and workplace setting. This study makes contributions to the work-family literature by exploring how the relationship between work and family experiences figures into foreign-born experiences.

Overall, the study sheds light on why foreign-born faculty are more productive but less satisfied with work. Their high productivity may come at the expense of their family, which would lead to work-family conflict and in turn affect their work satisfaction. Further, the study provides implications for U.S. higher education institutions to manage diversity in academic science. It is not enough to simply create a representative workforce. More importantly, university leaders and policymakers should better understand the unique needs of faculty members with different identities and backgrounds, creating an inclusive work environment for improving the support and retention of these scientists.  

Acknowledgments

Data used for this paper come from the National Science Funded Project: “Netwise II: Empirical Research: Breaking through the Reputational Ceiling: Professional Networks as a Determinant of Advancement, Mobility, and Career Outcomes for Women and Minorities in STEM,” a project funded by the National Science Foundation (grant no. DRL-0910191; co-principal investigators: Julia Melkers, Eric Welch, and Monica Gaughan).

References:

Batalova, J. (2012). Senior Immigrants in the United States in 2010. Migration Policy Institute. https://www.migrationpolicy.org/article/senior-immigrants-united-states-2010

Carr, P. L., Ash, A. S., Friedman, R. H., Scaramucci, A., Barnett, R. C., Szalacha, L., Palepu, A., & Moskowitz, M. A. (1998). Relation of family responsibilities and gender to the productivity and career satisfaction of medical faculty. Annals of Internal Medicine, 129, 532–538.

Corley, E. A., & Sabharwal, M. (2007). Foreign-born academic scientists and engineers: Producing more and getting less than their U.S.-born peers? Research in Higher Education, 48, 909–940.

Fox, M. F., Fonseca, C., & Bao, J. (2011). Work and family conflict in academic science: Patterns and predictors among women and men in research universities. Social Studies of Science, 41(5), 715–735.

Feeney, M. K., Jung, H., Johnson, T. P., & Welch, E. W. (2023). U.S. Visa and Immigration Policy Challenges: Explanations for Faculty Perceptions and Intent to Leave. Research in Higher Education. https://doi.org/10.1007/s11162-023-09731-0

GreenhaU.S., J. H., & Beutell, N. J. (1985). Sources of conflict between work and family roles. Academy of Management Review, 10(1), 76–88.

Kim, D., Wolf-Wendel, L., & Twombly, S. (2011). International faculty: Experiences of academic life and productivity in U.S. universities. The Journal of Higher Education, 82, 720–747.

Kim, D., Twombly, S. B., Wolf-Wendel, L., & Belin, A. A. (2020). Understanding Career Mobility of Professors: Does Foreign-Born Status Matter? Innovative Higher Education, 45(6), 471–488.

Lawrence, J. H., Celis, S., Kim, H. S., Lipson, S. K., & Tong, X. (2014). To stay or not to stay: Retention of Asian international faculty in STEM fields. Higher Education, 67, 511–531.

Marvasti, A. (2005). U.S. academic institutions and perceived effectiveness of foreign-born faculty. Journal of Economic Issues, 39,151–176.

Mamiseishvili, K., & Rosser, V. J. (2010). International and citizen faculty in the United States: An examination of their productivity at research universities. Research in Higher Education, 51, 88–107.

Netemeyer, R., Boles, J. & McMurrian, R. (1996). Development and validation of work–family conflict and family–work conflict scales. Journal of Applied Psychology, 81, 400–410. 

NETWISE. (2012). Netwise Project Overview. Georgia Institute of Technology. https://netwise.gatech.edu/netwise-ii-empirical-research/

National Center for Science and Engineering Statistics (2021). Survey of Doctorate Recipients. https://ncses.nsf.gov/pubs/nsf23319 

National Science Foundation (NSF). (2022). International S&E Higher Education. https://ncses.nsf.gov/pubs/nsb20223/international-s-e-higher-education

Post, C., DiTomaso, N., Farris, G. & Cordero, R. (2009) Work–family conflict and turnover intentions among scientists and engineers working in R&D. Journal of Business and Psychology, 24, 19–32.

Sabharwal, M. (2011a). Job satisfaction patterns of scientists and engineers by status. of birth. Research Policy, 40, 853–863. Sabharwal, M. (2011b). High-skilled immigrants: How satisfied are foreign-born scientists and engineers employed at American universities? Review of Public Personnel Administration, 31(2), 143–170.

Sallee, M., & Hart, J. (2015). Cultural navigators: International faculty fathers in the U.S. research university. Journal of Diversity in Higher Education, 8(3), 192-211. 

Sarwar, F., Panatik, S. A., Sukor, M. S. M., & Rusbadrol, N. (2021). A Job Demand–Resource Model of Satisfaction With Work–Family Balance Among Academic Faculty: Mediating Roles of Psychological Capital, Work-to-Family Conflict, and Enrichment. SAGE Open, 11(2).

ABSTRACT. Background & rationale

Over the last four decades, the city of Shenzhen in southeastern China has enjoyed exceptional success as the country’s first special economic zone (SEZ). Despite humble beginnings, Shenzhen has emerged as the locus of China’s technological revolution, evolving a dense and idiosyncratic innovation ecosystem founded upon export-led manufacturing, labor and capital flows, industrial policy, and a distinct entrepreneurial spirit. However, while more recent literature has remained largely positive on Shenzhen’s prospects in the near term, the last several years have seen sweeping socioeconomic, political, and cultural change threaten ambitions for the city’s innovation-centered development. Within these new contemporary contexts, Shenzhen’s historical model of growth has become increasingly unsustainable. As such, on March 11th, 2021, China’s National People’s Congress endorsed the 14th Five-Year Plan of the Chinese Communist Party (CCP). The plan, which is to guide development policy for the period spanning 2021 to 2025, represents an important economic transition and a remarkable departure from the traditional instruments of Chinese development since the country first began its economic reforms. Rather than the investment-led, hyper-globalized model of growth upon which Shenzhen has relied and the country itself has staked its developmental legitimacy, the 14th Five-Year Plan paints a new vision of China’s near future: one of self-sufficient supply chains, consumption-led growth, and (perhaps most sweepingly) technological and innovative preeminence.

Therefore, by virtue of its central place with regard to China’s technology and innovation, Shenzhen will be embroiled necessarily in the sweep of this most recent Five-Year Plan. In the forty odd years since paramount leader Deng Xiaoping designated the small urban center the site of one of China’s most consequential market-oriented reforms, Shenzhen has grown beyond all imagination—indeed, in the decades after the SEZ’s establishment, even the CCP’s wildest conjectures failed to estimate the city’s meteoric ascent. Today Shenzhen is the third-largest city in China, home to some 17 million residents and counting, and is the professed nucleus of the country’s domestically cultivated technology sector. Global heavyweights such as Huawei, Tencent, and DJI headquarter there, to say nothing of other major conglomerates like Ping An Insurance and SF Express (both of which, incidentally, are looking to rebrand as technology companies). The streets bustle with entrepreneurial life, and scores of ambitious professionals flock from across the country and the wider world to make their fortune. Since its nascent days, Shenzhen has doggedly stood at the forefront of China’s reform and opening-up, and its urban character has been shaped by decades of immigration, Party taglines, and a pervasive sense of economic opportunity. Key to understanding China’s technological, innovation-driven ambition on the world stage, then, is understanding the city of Shenzhen: its history, its economy, and how the city’s contemporary contexts and the variegated trends that inform them lay the groundwork for the next stage in Shenzhen’s ongoing evolution.

Methodologies

The two primary methodologies utilized for this research were literature review and qualitative, semi-structured interviews. The first stage of the study consisted of a detailed assessment of existing published work, noting broader trends in the archival research, and developing a conceptual framework with which to analyze Shenzhen’s innovation ecosystem. This assessment was done to provide a holistic understanding of the city’s history and political economy, and helped ascertain Shenzhen’s position within the wider Chinese and international technology industries. However, to obtain a more granular view of Shenzhen and its people, and, moreover, to identify the cultural and societal factors behind the city’s success, the second stage of the study included semi-structured, 60- to 90-minute-long interviews with academics, experts, and industry professionals. These interviews furnished the contextual background upon which we interpreted recent events in Shenzhen, and were instrumental in characterizing the sociocultural conditions of city life. Both methodologies helped mitigate the lack of available or trustworthy data over this most recent period of Shenzhen’s development, and are commonly utilized for research of this nature.

Results

Our findings indicate that—contrary to assumptions of smooth, steady development over the next several years—innovation in Shenzhen will enter a “tightrope era” of pronounced uncertainty as the city seeks to transition from one model of growth to another. Virtually every factor of innovation that we identified as crucial to Shenzhen’s success will stall or has already begun to stall over the next several years. Shenzhen’s previously exceptional demographics will be haunted by low birth rates, flagging migration, and soaring housing prices and cost of living. Rising costs in its manufacturing sector have made Shenzhen less cost-competitive for firms who increasingly look outside Shenzhen or even China for their manufacturing needs. The city’s proximity to Hong Kong has become a liability as a harsh government response to protests there roiled investor confidence in what was once a sure thing, and even the distinctive shanzhai networks which built Shenzhen’s early technology industry have become less relevant as large corporations and reduced market competition become the norm in the city today.

The political and cultural indicators have fared no better. The progressive policies which had historically fueled the city’s entrepreneurial culture are no more as the government under Xi Jinping has demonstrated a willingness to crack down upon the technology sector, empowering regulatory authorities and intertwining CCP cadres ever closer with designated private enterprises. The cultural backlash is apparent as well, as public dissatisfaction with lifestyles like the 996 working hour system prevalent throughout Shenzhen bubbles to the surface: in the last two years, both the tang ping and bai lan movements have proliferated online, spurred on by poor labor conditions, long hours, and the significant societal pressure to overwork in China.

Even the international business environment has become far more hostile. The European Union labeled China a “systemic rival” and froze an investment deal in 2021. India has banned hundreds of Chinese apps from the country’s digital infrastructure. In 2018, the United States ignited a trade war with China during which several of Shenzhen’s major corporations were caught squarely in the crossfire, and the Biden administration’s 2022 ban on semiconductor exports has signaled no end to the conflict in sight. All of this is occurring as the government implements draconian “zero-COVID” measures to address the ongoing pandemic, and as the country faces a sharp downturn in its property sector (triggered by the default of the China Evergrande Group, a company that is headquartered—coincidentally—in Shenzhen).

Significance

Shenzhen’s symbolic, entrepreneurial, and innovative importance to China is difficult to estimate. It stands as a testament to the country’s enormous progress and its towering ambition, and the city’s macroeconomic trajectory serves as a guidepost toward understanding trends prevalent throughout all of China. As one of the most prominent engines of Chinese technology and innovation, how it weathers what we have coined “the tightrope era” will be indicative of how the entire country fares in the near future, and informs others countries that have since adopted China’s SEZ model of development of how they should or should not implement policy. It remains to be seen whether Shenzhen can navigate the myriad of challenges it faces, or if it is destined to fail, but the city’s future will have widespread implications for innovation, technology, and urban development, regardless of result.

ABSTRACT. Background and Rationale In the mid-2010s, companies producing autonomous vehicles (AVs) were projecting the widespread deployment of this technology to happen very soon, which did not occur. Current company projections focus on narrow AV deployments in limited areas and circumstances. As companies have missed their deployment goals and commercialization becomes more distant, enthusiasm for AVs has waned. Some companies are narrowing their focus to lower levels of automation, and none have commercialized high automation at a large scale. Investor enthusiasm has cooled, and several companies have gone out of business. The industry recently realized that this deployment would not happen as fast as projected.

The public, and in some cases, policymakers, have yet to recognize the lack of progress in AV development after such widely reported, bullish projections last decade. The lack of technological advancement can help clarify the federal, state, and local policymaking process. While many companies have identified the lack of policy progress as a barrier to development, the policymakers at NHTSA have argued that policy will not make sense until development reaches a practical level. Georgia Tech currently has a concurrent research project assessing state policies and reports. That research can help bridge the gap on how policy has responded to the stagnation witnessed in these projections. The research indicates that states produced legislation and reports with the early AV technology hype cycle. An analysis of industry projections and their evolution can give insight into the AV technology hype cycle and guide policymakers faced with a similar hype cycle in the future.

Methods While company projections are narrow today, Georgia Tech and the Center for Automotive Research collected all initial forecasts, including projected timelines and their fulfillment. The data consists of estimates from auto manufacturers, tech companies, parts suppliers, and startups. These projections can be aggregated and cleaned to illustrate the technology and innovation hype cycle. Additionally, the underlying data can provide a deeper understanding of trends throughout the industry to which governments seek to respond. The research team at Georgia Tech will utilize the data on these early projections for quantitative analysis of how projections have scaled back and which sectors missed their forecasts. The data will likely be split into two parts: the initial stage of the AV hype cycle (2014-2019) and the most recent stage of development (2020-2022). The comparative data analysis of the two phases will also enhance the understanding of how AV deployment has changed and the proper expectations for the future. Lastly, the data can be utilized to understand which sectors have given up on the deployment of automation.

Anticipated Results The current data shows that all sectors missed most of the projections early in the hype cycle. However, it is expected that auto manufacturers, startups, and tech companies missed their forecasts more than parts suppliers. There is also an assumption that results will show projections have begun to decrease in time and milestones. It is unclear which sector has the lowest success rate in its commercial viability. Still, it is likely that the companies like Waymo and Zoox, funded by big tech companies like Google and Amazon, have the most robust success rate. The results can help inform policymakers on the imminence of AVs and how to adjust expectations to industry projections properly. A more practical understanding of AV development will allow policies to occur at a rate more in line with their development.

ABSTRACT. Background and rationale

Determining patent eligibility for artificial intelligence (AI) inventions has become increasingly difficult to predict since the 2014 Alice Corporation v. CLS Bank International United States Supreme Court case. The case brought about concerns about abstract ideas being unacceptable for patent protection, yet foundational to computer software and technological inventions. At the same time, there has been both growing public debate and increased scholarship and policy dialogue on public and societal concerns associated with the rise of AI and its applications. This poster will present the results and insights gained from a systematic literature review on the societal dimensions of AI invention and innovation, focusing on the public values expressed in AI patent documents. We build on public value frameworks to conceptualize public values expressed in patents as statements regarding the potential social objectives and benefits of an invention, beyond the private value to patent holders. Although the United States Patent and Trademarking Office (USPTO) has no formal requirements for expressions of public values in patent documents, patent applications often describe the broader objectives or problems that the invention aims to address. Such public value statements inform the context for understanding the potential utility of a patent.

The United States and other countries spearheading technological innovation have begun widely developing AI, even promoting its proliferation. This is evidenced in the United States by growth of R&D investment in AI research and in the rapid expansion in recent years in AI patent applications and grants. Generally, in return for the private protection of intellectual property rights, patented inventions (and the innovations that flow from) are expected to contribute to R&D knowledge and, broadly, to the construction of social and public value. Yet, this is not straightforward, and there is much discussion about the heightened dilemmas that AI raises. For AI innovations, there have been concerns about the reliability of AI, the potential for bias across machine learning algorithms, and the loss of human agency and control. On the other hand, AI developers frequently promise that their AI inventions will address not only data security and bias concerns but also health, environmental, sustainability and other major societal challenges. In our literature review, we aim to analyze the emergence of literature that addresses the dilemmas presented by AI, considering how public value impacts associated with AI are framed and discussed, and how public values might be mapped in patent documents.

Methods

The methods used for the analysis that will be presented in the poster begin with formulating strategies for literature searches in the fields of public value theory, the public value of science, technology and innovation, the societal and public value implications of AI, and the societal and public value implications and assessment of patents. An iterative and structured search for relevant documents will be conducted in key scholarly databases (i.e., Scopus, Web of Knowledge). It is recognized that these topics represent potentially large swathes of literature. Bibliometric text mining tools (such as VantagePoint) will be used to organize, describe, and synthesize these literature landscapes. This approach will assist in identifying central and influential works and discerning diverse perspectives and schools of thought. Annotations will be developed from reading key works and the thematic literature review will highlight the recent developments in AI patent applications. A report will be produced that summarizes the findings of the literature review and contributes to the conversation about public value implications of AI and the associated concerns. From this, a visually communicative poster will be developed that details the method, the landscape of ideas, and the findings and implications. This research stands on it own as a deliverable poster but will also feed into an international project (undertaken by researchers from Georgia Tech, the University of Manchester, and the SKEMA Business School) to determine how text used in patent applications for AI technologies references public and private values. The broader project is using a machine learning method to text mine large volumes of AI patents. The project involves conceptualizing the public value frameworks that are used in patent applications and analyzing the anticipated uses, from a public values perspective, of AI inventions. The project will contribute to the existing literature surrounding AI and public values as described in the curated literature review portion of this research.

Anticipated results and significance

The research for this poster will identify, bring together, and evaluate aspects of the literature concerning patent protection, AI, and public values. After completing a thematic narrative literature review and considering how public value terminology is presented in the literature, the study will inform the algorithm training used for text mining expressions of public values in AI patent documents. This research will contribute to frameworks that evaluate ethical practices and morality for emerging technological innovations. This research will help to analyze and describe the textual evidence from patent applications and provide an opportunity to further explore anticipated societal implications for inventions and the effects of AI patents on human life. This presents an opportunity to explore positive and negative externalities associated with the increase of AI patents for healthcare use, digitalization, and other societal domains and to develop new concepts and methods that can contribute to evaluating the harm reduction, equity, and transparency aspects associated with AI patenting.

ABSTRACT. Background How will state-level administrators respond to the arrival of emerging technologies, such as autonomous vehicles (AV)? With the expectancy of new technology, state actors are likely to face Collingridge’s dilemma (Collingdridge, D. 1980): at the early stage of technological development, it is difficult to forecast the social impacts of the technology, while at the later stage after the technology matured, social control is costly. In other words, state actors, who are responsible for public roads, must learn and react in a situation of great uncertainty about how the emerging AV technology will be developed and how it will affect society. In the study, we explore strategies in which state actors respond to uncertainty, learn the technology and its social possible impacts, and adapt to innovation.

Method & Data In this study, we collect two classes of policy documents 1) state laws and executive orders and 2) the reports about technical and socioeconomic aspects of autonomous vehicle technology, which are commissioned by states, both from 2012 to 2021. We use content analysis and topic modeling to analyze the contents of these policy documents. The core text of each document was extracted for modeling, and the text was preprocessed using snowball stemming. We found a peak at about 30-35 topics; 35 topics were specified in the final model.

Anticipated Results In our analysis, we take two approaches: first, we examine how states legislate for the arrival of autonomous vehicles, and second, in state reports, we explore topics states share and specific patterns in the content. We find a remarkable similarity of texts, especially in laws across states, suggesting common sources outside the states and/or institutional isomorphism. Also, we find the extent to which states track actions in other states in the reports, which might be a process leading to isomorphism. By combining two approaches, this study will provide an overview of strategies that various states are taking in anticipation of autonomous vehicles on the road.

Reference Collingridge, D. (1980). The social control of technology. St. Martin's Press

ABSTRACT. Background and rationale Academic publication is central to higher education as it serves various purposes that affect and determine the life and career of academics. Commonly known as an output of knowledge production, academic publications also play multiple administrative roles in determining individual promotion, funding, and institutional performance (Ramsden, 1994). In addition to its centrality in determining individual academics' accountability, earlier studies have also shown that, to varying degrees, academic publication contributes to knowledge governance through evaluative systems (Geuna & Martin, 2003) and rankings (Marginson, 2014). With such multiple roles, various policies and programs have been designed and established to improve publication performance, such as collaborative research initiatives (Lee & Bozeman, 2005), academic writing interventions (McGrail, Rickard and Jones, 2006), and publication incentives (Andersen & Pallesen, 2008; Xu, 2019). However, it remains unclear how massive incorporations of academic publications as output and administrative means in various policies and programs improve academic research governance, especially with the increasing influences of public sector narratives. Against this background, this study argues that consistent narratives are essential to facilitate the implementation of government policies and programs. Accordingly, this study uses regulatory mapping to uncover official narratives underpinning various laws and regulations in constructing and advancing our understanding of academic publications' roles in academic research governance. More specifically, the study aims to address two objectives: first, to identify and map legal provisions governing academic research and publication. Second, to assess the cohesion of narratives and values of corresponding legal provisions across regulatory mechanisms in understanding implementation approaches. In doing so, the study will assess Indonesia's regulatory mechanisms on higher education and research enacted between 1999 and 2019. Methods This study reviewed 30 authoritative documents governing Indonesia's higher education and research enacted between 1999 and 2019 to generate insights on how academic publication governs and, conversely, being utilized in research governance. Given the complexities and densities of Indonesia's research governance, obtaining the authoritative documents involved accessing various ministries' and agencies' databases. Therefore, the mixture of authoritative documents used in this study arguably represents the broad interests of different ministries and agencies on academic publication while also showcasing the propagation of academic publication in various government policies and programs. The review began with identifying various provisions related to research and academic publication. Keywords used in this study include academic publication, journal articles, scientific publication, and more ambiguous references such as 'research output' to identify relevant regulatory provisions. In making sense of extracted information, the study employed 'horizontal review' (Molinuevo & Saez, 2014) and 'vertical' classification of the authoritative documents. The horizontal review was used to uncover and identify different dimensions and perspectives of academic publication. Meanwhile, 'vertical' categorization was essential to analyze consistency and coherence between narratives, objectives, and provisions between 'high-level' policy and 'lower-level' technical documents (Howlett, 2009). Such an integrated approach provided this study with a comprehensive tool to unpack the dynamics of governance and implementation. Preliminary result This study identified significant proliferations of New Public Management (NPM) logic and values in various provisions governing research and academic publication, confirming public sector narratives' influences on higher education (Ferlie et al., 2006). It is also found that the Government has introduced numerous programs to enable and accelerate academic publications, such as publication incentives. However, as NPM values, like efficiency and effectiveness, arguably do not resonate with higher education traditions and values, this study discovered profound applications of control and audit in various programs, implying a strong sense of distrust. While some might justify such restrictive approaches with accountability reasonings, distrusts are deeply embedded in the administrative and bureaucratic obligations and practices associated with funding and performance assessment, thus, potentially reducing or crowd-out motivations (Frey, 1997) in conducting research and producing academic publications. Research funding Indonesia's research funding has been incrementally increasing controls on the applicants and quality demanded using track records and required outputs that are exercised through frequent revisions of funding guidelines. While applicants' track records and research outputs were arbitrarily defined for years, Guidelines Edition XII, published in 2019, requires the applicant's publication track record as a condition to apply for funding and specifies the type of publication criteria. For some schemes, these arrangements go into more detail, such as the obligation to state Hirsch-index or to publish at top quartile journals indexed by Web of Sciences or Scopus. With such increasing specificities, the funding arguably creates higher barriers to applying, especially among academics whose work is not publication oriented. Lecturers' performance assessment system The performance assessment system audits academic performance by aggregately measuring the outputs of research, teaching, and community services. However, the system sends mixed signals about academic publications. Although the system substantially rewards academic publications, the pre-2013 system required a review process to determine the 'real value' of the academic publication. While such a provisional mechanism no longer exists in the current system, the bureaucratic review of academic publications and other research outputs to assess academics' performance continues. As a result, the system arguably discriminates against academic research with excessive audit and gatekeeping on academic publications. Significance This contribution of this study is trifold. First, it aims to advance behavioural public administration research, which remains underexplored, especially in the higher education and research policy contexts. Empirically, the study also serves as an empirical contribution to the study of Indonesia's higher education, which is rarely discussed academically despite being one of the largest and most dynamic higher education systems. Lastly, the regulatory mapping performed in this study provides a foundation for further policy developments.

ABSTRACT. Background The relevance of government interventions and the catching up of emerging countries has been the subject of a significant amount of study that has been published in academic circles. However, the majority of articles concentrate on making up ground on a national level. Studies on sectoral catching up, particularly those dealing with distinct industries in different nations at various stages of development, on the other hand, are fairly sparse. Furthermore, research has focused on industrial policies, whereas just a limited number of authors have specifically addressed STI policies. With the presumption that the type of government policies strongly varies based on sectoral characteristics and learning objectives during each phase of catching up, the purpose of this study is to examine the applicability of STI policies that account for variation in stage of development and different sectors. The textile and apparel and the pharmaceutical industry are selected for investigate because of the vast differences between their respective knowledge bases: the former is supplier-dominated and the latter is science-based. The research attempts to answer the following research questions. 1. What are STI policy instruments have been adopted in each stage of sectoral catch-up in textile and apparel and pharmaceutical industry? 2. How the adoption of STI policies varied between the catch-up process of different knowledge sectors: textile and apparel and pharmaceutical industry?

Methods This study conducts a systematically review of academic literature on the STI policies for catch-up of pharmaceuticals and textile and apparel considering two primary criteria: i) papers including a discussion of the government's STI policies for sector catch-up; and ii) the records need to have information indicating the stage of catch-up at which certain measures were implemented. A search of Scopus and Google scholar databases for articles published up to May and 22 to the textile and apparel industry. By utilizing the three stages catching up model of learning objectives, i.e. operational skills & process technology, design technology for existing products, and new product development technology, those articles were examined to explores relevant STI policies that facilitate catching-up process at each phase of development for textile and apparel and pharmaceutical industry.

Findings The results show that a variety of STI policy instruments have been utilized at each step of the catching-up process in both the textile and apparel industries as well as the pharmaceutical industry. Moreover, although certain policy instruments, such as strategies for developing a public education system or policies that promote quality and standards, are relevant in each phase of the catching-up process, the goals of these policies are fundamentally distinct from one another and depend on different sectors as well as specific learning objectives associated with the stages of development. When compared the textile and apparel and the pharmaceutical industry, despite the fact that the two sectors have distinct sectoral characteristics, catching up strategies for both follow a fairly similar pattern. In the early phase of catching up, education system development was a top priority and contributed significantly to the growth of industries in latecomer nations. During this time period, R&D funding has been limited. Only when nations enter a later phase of catching up do their governments starting to explore more ways to encourage R&D investment as domestic firms have accumulated basic and intermediate R&D capabilities. These strategies may include competitive financing programs, tax incentives, venture capital, and policies that facilitate access to capital for R&D activities. In addition, in last phases of development, policies promoting linkages within the sectors become a crucial tool for fostering the development of sophisticated R&D skills among domestic players. Still, there are differences in which the textile and apparel industry and the pharmaceutical industry diverge from one another in terms of STI policy implementation. For example, intellectual property law plays a vital part in the growth of the pharmaceutical business, but it has played a lesser role in the other sector. On the other hand, one distinguishing feature of nations that have successfully made the transition of their textile and apparel industry into the later phase of catching up is government’s efforts to increase awareness about the value of new products and scientific and technological innovation as well as elevating design as a cultural and artistic symbol.

Contributions This research review research on government's initiatives to develop the textile and garment sectors, as well as the pharmaceutical industry of latecomer nations, and discusses the lessons that may be learned from those studies. The objective of this study is to investigate implementation of STI policies for catch-up purposes in a manner that takes into account differences in both the phase of development and sectors. The findings will shed light on when and how and when the government should employ policies on STI to facilitate the growth of the textile and apparel and pharmaceutical industry.

ABSTRACT. This paper is submitted to the panel entitled "The Dynamics of Science and Technology Governance in Emerging Sectors" (submission 6247)

This study explores how the innovation ecosystem of emerging technology industries are formed and developed and how does the evolution of the innovation ecosystem in the course of technological progress depend on institutional factors? Taiwan has long experience in the ICT field and the technology of wheelchair assistive device manufacturing has led to the rapid development of smart medical devices by introducing smart technology. In this empirical study, 7 elite group interviews and 10 multi-criteria decision-making questionnaires are conducted an in-depth study to understand the formation of the medical device industry innovation ecosystem. In Taiwan, however, there is no significant growth trend, though the medical device firms adopted intelligent technologies in their product developments in recent years. we have found the medical device industries in Taiwan have gradually developed over the past 20 years, and has established its own international partners, mostly from the U.S. and Japan. Therefore, we hope that this joint research will open up opportunities for international research collaboration of in the medical device sector.

ABSTRACT. Background

Policy makers and scholars are increasingly embracing the idea of using science, technology, and innovation (STI) policy to tackle grand challenges (Castellacci, 2022; Diercks, Larsen, & Steward, 2019; Edler & Fagerberg, 2017; Fagerberg, 2018; Hicks, 2016; Kuhlmann & Rip, 2018; Mazzucato, 2018; Schot & Steinmueller, 2018). Such ambitious initiatives often come under the label transformative innovation policy (Diercks et al., 2019). Design, implementation, and evaluation of transformative innovation policies are more challenging than traditional innovation policies because they span across multiple policy subsystems, levels of government, and industrial sectors (Flanagan, Uyarra, & Laranja, 2011; Haddad, Nakic, Bergek, & Hellsmark, 2022; Howlett & Del Rio, 2015). The literature has largely taken a rationalistic approach (Geels & Schot, 2007; Mazzucato, 2018; Nunez-Jimenez, Knoeri, Hoppmann, & Hoffmann, 2022) to studying transformative innovation policies, and political failure has been suggested to be a ‘missing piece’ in the innovation policy analysis scholarship (Bonvillian, 2011; Karna, Karlsson, Engberg, & Svensson, 2022; Marburger III, 2011).

Aim

The purpose of this study is to introduce the policy regime approach to analyzing governance challenges in transformative innovation policies and demonstrate its utility to shed light on political dynamics of transformative innovation policies. It operationalizes the original policy regime approach proposed by Jochim and May (2010) into a two-by-two policy regime matrix with policy legitimacy and policy durability axes so that it can be used as a heuristic to assess how favorable the political environment is for a transformative innovation policy.

Method

This is a conceptual study based on a literature review of studies related to transformative innovation policies. It primarily focuses on the mission-oriented innovation policy (MOIP) framework (Mazzucato, 2018), the multi-level perspective (MLP) framework (Geels & Schot, 2007; Kanger, Sovacool, & Noorkoiv, 2020), and the policy mix scholarship (Flanagan et al., 2011). Then it attempts to apply the concept of a policy regime (Jochim & May, 2010) in the context of transformative innovation policy.

Results

Innovation scholars have attempted to understand transformative innovation policies from largely two directions: a Mission-Oriented Innovation Policy (MOIP) framework and a Multi-Level Perspective (MLP) framework (Haddad et al., 2022). The MOIP framework suggests a list of prescriptive advice on desirable features of a transformative innovation policy (Mazzucato, 2018); the MLP framework contributes to a holistic description of the process of socio-technical change (Geels & Schot, 2007) and identification of potential intervention points (Borras & Edler, 2020; Edmondson, Kern, & Rogge, 2019; Kanger et al., 2020).

However, both frameworks remain short to explaining how the state can be successful in mobilizing relevant actors and their resources to transform the socio-technical system in the intended direction. While the MLP framework provides a comprehensive lens to understand interactions across multiple levels, it does not well explain when meaningful interactions are likely to occur. The prescriptive advice provided by the MOIP framework insists a normative role for the state that could potentially be used to explain implementation failure; however, the framework does not directly address how to respond to challenges that arise from boundary spanning activities.

In response, it has been argued that STI policy in general has not given enough attention to political design (Bonvillian, 2011; Marburger III, 2011). The policy mix scholarship has responded to this challenge by taking into account interactions across levels of government (Borras & Serger, 2022; Magro, Navarro, & Zabala-Iturriagagoitia, 2014; Magro & Wilson, 2019; Uyarra, Shapira, & Harding, 2016), documenting all available policy instruments and how they interact with each other (Borras & Edquist, 2013; Edmondson et al., 2019; Kern, Rogge, & Howlett, 2019; Magro & Wilson, 2019; Meissner & Kergroach, 2021; Nunez-Jimenez et al., 2022; Rogge & Reichardt, 2016; Scordato, Klitkou, Tartiu, & Coenen, 2018), investigating the politics of tool choice (Acciai, 2021; Capano & Lippi, 2017), and proposing various strategies and roles for the state to accelerate socio-technical transitions (Borras & Edler, 2020; Hayter & Link, 2020; Kanger et al., 2020; Mowery, 2012; Nunez-Jimenez et al., 2022; Parks, 2022).

A policy regime approach sheds light to whether the current environment is supportive of a policy of interest and can potentially inform what strategies or alternatives are feasible. A policy regime is defined as “a governing arrangement that spans multiple subsystems and fosters integrative policies” (Jochim & May, 2010, p. 307). A policy regime differs from a socio-technical regime in the MLP in that it is much more expansive to the extent that it could be viewed as network of socio-technical regimes. The motivation behind a policy regime is to go beyond policy subsystems as the unit of analysis to make sense of comprehensive governmental efforts that are implemented through boundary-spanning governance arrangements (Carter & May, 2020; Jochim & May, 2010). Due to this, the locus of a policy regime is a wicked problem, whereas the locus of a policy subsystem is a public policy (May & Jochim, 2013). This makes a policy regime a good fit to analyze the governance of transformative innovation policies.

A policy regime consists of ideas, institutional arrangements, and interests. Ideas are shared commitments around a policy goal. Institutional arrangements refer to the structures of authority, attention, information, and organizational relationships. Interests are the actors that provide support or opposition. The outcome resulting from the dynamics among ideas, institutions, and interests is described across three dimensions: policy legitimacy, policy coherence, and policy durability (Jochim & May, 2010).

A shortcoming to this approach is that the relationship between components and outcomes in a policy regime is not clear. In response, this study operationalizes the concept into a policy regime matrix. The three components of a policy regime are reduced into ideas and institutions. There is high level of policy legitimacy when ideas are shared as opposed to being disjointed, and there is high level of policy durability when institutions are aligned versus fragmented. Policy coherence is considered to be high at the intersection of high policy legitimacy and high policy durability.

Among the three outcome dimensions, policy coherence and policy durability have been identified to be critical for success in the transformative innovation policy literature. Incoherence of policy mixes within government has been cited as a cause of policy failure that aim for sustainability transitions (Magro & Wilson, 2019; Uyarra et al., 2016). Also, strong commitment from the government (i.e., a high level of policy durability) that provides a clear direction and strong signal to the private sector is thought to be critical for MOIPs to succeed (Mazzucato, 2018).