ABSTRACT. Background and Rationale Team based project organization plays a critical role in the advancement of scientific knowledge, and the allocation of research work within teams affects the training of graduate students and postdoctoral researchers, which has implications for their ability to pursue and success in scientific careers. While a majority of new doctorate holders rate academia as their top choice for a post-PhD career (Woolston, 2019), an increasing number of recent graduates find the pathway inaccessible (Cyranoski et al., 2011; Doctorate Recipients from U.S. Universities: 2018, 2019; Schillebeeckx et al., 2013). This disconnect may be exacerbated by the division of labor on research teams where the supporting research roles that early career researchers occupy may not necessarily be valued during hiring and promotion decisions (Leahey et al., 2010; Milojević et al., 2018; Robinson-Garcia et al., 2020). While the devaluation of supporting research work is well-established, the extent to which scientists’ strategic decisions about their next career steps are affected by their ability to dedicate time to more prestigious research tasks is unknown. Do scientists allocate their time based on what they value most or what they feel will advance their career within in academe? We surveyed academic scientists in the fields of Biology and Biomedical research in the United States about their usual and desired roles on project teams, and the extent to which they see their current work as contributing to their desired career. The respondents’ perceptions of the roles that they have performed during their training period will shed light on the extent to which researcher expectations and career decisions are affected by their positioning on project teams and seniority. Understanding the distribution of labor on research teams can inform policy decisions that incentivize the types of work that will advance scientific innovation and provide opportunities for early career researchers.

Methods To prepare a list of potential survey takers, we identified a sample of peer-reviewed scientific papers in the Web of Science that were published in the period from 2018 through 2020 in Biology or Biomedical Research according to the paper disciplinary classification that was developed for the National Science Foundation. We focus on these fields due to ubiquity of team-based science in these areas and their discussion of the issues around research training in a team science environment (Alberts et al., 2014; Bourne, 2013). The final sampling frame contained 138,387 unique researchers with email addresses and affiliations in the United States. We emailed each author a link to participate in the survey and a request to forward the survey and message to collaborators who are currently or who had been graduate students or postdocs in the past two years. We collected 5,088 total responses to our survey, with 4,150 fully completed surveys. Our survey asked researchers at different career positions to rank the prominence of 14 different research-related tasks as defined by the CRediT taxonomy (Allen et al., 2019). After parsing the survey responses, we were able to calculate the proportion of researchers who rank each of the 14 CRediT tasks among their top three tasks. We clustered the roles into Lead, Direct Support, and Indirect support based on a previous classification (Xu et al., 2022), and performed a linear regression to identify which factors and project roles were associated with a higher degree of confidence for pursuing desired career, and which roles were associated with higher numbers of publications.

Results Most notably, we found that while publishing higher number of lead publications is weakly associated with greater a sense of preparedness for pursuing an academic career, there is virtually no correlation between publication counts and a sense of preparedness for pursuing a non-academic career. Nonetheless, the number of lead publications is very often a prerequisite for employment in university contexts. 85% of graduate students and postdocs report that that at least one of the top three roles they spend time performing is a lead role; in fact, the most common role for graduate students and postdocs, was “Writing the original draft” (22%), followed by the direct support roles, “Formal Analysis” (21%) and “Investigation” (18%). However, we found that the degree that early career researchers spend time performing lead roles was not associated with higher numbers of lead publications for early career authors. This counterintuitive negative result suggests that for early career researchers, spending more time on lead roles alone may not actually lead to a higher number of lead author papers. We argue that the reason for this may be that the particular lead author tasks that early career researchers perform were the most time-intensive task on a per-paper basis: Writing the original draft of the manuscript. On the other hand, as senior researchers perform lead roles, their most common lead roles were conceptualization and designing methodology, tasks that are less time-intensive on a per paper basis. For researchers targeting non-university settings, receiving credit for conceptualization is positively associated with their sense of preparedness for their intended careers.

Significance A common conception of doctoral training is that it is designed towards preparing for academic careers. The focus on scientific publications reinforces this conceptualization. Furthermore, establishing oneself as a lead author is essential for retention in academic careers (Robinson-Garcia et al., 2020). This work, however, demonstrates a misalignment between lead author labor and output. Furthermore, few doctoral students obtain academic positions. Whether training for these students is appropriate for their career needs is unclear, especially as trainees may have less autonomy to select the roles that most directly prepare them for their career. Senior researchers, policy makers, and academic institutions must take into account their students' future career trajectories (desired and real) in the distribution of labor.

References Alberts, B., Kirschner, M. W., Tilghman, S., & Varmus, H. (2014). Rescuing US biomedical research from its systemic flaws. Proceedings of the National Academy of Sciences, 111(16), 5773. https://doi.org/10.1073/pnas.1404402111 Allen, L., O’Connell, A., & Kiermer, V. (2019). How can we ensure visibility and diversity in research contributions? How the Contributor Role Taxonomy (CRediT) is helping the shift from authorship to contributorship. Learned Publishing, 32(1), 71–74. https://doi.org/10.1002/leap.1210 Bourne, H. R. (2013). A fair deal for PhD students and postdocs. ELife, 2, e01139. https://doi.org/10.7554/eLife.01139 Cyranoski, D., Gilbert, N., Ledford, H., Nayar, A., & Yahia, M. (2011). Education: The PhD factory. Nature, 472(7343), 276–279. https://doi.org/10.1038/472276a Doctorate Recipients from U.S. Universities: 2018 (No. 20–301; Special Report NSF). (2019). National Science Foundation, National Center for Science and Engineering Statistics. https://ncses.nsf.gov/pubs/nsf20301/ Leahey, E., Keith, B., & Crockett, J. (2010). Specialization and promotion in an academic discipline. Research in Social Stratification and Mobility, 28(2), 135–155. https://doi.org/10.1016/j.rssm.2009.12.001 Milojević, S., Radicchi, F., & Walsh, J. P. (2018). Changing demographics of scientific careers: The rise of the temporary workforce. Proceedings of the National Academy of Sciences, 115(50), 12616. https://doi.org/10.1073/pnas.1800478115 Robinson-Garcia, N., Costas, R., Sugimoto, C. R., Larivière, V., & Nane, G. F. (2020). Task specialization across research careers. ELife, 9, e60586. https://doi.org/10.7554/eLife.60586 Schillebeeckx, M., Maricque, B., & Lewis, C. (2013). The missing piece to changing the university culture. Nature Biotechnology, 31, 938. Woolston, C. (2019). PhDs: The tortuous truth. Nature, 575, 403–406. https://doi.org/10.1038/d41586-019-03459-7 Xu, F., Wu, L., & Evans, J. (2022). Flat teams drive scientific innovation. Proceedings of the National Academy of Sciences, 119(23), e2200927119. https://doi.org/10.1073/pnas.2200927119

ABSTRACT. Introduction This paper analyses the conditions and factors influencing non-university researchers' career transitions back into academia. In knowledge-intensive economies, research is performed in various institutional settings, in private and public organisations and covers a multitude of different purposes. Research in some key scientific research areas like artificial intelligence or big data is nowadays no longer driven by public research alone, but is intricately connected to public applied research and industrial R&D. Thus, research careers are increasingly forged in diverse knowledge production environments. However, previous scholarship on researcher mobility has focused predominantly on single-sector careers within universities. Inter-sectoral career transitions, in particular those of applied researchers from public research organisations (PROs) and industry into higher education, are still poorly understood. Given the variety of knowledge-intensive sectors where research careers are de facto built, this is a serious limitation.

Conceptual Approach In order to explain why some researchers may choose to transition back to universities, we draw on the notion of instutional proximity. Although all researchers, regardless of their domain of occupation, adhere to fundamental rules concerning the scientific method, in practice, their work is shaped by sector-specific institutional logics. Since career scripts associated with professional success differ across institutional domains, over the course of their career researchers develop distinct intellectual and social capital portfolios tailored to the respective institutional referents and optimised to enable career advancement within their sector. Opportunities and desirability of cross-domain transitions are determined by the possibility for researchers to advance their careers within the institutional referents of a different sector. Institutionally proximate sectors share formal and informal rules, goals, systems of meaning and a professional language. As proximity decreases, institutional referents begin to differ more strongly and the transaction cost of knowledge exchange and collaboration increases. Therefore, the higher the degree of proximity between domains, the more similarities there will be between expectations of successful research careers, the higher the synergies in the researchers' portfolios of skills, capacities and networks. Cross-sectoral transitions between institutionally more proximate domains will therefore have lower barriers. However, hiring decisions will also likely differ in specific situations, e.g. for early career hires vs senior hires since these types of researchers bring different kinds of value to universities.

Empirical Approach Germany offers a rich context for the study of cross-domain transitions due to the functional differentiation of its national research system. We focus on three pillars of the German system: universities, industrial applied research sector, and applied PROs. The latter are represented by Fraunhofer-Gesellschaft in our study. In academia, performance evaluation and, consequently, career advancement, depend in similar measures on scientific reputation, visibility within relevant peer community, and strong research metrics. Corporate researchers tend to work in an environment, which is functionally separated from academia and shaped by its own, dominant institutional reference system that is characterised by profit-orientation and intellectual property protection. Industrial researchers are less likely to publish research papers, apply for public funding or present their work at conferences due its proprietary nature. PROs are characterised by a degree of institutional hybridity. They are involved in producing scientific knowledge, but are also in contact with user environments. In PROs, institutional logics from various societal spheres co-exist and overlap, meaning that they may complement or contradict each other. Fraunhofer-Gesellschaft, the research organisation with 76 institutes in Germany, embodies key characteristics of an applied research PRO. Its mission is "to partner with companies to transfer original ideas into innovations" that benefit society and strengthen the economy.

Hypotheses Barriers to research career transitions into academia are high due to the idiosyncratic features of the academic career scripts. However, some parts of the academic sector look for high potentials that bring in complementary skills, which academics struggle to acquire. Therefore, in some cases cross-domain transitions into academia take place. We offer several hypotheses regarding the factors that might ingluence them. First, a researcher's publication activity is likely to matter as high-impact scientific publications are the cornerstone of scientific credibility: H1. Applied researchers who publish more and in more acclaimed journals are more likely to transition to universities. Second, there will likely be sectoral differences among transitioning researchers. Since institutional referents of applied PROs overlap with those of universities', we propose: H2a. The rates of transition of applied PRO researchers are higher than of applied industrial researchers. However, institutional proximity will be likely to have dynamic effect. Early career applied researchers should be attractive to hire for universities because of similarity in capacities. At senior career stages, capacities and networks of applied researchers become very different that those of academics regardless of the sector. Therefore: H2b. Differences in transition rates between applied PRO and industrial applied researchers decline with scientific age.

Method We use publication data from Elsevier Scopus ranging from 2010 to 2021 to identify instances of career transitions of 18,595 German authors who are affiliated either with industry or with Fraunhofer-Gesellschaft in the beginning of their career, but that later acquire university affiliations. We identify 927 individuals who have transitioned to universities. Based on this data, we create a panel dataset at the author-year level comprising a total of 30,430 observations. We split the data into a Fraunhofer sample and an industry sample. We define our dependent variable University Affiliation as a binary variable taking the value of 1 if a researcher is affiliated with a university at an author-year and 0 if no university affiliations are identified. The change from 0 to 1 measures acquired university affiliations or co-affiliations and therefore, we interpret them as instances of career transitions. We include the following independent variables in the analysis: Publication Stock; Mean Journal Impact Factor; Scientific Age. The following controls are included: co-publications between Industry researchers and university researchers; co-publications between Fraunhofer researchers and university researchers; affiliations with Max-Planck institutes and Helmholtz Centres; Female Gender. We estimate the models using probit regressions.

Results Around 8% of Fraunhofer researchers and around 3% of industry researchers transitioned to universities at some point in their careers. The three hypotheses are supported. Fraunhofer researchers transition to universities more frequently due to their greater institutional proximity and their competences presumably have a better fit with the academic system. Transitioning to universities is an easier - and hence more viable - career choice for them than for their peers in industry, especially during their early years. Later on, researchers moving to long-term contracts in Fraunhofer institutes are likely to begin to align more strongly with non-academic incentive systems and the number of transitions converges to those of industrial researchers.

Significance The lack of academic appointments of external researchers indicates missed opportunities, because recruitment of professionals from outside academia can offer tangible benefits to universities. Researchers with cross-sectoral experience are able to facilitate attitudinal alignment, knowledge exchange between domains and act as drivers for collaboration. Thanks to their external experience, they can identify original research questions, new impulses for application-driven research directions, which can lead to novel outputs characterised by both scientific excellence and societal relevance. Lowering the barriers to cross-sectoral transitions will improve the quality of knowledge production process in academia. Our findings further provide an indication of how different degrees of domain proximity influence the scale and the driving factors of cross-domain career transitions.

ABSTRACT. The realities of periodic research audit in operation in many research-intensive countries means that HEIs are incentivized to implement policies and mobilize their research workforce towards achieving a high return within these evaluations. This preliminary research uses a global comparative approach using HEI case studies in the United Kingdom, Australia, Norway, New Zealand, Hong Kong and Italy to investigate the productivity and visibility benefit of a competitive hires to university departments. Competitive hires are identified as a mobility event into a department two years prior to an evaluation event (T1), and their value to the department (the extent that the competitive hire was successful) is determined by comparing their productivity and visibility value three years post hire (T2) when the evaluation event is complete. Particular attention was paid to the value-added by international, versus locally based hires, especially in countries with a research workforce that is internationally diverse (UK, Australia & New Zealand). Results show an interesting interplay and conceptualization of the potential benefit of hiring individuals to a department that extend to a consideration of their wider networks. These results provide alternative considerations for research recruitment policies, as well as extend current thinking of the value of an individual vis-à-vis their wider research capacity and existing network.

ABSTRACT. 1. Introduction The paper is based on the main results of the research project called "The global dynamics of production and innovation and the role of territory and national states: challenges for the development of the Health Economic Industrial Complex in Brazil" developed under the project "Challenges for the Brazilian Public Health System (SUS) in the national and global context of social, economic and technological transformations”. The project was supported by Fiocruz (Oswaldo Cruz Foundation), one of the most important public research institutions in Brazil.

Some of the global transformations in production and innovation in the last twenty years are related to the deepening of the financialization process and the reduction of the world trade and world GDP growth rates. These trends have characterized the global economic scenario since the international crisis of 2007/08 and were exacerbated by the Covid 19 Pandemic crisis. The changes in global production and innovation dynamics have provoked significant impacts on the public policies focused on productive and innovative development of developed countries and China.

In this sense one of the most important change in the role of state since the international crisis of 2007/08 was the increase in protectionism in the industrial and innovation policies of the most developed countries. It was observed that the governments of most countries, especially of the G-20, have significantly increased the use of barriers (tariff and non-tariff) to minimize the impact of changes in the global dynamics of production and innovation and of international crises on productive structures and to protect domestic companies.

Another policy instrument increasingly used by the most advanced countries since 2016, especially since the Covid 19 pandemic, is related to the control of foreign capital in strategic economic activities related to national security. The sectors related to health (pharmaceutical products and medical equipment, for example) are increasingly subject to control of direct foreign investment entry.

While openness to foreign direct investment has not been eliminated outright, in recent years many governments have established or strengthened mechanisms to increase control of foreign direct investment (OECD, 2020). The most common explanation for doing this has been related to national security, but it can be observed that the protection of domestic firms and the strengthening of the domestic productive base have also been stimulating more developed countries to increase control of foreign capital inflows (especially foreign direct investment).

In addition to using trade policy and increasing control over the entry of foreign capital to protect local industry and domestic firms, the different governments of the developed countries have also adopted policies aimed at stimulating innovation, especially since 2016, aimed at the development and diffusion of new digital technologies (the so called Industry 4.0). In these cases, public resources are articulated with the measures to protect local industry and with other mechanisms, such as the use of the public purchasing power. Those policies are mostly restricted to locally owned companies, excluding companies controlled by foreign capital.

Another important change observed in recent years in the industrial and innovation policies of developed countries and in the strategies of multinational companies is related to attempts to stimulate local companies to return to their countries of origin important parts of their productive systems previously re-located for countries with low costs, mostly in the Southeast Asia. This process was part of the globalization of production and expansion of global value chain. Such practices, treated in the literature as "reshoring", were already a policy objective of some countries since the middle of the last decade, but received a greater impulse with the crisis of the pandemic, that demonstrated the importance of local productive and innovative capabilities in some strategic activities.

2. Objectives

The paper has three main objectives:

(i) To analyse the characteristics of industrial and innovation policies of developed countries, with respect to: increase in protectionism, increased in the use of instruments aimed at controlling the entry of foreign capital and stimulus to the generation and incorporation of technologies associated with Industry 4.0. (ii) To analyse the reshoring strategies (mechanisms aimed at the internalization of stages of the production process previously displaced to countries with lower production costs) adopted by multinational companies and governments of some developed countries from the mid-2010s on strengthened after the crisis of the Covid 19 pandemic. (iii) To suggest inputs for the design of an industrial and innovation policy in developing countries considering the recent transformations in the global dynamics of production and innovation and in the public policies focused on industrial and innovation development of developed countries.

The context of change in the role of the state in the production and innovation dimension of the developed countries has to be considered in the design of industrial and innovation policy in Brazil. Traditionally, international agencies and institutions suggest that developing countries should liberalize their economies and reduce tariff and non-tariff barriers to increase their competitiveness in the global economy.

In this sense, this paper has the objective of contributing with inputs to new designs of industrial and innovation policies in developing countries (especially Brazil), which take into account the new global trends in terms of productive and innovative dynamics and the new characteristics of the industrial and innovation policies from developed countries and contribute to economic and social development.

3. Methodology The paper uses the innovation system approach to analyse the systemic character of the recent innovation policy adopted by developed countries and to reflect upon the design of new industrial and innovation policy in developing countries.

The methodology of the paper is based on two main axes. The first will be based on the mapping of references about the changes in the role of the State since the international crisis of 2007/08, as well as about the main characteristics of the industrial and innovation policies of the most developed countries. In this case, bibliographical research in international information sources was carried out. The main international information sources to be consulted are the Organization for Economic Cooperation and Development (OECD) and the United Nations Conference on Trade and Development (UNCTAD).

The second axis of analysis of this research project is based on the use of databases to analyse in greater detail the use of protectionist instruments (tariff and non-tariff barriers) in industrial and innovation policies of selected developed countries. For this, the Global Trade Alert database will be used (www.globaltradealert.org). This database gathers, since 2009, information on protectionist and liberalizing measures in relation to international trade adopted by a set of developed and developing countries. This information allows an analysis of the growth of protectionism in the context of industrial and innovation policies, as well as the main instruments used to protect domestic industry.

The main expected result of the paper is to suggest analytical inputs and proposals for the design of a new set of public policy for productive and innovative development in developing countries. This new industrial and innovation policy must consider the international context of increasing protectionism for the domestic firms of developed countries and the need of new approaches for industrial and innovation policy. The paper concludes that these new approaches should be focused on solving national challenges and seek to articulate social demand with the economic development in developing countries.

ABSTRACT. Purpose: This paper investigates the role of the regional endowment of technological capabilities and co-invention network structures in the emergence of new regional technological specializations. We consider a region specialized in a technology when its share of patents in a given field is higher than the share in the country.

Literature Review: Regional technological diversification has been addressed in the literature as a path-dependent process where new technologies congruent with the extant local endowment of capabilities are more likely to be successfully implemented in a region (Hidalgo et al., 2007; Neffke et al., 2011). The rationale is that regional diversification is like a branching process whereby the chances for the successful development or adoption of new technologies rise with the cognitive proximity to those in which the region is already specialized. This, for instance, is the core idea behind the “smart specialization” movement in the European Union. A significant literature has already explored this argument, emphasizing the endogenous dynamics of the regional diversification process. They empirically show that the regional endowment of knowledge and other capabilities is crucial to developing new economic specializations (Balland et al., 2019; Neffke et al., 2011). In parallel, another stream of literature has been pointing out the benefits of inter- and intra-regional co-invention networks for regional innovation performance (Bathelt et al., 2004). However, the impact of these network configurations on regional technological diversification has not yet been widely discussed in a branching process context. While some recent attempts on this matter have concentrated on external linkages (Balland & Boschma, 2021; Whittle et al., 2020), the role of co-invention networks at the inter- and intra-regional levels remains unclear.

Methodological Procedure: We use patent data from Brazil (2000 – 2019) to design two types of networks: one based on inter-regional and the other based on intra-regional co-invention linkages. We then conduct an econometric analysis at the microregional level to examine the effect on regional technological diversification of four main variables: centrality degree, betweenness centrality, transitivity, and technology flexibility. Centrality degree is the number of connections a region establishes with others, capturing the region's access to external knowledge. Betweenness indicates how well-positioned a region is in the inter-regional co-invention flows in Brazil, meaning that the region intermediates knowledge flows between others. Transitivity shows how well knowledge disseminates inside a region based on the network’s cohesiveness. Technology flexibility captures the average cognitive proximity between the regional technological capabilities and the technologies in which the region is not specialized.

Findings: Our results indicate that the regional technological endowment and co-invention network structures at both the inter- and intraregional levels are essential for diversification. High inter-regional betweenness is shown to be important in all contexts. High intra-regional transitivity is shown to be important in most contexts. Network centrality is statistically significant and positively affects diversification when interaction terms regarding the regional development level are included. Our results suggest a substitution mechanism between technology flexibility and betweenness, on the one hand, and GDP per capita and betweenness, on the other. This effect also holds when we change betweenness for centrality degree. It means that less developed regions with lower technological capability endowments benefit from establishing inter-regional co-invention linkages in general, regardless of control and intermediate positions. Our analysis corroborates with findings from two strands of literature. First, the literature on evolutionary economic geography which has empirically shown that the diversification process is path-dependent and is strongly impacted by the relatedness between different sectors, technologies, and products, as the current knowledge and capabilities regionally available will support the development of new ones. Second, the literature on regional innovation networks, in which the flows between physically close and distant agents contribute to enhancing knowledge recombination and production, encouraging learning and innovation.

Policy Implications: Our results can enrich regional diversification policies. For well-endowed regions the issue is what to aim for and with who. Less well-endowed regions face additional stress. Designing place-based policies can be especially challenging for them due to fewer knowledge and technological capabilities which, in turn, narrows the options for policymakers regarding diversification targets. Adding network-related aspects broadens the scope of policy by including practices that favor the connections of actors within and across regions.

REFERENCES Balland, P. A., & Boschma, R. (2021). Complementary interregional linkages and Smart Specialisation: an empirical study on European regions. Regional Studies, 55(6). https://doi.org/10.1080/00343404.2020.1861240 Balland, P. A., Boschma, R., Crespo, J., & Rigby, D. L. (2019). Smart specialization policy in the European Union: relatedness, knowledge complexity and regional diversification. Regional Studies, 53(9). https://doi.org/10.1080/00343404.2018.1437900 Bathelt, H., Malmberg, A., & Maskell, P. (2004). Clusters and knowledge: Local buzz, global pipelines and the process of knowledge creation. Progress in Human Geography, 28(1). https://doi.org/10.1191/0309132504ph469oa Hidalgo, C. A., Winger, B., Barabási, A. L., & Hausmann, R. (2007). The product space conditions the development of nations. Science, 317(5837). https://doi.org/10.1126/science.1144581 Neffke, F., Henning, M., & Boschma, R. (2011). How Do Regions Diversify over Time? Industry Relatedness and the Development of New Growth Paths in Regions. Economic Geography, 87(3). https://doi.org/10.1111/j.1944-8287.2011.01121.x Whittle, A., Lengyel, B., & Kogler, D. F. (2020). Understanding Regional Branching Knowledge Diversification via Inventor Collaboration Networks. Papers in Evolutionary Economic Geography.

ABSTRACT. Background and rationale

Science and innovation are core practices for agricultural development, and the extent to which knowledge is successfully produced, transferred, and applied often defines the trajectories of agricultural development processes. This subject has been captured by the concept of Agricultural Innovation Systems (AIS), which describes the interactions between multiple agents within a given institutional setting that shapes innovation processes, in this case for agricultural development (Rajalahti et al., 2008; World Bank, 2012; also see Tropical Agriculture Platform).

In emerging economies, the dynamics that take place within AIS are often shaped by variegated knowledge sources at the regional level, especially in rural agricultural systems where overlapping institutional settings and diverse actors perform a role from situated practice. In such settings agricultural scientific colleges are still weak (Goyeneche, et al., 2019), making it difficult to assess AIS on the base of scientific metrics.

This is the case in Colombian regions, where knowledge is subject to hybridization processes. In such settings local and scientific knowledges interact in the different phases and domains of the AIS. Therefore, the coexistence of multiple emergent patterns and diverse degrees of sophistication makes it difficult to track the specific features and configurations shaping the AIS’ capacity and performance.

The growing institutionalization of the Colombian AIS demands further consolidation of mechanisms for monitoring and evaluating (M&E) agricultural innovation to enhance informed decision-making (Ordóñez-Matamoros, 2013) at the regional level. This practice generates knowledge about the system, which favors strategic direction towards the objectives set to achieve innovation (Hekkert et al., 2006).

However, in a context of emerging practices, such measurement needs to transcend the traditional approaches that tend to overemphasize bibliographic scientific output, innovation patents, and results-based management, usually of a linear and short-term nature, to assess better the variety of practices that take place in regional AIS.

While there have been theoretical contributions to develop logical models for M&E mechanisms of the Colombian AIS, considering the particular institutional trajectory of the country in that regard (Londoño, Ordóñez-Matamoros & Uribe, 2021), further research needs to advance on the operationalization of such frameworks in order to assess their relevance in regional contexts.

This paper reflects on a pilot project implemented in the Casanare region in Colombia, assessing the conditions under which a M&E mechanism should be operationalized as a tool for the strengthening of regional AIS.

Methods

We rely on the framework of Londoño, Ordóñez-Matamoros & Uribe’s (2021) to examine the scope and limits of the proposed monitoring and evaluation model, as well as the conditions for its regional operationalization and possible challenges for its scalability to other regions.

Empirically, the paper builds on primary data from three workshops conducted to involve local actors in the definition of a set of indicators for M&E in Casanare’s AIS, as an instrument to make the M&E process effective. The first workshop aimed at identifying, validating, and formulating indicators for monitoring and evaluating science, technology, and agricultural innovation. The second workshop maintained the same general objective but focused on the validation of indicators. Finally, the main objective of the last workshop was to start the process of collecting information for the calculation of indicators. This last workshop collects information from the 47 most relevant actors of the AIS and is essential to see the effectiveness of the mechanism evaluated.

The initial battery of indicators was built by revising different sources of secondary information. Among them, national and departmental policies on agricultural innovation, laws, research products, and an advance that the National Government has been implementing in the form of a national survey of agricultural science and technology. Furthermore, the direct involvement of the authors in this project offers multiple lessons that allow a better interpretation of the gathered information.

The case of Casanare offers a relevant context in which agricultural innovation can be regarded as an alternative to extractive industrial practices that pose challenges for sustainability transitions at the regional level. Casanare is known for its oil extraction economy, as well as extensive livestock farming and monoculture crops of palm oil, which entail diverse social and environmental tensions.

Anticipated results

In Colombia, there is a theoretical and methodological approach to how the M&E mechanism for agricultural innovation should work. However, the operationalization of this remains a challenge for the country because it depends on the environment and the characteristics in which this mechanism is developed, and Colombia is a country composed of diverse regions that have different populations and geographical and productive characteristics. Therefore, the results will allow validating this methodological and theoretical instrument and show whether the country can implement a standardized M&E model for all departments based on knowledge of the internal and critical relationships of the system. Consequently, these show the requirements of capacity and coordination in various management levels of the tools of follow-up and monitoring to go deeper into its territorialization and homogenization.

Significance

This research adds to the growing literature on AIS, with a special interest on the regional scale and M&E practices, which remain underexplored. The regional scale in this case suggests possible pathways for the implementation of national AIS and policies in that realm, while considering emerging practices and interactions between actors whose knowledge is built upon multiple sources beyond the scientific. This research suggests possible ways to cope with complex emerging practices for monitoring and evaluation purposes, so that M&E mechanism in AIS can better inform policymaking.

This paper also contributes to a better understanding of innovation policy and practice in emerging economies. The Colombian case offers an interesting landscape of growing institutionalization of AIS at the regional level, along with multiple sustainability challenges that can be better addressed by means of agricultural innovation. However, such endeavor can only be fulfilled with relevant information and data on the performance of AIS, and M&E systems are a necessary condition to this.

ABSTRACT. Introduction

Science policies (or lack of) have the potential to maximise opportunities as well as expose society to risks presented by emerging technologies. The integration of artificial intelligence (AI) and biotechnology, whilst in its infancy, presents significant opportunities and risks, and proactive policy is needed to manage these emerging technologies. While AI continues to have significant and broad impact, its relevance and complexity magnify when integrated with genetic editing in particular; applying AI elements such as machine learning and deep learning can foster substantial benefits as well as daunting risks that range from ethics to national security. Yet, this combined field has not been adequately studied from a policy perspective. The components need clear definitions and analyses with respect to their practical combined implications for society. The study of genome editing, and its technological advances have been facilitated to a large degree by their underpinning bioinformatics platforms, which has allowed screening and detection and gene manipulation to become common practice. Application of AI to this technology will catapult the revolutionary potential of genome editing from ‘hypothetical’ to imminent. However, application of AI to this rapidly advancing field poses severe risks from potential weaponization and bioterrorism as well as opportunities to improve health and wellbeing. Both genome editing and AI technologies are being pursued at scale in various global markets and the genome and AI research agenda is moving towards a progressively permissive landscape, with AI and genome editing technologies being brought together to cut costs and time. For instance, DNAnexus, a cloud based commercial platform, offers products utilizing machine learning techniques for mutation predictions, diagnostics, and gene target optimisation. Verge Genomics, an AI driven therapeutics company, is bringing together human genome and machine learning to create predictive models for drug development with its first ALS drug just entering the clinical market. It is just a matter of time before these developments are further exploited with AI tools, not just to modify the human genome to cure disease but to develop resistance to diseases, enhance vision, create superhuman strength, and pave the way for a species evolution. There clearly is an urgency to addressing policy issues surrounding these emerging technologies. Given this gap, we are investigating the policy implications of the application of AI to genome editing in humans, in particular technology governance as a cross-cutting theme. This analysis will help regulators take stock of advances with AI-facilitated genome editing and develop a future scenario-focussed framework to protect human interest by considering the implications of these technologies being pursued at scale and globally. Our methods outlined below will ensure that this is not just an academic exercise, by engaging relevant sector and policy experts and ensuring that the outputs reach a variety of audiences.

Methods

Landscape assessment: We will use software-assisted horizon scanning for assessing the state-of-the-art of AI and genomic tools, creating a typology for their use, and assessing where they have been or integrated. We will complement this approach with interviews with up to 10 subject-matter experts. We will conduct a desk-based review of the current regulations surrounding genome editing and AI applications in the life sciences as a frame of reference for understanding the areas in policies that are unclear or uncertain. Landscape assessment will enable us to identify the most important factors that affect the trajectory of AI and genomic tools. Futures framework: Landscape assessment will form the basis for how the relationship between AI and genomic editing is currently configured, and a futures framework will then be used to explore how this relationship might change over time. We will use a scenarios-based futures exercise, where the landscape assessment will facilitate crafting of illustrative stories that outline how AI and genomic tools might remain the same, improve, or change in some other way based on the effects of the factors identified in the landscape assessment. A basic scenarios-based table-top exercise (TTX) will be conducted based on the trajectory of the landscape over the next 25 years, bringing together key stakeholders in genomics, AI, and regulatory sectors, to contemplate the various policy dimensions of these technologies. The TTX will identify opportunities, risks, and policy issues, with special consideration of technology governance challenges as a cross-cutting issue. The proposed approach will serve as a blueprint for future areas of research in multiple sectors where various policy lenses could be applied (e.g. impact on agriculture and food security). Going through such scenarios with stakeholders will enable us to characterise how well current policy addresses these scenarios and what changes may be needed taking this further than just an academic exercise. The project team will use MURAL software for scoring the scenarios, to enable the prioritisation of the most important policy responses or changes required. Key risks, opportunities, policy implications, and actions will be identified as outputs of the exercise.

Anticipated results

The current regulations and discourse are not appropriate to respond to the melding of AI and genome editing technologies and to regulate them at scale. Using a combination of landscape review and futures methodology, we will create a novel frame of reference for policymakers and the public. This work approaches the understudied sector of the confluence of AI and genome editing, and it explores technology governance as a broader policy challenge that could be applied to other sectors and technologies. The outputs of this work will be consolidated into a short report, summarizing opportunities, risks, and policy considerations for countries investing in R&D, thus supporting responsible innovation. The evidence will provide a steppingstone for a roadmap for policymakers and will serve as a mechanism for engagement with potential clients in Europe and the US such as DARPA, NATO, DoD, and more. We will pursue further dissemination via development of a podcast with sector experts delving into the output produced and assessing their views. This will further our research and help reach policy makers and the wider public to generate interest and a public discourse as well as policy considerations. We will actively engage our networks and potential clients to discuss our findings and create bespoke policy briefs and concept notes as a follow up activity.

ABSTRACT. Global value chains (GVC) have taken the policy world by storm with virtually all leading international organizations dedicating flagship publications to the topic. Yet, despite the huge enthusiasm about GVCs, there remains substantial ambiguity about their policy implications. In this article, we argue that the novelty of GVC-oriented policy lies in the elevated role that it gives to tasks, linkages, and firms. Such novelty pushes policymakers to adopt new task-based industrial policies, global connectedness policies and corporate due diligence policies that are challenging the current rule-based trading system.

A first novelty of GVC-oriented policies is the shift of attention from industries to tasks. Global production allows countries to functionally specialize in finer-grained value chain stages instead of entire industries. Yet the disproportionately large value-added that is captured by intangible-intensive tasks (e.g., R&D and marketing) presses policymakers to develop public policies that attract and retain the latter. In both developed and developing countries, policy makers are thus adopting a mix of task-based industrial policies – including ICT infrastructure investments, tax credits and subsidies – to expediate the upgrading of local industrial activity into more intangible-intensive tasks. For example, the Malaysian government has invested heavily in technology centers like the National Applied R&D Center that houses labs offering a wide range of instruments and infrastructures for advanced ICT testing services accessible to GVC suppliers at subsidized rates.

Second, GVC-oriented policies emphasize the role of linkages. GVCs link domestic and foreign tasks, and these linkages influence domestic firms’ economic and social performance. On the positive side, they act as powerful conduits for accessing foreign knowledge and resources that can be leveraged to improve technological and operational capabilities. Decent work parameters imposed by foreign value chain partners can also incentivize firms to improve local labor conditions. On the negative side, these linkages can reduce economic resiliency by transmitting foreign economic shocks to domestic firms. A focal concern of policymakers is thus how to properly regulate, deepen, and strengthen GVC linkages so that they can promote economic and social upgrading while guaranteeing economic resilience. On the one hand, such global connectedness policies aim to improve the access to knowledge and information that is transferred through linkages. On the other hand, they focus on strengthening the absorptive capacity of local firms. For example, for many years the Chilean government has promoted networks of suppliers to enhance inter-firm collaborations and reduce the gap between lead-firms and their input providers.

GVC-oriented policies finally elevate the status of firms. Multinational enterprises (MNEs) have enormous power in GVCs. Sitting at the apex of a hierarchical chain, they have the muscle to select which firms are included or excluded in GVCs, to determine the terms of supply-chain membership and to allocate where, when and by whom value is added. It is this power that allows MNEs to not only enhance the efficiency of the GVC, but also to promote social and environmental conditions extraterritorially. They can remediate poor labor and environmental conditions in global supply chain factories, for example, by pledging to work only with suppliers and sub-suppliers that adhere to strict codes of conduct and collaborate with them to reach this goal. Policymakers not only see this power as a challenge but also as an opportunity. By properly harnessing MNE power within GVCs, governments can use domestic policies that target MNEs to foster changes along GVCs. Several countries, for example, have started to impose strict corporate due diligence principles on MNEs that apply along their entire GVC. For example, the French government in 2017 implemented the Duty of Vigilance Law to impose a legal duty to exercise human rights due diligence.

These new policy directions challenge current trade policy practices in several ways. First, the intricate relations between tasks, linkages and firms require policymakers to move beyond traditional silo thinking where each Ministry pursues its own objectives independently. Now more than ever Governments need to develop coordinated trans-Ministerial efforts to synchronize their trade, innovation, industrial and social policies. Second, the global scope of GVCs requires countries to develop new supra-national policies to avoid distortionary arbitrage strategies that are rampant in GVCs. The recent G20 agreement for a minimum global corporate tax rate to close cross-border tax loopholes is a good example of new steps in this direction. Third, the rule-based multilateral trading system needs to modernize to tackle challenges that GVC-oriented policies pose to the foundational principle of non-discrimination. For example, corporate due diligence policies that apply throughout GVCs represent a legal obstacle to “national treatment” if they lead to the discrimination of imports from non-compliant businesses.

Taken together, even if GVCs reinforce several traditional trade policy elements, they in other areas force a fundamental redesign in policy thinking by accentuating the role of tasks, linkages and firms. We argue that these changes are so profound that the set of GVC-oriented policies has become a truly different concept that challenges the rule-based trading system.

ABSTRACT. Overview This presentation offers a comparative analysis of the President’s Council of Advisors on Science and Technology (PCAST) across presidential administrations, beginning with President George H.W. Bush through the first two years of the Biden administration. PCAST – a federal science advisory committee appointed by and advising the president – will be assessed for its changing balance of member social and professional perspectives and its role and impact in federal policymaking over the last thirty years. The goal is to understand the changing balance of PCAST’s expertise and the evolution of its advisory role and across time, as well as to employ this analysis to develop recommendations for PCAST’s organization and operations in future administrations.

Background and Rationale PCAST was created by President George H.W. Bush in 1990, although its origins date back to the Truman administration and the early Cold War. PCAST is co-chaired by the president’s “science advisor,” its only federal member, and consists of roughly 30 independent scientists, engineers, industry leaders, and other research professionals. PCAST is charged to advise the president – either directly or through the science advisor – on all matters related to science, technology, and innovation (STI).

While PCAST’s function has not changed significantly since its creation, its membership demographics, policy areas of focus, and influence on White House policymaking has shifted considerably with each presidential administration. Most recent research studying scientific experts in U.S. government decision-making focuses on “science for policy” technical advisory bodies within federal regulatory agencies. PCAST, however, is “policy for science” presidential committee responsible for guiding White House policy development and implementation across a broad range of issues that rely on STI data and analysis, including national security, public health, industrial competitiveness, environment and energy policy, and science education. PCAST’s proximity to the president, its high-profile membership, and the historical influence of its predecessors make it a unique case study for examining the role of scientists and other research experts working at the nexus of science, policy, and politics. Empirically evaluating scientific advice remains a challenging research question. This presentation offers data and a preliminary analysis of PCAST’s changing member perspectives and its evolving role and impact in White House STI policy decision-making. The goal is to understand how and why professional or social groups were represented on PCAST’s roster across time and how PCAST policy recommendations are developed and implemented in broader STI ecosystem through time.

Methods This research employs two analytical frameworks to understand PCAST’s membership demographics and the nature of its advisory function over time, from 1990 through 2022. The first framework establishes a measure of PCAST’s diversity in the context of its governing legislation, the Federal Advisory Committee Act (FACA). FACA mandates certain transparency requirements for committee operations, as well as guiding language for ensuring committee membership is “fairly balanced” in represented interests and technical expertise to address its intended function. As PCAST members are appointed based on both their scientific knowledge and personal achievements, member demographics are categorized into “professional perspectives” and “social perspectives.” PCAST membership will be compiled from the Federal Register and each individual will be coded by their professional (i.e., scientific expertise and career experience) and social perspectives (i.e. race or ethnicity, gender, and geographic location) to be collected from public biographical records. Data will be supplemented with public statements from PCAST members—such as oral history interviews and public statement from current and former White House officials—to provide comprehensive picture of how each president and their science advisor approached membership balance and if that balance is representative of the national STI workforce at the time of service.

The second analytical framework offers a methodology for examining PCAST’s intended function and its impact in White House policy development and implementation. Although FACA ensures PCAST’s formal independence from the administration, PCAST remains responsive to White House policy priorities through the leadership of the science advisor and sometimes direct engagement with the president. PCAST, in its alignment with administration priorities, could possess an “instrumental” role, providing substantive policy analysis that advances the public interest, or a “symbolic” role, legitimizing or otherwise supporting administration decisions – or a combination of both. To make this determination, each president’s PCAST will be analyzed based on four criteria: its leadership structure (e.g., Does the science advisor serve as its sole chair or co-chair?); its process for consensus building (e.g., How are policy recommendations vetted and agree upon?); the long-term impact of policy products (e.g., Do PCAST studies translate into concrete policy actions?); and the image of PCAST presented through official government press releases and its reception in national media (e.g., How and how often are PCAST members and their work represented in news outlets?). Anticipated Results

The two frameworks will help to provide a complete picture of PCAST’s shifting membership balance and its role in White House STI policymaking beginning with the George H.W. Bush presidency. With respect PCAST’s diversity, the council’s exceedingly broad charge to address all policy related to STI raises the question of how best design a representative council membership inclusive of all individual and career backgrounds and scientific disciplines. An initial analysis of PCAST membership demographics demonstrates a trend toward increased inclusion of both professional and social perspectives with time. However, the data indicate a persistent underrepresentation of both categories with respect to the national STI workforce and overall U.S. population. In particular, women and minoritized racial and ethnic groups are underrepresented in PCAST in most administrations. Notably, individuals of Asian descent were significantly underrepresented with respect to that group’s increasing participation in the national STI workforce. Additionally, individuals with expertise in the social sciences, while consistently represented in most PCASTs, are not included in PCAST at the level of active U.S. STI professionals.

Tracking the trajectory of PCAST research products from ideation, to development, dissemination, and implementation on downstream policy decisions is a challenging task. The second framework will provide a method for beginning to understand how PCAST has interfaced with high-level stakeholders in the STI community to deliver policy recommendations, and how those recommendations translated into actions or informed government decision-making. The analysis will allow for a deeper understanding of the de facto nature of PCAST’s independence and the limits of working within both FACA and the highly political context of the White House.

Significance This study presents the first longitudinal analysis of PCAST membership and policy products. PCAST stands as one of the U.S.’s most visible scientific bodies, unique in both its position within the federal STI policymaking apparatus and the preeminence of its membership. The anticipated results will offer an empirical basis for developing best practices for PCAST’s membership selection and operations. As PCAST is involved in high-level development of federal STI policy, optimizing its activities and policy products to increase its future impact will in turn work to advance the overall health of the U.S. national STI enterprise.

ABSTRACT. Summary To meet the demands of the ongoing green energy transition, policy makers need to know how to induce market transformation towards low-carbon technologies. However, the widespread diffusion of new technologies is hampered by a variety of barriers: e.g. high upfront costs, uncertainty around performance, etc. There are abundant policy tools designed to overcome these barriers, but relatively little is known about which policy tools to deploy when, for how long, and how much. A comprehensive approach to designing a policy mix could illuminate the path forward to improved efficiency in policy support by taking advantage of synergies between individual policy tools. Here, we model the owner-broker interaction in green building technologies in a variety of randomly generated policy contexts. We then search through this parameter space for patterns in the combinations of policies – their sequence, timing, duration and strength – that reliably yield market transformation. Our findings contribute to a growing literature on policy mixes, and to the body of policy knowledge derived from computational social science approaches. This paper is a work in progress with preliminary results. Full results are expected in time for the conference.

  Introduction / Background Governments have set aggressive timelines for net-zero decarbonization that requires widespread energy and sustainability transitions (H Res. 109; Weyant, 2011; Mallapaty, 2020; Dooley, Inoue & Hida, 2020). However, despite a clear acknowledgement of the need for energy and sustainability transitions, the institutions and policies that would drive these transitions remain unclear. The economics and innovation literatures call for a better understanding of “transitions management,” acknowledging the role of emissions pricing, cost reductions for low-carbon technologies, and information barriers that slow the diffusion of nascent technologies (Meadowcroft, 2009). The “valley of death” describes the challenges of moving a nascent technology from basic science to widespread market penetration. The valley of death is often approached as a cost problem. Large private R&D investment results in high per-unit prices, and low demand at high prices prevents scaling up and decreasing marginal costs of production (Nemet et al., 2018). The dominant perspective on reducing these costs, and thus traversing the valley of death, centers on market mechanisms such as subsidies, carbon taxes, or cap and trade schemes. However, the implementation and effectiveness of these approaches has been, at best, incomplete (Geels et al., 2017). Politicians have been unwilling to move on these strategies or set emissions prices that are too low, and as a result, socially efficient externality pricing covers only a small portion of global greenhouse gas emissions (Geels et al., 2017; Weyant, 2011). We contend that the failure of conventional pricing tools to pull nascent low-carbon technologies into widespread diffusion is the result of a policy mix that ignores the underlying information barriers that drive the valley of death. We frame the issue of high per-unit costs as stemming from “knowledge gaps” and information barriers that substantially increase risk and subsequently preclude significant investment from capital markets (Blackburn et al., 2020; Jaffe et al., 2005; Weyant, 2011). Many policy instruments, including public R&D investment, pilot projects, demonstration projects, public procurement, tax breaks and rebates, regulations and mandates, and other market instruments, have been shown to foster learning effects and reduce information barriers, risk, and costs (Bossink, 2002; Bossink, 2015; Edler, 2007; Aghion et al., 2009; Blackburn et al., 2020; Geels et al., 2017, Jaffe et al., 2005; Weyant, 2011). Despite the variety of tools and levers that can be pulled to induce market transformation, each instrument deployed in isolation may not generate enough learning or overcome sufficient informational barriers to drive the sustainability transition (Rosenow et al., 2017). The question of how hard and in what combination to pull these levers remains pressing (Nemet, 2010). We propose that synergies are likely to exist between various policy tools, and that coordinated efforts to implement carefully designed policy mixes can leverage these synergies to bridge the valley of death. Analyses of the interactive effects, temporal factors, and institutional dynamics of policy mixes are likely to shed light on how governments can design effective transitions management strategies (Flanagan et al., 2011) Here, we develop a theory of the valley of death with a particular focus on information issues that drive high costs, mechanisms that can be used to overcome these information issues, and specific policy tools that can be implemented to traverse the valley of death.

A New Approach: Building a Bridge with Policy Mixes Multiple market failures and barriers contribute to the technology VoD, thus, a “policy mix” of several instruments – some pushing from the supply side, some pulling from the demand side, and other acting to reduce informational barriers directly – are likely needed to bridge the VoD (Rosenow et al., 2017; Flanagan et al, 2011; Raven and Walrave, 2020; Fischer and Preonas, 2010; Weyant, 2011; Lehmann, 2012). Deploying a combination of cost-focused and information-based instruments has been shown to reduce the cost of transition management for emissions reductions compared to a single policy (Fischer and Newell, 2008). The design and implementation of policy mixes to spur nascent technology diffusions is beginning to take hold on the international stage (OECD, 2009). An attractive feature of the policy mix approach is that the effects-in-isolation of individual policies are unlikely to be additive when implemented together – rather the expectation is that synergistic interaction effects, temporal dynamics, and institutional dynamics will contribute to the success of the policy mix (Flanagan et al., 2011)

Methods Overview To demonstrate the framework for understanding the relationship between context and policy levels, we deploy several methods sequentially. First we build an agent-based model (ABM) of building development driven by owner-broker interactions in which we operationalize both contextual variables and policy levers. Second, we simulate the first two (of three) scenarios: a no-intervention scenario the establish a baseline and a random intervention scenario to explore the space of policy interventions. Next, we conduct an unsupervised cluster analysis on the cases with successful outcomes from the random intervention scenario to find groups of self-similar “pathways” to success. Next, we simulate a third scenario with strategic policy interventions based on the pathways to success identified in the previous step. Finally, we compare the distribution of successes across the three scenarios.

Preliminary Results Based on 10,300 simulations of our model, we identify solutions that improve the probability of successful market transformation. The random intervention scenario improves the expectation of successful outcomes over the no intervention baseline scenario from 15.2% to 22.3%. However, it does not do so systematically. The details of each random context are not used to inform policy design – this reflects a tremendously naïve approach to policy design. We improve upon the policy design by strategically connecting a policy prescription to a specific context, and in doing so demonstrate a further improvement to outcomes: increasing the expectation of success to 28.1%. These results can help inform how policymakers and other stakeholders can strategically deploy policy tools to accelerate green market transformation.

ABSTRACT. Evidence-based or evidence-informed decision-making is the paradigm by which public decision-makers leverage knowledge from various sources to craft policies and programs that are fair, efficient, and effective. Evidence can come in a multitude of packages and be leveraged with different calculuses about what matters most. In this work, we build on Head’s three lenses of evidence-based policy where he articulates that evidence can come from science (what we traditionally think of as “evidence”), from practice (the tacit and earned knowledge of front-line public servants), and from the political sphere (knowledge about political dynamics, authority, and feasibility) by examining policy documents from two consequential public policy domains in the United States: autonomous vehicles and state-level opioid strategic plans.

The focus is on how state governments, often in conjunction with knowledge intermediaries attempt to monitor and understand innovation trends and develop strategies to adapt to changes in science and technology. Policymakers and public administrators frequently draw from research in science and technology to address problems confronting society. Relatively little is known regarding how state public managers scan for, access, and employ consensus science. This panel explores the variety of sources of information that can be accessed, the development of intermediary organizations to help structure and access information at the state and local levels, and the methods local governments use to communicate that information to their constituencies.

Policy reports and plans can be used as artifacts of policy processes that capture the decisions and dynamics of complex policy activities, rather than as an end of the policy process (Freeman and Maybin 2011). As such, detailing the people and sources used to create the policy document can reveal important aspects of how the policy evolves in a location. We use policy documents from the two domains mentioned above to identify important aspects of how the policy deliberations occurred and the impact they had on the outcome (report construction). We created unique datasets for this investigation.

Data and methods. All reports from the 50 states and Washington DC relative to the two substantive areas were scraped from publicly available sources on the internet. All reports from 2013 until 2020, when the COVID-19 pandemic started) were considered for this investigation. Reports were then reviewed for their focus on program implementation and governance, rather than a technical focus. There were 68 reports on the AV side and 72 in the opioid case.

Reports were then analyzed on a number of dimensions to understand how they used evidence in their work. Our methods were centered on bibliometric techniques which catalog and enumerate the contents of scientific documents. We assessed images and figures, references, word/page count, topics covered using NLP, authors, and committee/task force membership. We also ran clustering models to find distinct patterns of reports.

We used topic modeling to characterize the contents of the state reports, state laws, and executive orders. The core text of each document was extracted for modeling. Wordstat 9.0.8 was used for text processing and topic modeling using the nonnegative matrix factorization (NMF or NNMF) method. The text was preprocessed using snowball stemming. Words that occurred at least 30 times in the corpus were retained, or about 2,000 words in total. Other settings remained Wordstat defaults. Examining coherence statistics showed a peak at about 30-35 topics; 35 topics were specified in the final model.

Findings. We found three distinct clusters of reports within each of the two substantive areas. While the specifics of the clusters across the two areas are slightly different, they are remarkable in that they have the hallmarks of Head’s three lenses of evidence for policymaking. On the AV side, we see three clusters that are expert-engineering (practice evidence), expert-academic (scientific evidence), and convening, legislative focus (political evidence). For the opioid reports, we see that reports that have a lot of private organizations (rather than state agencies) emphasize practice-based knowledge, while those reports that are oriented by task forces emphasize political knowledge, and finally, those reports authored by groups with heavy representation from public health departments emphasize scientific knowledge.

ABSTRACT. The ‘turn’ in policy towards addressing societal challenges calls for a radical rethink of how to do research and innovation policy. Our paper focuses on evaluation in this new policy environment. Challenges for evaluation include the increased scope of the policies needed, devising theories of change that are challenge- rather than push-based, tackling complexity, and new roles for evaluation in the life cycles of policies that tend to be longer than those of traditional research and innovation interventions. Policymakers and evaluators alike need to embrace and incorporate lessons from the fast-growing literature on transitions and missions. The paper will synthesise results from our own and others’ research on evaluating transition- and mission-like interventions, deriving operationally relevant conclusions for evaluators and policymakers.

ABSTRACT. The growing institutionalization of innovation systems in emerging economies has led to a more prominent role of the State in policy design and implementation to foster knowledge and innovation production, diffusion and use in different realms of society. In this context, Science, Technology and Innovation Policies (STIP) and instruments require the constant evaluation of its performance and results, in order to better inform the role of governments therein.

In emerging economies, the increasing number of policy documents contrasts with the limited capacity for the implementation of national STIPs, which is often related to the lack of use of policy evaluation results and its derived lessons and recommendations (Kuhlmann & Ordóñez-Matamoros, 2017). One of the reasons for that is the single-intervention approach of policy evaluations, which do not often reflect their implications for broader innovation systems. In that regard, translating policy learning into effective policy change demands more comprehensive lessons on the strengths and challenges of STIP with the use of STIP evaluations. In other words, the body of knowledge produced by evaluations may clarify what we know and do not know about STIP, in order to enhance STI policymaking and governance.

Meta-evaluation frameworks help us to systematically assess the results of evaluations in order to learn more about how STIPs perform within systems (Edler, Ebersberger, & Lo, 2008). Rather than focusing on the intrinsic technical merit of single interventions, these approaches emphasize the broader contribution of STIPs to innovation systems (Arnold, 2004). While such approaches are frequently used in mature innovation systems (Edler, Cunningham, Gök, & Shapira, 2016; Collin, Sandström, & Wennberg, 2022), these could be further operationalized in emerging economies, for instance in Latin America (Bin, Andrade, Vasconcellos, & Salles-Filho, 2019), where more knowledge is needed on the systemic role of STIPs.

With that in mind, this research presents a meta-evaluation of STIP in the context of Colombia, as a relevant and illustrative case of an emerging economy. It assesses the design of and results produced by the STIP evaluations conducted in the country. Specifically, this work has a three-fold purpose: first, to derive lessons learned from the STIP evaluations performed in Colombia; second, to understand and explain the factors that facilitate or hinder the conduct of successful evaluations, where success depends on a) whether the evaluations achieved their objectives and b) whether the evaluation achieved any impact; and third, explore the value or merit of different types of approaches and methods that are used to evaluate. In this regard, some key research questions are:

• Did the purposes of the STIP evaluations clear and achieved? What lessons can be derived from evaluating the effectiveness of STIPs? • Have the evaluation approaches been useful or not in understanding the underlying rationale of STIPs? • What methods have been used to understand the interventions? • What factors are determinants in the impact and use of evaluations?

This paper follows a meta-evaluation approach, i.e. “an evaluation of evaluations” or a systematic analysis of a series of public policy evaluations with the purpose of informing the scope and quality of the ways that the merit of public interventions is assessed (Scriven, 1991, p. 95).

To that aim, empirically we examine different public repositories of evaluations of the Colombian national government, including the Ministry of Science, Technology and Innovation, the National Planning Department and its evaluation system, and the Colombian Observatory of Science and Technology.

A total of 36 evaluations have been identified so far, conducted between 1997 and 2021. The basic relevant information of each evaluation report is extracted and systematized in a table, including categories such as the type of evaluation, the type of STIP evaluated, the date of publication, the time frame observed by the evaluation, the authors and its institutional affiliation, the purpose of the analysis, the evaluation methodology and the type of methods, the main results and conclusions, among others. Furthermore, a grounded literature-based framework is being developed to derive additional categories that allow a better assessment of the evaluations. The main themes of evaluation reports will be codified under these categories by using traditional content analysis software (e.g. Atlas.ti, NVivo).

The expected results include a complete data base with systematized and processed information on STIP evaluations in Colombia. This will allow to eventually build a STIP Evidence Gap Map (EGM) that graphically shows what we currently know and ignore about the STIP process and performance in the country. According to Snilstveit et al. EGMs provide thematic collections of evidence structured around a framework which schematically represents the types of interventions and outcomes of relevance to a particular sector. By mapping the existing evidence using this framework, EGMs provide a visual overview of what we know and do not know about the effects of different programs (2016, p. 120).

Preliminary findings show an evolution on evaluation approaches in the Colombian National STI System, beginning in the 1970’s with mechanisms such as expert panels to assess the merit of STI projects and instruments ex ante. During the 1980’s and 90’s peer-review is the main mechanism to evaluate STI policy, and by 1997 results and impact evaluations are put in place to assess the socioeconomic effects of STI programs. During the 2000’s the previous mechanisms are further institutionalized, along with institutional evaluations of actors in the National STI System, and by the 2010’s decade more evaluations are conducted using more standardized criteria, professionalizing the role of the evaluator, and complementing measurement mechanism of scientific capacity, performance and output.

The evolution of STIP in Colombia reflects what the body of literature on STIP evaluation stresses regarding the ever growing and changing nature of evaluations: as general STIP frames change (Chaminade & Edquist, 2010; Vehlo, 2011; Schot & Steinmueller, 2018; Kuhlmann & Rip, 2018), so are STIP evaluation approaches, which seem to reflect on the former, offering at least four broad streams on evaluation practice.

A first approach is inspired by the scientific tradition of peer review in which the value of policies lies in their intrinsic (scientific and technical) quality. Second, there is an approach derived from new public management thinking interested in the broader socioeconomic impact of research and innovation programs. Third, a systemic perspective concerned with the performance or 'state of health' of innovation systems, considering the multiple variables that compose them (Georghiou, 1999; Arnold, 2004; Edler, et al., 2016).

Fourth, with recent innovation policy approaches such as Mission-Oriented Innovation Policies (MOIP) (Mazzucato & Semieniuk, 2017) or Transformative Innovation Policy (TIP) (Schot & Steinmueller, 2018), there is a renewed interest in formative evaluation (Magro & Wilson, 2013; Molas-Gallart, Boni, Giachi, & Schot, 2021), oriented towards learning and reflexivity in experimental processes for transition to sustainability (Luederitz, et al., 2017; Mickwitz, Neij, Johansson, Benner, & Sandin, 2021).

This research intends to produce relevant evidence aimed at STI policymakers, program managers and evaluators to inform STIP process and its phases of design, implementation and evaluation, while contributing to the broader debate on the effectiveness of STIP in Colombia. Results can also be relevant for regional projects interested in STIP governance and evaluation, where there may be lessons for cross-learning between Latin American countries (Bin, Andrade, Vasconcellos, & Salles-Filho, 2019), while dialoguing with STIP meta-evaluations conducted in developed countries (Collin, Sandström, & Wennberg, 2022).

ABSTRACT. The objective of this paper is to study the effects of the micro policy of sectoral funds on the innovative and economic performance of firms. For this purpose, we are going to study the High-Tech Sectoral Fund (FSAT), one of the instruments of the Argentine Sectoral Fund FONARSEC), implemented by the National Agency for the Promotion of Research, Technological Development and Innovation (I+D+i Agency) of the Ministry of Science and Technology of Argentina. The period of analysis is between the years 2010 - 2017. Evaluations of policy instruments in general focus on conducting impact analyses, either from a market failure or systemic problems perspective. In turn, these studies also mostly focus on studying horizontal programs that do not promote particular sectors. This research analyzes a sectoral fund whose goal is also to strengthen the link between the scientific and technological sector and the socio-productive sector in order to contribute to the solution of social and economic problems. In particular, we will take as a reference the policy of support to isolated innovation projects to evaluate whether the promotion of public-private partnerships generates a better innovative performance. This type of analysis becomes relevant due to the scarcity of studies that analyze the impact of vertical innovation policies. In particular, given the challenge that the covid-19 pandemic implied for policy in general, and science and technology policy in particular. Currently, the policy has shifted to a much more focused scheme, and this requires knowledge about the results of this type of policy. This paper thus contributes to policy by shedding light on the results of a vertical instrument. The policy of sectoral funds is a recent milestone in Argentine science, technology and innovation policy. In particular, in relation to the way in which technological innovation was supported in companies. Three elements make it possible to dimension this change. First, innovation policy shifted from horizontal instruments to vertical instruments that sought to provide support focused on sectors, regions of lesser relative development, or technological platforms. Secondly, there was a shift from isolated support for companies to innovate, to instruments focused on public-private partnerships. Finally, the size of the non-refundable contributions was substantially larger than that of the pre-existing horizontal instruments: the minimum non-refundable contribution was USD400,000 and could reach up to USD10,000,000. This policy was manifested in the Argentine Sectoral Fund (FONARSEC), composed of three instruments: Sectoral Technological Innovation Funds (FITS). Regional Technological Innovation Funds (FITR) Sectoral Funds for High Technology (FSAT). FITS was designed with a focus on sectoral development, while FITR has a regional focus. Finally, FSAT has a focus on intensive technology platforms, such as biotechnology, nanotechnology and ICTs. Within this framework, this paper aims to analyze the impact of FSAT. To this end, we will study the causal relationships between access to financing, the formation of a public-private partnership and the trajectories of the firms. We will analyze whether the trajectory of their innovative effort and/or market performance improved as a result of their participation in the program. Understanding the impacts of the instrument on the companies will make it possible to identify the public policy criteria that have worked and those that need to be focused on in order to improve. The methodological approach is based on an econometric study that will determine whether the trajectory of a firm that accessed the instrument was different from one that did not. For the evaluation, a panel of data will be constructed with a balanced structure and information for two moments in time, before and after the FSAT considering the average of three years. The identification strategy will combine two impact evaluation techniques. First, a Propensity Score Matching will be implemented to identify those firms in the control group that, due to their observable characteristics, are not similar and comparable to the beneficiary firms. Then, once the database has been cleaned, a Differences in Differences estimation will be implemented to estimate the trajectory that would have been observed in the absence of FSAT. By comparing the two trajectories, the observed and the counterfactual, we will be able to identify the Average Treatment Effect attributable to FSAT, controlling for unobserved and fixed factors over time. The results confirmed an impact of FSAT on the trajectory of the beneficiary companies. On the one hand, the evidence showed that the growth in spending per employee on innovation activities was higher thanks to FSAT. In addition, the growth in employment and total sales was more intense thanks to the program. No evidence was found on the trajectory of spending per employee on R&D activities and firm productivity. The first result could be reflecting the positive synergies of the public-private linkage promoted by FSAT. R&D activities were undertaken by academic research groups, which took advantage of the funding received to incorporate scholarship holders, train researchers, incorporate equipment and improve their infrastructure. This allowed companies to rely on the capabilities of these groups, and to undertake the rest of the innovation activities by increasing their investment effort. The second result may reflect the short time that has elapsed since the end of the FSAT. Previous evidence suggests that on average it takes 6 years to begin to observe impacts on the productive efficiency of companies participating in innovation support programs. Future research plans to conduct an evaluation that takes into account these possible dynamic effects.

ABSTRACT. Background and Rationale: The effects of policies to support innovation must be addressed in the context of the entire policy mix (Flanagan et al. 2011). Therefore, it is important to establish whether the collection of policies that may influence innovation activities do so in a coherent and constructive manner (Falk 2009). At the same time, the actual manner in which policies are designed, implemented and their governance context will also either facilitate or constrain the ability of such policies to deliver on expected goals. To address the needs of countries to improve their innovation policy approaches, World Bank has developed a public expenditure review framework (PER) for innovation policies (Correa 2014). This framework proposes and integrated approach to the science technology and innovation policy environment of a country. It consists of four stages, namely, policy mix analysis, functional analysis, efficiency analysis and effectiveness analysis. The first three stages have been fleshed out into detailed analytical methods and applied in several countries around the world providing useful insights for governments to improve their practices. This paper illustrates the application of the methodology with the specific case of Romania. Methods: In Romania, the policy mix analysis and functional analysis were conducted of the set of policies for supporting and enhancing innovation, especially in the context of its entrepreneurship ecosystem. Each one of these components of a PER have specific empirical strategies and instruments to carry out the analyses. The policy mix requires cataloguing all the policies and identifying a full range of features of each one. These include the institutional context of the policies, period the policies were active, their stated general and specific objectives, intervention mechanisms, intended beneficiaries, supported activities, budgets, including their origin, among others for a total of more than 100 possible items. The functional analysis uses a model of the policy intervention with 31 categories covering design, implementation and governance. For each category, an assessment of performance is coded on a Likert scale using criteria derived from a broad base of recommended practices found in scholarly journals, agency reports and consultations with experts. Data and Results: The data for the policy mix analysis is obtained from the government documents describing each policy instrument and government databases that contain budgets and other administrative information. Consultations with relevant ministries and agencies are often required to complete information needed for all the categories in the policy mix catalog. The functional analysis uses the results from the policy mix, documents obtained during the first phase and is complemented by in depth interviews with the relevant public officials who are responsible for managing the instruments in the analysis. The information on each policy instrument is used to address each of the categories in the model and assess on a likert scale the degree to which they reach best recommended practices in each area. In the case of Romania, the results of the policy mix analysis show that, first, the current portfolio of policies is highly concentrated, with significant resources devoted to a few instruments and a ‘long tail’ of much smaller instruments which are likely to be sub-scale. Second, most instruments have multiple top-level objectives, such as productivity improvement combined with job creation. This may suggest a lack of clarity in the design of instruments. Third, there are gaps in intermediate-level objectives, especially regarding improvement of management practices and linkages with foreign firms and market access. Fourth, there are gaps in support for entrepreneurs, especially regarding early-stage, pre-profit, startups and individual entrepreneurs - as well as the intermediary organizations which support such startups, such as incubators and accelerators. Some ‘entrepreneurship’ instruments may potentially reinforce the role of incumbents rather than encouraging the growth of innovative startups. Fifth, grants are the dominant intervention mechanism, which may be appropriate in a relatively under-developed innovation and entrepreneurship ecosystem, notwithstanding their potentially distortionary effect. However, it is important to remember that not every element of the ecosystem can necessarily be resolved through the addition of public money: regulatory reform is also needed and factors such as cultural perceptions of innovation and entrepreneurship are also important. In the case of the functional analysis, the results of the analysis show, first, that Instrument-level evaluation could be improved. This is currently stronger at the Priority Axis level of EU supported instruments. Improved evaluation at the instrument level would also support rationalization of the portfolio, by providing information about which instruments should be scaled-up. Second, instrument design is consistently weak in the articulation of a theory of change. Developing and articulating these models should enable better accounting of inputs and activities by managing authorities, as well as improved targeting of the instrument. It should also prompt ideas for alternative instruments and mechanisms, which would assist with rationalization of the portfolio and the development of a wider range of mechanisms. Third, administrative burdens for many beneficiaries are still high. There has been a significant effort by managing bodies to simplify the application processes for support scheme and reduce the bureaucracy for applicants. However, there are still indications that these processes remain overly complex for applicants and could be further simplified. Fifth, bureaucratic burdens for administrators are also significant. In particular, multiple auditing of instruments appears to impose a significant burden on administrative teams. Bureaucratic friction is likely to be increased by the structure of many instruments which have both a Managing Authority and a separate Intermediate Body; this may be a necessary condition of some European funding but is unlikely to be optimal.

References Correa, P. (2014). Assessing Public Expenditures on Science, Technology and Innovation: A Guidance Note. World Bank Group. Falk, R., 2009. The Coherence of the Instrument Mix. Report Nr. 8 in the context of the Study: Evaluation of Government Funding in RTDI from a Systems Perspective in Austria. : Vienna. Flanagan, K., Uyarra, E., Laranja, M. , 2011. Reconceptualising the ‘policy mix’ for innovation. Research Policy, 40 (5): 702-713.

ABSTRACT. 1.Background and rationale Turning scientific research into innovation is a considerable challenge in the medical field (O’Connell & Roblin). Current developments in scientific research have not been mirrored by the same level of progress in drug development (Pammolli et al., 2011), especially with regard to cancer diseases, the most dangerous type of non-communicable disease (World Health Organisation, 2017). Even though research on genetic alterations in human cancers has led to a better understanding of molecular drivers of cancer diseases, and this knowledge should provide more useful drugs, the effectiveness and success rate of cancer drug developments are remarkably low (Hutchinson & Kirk, 2011; Begley & Ellis, 2012). Previous investigations have also confirmed that most medical research organisations focus on publishing novel scientific research instead of developing new drugs (Venditto & Szoka, 2013). This state of affairs is the motivation for the present research to explore the knowledge transfer from scientific research to drug development in cancer diseases. In academia and industry, all parties support the idea that new drug developments rely on the improvement of scientific research. However, how scientific research can be transferred into drug development is still being discussed. Some scholars support that scientific organisations should participate in drug development directly, as their scientific research helps them better understand pre-clinical results and match the patient conditions with in vitro tests (Van Dongen et al., 2017; Haeussler & Assmus, 2021). However, others believe publishing scientific research and developing drugs both require a great deal of effort, and that organisations do not have enough resources to cover both areas well (Du et al., 2021). Thus, this investigation will explore to what degree the basicness and the scientific impact of research influence organisations to engage in drug development and their effect on its success. Given the limitations of knowledge and resources, some organisations prefer to co-operate with others to develop drugs jointly rather than in isolation. Knowledge spillovers can be generated via co-operation activities (Hájek & Stejskal, 2018; De Noni et al., 2018). According to Smith (1994), the definition of knowledge spillover is the process of knowledge transfers from producers (knowledge sources) to users (knowledge receivers) through sharing, interaction, and the exchange of knowledge. In a co-operation network, knowledge spillover is always produced as a phenomenon in which the existing research efforts of co-operators may allow a given organisation to achieve results involving less research effort on their part than they would otherwise require (Jaffe, 1986). In this paper, research co-operation networks are built to observe and analyse the knowledge spillover through co-operative relationships. However, the efficiency of knowledge spillover is different with regard to basic and to applied knowledge, and it is still not clear whether the scientific impact of research can effectively spill over through the co-operation network. Thus, this paper also explore the spillover effects of basicness and the scientific impact of research on the success of drug development in the co-operation network 2. Method Since cancer diseases are very dangerous and the efficiency and success rate of cancer drug development are remarkably low, it was decided to choose two typical cancer drugs for the sample: alkylating and immunological cancer drugs. Variables were built with reference to the process of drug development through information from publications, clinical trials and FDA-approved drug products in oncology. Publications data was collected from PubMed, since it is the optimal publications database in the biomedical field (Falagas et al., 2007). The U.S. Clinical Trials Registry was chosen due to collect clinical trials data. For the purposes of this study, only the clinical trials from 2005 to 2018 were considered. The final database contains 104 drugs, with 250,257 publications and 14,345 clinical trials. After matching publications and clinical trials, there were 29,723 organisations with publications, of whom 832 had develop clinical trials in alkylating cancer drugs and (or) immunological cancer drugs. The co-operation networks were built according to the records of “Sponsor/Collaborators” in ClinicalTrials.gov. There are 591 organisations in clinical trials co-operation networks. There are two dependent variables, Engagement of drug development and Success of drug development. For a given publishing organisation, Engagement of drug development means the organisations decide to develop clinical trials. This takes a value of 1 if that organisation develops clinical trials, and 0 otherwise. The second dependent variable is Success of drug development, which is calculated by the number of FDA approved drugs or Phase IV clinical trials drugs. The first independent variable is Basicness. This is calculated according to the Triangle of Biomedicine, which maps PubMed papers onto a graph to determine the basicness of the organisations’ scientific research (Weber, 2013). The basicness of a given organisation is measured by the average of the basic scores of their publications. A higher score means it is more basic. The second independent variable is Impact. The scientific impact of an given organisation is calculated by the average of the Relative Citation Ratio (RCR) of publications. Other factors which influence the engagement and success of drug development are monitored: Publications are used to reflect the number of scientific research articles; Organisation Type to reflect the types of organisations; Country to reflect the location of organisations; Drug, Clinical Trials, Participants and Biomedical Percent to reflect the factors of clinical trials; Degree Centrality and Betweenness Centrality to reflect the characteristics of co-operation network; and NIH, Other U.S. Fed, Industry and Other Funding to reflect the funding sources of organisations in clinical trials development. Since the dependent variable, Engagement of drug development, is a dummy variable and there are only a few values equal to 1, rare events logistic regression was run to test the effect of basicness and scientific impact on the engagement of drug development. The zero-inflated negative binomial regression and spatial Durbin model were used to test the direct and spillover effect of basicness and scientific impact on the success of drug development. 3. Results The results show that although engagement in drug development is an important concern, only a few medical research organisations (less than 3%) actually engage in drug development. The lack of applied research is the reason behind this lack of engagement. Applied research fosters organisational engagement in drug development. In drug development process, basic research increases the success rate of drug development. The scientific impact of research not only stimulates the organisation into engaging in drug development, but also provides novel solutions to increase the success rate. In the co-operation network, applied research is easier to transfer and is more successfully exploited than basic research, thus the spillover effect of the basicness of scientific research is negative on the success of drug development. An efficient way to obtain frontier research from co-operators is through co-operation; thus, the spillover effect of the scientific impact of research is positive on the success of drug development. This investigation improves understanding about which scientific research — basic or applied — leads to engagement and the success of drug development in cancer diseases, and whether high-impact organisations also develop drugs. Methodologically, this study offers a new approach that overcomes the huge computational effort to empirically test these links, exploiting the spatial Durbin model to test the spillover effect in the co-operation network.

ABSTRACT. Background

New technologies, as they are applied and scaled-up, especially if they represent disruptive developments, invariably raise concerns about broader public implications, distributional effects, and governance. In recent years, attention has turned not just to the societal consequences of new technology deployment but also to the anticipation (and mitigation) of potential consequences in research, design, and early emergence phases. In parallel, frameworks for considering the implications of new technologies have evolved from expert technology assessment to more diverse (and arguably more encompassing) constructions such as public values mapping and responsible research and innovation.

Advances in analyzing scientific literature outputs and patent documents can now shed light on the emergence, significance, networks, and trajectories of rising topics in research, invention, and innovation. Building on this work, we add a new dimension by identifying and analyzing the public values expressed in patent documentation. We put forward an approach to recognize early signals as to the public values expressed in patents as a new technology emerges and scales. Resulting insights can contribute towards assessing the societal implications of technological development.

For empirical exploration, we focus on artificial intelligence (AI) - a domain that has received growing investment from the private and public sectors and has substantially increased its patenting activity in recent years. Science, technology, and innovation (STI) scholarship has drawn attention to ethical, privacy, bias, control, and distributional consequences of AI applications, as implemented by private sector organizations and governments. AI patenting studies have also examined relationships with business development, economic growth, regional innovation clusters or global innovation dynamics. However, the social sciences have yet to comprehensively probe the relationships between AI patenting and public values and how AI patents express public value claims.

Given this knowledge development opportunity, the paper addresses three research questions. First, what kinds of public values are embedded in AI-related patents? This involves delineation of the AI patenting domain, the conceptualization of public values, and the operationalization of an approach that can identify and contextualize public values within a patent text corpus. Second, how are public values expressed in patents distributed across time, geographies, and areas of AI applications? AI patenting has developed extensively over recent decades, spreading across countries, sectors, applications, and assignees. There is rising public debate about AI’s societal implications, with – varying by country and sector – policy interventions and the introduction of good practice codes. Public debate has also grown on sustainable development goals and global challenges. We anticipate that the public value rationales for AI inventions may in some ways evolve to reflect these public debates. Third, how do public values expressed in AI patenting speak to different dimensions of AI governance and policy? Here, we seek to distill insights and recommendations that can inform AI governance and policy, including implications for early anticipation and responsible innovation.

Methods

We use a peer-reviewed method combining a systematic keyword-based search with patent classification codes to select AI-related patents. The method is comprehensive in that it captures a range of approaches relevant to the domain, including deep learning, machine learning, supervised leaning, and other advanced techniques. We also conceptualize and operationalize an ontology of public values, building on available literature and prior work by research team members. To address the core research questions, we iteratively tune a BERT (Bidirectional Encoder Representations from Transformers) language model. This a machine learning method for natural language processing developed by Google and now in widespread use. We use BERT to classify USTPO AI patents (comprising patent applications and grants filed by US-based and non-US entities) according to whether and how public values are expressed. Text expressing public values can be found in patent background text, although it is also in other patent parts. An AI patent may express more than one set of public values, while not all AI patents express identifiable public values. A topic model identifies AI patents’ main public value themes (e.g., healthcare, privacy, discrimination, environment, etc.). Model performance is tested and iterated.

Results and significance

Research is advanced in identifying and classifying public values in US AI patents, drawing on patent records contained in PatentsView. We have developed a database of more than 100,000 US AI full-text patents records with application filing dates between 2005 and 2021. We have identified more than 4.6 million patent text sentences which potentially contain expressions of patent values. We anticipate completing the testing and refinement of the model by the end of 2022, with analysis of results in early 2023. The resulting consolidated database of public values expressed in US AI patents, along with other associated patent information (including filing year, award status, patent class, inventors and assignees, address information, forward and backward citations, and whether public funding is attributed) opens multiple opportunities for investigation. This includes addressing our core research questions related to what kinds of public values are expressed in AI patents and variations by time, sector, and inventor/assignee geographies, and policy implications. We will probe factors or motivations that might lead inventors to express or address public values in their inventions or to recognize social, legal, and ethical implications. One of these might be the presence of public funding; another could be the continuous v. discontinuous characteristics of the patent (using methods related to emergent word combinations), where we might expect the latter to be more associated with attention to public values.

While we anticipate that the approach presented in the paper will present a methodological advance, a key goal of the paper is also to advance frameworks for conceptualizing and operationalizing public values that may have broad applicability. While social scientific analysis of patents and patenting practices began to emerge in the 19th century, and began to significantly increase in the 1950s, much of the social sciences literature on patents is focused on their legal and organizational aspects, complemented by management and economics of innovation scholarship on business strategy and private economic value considerations. Nonetheless, the early 1970s did see the growth of articles looking at the implications of patents to society, with a focus on developing countries. It was argued that, until then, not enough was known regarding the distribution of benefits of patent systems. Critical concerns were raised about the benefits of patenting to the economies of developing countries. Social sciences scholarship on patenting grew more rapidly and diversified in focus from the 1990s, with recent increased interest in overlooked topics, such as the impacts of patents on human rights or gender inclusion. Scholarship more generally on public values in science and technology has also grown, although there has been only limited work on public values in AI patents. As well as providing responsible innovation and policy insights about how AI patents are expressing public values, we hope that the concepts and approach elaborated in the paper will stimulate debate and open new pathways for further work on the relationships between innovation in emerging technologies and public values.

ABSTRACT. Whether research funding is targetable is one of the central unresolved questions of science policy. A particular question is how often research aimed at understanding one disease or problem spills over to others. This has been a perennial topic of debate at the world’s largest single funding body of biomedical research, the U.S. National Institutes of Health (NIH). Critics of the agency’s priority-setting process have repeatedly called for better alignment between funding and disease burden, and patient advocates for specific diseases for more funding for their causes. In response, opponents of planning have argued that research in one area frequently leads to advances in others. In this paper, we provide new evidence to inform these debates by examining the extent to which research funding (grants) in one scientific or disease area leads to research findings (publications) in another. We used the NIH’s Research, Condition, and Disease Categorization (RCDC) to identify categories for NIH grants awarded between 2008 and 2016. We applied machine-learning to map text to these categories and use this model to categorize publications resulting from these grants. We categorized over 1.2 million publications, resulting from over 90,000 grants. We found that 70% of the publications have at least one RCDC category not in its grant, which we termed “unexpected” categories. On average, 40% of categories assigned to a publication were unexpected. After adjusting for similarity across some of the RCDC categories by empirically clustering the categories, we found 58% of the publications had at least one unexpected category and, on average, 33% of publication categories were unexpected. Our results suggest that disease-orientation and clinical research were less likely to be associated with spillovers. Grants resulting from targeted requests for applications were more likely to result in publications with unexpected categories, though the magnitude of the differences was relatively small.

ABSTRACT. Background and rationale

Agricultural innovation policies are defined as those policies that are designed to improve a country’s agricultural innovation capabilities. In most countries, agricultural innovation is usually managed by several government agencies, but principally by those in charge of agriculture or science, technology and innovation. While Science Technology and Innovation (STI) policies at the national level may primarily focus on the establishment of an enabling environment for innovation in the country, sectoral agricultural policies on the other hand may focus on encouraging and helping farmers to enhance their outputs.

From a developing countries point of view, agriculture has been recognised as a sector that may contribute to economic growth by creating jobs, food security, and promoting the preservation of bio-diversity and scarce natural resources. While agriculture plays a significant role in the socio-economic developments of these countries, the sector’s contribution to the economic growth in Sub-Saharan Africa has largely stagnated and in some cases, significantly decreased in recent years due to a myriad of factors and challenges including, high and rising input costs, lack of support, droughts caused by climate change, water scarcity, crop or livestock diseases, and global competition. For example, in South Africa, agriculture’s contribution to the GDP has decreased from a total share of 7.7% of gross domestic product (GDP) in 1969, to approximately 3% every year in the last ten years (DAS, 2012). According to the OECD, South Africa’s agriculture sector is among the least supported in the world. Moreover, the OECD estimates that South Africa’s Producer Support Estimate is currently 3,2%, versus 4,6% for Brazil, 7,1% for the US (OECD, 2021). A recent FAO report, maintains that agriculture and aquaculture sectors will need to innovate not just to enhance the efficiency with which inputs are converted into outputs, but also to conserve precious natural resources and minimize waste in order to adapt to future global issues (FAO, 2017 ).

Realising these problems, many sub-Saharan developing countries including South Africa, have moved swiftly to make a number of policy adjustments, notably in the areas of agriculture and innovation. For example, one of the key objectives and justification of South Africa’s innovation policy has been the boosting of declining economic growth through innovation and the increase of productivity as well as competitiveness. More specifically, the new government white paper on Science, Technology and Innovation (DSI, 2019) as well as the decadal plan both outline important innovation policy priorities which puts innovation at the centre of seeking solutions to South Africa’s main pressing challenges of poverty, rising inequality, unemployment and economic growth.

Spielman and Birner argue that innovation policies are typically classified into three types, namely: (a) policies aimed at establishing and strengthening the formal structures and institutions required to collect and apply new and existing knowledge; (b) policies that encourage and enable the development of new innovations among farmers and other stakeholders; and (c) policies that incorporate and coordinate governmental, private businesses, and other stakeholders that are involved in innovation processes. Given that agricultural innovation has grown more intertwined with national innovation priorities in recent years, it has naturally become vital to integrate it into national innovation programs. Improved national and sectoral policy coordination between agricultural and innovation policy priorities can help policymakers close gaps and focus on sectoral challenges that are usually overlooked in both national STI and agriculture policy circles.

Objectives

The objective of this study was to investigate how policy problems in national STI policies and sectoral agricultural policies have been defined in order to stimulate innovation in the agricultural sector, to enable agricultural businesses to participate in the innovation process. In addition, the study examined the assumptions behind the policy measures offered in the STI policies in order to identify asymmetries in agricultural and STI policy coordination. An efficiently coordinated agricultural innovation policy would ensure that policy makers avoid making inaccurate assumptions about important agricultural policy priorities which may prove to be costly, and may inadvertently result in inefficient policy instruments that do not address the fundamental needs of agricultural sector stakeholders. As a result, the purpose of this study is to try to bridge this knowledge gap.

Methods This study used the critical discourse problematization framework (CDPF), an approach which merges two complimentary techniques to policy analysis, namely: (a) the critical discourse analysis and (b) policy problematization approach to problematise agricultural innovation in existing innovation policies. Policy problematization can be defined as the process of breaking down specific policy issues into problems that need to be solved. This approach recognizes that all policies have inherent problems and suggests that these problems have implications for how citizens are treated by governments and how they are conditioned to understand the social world and their roles as citizens. The main goal of studying problematizations, therefore according to Bacchi, (2012, p. 2) “... is to dismantle taken-for-granted fixed essences and show how they have come to be”. While traditional policy analysis techniques use quantitative methods to seek for causal explanations for why policies succeed or fail, on the contrary, qualitative discourse approaches look to analyse policy in terms of how dominant narratives emerge and come to define as well as shape policy practices. As such, to investigate the research problem, the study took a qualitative research approach, while using secondary data sources including key copies of the national innovation policy documents, national development plans and agricultural policies. This method was commensurate with other policy analysis research studies that employed secondary data (see Bacchi, 2015; Makoza 2013; Midgley, 2013). The documents were sourced from the South African Government departments of Science and Innovation as well as the department of agricultural land reform and rural development (DLDR). The researchers purposefully selected materials that were (a) related to the national STI policy and national development (b) produced as part of policy development process. Additionally, the set of documents used for the analysis enabled to examine the discourses, practices, and contextual settings that shaped the development of the problems that the national innovation and agriculture policies aimed to solve (Bacchi, 2009; 2011; 2015).

Contributions and implications

This study contributes to the debates on agricultural innovation policy by shedding light on the importance of policy coordination between national STI policies and sectoral agricultural policies in developing countries. It uses the case of South Africa’s experiences with problematization of the role of STI policy in agricultural innovation to further show how problematization may be used to identify asymmetries in priorities and policy coordination issues that hinder agricultural innovation. These coordination issues may hinder the development of appropriate agricultural innovation policy instruments intended to stimulate innovation and address challenges faced by the agricultural sector.

ABSTRACT. Background and rationale The Covid-19 pandemic has shown the vulnerability of nations, global institutions, and humanity at large to cope with this disease. It has underscored the need for a Great Reset that calls for a revisit by nations, the global institutions of the premise and assumptions of the growth discourse, and the translations based on them. This need for a reseat has long been echoed by low and middle-income economies in their struggle for addressing developmental challenges such as health, and climate change, for achieving the target set for SDG goals, etc. The pandemic has shown the vulnerability of coping with the pandemic even in OECD economies and exposed various fault lines in these economies. A large consensus is emerging that contemporary growth models which underpin contemporary policy framing are not resilient enough to cope with the various shocks. New innovative approaches during the pandemic have played a major role in fighting the pandemic against adverse circumstances. They have challenged the dominant models of innovation which are centered on growth models. The paper is motivated by the issues raised above to look more closely into the research and innovation systems. Primarily the paper centers on to what extent the conceptual/theoretical framework that guides present policy discourse and implementation needs a revisit. The paper draws from these to make a more informed argument, suggestive pathways in the context of innovation and innovation policy Method The paper is exploratory research drawing from a large body of scholarly work across different strands of research within innovation and STS, and development studies. It has selectively drawn from a huge body of literature covering COVID-19 and past-pandemic discourse. The challenges of climate change and green growth, and issues of technology transfer therein, were also examined to see to what extent they echo the struggles and challenges faced in containing the pandemic. Results A ‘New Normal’ emerged for fighting the disease. Collective global response, local networks, frugal innovations, open sharing, exploring new daring pathways, novel public-private partnerships, etc. that in some ways the pandemic compelled to take, provided solutions and hope to mitigate the severe impact of this pandemic. Research became more OPEN with the global sharing of research findings, genome sequencing, data, tools, and techniques which gave leads for drug/vaccine development. Many innovations happened in the face of institutional voids and resource constraints (frugal innovations) that were characterized by being low-cost, affordable, and ‘know-how’ that is easily transferable and replicable demonstrating an alternative pathway to prevention and costly healthcare. Face masks, sanitizers, and PPE kits were made from reuse of existing material, make-shift hospitals, containment zones, low-cost oxygen cylinders, and ventilators that were not as sophisticated but could provide critical make-shift support that became life savers in the face of the weak health infrastructure. The development of Covid-19 vaccine/drug was more complex and difficult due to high levels of uncertainty surrounding the virus i.e. its effect on the human body, its rapid mutation, pathways it chose to enter and target different functions, etc. Inspite of these challenges, different treatment options, and vaccines were available within a year or so which in normal development takes between 10 to 15 years. This was possible due to various novel strategies and radically different approaches adopted. Repurposing Strategy “old drugs, new use”, Global coordinated among them, the WHO ‘Solidarity Trial’ and Access to Covid-19 tools (ACT) accelerator and new platforms for drug/vaccine development, the proactive regulatory system can be cited as key enablers in bringing new vaccines within the unimaginable time frame. Discussion and Conclusion The covid-19 exposed various fault lines. However, drawing from the stories of resilience, unprecedented response, and some extraordinary moves from countries to fight this pandemic have also suggested new ways forward. Many of these were transient measures to address the alarming challenges. Frugal solutions are more accurately felt now not only in Global South but also in the Global North. The pandemic also highlighted the need to move towards a more inclusive innovation system framework, which would be open to incorporating local expertise and informal knowledge systems beyond the actors and institutions of the formal economy. As spaces of frugal innovations vary widely and are of variable quality, their adoption and scalability call for innovation systems to strengthen these frugal innovations and create institutional mechanisms for supporting them. The innovation systems on the other hand need to see that adherence to formal standards should not lead to eroding the essential essence of these frugal innovations. Greater attention is also needed for other forms of innovation, such as user-led and open innovation (see e.g. von Hippel, 2005). Innovation Systems (IS) approach has given a more nuanced understanding of factors that leads to the failure of the research-innovation ecosystem that hinders the creation, translation, and diffusion of technologies. The attention to different forms of failures in IS approach is distinguished under ‘system failure’ (Woolthuis et al. 2005) and augments the ‘market-failure’ rationale for policy intervention. Infrastructure, institutional, network, and capability failure are the key domains covered under System failure. Many instances were observed wherein the STI ecosystem could not perform due to these failures. Many other types of failures were observed which have been pointed out by Weber and Rohracher (2012) namely Directionality, Demand articulation, Policy coordination, and Reflexivity failure that could be seen impeding the institutions effectively during the shock exhibited by the pandemic. Informal innovations have been largely neglected within the IS framework. Greater attention is also needed to other forms of innovation, such as user-led and open innovation (von Hippel, 2005). The post-pandemic transformation calls for rethinking development more broadly. Long-term transformative change in our industrial society needs to expand the criteria of “success” in economic development beyond growth and profit to include inclusiveness, equity, trust, and sustainability. The important role of public funding behind successful Covid-19 vaccine development has largely remained invisible (Cross et. al., 2021). This is also true of many vaccines and drugs in the market including blockbuster drugs. The voices that are suppressed when price reduction is demanded by activists, by low and middle-income economies in the access and equitable debate have high merits. Inflating the pricing of patented drugs which results in deadweight loss, costing of drug development which suppresses the huge public funding, equity and access are issues that are raised in these debates and calls for globally coordinated action plans and sensitivity of high-income economies to regulate the big drug MNCs which operate as a cartel to inflate prices. The covid-19 pandemic has shown the importance of ‘commons’ that calls for shared resources managed collectively in providing economic activity and livelihoods, mutuality, and building trust and solidarity. Amartya Sen (Sen, 1999) who had cautioned the uncritical acceptance of the notion of ‘growth identical to development’ makes one rethink the new policy framework which is just, equitable and inclusive and at the same time creates/promotes innovation and entrepreneurship.

ABSTRACT. Background and Significance “That’s not ethics! It’s innovation! And it’s dangerous!” These phrases, addressed by an engineer to a philosopher – me – have served as a kind of Zen koan, which means they call for a response. What was it that I, the philosopher, had said to provoke such a reaction? I had talked about going beyond merely following rules, codes, or laws and making ethical decisions rooted in one’s autonomy. How else could one decide, after all, if one did not own one’s decisions? Would simply following the rules not take the decision out of decision making? What sense of ‘innovation’ underlies the contrast with ethics, conceived as rule following? Must all innovation violate a rule or rules? And what danger comes with suggesting that people ought to make their own (and ownable) ethical decisions? Of course, one could make the wrong decision. Making wrong ethical decisions could be dangerous, too, in the sense that doing so could lead to other bad consequences. But making correct ethical decisions also often means facing danger. Whistleblowers, to choose but one example, often face reprisals, even if they do everything right. We could argue that anyone who punishes whistleblowers is themselves wrong for doing so; but that hardly removes the danger that accompanies whistleblowing. So, what is ethics? What is innovation? And what makes innovation so dangerous?

Methods This presentation addresses these questions by considering three cases.

1. NSF’s new TIP Directorate In 2022, the US National Science Foundation (NSF) established its first new directorate in 30 years – the Directorate for Technology, Innovation and Partnerships (TIP). According to NSF’s website, the point of the TIP Directorate is “maximizing NSF’s impact.”

"NSF has advanced the full spectrum of fundamental research and education in all fields of science, technology, engineering and mathematics, or STEM, for more than 70 years — from foundational, curiosity-driven research that has led to new knowledge about our world, to use-inspired, solution-oriented research that has directly impacted people's everyday lives. At every stage, investments across this spectrum have been deeply intertwined.

NSF's TIP Directorate doubles down on the agency's commitment to support use-inspired research and the translation of research results to the market and society. In doing so, the new directorate strengthens the intense interplay between foundational and use-inspired work, enhancing the full cycle of discovery and innovation." (https://beta.nsf.gov/tip/latest, accessed 10/31/22)

In what sense is the new innovation directorate itself an innovation? How might it be dangerous? Is NSF wrong to double down on impact?

2. The Ethics of Community-engaged Research NSF is increasingly recognizing that community-engaged research – itself arguably an innovation – raises ethical concerns (https://www.nsf.gov/geo/opp/arctic/ace/community.jsp, accessed 10/31/22). Although Indigenous communities may have their own policies governing the ethics of community-engaged research, and some countries (such as Brazil) have laws governing researchers’ engagement with Indigenous communities, there is no policy governing the ethics of community-engaged research as a whole (i.e., beyond Indigenous communities). To put the point differently, there is nothing resembling the Common Rule, which governs the ethics of human subjects research, for community-engaged research. This section of the presentation discusses an effort to fill this gap.

3. Incorporating Social Justice into Engineering Ethics Education This section of the presentation discusses an effort to integrate considerations of social justice into an engineering ethics class. In collaboration with an engineer, the author has re-designed an engineering ethics class around the idea of social justice to test whether doing so may allow us 1) to reorient the focus from professionalism to include social and cultural impacts; and 2) to integrate discussions of ethics and social justice in novel ways through the engineering context. For reasons to be addressed in the presentation, ‘ethics’ and ‘social justice’ are treated as separate subfields within the discipline of philosophy. Although it suggests an innovative approach, perhaps bringing these subfields together is dangerous (for the students in the classes, for the instructors, and for society once engineers educated in this way are unleashed on society). Although all three cases relate to the idea that innovation is dangerous, this last case is the one most obviously related to the fear that innovation in (or in place of) ethics is dangerous.

Results All of these cases represent nascent examples of the interplay between ethics, justice, and innovation. We have yet to see how any of them play out. For this reason, it is too early to report definitive results.

Conclusion To claim that innovation is dangerous and for that reason should not replace ethics misconstrues ethics, innovation, and danger. We should avoid the contrast between ‘safe’ ethics conceived as rule-following and ‘dangerous’ innovation conceived as rule-breaking. We should also avoid the contrast between ethics and innovation, since innovation within ethics is not only possible, but also desirable. Moreover, innovation can be ethical, as well as dangerous. In fact, we will be better off if innovations incorporate concern for ethics from the beginning.

ABSTRACT. Background and rationale Introduced in the year 2000, the European Research Area has become the overarching framework for research and innovation policy (R&I policy) in the European Union. In a nutshell, it aims to better harmonize national and EU-level R&I policy in order to avoid duplication and contribute to the formation of critical mass in research and innovation in Europe. Even if the support for the ERA framework has varied with the changing mandates of the European Commission, it was renewed and updated first in 2010 and most recently in 2020 with a view to the time horizon 2030. The future of R&I, of European R&I policy and of the European Research Area was also the focus of two forward-looking projects funded by the European Framework Programme for R&I between 2012 and 2014: RIF (Research and Innovation Futures 2030: From Explorative to Transformative Scenarios) and VERA (Visions of the European Research Area 2030). Both projects explored alternative scenarios of how practices, organizational forms and institutional frameworks of research and innovation might evolve up to a time horizon of 2030, and what implications these scenarios may entail for R&I policy. The focus of analysis differs somewhat between the two projects, with RIF putting more weight on the practice and organization of research and innovation in a global context, and VERA focusing on the practices, organization and governance of R&I at the European level, but both projects focused at the future of European R&I policy and the European Research Area in particular. In this paper, we analyse the findings of these two projects with a twofold intention. First of all, we want to compare in what regards the two projects cohere in terms of anticipating a departure from strongly held beliefs in what should characterize European R&I policy in the early 2010s, i.e. whether the projected future scenarios differ or resemble each other in terms of their underlying principles (what we call “key tenets”). Secondly, we compare the scenarios with what was actually decided in 2020 as the guiding orientations for the European Research Area in 2030, i.e. at the time horizon that the projects addressed.

Approach and method The methodology of the paper is based on four main elements. Starting point are the fundamental principles of European Research Areas policy (key tenets) in the early 2010s. As a second step, and using these key tenets as a reference point, a meta-analysis of the RIF and VERA scenarios was conducted in order to analyse whether the scenarios maintained or departed from these key tenets. The insights from both projects are used in combination in order to have a more reliable and comprehensive basis of emerging ideas in relation to the future of R&I (policy) in Europe. The third step consists of analyzing the most recent update of ERA policy targeting a time horizon of 2030, and again focused on the five key tenets of ERA policy (and on possible new ones). Finally, by comparing the findings of RIF and VERA with those of the actual evolution of ERA policy, we are able to assess whether the two foresight projects were able to anticipate the actual evolution of ERA policy as part of the portfolio of scenarios developed.

Results The first result of our approach consists of the identification of fives key tenets of ERA policy in the early 2010s: • Key tenet 1 “Scientific excellence is a central pillar on which European RTI policy builds and from which major long-term benefits are expected.” • Key tenet 2 “Public funding of basic and frontier research is justified by market failure arguments, and is not questioned.” • Key tenet 3 “(Academic) scientific knowledge is claiming primacy over other forms of knowledge production.” • Key tenet 4 “The integration and/or coordination of resources at European level are a pre-condition for effective and efficient ways of organizing research by avoiding duplication of efforts, concentrating on harmonized roadmaps, and ensuring critical mass.” • Key tenet 5 “The main purpose of R&I is to create jobs and growth (in the old industrial economy).” The analysis of the five (RIF), respectively four (VERA) scenarios indicate first of all, that the emerging changes and weak signals in the ways of doing and organizing research and innovation point to the possibility of much more transformative changes ahead. We conclude that we should not take current wisdom on the fundamental beliefs in what makes up good R&I at face value, but continuously explore and question whether these beliefs are still valid. Secondly, RIF and VERA are quite coherent in terms of the projections and underlying principles of what might characterise R&I in 2030. The scenarios from the two projects do not provide identical views on how the five key tenets might change in the future, but the majority of projected future features are similar. Moreover, three pairs of scenarios feature similar characteristics and implications. As the projects drew on similar conceptual foundations and applied different, but mutually compatible methods for scenario-building, finding this congruence of internal scenario logics supports the overall consistency and validity of the two scenario sets. As regards the most recent revision of ERA Policy 2030, several of its novel elements were already identified in at least one of the two foresight projects. This also underlines the usefulness of foresight in anticipating sensible futures, even if they were regarded as “unrealistic” at the time of development. We also look into how the results of the RIF and VERA were taken into account in the design of ERA policy 2030, which would support the ambition of the ERA Policy 2030. In parallel, looking at the fundamental principles ("key tenets"), which remained untouched in the policy development so far, can give focus to policy discussions today. These principles should be revisited whether they are indeed still "things we can take for granted" or would need political attention and policy change as well.

Significance of findings The paper provides two significant insights into the anticipatory quality of the scenario development approaches used: (1) Scenario approaches with scenarios co-created by stakeholders and experts are suitable to develop disruptive future projections that question fundamental beliefs (i.e. they can go well beyond incremental change); (2) While disruptive projections in scenarios shall serve at the time of scenario-making mainly as tools to challenge current thinking and implicit assumptions, and thus do not need to be realistic, from a hindsight perspective as we take it here, it is relevant to know how future projections have materialized in order to assess the further ambitions of policy development and identify open issues that still should be on the political menu.

ABSTRACT. Background and rationale

Tackling grand societal challenges requires new conceptual foundations encouraging a debate on the need for modified innovation policy approaches. While the discussion on mission-oriented innovation policy originated as an observation of an empirical phenomenon, only a few theoretical underpinnings have been undertaken so far. Some research focuses on the definition and operationalisation of missions and objectives of policy-making. Other scholars, including our previous research, attempt to improve the analysis of policy instrument mixes and their agile coordination, implementation and adaption in the course of dynamic innovation processes. Missions and associated failures are often rather broadly conceptualised and have only limited insights into the instrument design processes of political practitioners. A practice-oriented conceptualisation of types of innovation policies needs to encompass more analytical dimensions of importance for the policy design. Such policy intervention points are particular areas in the socio-technical system or its environment along with the dynamics of transformation processes (such as the stimulation of niches or regime destabilisation). At these points, suitable policies are supposed to facilitate or support transformative change (Kanger et al., 2020). However, a situation analysis is needed for innovation policy practitioners, which requires identifying and prioritising technologies, relevant target groups, and specific functional strengths (or problems). In that way, the strategic directionalities and the relevant operative knowledge are provided as a basis for the instrument design processes and policy mix orchestration.

Our previous research on the agility of innovation policy has shown that successful instrument selection and design processes depend highly on clear and operationalized strategic objectives (Weber et al., 2021). To guarantee the practicability of recommended policy instruments, policies need to be developed in the context of existing policy mixes and to include the perspective of polity (Which innovation policy actors and structures of the innovation policy system are concerned?) and the required dynamic adjustment processes of policy instruments (politics) to guarantee necessary adaptions to the dynamics of the addressed innovation processes.

Research questions and objectives

The proposed research is developed within the "Innovation Policy Orchestra" project funded by the German Ministry for Research and Education. It aims at considering the complexity of design features and implementation practices of innovation policy instruments, including the role of the implementing policy actors and the dynamics of policy processes for adjustment and orchestration of instruments in the course of the addressed innovation processes. The project addresses the following research questions: • What kinds of analytical dimensions does the literature discuss for a situation analysis of policy intervention points in innovation systems, processes and mission-oriented socio-technical transformation? • What kinds of policy instruments and their specific design features can be identified to develop practical policy recommendations addressing the results of the situation analysis? • How can the situation and the instrument analysis be combined into a morphological design process that includes the orchestration of existing and new policy instruments? The objectives of the research project are two-folded. For the practice of innovation policy, a comprehensive morphological policy design process shall be developed, such that innovation policy designers, for example, in ministries and executing agencies, have a viable method to operationalise innovation policy strategies and the instrument selection and design. For innovation policy research, developing a holistic innovation policy design process helps to understand the practical challenges of innovation policymakers, which have been discussed as a crucial research gap in the literature. Existing typologies of innovation policy instruments, such as supply- and demand-sided instruments or technology-specific and neutral instruments, provide only limited insights on the instrument selection, as not all possible instruments and design principles are included. Through the in-depth discussion on instruments and their impact mechanisms, we aim to develop a holistic overview of instruments and design features to improve policy design processes for scientific policy recommendations and for the practitioners of innovation policy-making. Though new trends, such as the mission orientation, have been discussed in the literature, there is still a lack of practicable policy recommendations to implement an agile mission-oriented innovation policy. However, such policy recommendations require considering the specific national contexts of the innovation policy system. Therefore, the objectives of the proposed research project focus on the case of Germany. To share experiences and discuss the relevance of our research for other countries, we are motivated to present our research at your conference.

Methods

The proposed project follows a qualitative case study research design. In the first step, two structured literature reviews were conducted to collect the criteria for the situation analysis and the possible instruments and design features for the instrument analysis. As the literature reviews have demonstrated important research gaps, notably for polity and politics, our research project uses semi-structured expert interviews to consider the practitioners' perspectives on innovation policy. We are currently conducting interviews with the German Ministries for Research and Education, Economic Affairs and Climate Protection and executing agencies. Together with relevant policy documents, the interviews are transcribed and coded in an iterative process such that evidence contributing to the research questions can be structured and added to the morphological design process.

Preliminary results and significance

For structuring the morphological design process, we follow Zwicky (1948) and others using matrices as a creativity tool to analyse and solve complex problems. The morphological approach facilitates decision-making by providing relevant analytical dimensions and their specifications. For example, the analytical dimension of "target group" is subject to different specifications, such as SMEs or research institutes, which can be selected to know what kind(s) of target groups are relevant to address by policy-making. Two matrices have been developed, one for the situation analysis and one for the instrument analysis. The interviews aim at collecting the tacit knowledge and experiences from policy practitioners in order to get to know tacit and unknown design criteria, practical decision rationales and aspects from the policy design and implementation process itself. This new empirical perspective adds to the existing conceptual debates on operationalising mission-oriented innovation policy-making. As a result, our instrument analysis considers the entire scope of innovation policy instruments and design features by assigning the different kinds of possible policy inputs (financial, guidance, information) to specific activities to be addressed by the instruments (physical installations/infrastructures, knowledge generation, capacity building, market formation, innovation system building). Both dimensions for the assignment of instruments are subject to specific design features, which we have collected from the literature and the qualitative interviews. The design features include relevant aspects of polity (Who of the innovation policy system is deciding and who is implementing an instrument? What kind of political structures and existing rules and regulations need to be considered?) and of politics (How are learning cycles based on monitoring and evaluation processes institutionalised?). We aim to finalise the interview analysis and the matrices in the spring of next year to present the results at the conference.

References

Kanger et al. (2020): Six policy intervention points for sustainability transitions: A conceptual framework and a systematic literature review. Research Policy 49(7), 104072. Weber et al. (2021): Agilität in der F&I-Politik. Konzept, Definition, Operationalisierung. Study on the German Innovation System 8-21 for the German Expert Commission for Research and Innovation. Zwicky (1948): Morphological astronomy. The observatory 68, 121-143.

ABSTRACT. Under-representation of women in science has been a longstanding phenomenon (Bayer & Astin, 1975; Etzkowitz et al., 2000; Larivière et al., 2013; Long & Fox, 1995; Preston, 2004; Zuckerman & Cole, 1975). Not only do women face more structural obstacles when entering science (Cimpian et al., 2020; Leslie et al., 2015; Reuben et al., 2014; Zuckerman & Cole, 1975), but they are also more likely to leave science and pursue non-scientific careers (Etzkowitz et al., 2000; Huang et al., 2020; Preston, 2004).

Prior studies examine gender inequality in science in terms of non-universalistic evaluation standards (Jappelli et al., 2017; Long et al., 1993; Long & Fox, 1995) or training opportunities (Moss-Racusin et al., 2012; Sheltzer & Smith, 2014; Stockard et al., 2021), family obligation (Cech & Blair-Loy, 2019; Fox et al., 2011), unequal positions in social network (Fox, 2006; Leahey, 2007; Xie & Shauman, 1998), and returns to their accomplishments (Ghiasi et al., 2015; Hofstra et al., 2020; Lerchenmueller & Sorenson, 2018; Reskin, 1976; Van der Lee & Ellemers, 2015).

Although those prior studies point out important factors that are attributed to perpetuating gender inequality in science, they often view gender inequality among scientific workforce in isolation from the changing nature of scientific work. As science is increasingly becoming a team-based, collaborative activity and the size of teams gets larger and larger (Adams et al., 2005; Wuchty et al., 2007), scientific work becomes increasingly bureaucratized, which is characterized by hierarchy, standardization, and division of labor (Pugh et al., 1968; Walsh & Lee, 2015; Weber, 1978). In particular, division of labor generates specialization in supporting roles (Hackett, 1990; Hagstrom, 1964; Walsh & Lee, 2015). At the same time, the increasing demands for productivity generate a growing need for supporting scientists whose primary role is supporting the leading scientists of projects or labs (Lee & Walsh, 2022). Therefore, the changing nature of science has been generating more and more supporting scientists whose work is critical to the production of science but who holds a more vulnerable status with unstable career paths than the leading scientists, e.g., academic principal investigators (PIs) (Lee & Walsh, 2022; Walsh & Lee, 2022). Milojević et al. (2018) show that a growing fraction of researchers spend their careers only in supporting roles, measured by those who have never been lead authors in their publications, and that these supporting scientists tend to have shorter careers than lead scientists.

This change in the composition of the scientific workforce may also exacerbate gender inequality in science. If those who have a more vulnerable status may get pushed into more vulnerable positions, female scientists may be more likely to be supporting scientists, and further have higher attribution than male scientists. Therefore, in this study, we empirically explore gender and attrition in science in terms of different roles in research collaboration, i.e., leading scientist vs. supporting scientist.

Using data on publishing careers from 1971 to 2012 from selected natural and social science fields (2,268,176 scientists in total), we find that women were historically more likely to spend their careers as supporting authors. However, we find this gap has been converging over time and recently flipped in natural sciences. Interestingly, this convergence has been overwhelmingly driven by a decreasing share of lead authors among men. Furthermore, statistical analysis using survival models shows that both women and supporting authors are more likely to leave publishing careers. However, among women, the supporting role has an especially high exit risk in social sciences, while the lead role has an especially high exit risk in natural sciences. In other words, female supporting authors are relatively worse off in social science than in natural sciences. We argue that natural science fields have provided a more rationalized and standardized career path for supporting scientists than social science, which is relatively more beneficial for the group that has traditionally a lower status, i.e., female supporting scientists.

We will further examine the primary tasks performed by male and female supporting scientists and other characteristics such as their novelty at the beginning of their careers and see how these may explain different representation and different rates of attrition. The results of this study will help us understand gender inequality in science in the context of the changing nature of science. Future research can examine how traditional drivers of gender inequality such as non-universalistic evaluation standards, training opportunities, and family obligation can explain representation and attrition in science between male and female scientists in different roles.

ABSTRACT. Science is a risky business by nature. Scientists explore and cultivate uncharted space of knowledge through trials and errors, in which their original ideas are often rejected and expected goals are not fulfilled for various reasons (Franzoni and Stephan, 2021; Machado, 2021; OECD, 2021; Reinhilde et al., 2022; Wang et al., 2019). Such risk and uncertainty tend to be especially high when scientists aim at novel discoveries, which have the potential to open up new avenues and make substantial advancement (Bourdieu, 1975; Hagstrom, 1974; Kuhn, 1970; Merton, 1973). Thus, there is a growing concern over scientists' risk-averse behavioral patterns, and science communities and policymakers emphasize that efforts should be made to facilitate high-risk-high-return research (Franzoni and Stephan, 2021; Gewin, 2012; Machado, 2021; OECD, 2021). Despite its fundamental role, risk and uncertainty in science have been poorly understood (Althaus, 2005; Aven, 2011; Franzoni and Stephan, 2021; Hansson, 2018). Thus, this study aims to contribute to advancing our understanding in scientists' risk-taking behavior. In particular, we investigate (1) how risk-taking is socialized in the academic training context, being transferred from a generation to the next generation, and (2) how risk-taking behavior is translated into research output.

To achieve the goal, we carried out a questionnaire survey with a sample of PhD students in life sciences in Sweden. We collected responses from 1,200 current PhD students concerning their research experiences, lab environment, risk attitudes, and their supervisors. We also collected information as to their supervisors based on secondary data. Finally, we collected bibliometric data for students and their supervisors. With the bibliometric information, we computed several features of research output from both generations. Among others, we developed a bibliometric measure for "riskiness", which we validated against a self-reported perceived risk. This is to quantify the risk-taking behaviors of both students and supervisors. We also highlight the novelty of research output (Wang et al. 2017; Shibayama et al. 2021) to investigate whether novel discoveries are facilitated by risk-taking behavior.

With the data complied, we first analyze how the risk attitudes of the younger generation (i.e., students) is shaped. We test the impact of various explanatory variables concerning the research training environment, including their supervisor's risk attitudes. In particular, we investigate in what situation supervisors' risk attitudes are likely to be transferred to their students. Second, we investigate how student's risk attitudes are associated with the nature of their research output. In particular, we examine whether risk-taking is associated with an increasing chance of novel discoveries. Based on the empirical findings, we discuss policy implications concerning funding, career development, etc.

ABSTRACT. This paper investigates gender bias (if any) when teams are formed. Data from the European Science Foundation are used to estimate if female scientists have the same opportunities as their male colleagues to join a team when applying for funds. To assess gender bias, we construct a control group of scientists with the competencies for being invited to join the team by the project leader, the researcher in charge of forming the team for the grant application. By comparing the proportion of female scientists in the control group with the one in the observed teams, we find a gender bias against female scientists only when the project leader is a man, while we do not observe any gender bias when the project leader is a woman.

ABSTRACT. Evaluation of proposals for allocation of research funding is done mainly through peer review. Some research concludes that there is gender bias in research funding; others sustain that gender differences exist but bias evidence is elusive and findings are contradictory. Despite some conceptual ambiguities, the debate revolves around to what extent the (potential) existence of gender bias in research assessment is causing gender differences in outcomes.

Evidence in observational approaches is either based on outcome distributions (e.g. success rates by applicants’ gender) or at best on modelling bias as the residual, once all relevant variables are considered. In observational research causal claims are usually mixed with simple statistical associations.

The standard problems of selection bias, confounding, lurking or unobserved heterogeneity have not always been properly addressed in previous observational research on funding research and peer review. Research addressing the links between research funding and gender bias has often faced problems of validity, both external validity (as it usually referred to a single country, funding agency or funding instrument) and internal validity, more related to the research design and the causality approach.

The funding organizations usually define the criteria for quality or merit assessment and their weighting. If gender bias exists in research funding, it would be the outcome of the reviewers’ assessment, under the evaluative framework set by the funding agency.

Trying to overcome validity shortcomings, in this paper we use an experimental design in which, instead of trying to identify the causes of observed effects (observational approaches strategies), we aim to measure the effects of a cause, the treatment effect of the gender of the principal investigator (PI) in a research funding proposal score.

We embedded a hypothetical research proposal description in a field experiment and with the research design we addressed some of the most important challenges of the “experimental approaches” that help to avoid bias estimates: 1) authenticity of the treatments: whether the treatment used in the study resembles the intervention of interest in the real word; 2) the realism of participants: whether the participants in the experiment resemble the actors that usually participate in this type of process; 3) the genuineness of the context: whether the context within which subjects are receiving the treatment resembles the context of interest, and 4) the truth of the outcome measures: whether the outcome measures resemble the actual outcome of theoretical or practical interest.

Our subjects were the reviewers selected by a funding agency, and the scoring task was done using the same criteria, framing and weighting of the call, additionally it was implemented simultaneously to the agency peer review assessment.

We manipulated the item in the proposal, which described the gender of the PI, with two designations: female PI and male PI. Treatment was randomly allocated with block assignment and response rate was 100% of the population, avoiding problems of biased estimates in pooled data.

Our results show that proposals led by female PIs received a lower mean score than the ones from male PIs, but the differences were not statistically significant. The origin of the small difference was that female reviewers assessed female PI proposals less favorably than male PI proposals.

Contrary to previous research, we find no evidence that male or female PIs received significantly different scores, nor did we find evidence of same-gender preferences of reviewers regarding the applicants’ gender. Thus, we cannot reject the null hypothesis that PI gender has no significant effects on scoring, nor do we find support for the matching hypothesis between applicant and reviewer gender.

ABSTRACT. Technology Sovereignty: empirical implementations of a conceptual framework

Background The global economy is in crisis mode due to the pandemic, the Ukraine war and the energy crisis. Even before that, an ever growing shift in the global economic focus already begun in the early 2000s, particularly in the area of research- and knowledge-intensive products as a result not only of the globalization of markets, but also the globalization of knowledge and supply chains. Supply chains could no longer be regarded as secure and dependencies were questioned. Protectionist tendencies in numerous countries, a politicization of the global economy - not least as a result of a newly evoked system competition between China and the "Global West" - as well as a specialization of individual countries resulting, among other things, from cost considerations, which in turn has led to major dependencies in individual technologies, had already presented many companies with major challenges before the current crises. The multitude of challenges at the business level aggregate into challenges and starting points for innovation and economic policy action at the macroeconomic level or at the level of sectors. The question of technological sovereignty - defined as the preservation of options for action based on one's own competencies and stable networks - is of particular importance against the backdrop of the above-mentioned challenges. However, internationally integrated supply, production and ultimately innovation chains have created dependencies and links which, against the backdrop of the economic and political challenges mentioned above, can restrict or even jeopardize secure innovation processes and thus competitiveness. Changing markets and new competitors also have an impact on both technological sovereignty and competitiveness, and thus ultimately also on most countries' security of supply.

Concept With its policy paper , Fraunhofer ISI had suggested a concept for Technology Sovereignty (TS) and a first empirical implementation for its monitoring. That paper defines Technology Sovereignty as the freedom to choose between different options instead of being directly or indirectly single-dependent. The definition makes a strict distinction to autarky and protectionism. It stresses the need for collaboration and exchange as well as knowledge and competence building for securing a country's technological sovereignty. In a second - scientific - paper , we offered a theoretical underpinning derived from economics and sociology, when we emphasized Technology Sovereignty as being defined by agency at the state-level (government action). This makes TS subject to policy action, required to be taken by government, industry and science in a country alike. The theoretical and the first empirical implementations of the concept revealed, however, a need for fine-grained (at the technological level) demarcations of crucial inputs (natural resources, pre-products, system components) at the level of technologies or technological system. This paper aims to suggest an empirical approach that will, in addition, be implemented for a set of selected technologies, building on networks and collaboration structures of national economic/innovation systems. With reference to the basic concept for defining, measuring and evaluating technological sovereignty presented by Fraunhofer ISI, this paper compiles and evaluates key statistical indicators for describing and assessing technological sovereignty versus dependency.

Methods To be able to enter at the level of technologies, IPR data (patents, trademarks) are used as one of the main data sources. The development or stock of competencies in key technology fields is an essential building block in securing the technological sovereignty of nations. Hence, bibliometric analyses - i.e. not only the counting of scientific journal publications, but also the use of citations and thus an assessment of the visibility and relevance of corresponding contributions - provide an indication of the intensity and quality of knowledge generation. In addition, the feasibility of a selective analysis of trade data is examined. While patents can be used to implement the delimitation of specific technologies in a fairly targeted manner, it is generally necessary to switch to more highly aggregated product groups (e.g. 'vaccines' in their entirety) in the case of trade data. We develop a technology-specific set of networks at the level of companies and research institutions with the help of co-patents, co-publications, and trademarks. For this purpose, we will resort to technology definitions that can be demarcated for all these indicator dimensions. In a next step, we aggregate each of these networks to the level of countries. At this level, we will also take into account the ownership of companies from BvD Orbis as well as inventor versus assignee country of patents, so that a potential international command of national technological competences can be modelled as well. The resulting networks will be merged in a multilayer network analysis. This will be set at the technology or at least technology field level, allowing a focus on essential technologies. The goal is to determine the extent to which countries have control over and/or access to particular technologies. With this multi-layer network analysis, we hope to be able to work out who collaborates with whom (and who does not) and also what the ownership relationships are. Inputs (value chains) cannot be directly mapped in this way, but the linkages between countries (based on links of companies and research institutions) are able to show technological collaboration.

Results As the project is still in a very early stage, comprehensive results are not yet available. We expect this analysis, however, to provide better insights into the interdependencies and also the network density itself. If there are many unconnected players in a technology field, then there are at least potential alternatives in the global network - hence a larger freedom of choice and room for policy action. From bibliometric and patent data analyses we already learned that international collaboration and exchange has increased in the past years. These kind of analyses, however, have employed individual indicators only or analyzed them separately, while an integrated assessment of a country's Technology Sovereignty based on multiple dimensions has not been attempted in this way, to the best of our knowledge.

ABSTRACT. Background and rationale Public funding for Science, Technology and Innovation (STI) is generally assessed in terms of its pertinence for the successful termination of inventive outcomes. Nonetheless, its projected influence on other key factors that might also shape innovation performance across companies remains largely unexplored. These include the role played by such subsidies in stimulating likelihood to innovate and even in fostering a higher resilience to sort out those financial and non-financial obstacles usually encountered by firms along the road. The primary purpose of this paper is to fill such existing gap.

We therefore study firm-level elements configuring propensity to engage on innovation strategies (product and process innovation), together with the adaptability of these in the presence of barriers to innovate, both for the case of firms subject to public funding as well as for those that are not. Chile’s Innovation System is herein referred as a case study due to its recent noteworthy performance as one of Latin America’s inventive hubs. The 9th to 11th module of the country’s innovation survey (spanning the 2013-2018 period) constitute our main source of micro information.

Methods Our econometric strategy is based upon a Multinomial Logistic Regression analysis (MLR). Two type of model specifications are derived from such approach: a baseline model solely accounting for innovation probability determinants, and a second multinomial regression where each of these latter elements are individually interacted with a given obstacle. In line with the MLR method, our dependent variable (inno_outcome) is built by classifying manufacturing firms into four categories that signal the execution of different innovation alternatives: (1) firms exclusively practicing product innovation; (2) those following process innovation only; (3) companies being able to implement both (top inventive firms), and (4) those other without either strategy. This latter fourth group of companies (labeled as potential innovators) are included in the form of a baseline sample within each of our econometric specifications.

Individual expenditures on intramural and extramural R&D, the use of intellectual property rights, the total amount of labor employed by the establishment (proxy for size), cooperation agreements with other firms along with the presence of scientific ties with research institutions comprise the set of independent variables explaining probability to innovate. Obstacle-related MLR estimators are then generated by interacting each of these with the following barriers: financial difficulties (lack of funding, prohibitive costs to innovate), knowledge disincentives (scant information about the availability of highly skilled personnel and/or latest technologies), network restrictions (failure to cooperate with other economic units) and, market obstacles (prevailing uncertainties with regard to future demand, market being dominated by well-established producers, etc.). Given our research objective, this empirically strategy is applied for two types of firms (Chilean firms with public subsidy and those without).

General results Alleviating size-related limitations to innovate, strengthening the relevance of intramural R&D funding and, encouraging the signing of more cooperation agreements with other firms (in spite of the presence of monetary and non-monetary barriers) represent the main supplementary benefits that stem from access to government support. While extramural R&D and intellectual property protection are found as critical and generally resilient elements to increase likelihood to innovate for the average inventive firm, our research notes that publicly supported (less innovative) companies fail to also utilize these two as pivotal instruments for the completion of scientific outcome (their respective regression coefficients are non-significant). Surprisingly enough, possibility to innovate (for all type of firms) seems to be reduced by the presence of scientific ties with research institutions. Such negative impact is neither drastically deepened by the presence of innovation barriers nor improved by public funds.

Policy implications These general results have strong implications for those developing economies seeking to evaluate the effectiveness of their innovation policy beyond mere indicators of R&D investment and their resulting output. Our empirical evidence illustrates additional indirect effects that can originate from granting public resources to less innovative companies. Alleviating barriers to size, nurturing cooperation skills coupled with the advancement of superior abilities to efficiently exploit intramural R&D investment, irrespective of obstacles of any sort, encompass some of the extra benefits that can be induced by government funding. Areas of innovation policy enhancement are also here put forth. To properly rely on intellectual property protection, extramural R&D as well as on collaborative projects with research institutions, less innovative companies require additional and tailored specific instruments of government support. Public funding, alone, does not seem to have a profound effect on these latter set of factors across Chilean innovators.

ABSTRACT. Online platforms have become essential infrastructure for social, economic, and political activities. In doing so, gatekeeper (dominant) platforms started fulfilling functions that used to be the sole purview of the state, such as solving disputes and regulating content, “taxing” transactions, and regulating ever greater parts of the economy. Various states are responding according to their own sensibilities and internal pressures resulting in divergent national internet governance systems. We explore these evolving systems by introducing a framework that identifies three strategies through which governments respond to the infrastructural power of platforms: accommodative, defensive, and generative. We use our framework to make sense of the evolving nature of governance models in four key jurisdictions: the US, the EU, China and India, and to explore the general direction of change and its implications for the global digital economy and international trade. Ultimately, our goal is to use our analysis to raise intriguing new questions: Moving forward, will the previously largely open global market for platforms continue, or is a fracturing of the global market emerging? And what will be the major fault lines?

ABSTRACT. _Background and Rationale_: Although studies have examined the benefits of digital transformation and the effectiveness of certain policy instruments for digitization, the policy implementation capabilities of public agencies have received little attention despite their critical importance in the effective delivery of support programs. Using a new framework developed by the World Bank for assessing the organizational effectiveness of innovation agencies, this research aims to address this knowledge gap with a case study of a Korean implementation agency tasked with supporting firms' adoption of digital technologies. This study analyzes the effectiveness of the Korean agency in four key dimensions–delivery of goals, stakeholder satisfaction, processes and functionality, and resources–of organizational effectiveness and draws lessons in each of the dimensions, as well as from a systemic point of view. The findings show that phased efforts that start with a focus on building critical mass across the digital economy and that are based on existing capabilities are key to improving the effectiveness of public agencies supporting firms' digitization.

_Methods_: From the experience of assessing the policy mix and functionality of science, technology and innovation (STI) policies in developing and middle-income countries, it became clear that there was a need to develop a methodology focused specifically on the features of the agencies that played a key role in various phases of these policies, ranging from design through implementation and governance. Especially important was the need to gauge their effectiveness in fulfilling their mission and goals. With this a novel analytical framework was developed inspired in the organizational effectiveness approach from Quinn and Rohrbaugh (1981) adapted to the specifics of STI policy agencies. It takes into account recent reports on observed features of these agencies by NESTA (Glennie & Bound 2016) and the World Bank (Aridi and Kapil 2019). The analytical framework develops the framework of organizational effectiveness into a model with specific categories relevant to STI agencies that guide an empirical strategy for assessing their effectiveness.

_Data and Results_: The project gathered data from available agency reports and interviews with relevant agency personnel. Reports were used dating back to 2104 regarding budgets, mission transitions, project catalogs, statements of objectives and goals and evaluations of beneficiary satisfaction. Several rounds of interviews were conducted and followed up with written questions which received written responses and supplementary information on the available reports. Two sorts of results were of interest in this project: First, an empirically grounded assessment of the effectiveness of NIPA as a digitization innovation agency in Korea was sought. Second, the project aimed to establish whether the proposed analytical framework is useful for determining the profile and effectiveness of innovation agencies generally in other countries.

On the first result, the project found that NIPA has a very specific profile that is not found in other innovation agencies generally. It is an implementation and services organization that maintains close connection with the beneficiaries of the programs it implements and manages. It takes significant feedback directly from beneficiaries to adapt its approaches to better serve them and caters to their perception of what is most important and useful. Understanding the profile of the organization in this way, in its tightly constrained policy context, it can be said that it is very effective in its role. However, it is a limited role lacking many of the functions and capabilities that would be required for an organization that would be expected to have greater responsibility in broader technology policy strategies. The broader role of innovation policies is distributed across policy domains under the purview of several ministries giving implementation agencies a very narrowly focused role closely tied to their beneficiaries’ satisfaction.

The second conclusion is that method is useful to determine the profile and effectiveness of innovation agencies generally. Even though there were some limitations in the availability of information, we were still able to determine the profile of the agency and get a sense of its effectiveness and how it may be considered as a model in other contexts. We were able to conduct interviews but were not able to run surveys or other more intrusive approaches to information gathering. Future use of the method would require fuller access to take advantage of its potential. Interestingly, the approach seems sufficiently general and, at the same time, detailed enough, that it did not require much ex-ante knowledge of the organization to arrive at the key findings. It seems the method is flexible enough for use in almost any government context if enough information is made available.

References: Aridi, Answer and Natasha Kapil. (2019). Innovation Agencies: Cases from Developing Countries. World Bank Group. Glennie, Alex and Kirsten Bound. (2016). How Innovation Agencies Work: International Lessons to Inspire and Inform National Strategies. NESTA, UK. Quinn, Robert E. and John Rohrbaugh. (1981). “A Symposium on the Competing Values Approach to Organizational Effectiveness.” Public Productivity Review 5, 2: 122-140

ABSTRACT. Background and rationale Evaluative processes struggle with the notion of diversity of a scientific workforce (Walsh et al., 2019). Despite overwhelming evidence on the need for diverse teams in terms of division of labor (Robinson-Garcia et al., 2020), ethnic mix (Freeman & Huang, 2015) or gender (Díaz-García et al., 2013; Maddi & Gingras, 2021) among others, bibliometric methods have traditionally been focused on the development of impact and productivity indicators. Recently, the development of machine learning algorithms, new data sources and strong calls for action favoring an open and diverse scientific ecosystem, have given room to a stream of studies focused on studying different aspects of career trajectories, diversity of profiles or biases in science. In this paper we attempt at reviewing recent advances in the development of novel informetric approaches and methods to study diversity in science. Specifically, we focus on those related to the scientific workforce.

This focus on diversity of the scientific workforce differentiates this work from other reviews focused on individual level indicators (Gauffriau, 2020; Wildgaard et al., 2014), as we deliberately ignore productivity or impact indicators to focus on methods and approaches aimed at characterizing individuals and the context in which they work. For this, we first review the changes with regard to data sources which have made possible such approaches. Here we focus on two specific aspects: 1) the expansion of author identifiers, and 2) the improvements made with regard to author name disambiguation algorithms. Then we revise the different proposals made by organizing these approaches into three groups: 1) informetric methods related to individuals’ characteristics (e..g, gender, age, mobility); 2) methods related to individuals’ context (e.g., career trajectory, social engagement); and 3) approaches related to team dynamics (e.g., author position, contribution statements). We then conclude by identifying the main gaps in the literature and pointing towards potential areas to explore. Data sources The development and expansion of individual level-metrics is closely linked to that of data sources. Their inclusion of metrics, the launch of author profiles and the improvement of the bibliographic metadata have expanded both the popularity of certain metrics and the possibilities for quantitatively studying individuals’ academic activities. Here we review the main milestones on how data providers have contributed to expand the use (and misuse) of individual level metrics (Haddow & Hammarfelt, 2019), while at the same time increasing the opportunities for more detailed and fine-grained analyses. We specifically focus on two phenomena that are key to understanding the renewed interest of scientometric studies on individuals: the expansion of author identifiers and the development of name disambiguation algorithms. Individual characteristics The context surrounding the researcher should be considered to offer a more holistic view of the individual level performance (Ràfols, 2019). This section discusses works which have studied intersecting individual categories (age, gender, ethnicity or national background, or cultural identity) influencing the academic career and their activities, aiming to clarify which indicators have been used to study the context and personal features of the researcher. Context Two factors have been essential for the study of individuals’ context. First, the development of author identifiers and name disambiguation algorithms. These have allowed the study of changes in the bibliographic metadata associated with publications of a single individual. These changes are monitored by studies using bibliometric methods to study career trajectories. For instance, we find studies focusing on geographic mobility (Moed et al., 2013), changes between sectors (Jurowetzki et al., 2021) or topic mobility (Yu et al., 2021). The second development has been the launch and expansion of new data sources. For instance, altmetric data allows mapping interactions of academics with other stakeholders (Robinson-Garcia et al., 2018), open access data allows the study of open practices (Ramos-Vielba et al., 2022), and author registries allow looking into individual funding (Costas et al., 2021). Team Dynamics There is an abundance of literature with regard to team dynamics. Since Derek de Solla Price published his seminal work ‘Little Science, Big Science’ (1963), a field has grown in relation with team science (Hall et al., 2018). Here we discuss on approaches related to individuals’ and their role when collaborating. We focus on four specific aspects: the use of author order, contribution statements, disciplinary differences and types of collaborations. Conclusions The present study represents a first attempt at compiling recent developments in the field of scientometrics with regard to the development of new approaches and methods to characterize the diversity of the scientific workforce. These new developments represent a unique opportunity to better study and understand how new scientific knowledge is produced from a sociological point of view. From an evaluative perspective, the present review has important implications, as it allows us to identify major methodological gaps and limitations when approaching research evaluation from a quantitative perspective. We conclude by suggesting an agenda for further development and potential areas of interest in which these methods can be applied. References Costas, R., Corona, C., & Robinson-Garcia, N. (forthcoming). Could ORCID play a key role in meta-research? Discussing new analytical possibilities to study the dynamics of science and scientists. In A. Oancea, G. E. Derrick, & N. Nuseibeh (Eds.), Handbook on Meta-Research (pp. 1–20). Elgar Publishing. Díaz-García, C., González-Moreno, A., & Jose Sáez-Martínez, F. (2013). Gender diversity within R&D teams: Its impact on radicalness of innovation. Innovation, 15(2), 149–160. https://doi.org/10.5172/impp.2013.15.2.149 Freeman, R. B., & Huang, W. (2015). Collaborating with People Like Me: Ethnic Co-authorship within the U.S. Journal of Labor Economics, 33(S1), S289–S318. https://doi.org/10.3386/w19905 Gauffriau, M. (2021). Counting methods introduced into the bibliometric research literature 1970 – 2018: A review. Quantitative Science Studies, 1–52. https://doi.org/10.1162/qss_a_00141 Haddow, G., & Hammarfelt, B. (2019). Quality, impact, and quantification: Indicators and metrics use by social scientists. Journal of the Association for Information Science and Technology, 70(1), 16–26. https://doi.org/10.1002/asi.24097 Hall, K. L., Vogel, A. L., Huang, G. C., Serrano, K. J., Rice, E. L., Tsakraklides, S. P., & Fiore, S. M. (2018). The science of team science: A review of the empirical evidence and research gaps on collaboration in science. American Psychologist, 73(4), 532–548. https://doi.org/10.1037/amp0000319 Jurowetzki, R., Hain, D., Mateos-Garcia, J., & Stathoulopoulos, K. (2021). The Privatization of AI Research(-ers): Causes and Potential Consequences -- From university-industry interaction to public research brain-drain? (arXiv:2102.01648). arXiv. https://doi.org/10.48550/arXiv.2102.01648 Maddi, A., & Gingras, Y. (2021). Gender Diversity in Research Teams and Citation Impact in Economics and Management. Journal of Economic Surveys, 35(5), 1381–1404. https://doi.org/10.1111/joes.12420 Moed, H. F., Aisati, M., & Plume, A. (2013). Studying scientific migration in Scopus. Scientometrics, 94(3), 929–942. https://doi.org/10.1007/s11192-012-0783-9 Price, D. J. de S. (1963). Little science, big science. Columbia University Press New York. Ràfols, I. (2019). S&T indicators in the wild: Contextualization and participation for responsible metrics. Research Evaluation, 28(1), 7–22. https://doi.org/10.1093/reseval/rvy030 Ramos-Vielba, I., Robinson-Garcia, N., & Woolley, R. (2022). A value creation model from science-society interconnections: Archetypal analysis combining publications, survey and altmetric data. PLOS ONE, 17(6), e0269004. https://doi.org/10.1371/journal.pone.0269004 Robinson-Garcia, N., Costas, R., Sugimoto, C. R., Larivière, V., & Nane, G. F. (2020). Task specialization across research careers. ELife, 9, e60586. https://doi.org/10.7554/eLife.60586 Robinson-Garcia, N., van Leeuwen, T. N., & Ràfols, I. (2018). Using altmetrics for contextualised mapping of societal impact: From hits to networks. Science and Public Policy, 45(6), 815–826. https://doi.org/10.1093/scipol/scy024 Walsh, J. P., Lee, Y.-N., & Tang, L. (2019). Pathogenic organization in science: Division of labor and retractions. Research Policy, 48(2), 444–461. https://doi.org/10.1016/j.respol.2018.09.004 Wildgaard, L., Schneider, J. W., & Larsen, B. (2014). A review of the characteristics of 108 author-level bibliometric indicators. Scientometrics, 101(1), 125–158. https://doi.org/10.1007/s11192-014-1423-3 Yu, X., Szymanski, B. K., & Jia, T. (2021). Become a better you: Correlation between the change of research direction and the change of scientific performance. Journal of Informetrics, 15(3), 101193. https://doi.org/10.1016/j.joi.2021.101193

ABSTRACT. Background and Rationale: The identification of hot or emerging topics is of interest to a broad variety of stakeholders. As such, it is one of the primary drivers of bibliometric analyses, especially those that attempt to characterize research areas through clustering of document sets. Once clusters are obtained, the hot or emerging clusters are most often identified as those that are growing the fastest or those with the youngest average age. In other words, most current studies identify hot topics as a production-based phenomenon where such topics represent ‘areas of knowledge.’

Our previous work has also looked at the growth rate in the number of papers in a document cluster as an indicator of emergence. Using our comprehensive detailed global model of the research literature (Scopus, 52.9 million documents, 98,767 clusters) we improved upon a method to predict which clusters of papers would experience high growth (productivity) over a future 3-4-year period (Boyack & Klavans, 2022). Among the 81 independent variables tested – which included author-, semantic-, application-, network- and gender-based cluster level features – the best predictor was related to historical growth rate, a production-based variable.

More recently we have begun to investigate the citing and cited relationships between clusters. Other studies that examine citations between groups of documents have typically been framed as investigating ‘knowledge flow’ or ‘knowledge diffusion’ and have been done at a highly aggregated field or discipline level. This is consistent with the notion of document clusters as ‘areas of knowledge’. We view clusters of documents differently. To us, each detailed cluster represents a research problem, and each paper is contributing a micro-solution to the overall problem. However, some micro-solutions are used outside the cluster. When a paper from one cluster cites papers in another cluster it represents the exchange of a ‘micro-solution’ between clusters. We refer to this as the export of a micro-solution from the cited cluster to the citing cluster. Concurrently, it is also an import to the citing cluster from the cited cluster. For example, when deep learning algorithms first appeared, they heavily imported micro-solutions from research on neural networks. Once deep learning was established, it began exporting micro-solutions to a variety of applications, for example facial expression recognition and online fashion recommendation. When considered in bulk, these historical imports and exports of micro-solutions may provide insights into how a research problem evolves over time.

This has caused us to reflect on the definition of ‘hot topics.’ Are hot topics those that experience extreme growth in production (micro-solutions to the problem at hand), or are they those that have high degrees of exports to other topics? This study pursues the notion that it is just as important to predict topics that will have exceptional growth in export activity (as enablers of broad application) as those that will experience exceptional growth in production. In order to explore this issue further, we conduct two experiments that are described in the following sections.

Experiment #1: In the first analysis, we chose to use the 71 document clusters from our 2014 study (Small, Boyack, & Klavans, 2014) because they had been shown to be emergent and had been validated with external information. However, these document clusters were identified in an older model. We thus took the papers from those clusters and found them in a model used in a more recent study (Boyack & Klavans, 2022). In most (57/71) cases, at least 60% of the papers from the original set of emerging document clusters remained together in the new model. Thus, the clusters in the new model remain relatively true to the emergence identified in 2010. In several cases (e.g., for graphene oxide clusters), multiple of the original emerging clusters had merged in the new model.

We then looked at the publication trends of these 57 documents clusters using the method for characterizing imports and exports that were described in a third study (Klavans & Boyack, 2022). In that study, we defined an import or export link between clusters as cases in which a paper cites at least 4 papers in another cluster. These instances were summed over pairs of clusters to identify import and export patterns by year. We then calculated production, import and export shares for each cluster (in reference to the entire model). This process led to several observations. 1) Production activity peaked within several years of identification as an emergent cluster for all but five clusters. In roughly half of the cases the peak lasted a single year and was followed by a decline in production. In the other cases the peak time was flat for several years. 2) Import and production activity were roughly parallel and higher than export activity up until the year of emergence. 3) Export activity dramatically increased after the year of emergence and continued to grow at a very high rate until several years past the peak year.

The overall story here is that, in general, production grows along with import of micro-solutions from other clusters as a cluster is emerging. Once it has emerged, production growth continues but is dramatically outpaced by export of micro-solutions, especially once production in the cluster starts to tail off. This provides evidence that cluster exports continue to be important well after the time that a cluster would no longer be considered hot from a production standpoint, and that tracking and prediction of such exports may be very important from a decision-making perspective.

Experiment #2 To follow this up we are in process of reproducing the recent study in QSS (Boyack & Klavans, 2022) in which we published a method of predicting which clusters in a large-scale detailed model of the research literature would experience exceptional growth. However, in this experiment we will use export activity as the dependent variable rather than production activity. Dependent variables will be the same 81 features used in the previous study plus two new variables – historical trends in import and export activity for a document cluster.

Significance: We expect to find that the sets of clusters that achieve exceptional growth in production and in exports will be partially overlapping and partially unique. Examination of those clusters that are unique to the set of exceptionally growing exports will help us to know if we should be predicting exports in addition to or rather than production. Vetting of those clusters with expert will enable us to understand better the potential policy implications of exports.

References: Boyack, K. W., & Klavans, R. (2022). An improved practical approach to forecasting exceptional growth in research. Quantitative Science Studies. doi:10.1162/qss_a_00202 Klavans, R., & Boyack, K. W. (2022). Predicting trends in research using research community imports and exports. Paper presented at the 26th International Conference on Science, Technology and Innovation Indicators (STI 2022), Granada, Spain. Small, H., Boyack, K. W., & Klavans, R. (2014). Identifying emerging topics in science and technology. Research Policy, 43, 1450-1467. doi:10.1016/j.respol.2014.02.005

ABSTRACT. Rationale: How might one identify translational pathways and evaluate translational research strategies? A translational research strategy is, in essence, a story about how one plans to commit resources over time, and across many related research problems, with a specific goal in mind. Translation, from the Latin trans (across) and latus (to carry or bear), simply means to carry or bear something from one place to another. Translation is generally defined as the process by which initial observations are turned into tangible interventions and applications. Central to this definition is the concept of ‘process’. Translation is not about where something is carried from or where it is being carried to. Neither research topics, communities, nor problems are translational in and of themselves. Rather, translation is about the connections between topics, communities, and problems. Translational research is about the carrying across of (micro)solutions from one problem to another—from one community of researchers to another. A translational research strategy is therefore an exercise in pathfinding. From the perspective of strategy, a ‘good’ translational strategy is one where the organization has expertise in specific research problems and the micro-solutions that link these problems. In other words, the path is navigable. A ‘bad’ translational strategy is one where the problems are stated vaguely and/or there is no way to evaluate whether one has expertise in the relevant set of micro-solutions. This study proposes a systematic method by which translational pathways can be identified using a large-scale detailed model of the scientific literature. It also focuses on storytelling based on linkages as a means of evaluating potential translational pathways from a strategic perspective.

Background: Most bibliometric analyses focus on characterizing research areas. This is often done by identifying a (small) set of papers, clustering those papers, and characterizing the actors and content of the clusters. Relatively little work has focused on the linkages between clusters even though translation is inherently an inter-cluster phenomenon. Moreover, most of these studies have little chance of accurately characterizing translation simply because their data do not cover the entire potential translational pathway.

We overcome these difficulties by using a global model of science, one in which 19 million documents from PubMed have been grouped into 29,013 clusters. Each cluster represents a research problem to which a group of researchers is contributing micro-solutions. (No single study solves the research problem, but each study contributes a piece of the solution—a ‘micro-solution.’) The clusters represent intellectually distinct problems along the basic-to-applied spectrum. For instance, one cluster may represent a problem in understanding the role of R-loops in DNA damage while another addresses the role of silica exposure in lung cancer among coal workers. When a paper from one cluster cites papers in another cluster it represents the potential exchange of a ‘micro-solution’ between clusters. This is an example of what is ‘carried from one place to another’. When considered in bulk, we refer to this phenomenon as the importing and exporting of micro-solutions between sets of research problems.

Consider, for example, significant events in the history of the COVID-19 mRNA vaccine. Between the initial observation that started it all—the discovery of mRNA in 1961—and the emergency authorization of the mRNA vaccine in 2020, many different groups of researchers with many different backgrounds and goals contributed over many years. But while the story of the mRNA vaccine would not exist without them, it isn’t solely about them. It’s about those discoveries and micro-solutions carried from one community of researchers to another. Thousands of pathways or stories like this one have occurred across the research landscape. More importantly, many currently incomplete potential pathways exist. Progress in translational science can be accelerated and improved by identifying these potential pathways in a strategic manner.

Method and Results: We have identified one translational pathway of strategic importance to the University of Michigan Medical School (UMMS) by exploring the PubMed model of science manually. UMMS has a specific competency in the general topic of chronic pain. We started by identifying a cluster in the PubMed model on pain-related aversion – a basic science cluster in which UMMS has noted expertise. We then identified other clusters that heavily cite the pain-related aversion cluster, finding that the strongest linkage was to a cluster focused on the problem of pain anticipation. This cluster is also one in which UMMS has a strong presence. We then carried on this process for three additional steps, leading to a five-node pathway.

While we were able to easily ascertain the particular focus of each cluster, to tell the story of translation one must have a more detailed understanding of the topical focus of the links between clusters. We thus identified the subset of papers that comprise these citing pathways and summarized their content. For instance, the link from pain anticipation to pain-related aversion is based on 320 citing instances and can be summarized as “the anticipation of pain prioritizes awareness and attribution of somatosensory sensations as painful…”

From a strategic standpoint, this pathway—which has been validated by administrators at UMMS—is important because, while the institution currently has faculty expertise in the first four nodes in the pathway, it has almost no activity in the final (bedside) node. In other words, UMMS discoveries are being translated to tangible patient health impacts by other institutions, potentially depriving UMMS researchers of valuable insights into the clinical applications of their work and precluding them from the clinical collaborations that could make their future work more impactful. If UMMS wants to develop expertise along a complete pathway, it will need to add research in that area to its portfolio, either through recruitment or collaboration. This is key information for UMMS.

While this translational pathway was identified manually, our next step is to develop methods to reproduce these results algorithmically. Once done, we will apply the method to other areas, to identify additional translational pathways relevant to the UMMS strategic vision. These will be validated by experts in-house. The algorithmic method and additional results will be presented at the conference in May.

Significance: In summary, we argue that these intellectual links, which we can quantify through citations, represent dependency relationships. Research communities are formed around difficult to solve problems and these problems cannot be solved in intellectual isolation. These research communities therefore rely on the continuous exchange of micro-solutions, the record of which can be found in article references. These exchanges represent literal instances of translation: the carrying across of information from one community to another. Ensembles of these exchanges become translational pathways—a directed process network of dependency relationships that we call “translation”.

Significantly, by examining the papers that comprise these dependency relationships between communities, we can summarize and understand them, effectively describing the way in which solutions from one community are used by another. The entire translational network, if visualized with all the edges, would look like an impenetrable hairball. We would, in effect, be attempting to describe all of science. The methodological solution is therefore to distill the narrative by screening out nodes and links, both simplifying and strengthening it, so that effective translational strategies can be identified and characterized.

ABSTRACT. Acknowledged in the contribution dynamics of authorship is an asymmetry between the Global North and the Global South (also referred to as the ‘scientific periphery’) in the production of new knowledge. Marginson (2022) points to a dominant global discourse that frames science as a centre-periphery hierarchy. Baber (2003: 621) refers to a perpetuation of ‘an unstated but real global division of intellectual labour’. Others argue that theoretical and methodological innovations are considered legitimate tasks for Global North scholars while data collection is the designated role for those from the Global South. This paper examines the contribution of scientists from the Global South by relying on multi-authored scholarly publications to assess their changing contributions to global science over time. The focus on the contributions of scholars in the Global South in an asymmetrical global science system provides a unique and empirical perspective. It is a perspective that accounts for the networked nature of science that shapes the relative contributions to the global scientific network and provides important insights for funders and policy-makers seeking to establish equitable partnerships between the Global North and the Global South in global science.

ABSTRACT. Background and Rationale: The sustainable transformation of the economy is increasingly a relevant topic globally, including for the Global South. This topic has particular relevance to natural resource-rich countries because these countries are challenged with distinctive pressures globally and locally. These are 1) market pressure to produce more extractive resources (in particular, mineral resources required for renewables) by the Global North for the energy transformation; and 2) growing local tensions between the extraction of natural resources and local environmental sustainability and society. It is urgent to identify a new pathway toward their own transitions, leveraging natural resources.

The management of revenues from natural resources, in this context, plays a critical role in transforming the economies of resource-rich countries. Traditionally, the revenues--mineral royalties--had been levied and these were used to stabilize the macroeconomic fluctuations (e.g. stabilization fund, sovereign wealth fund, natural resource fund) or economically compensate the people who live in the localities (usually poor communities) where extraction activities take place. These revenues, however, were rarely invested actively for innovation or knowledge for transforming the existing structure of the economy.

In the public policy literature, the importance of the dynamic capability of the public sector had been identified as a critical factor in adapting society to the new context. The management of finance and how to invest in knowledge would also require such dynamic capability of the government, whichever level it is situated either at national or local. The existing literature attempt to classify the types of capabilities and actors necessary for this purpose; however, there are still limited cases from the Global South.

Several Latin American countries, on the other hand, had introduced policy that earmarks certain parts of natural resource revenues for science, technology, and innovation purposes. In this paper, we call these initiatives as Natural Resource for Knowledge Fund(NR4KF). The mechanism, in distinctive ways, is introduced with the purpose to strengthen knowledge and innovation capacities in the country, especially in remote regions. The cases we study in depth are Bolivia’s direct tax on hydrocarbons (IDH), Chile’s Innovation for Competitiveness Found (FIC), Colombia’s Science, Technology and Innovation fund (FCTel), and Peru’s Canon and Mining royalties. These mechanisms are introduced in distinctive moments between 2004-2012, allowing sufficient time to observe the outcomes of these initiatives.

The purpose of examining above mentioned Latin American cases are to identify the types and moments of policy interventions and their design that are needed to avoid various difficulties that are associated with natural resource revenues. We consider that studying the case of NR4KF in the four Latin American countries mentioned above would give some insights into the challenge of transformation in the Global South, especially for those with natural resources, and provide policy suggestions.

Methods: Building upon the literature on the management of natural resources, industrial policy, and public management, we developed a framework for the ideal design criteria for NR4KF. These consist of four static design criteria (clear statement of purpose, rule-based design, multiple stakeholder governance, and transparency), and three dynamic design criteria (monitoring and evaluation system, Institutional ma/managerial capacity building, coordination with external actors). This framework is used to compare four Latin American NR4KFs.

In order to understand the mechanism, procedure, and outcome of NR4KF in each country, the following information is collected and analyzed: 1) gray literature on legal and policy; 2) interviews with policymakers involved in the process, and 3) data on the flow of NR4KF and STI performance indicator. Point 3) was done to identify the short-term outcome of STI activities to indicate some aspects of transformation.

Results or anticipated results, significance:

This study compiled rather a unique set of data, the revenue flow to STI areas, connecting back the natural resources. The observation of complied data on the flow of NR4KF in relation to STI performance indicators (number of scientific publications, exported products) did not show strong relationships. This can be interpreted as the lack of impact in providing finance for transformation, or simply the time span in which the observation was made was not long enough to mark the difference.

As the quantitative measures to assess NR4KF fail to show a clear outcome of NR4KF, the qualitative comparative analysis was conducted to identify differences in mechanism in detail based on the NR4KF design criteria.

While the static design principle of NR4KF ensures consistency, and transparency, and diminishes the possibility of corruption, the rigid rule-based design hampered the effective use of the funds. For example, in Bolivia and Peru, due to the shortfall of capacities in research management, universities appear unable to cope with sudden increases in the inflow of research funds, especially lacking the means to convert them into research output. This problem is compounded by the fact that the funds cannot be used for salaries to hire researchers. The lack of flexibility in management practice and capacity to manage it effectively was especially noteworthy in the regions, where the funds are allocated. As the result, a higher proportion of funds is left unused and even if these are used, funds are used on items that may not effectively lead to knowledge outcomes leading to transformation (such as physical infrastructure on research i.e. ICT equipment and research facilities). This means that the dynamic capability of knowledge institutions needs to be built in tandem with the creation of the static rule-based design of the fund.

In contrast to the above, dynamic design criteria can be used to examine institutional abilities to adapt to the new “evolving “policy goals. Under this category, the striking finding across the cases is their absence. For instance, monitoring and evaluation systems are mentioned in the design but not implemented. Even with the system installed, the political cycle often interrupts projects preventing them from undergoing an adequate evaluation process to ensure policy learning. In other words, nurturing the dynamic capability of the state would require a well-functioning policy setting, a big challenge for the Global South.

This paper illustrated the capability of transformation requires the dynamic capability of public institutions, through a comparative analysis of NR4KF examples in four Latin American countries. The results would have the following implications: one is to show in detail both static and dynamic capabilities are necessary for the resource-rich Global South to develop to enable transformation; second, if transformation were to be implemented effectively, the capacity building at implementing agencies is critical prior to mobilizing large sum of finances and, the third, long term aim for policy should be combined with a short-term policy. This will provide the means to maintain a good balance between flexibility in rule-based design and space for dynamic interaction.

ABSTRACT. Technology-based entrepreneurial firms face considerable barriers in commercializing their innovations. Time, in particular, is a scarce resource for start-ups (Levesque & Stephan, 2020). New firms operate under high risk and severe capital constraints as they try to outpace competitors and generate economic profit. Research has shown that start-ups’ capital structures influence firm outcomes and development speed (Hecchavaria et al., 2016). This is especially the case during the early stages as a company tries to move from proof of concept to early seed stages when the pot of money available to firms is quite small and their ideas are quite risky. Hearkening to Benjamin Franklin’s famous quote, “time is money” quite literally to an entrepreneur. The persistent qualm of the entrepreneur is that they rarely have enough of either.

This is where endogenous growth theory suggests government has a role. Government intervention is seen as necessary to reduce barriers in helping firms develop technologies, based on the idea that the creation of new knowledge is a public good that the private sector will not effectively support due to the high risk entailed (Mazzucato, 2011). Government enters at an earlier, riskier stage than private funding is willing to enter. But public funding operates under different institutional norms than private funding and we posit in this paper that these differences have implications for how the funding impacts firms’ subsequent milestones. Qualitative evidence from interviews with entrepreneurs indicates that federal funding through the United States (U.S.) Small Business Administration’s (SBA) Small Business Innovation Research (SBIR) and Small Business Technology Transfer (STTR) programs may be too slow to attract high quality applications relative to alternative sources of early funding (NASEM, 2022). This slowness can be attributed to the nature of the routinized bureaucratic forms that federal agencies follow (Perrow, 1986) which make the minute-by-minute decisions of an angel or venture capitalist difficult, if not impossible, to replicate. Evidence from Europe indicates start-ups that receive public venture capital (VC) sooner have higher sales-growth (Grilli & Murtinu, 2015). These questions have not been investigated in the U.S. context.

In this paper, we ask whether shorter time periods between proof-of concept and seed stages in public VC allow firms to achieve greater technology and financial milestones. We exploit a unique programmatic shift in the U.S. SBIR and STTR funding to answer this question. Owing to the concerns that important technologies were lying dormant, the federal government began experimenting with new models to support high tech firms more swiftly by offered “Fast-Track” awards, which allow firms to bypass Phase I (averaging $150,000) and move directly to Phase II of the program which offers significantly more funding (averaging $1 million). We take an exploratory approach to examine performance differences between firms who secure these special awards, called Fast-Track awards, and comparable firms who received the regular Phase II award after a Phase I. We isolate our setting to SBIR/STTR awards made through the National Institutes of Health (NIH). This allows us to examine a range of firm outcomes.

First, we present a descriptive overview of these special awards, including an exploration of the Fast-Track program to understand which firms are likely to receive them. Next, we estimate a series of two-way fixed effects panel regressions to assess whether speedier SBIR funding increases the likelihood that a firm will a) produce clinical trials or b) medical devices as the result of the funded application; c) obtain patents; d) attract subsequent funding from private sources; and e) be acquired compared to the control group (extensive margin). Finally, we use a unique dataset on state match programs in a differences-in-differences analysis to examine the impact of this exogenous windfall funding (Myers & Lanahan 2022) on the Fast-Track recipients (intensive margin). We use NIH’s RePORT data archive for detail on award type and corresponding patents and clinical trials, as well as the USPTO PatentsView database. We construct further outcome measures using the SBA SBIR award database as well as information on VC, angel investment, and other private funding support from Crunchbase. While numerous studies document the impacts of this program on follow-on financing (Lerner, 2000; Howell, 2017), patents (Myers & Lanahan, 2022), and employment (Lanahan et al., 2021), consideration of the influence of the timing of funding rounds is largely absent from the public VC literature.

We find that among firms that receive a NIH SBIR/STTR Phase II or Fast-Track award, the marginal Fast-Track dollar is associated with a greater number of follow-on clinical trials and patents. We also find that among Fast-Track award winners, those that receive an exogenous increase of funding through variably available state match programs have even greater number of clinical trials than those that do not receive the state match funding, but receive less private venture finance. These are important outcomes for small, young firms and illustrate that faster SBIR funding can be beneficial if firms are pursuing clinical trials and patenting as part of their development process.

This paper makes three contributions. First, we contribute to the field of entrepreneurship by responding to recent calls for a “time-based lens for entrepreneurship research” (Lévesque & Stephan, 2020). We offer new insight into the importance of the timing of public, early-stage, proof-of-concept, and seed funding. Second, we contribute to the literature on public VC by exploring a previously understudied method of hastened public funding. Finally, we draw practical implications for entrepreneurs and for policy. We provide new understanding about how the SBIR program may be meaningfully enhanced by quickening the pace of funding, if the SBIR wants to increase products, technologies, and follow-on funding as a goal.

ABSTRACT. The paper proposes a novel, web-based approach to innovation policy evaluation. The approach overcomes one major limitation affecting current evaluation methods, namely the tracking of invention commercialization. We implement it to study the impact of the U.S. DoD-SBIR program on technology commercialization. We start by identifying the universe of USPTO patents that acknowledge support by the program, and construct a set of control patents as benchmark. We then track whether these patents are mentioned in relation to commercial products in virtual patent marking web pages. We interpret the latter event as signal of commercialization. Finally, we compare the commercialization probability of SBIR-funded and control inventions. The results support the view that the SBIR program is quite effective at stimulating the commercialization of federally-funded scientific discoveries. The effect is particularly strong for grants covering development expenditures and Phase II awards.

ABSTRACT. IPOs allow companies to raise new capital, but this money comes with strings attached. Disclosure requirements and pressure from the shareholders affect your scientific strategies and consequently your scientific output. I test the causal impact of going public on firms’ scientific output, using data on 1,919 US IPO firms, which is the population of IPOs from 1994 to 2010 with at least one patent. My empirical strategy involves a treatment group of firms that successfully completed an IPO and a control group of firms that filed for an IPO but then afterwards decided to withdraw their filing. Identification is achieved instrumenting the IPO decision with stock market average returns in the two months after the filing. Preliminary results show a positive effect of IPOs on scientific output, measured as patents and scientific publications.

ABSTRACT. The Federal Government spends more than $150 billion on research and development. This funding has fueled the innovations and technologies that have transformed our societies. However, the pathways from research to impact can be arduous and complex. Way to effectively engage, work with, and leverage resources from the private sector, including small businesses, for example through public private partnerships, continue to be critical to the maturation and commercialization of new technologies and the supply chains necessary in their development.

The Federal Government continues to look for innovative mechanisms to engage with non-Federal stakeholders to enable and improve engagement and opportunities with the private sector. DOE has been working to better understand the lessons learned and identify ways to enhance the use of a few of these mechanisms--including 1) establishing an agency-affiliated foundation, 2) using the other transactions authority (OTA); and 3) using the partnership intermediary agreement (PIA) authority. This presentation will discuss cumulative lessons learned for planning and implementation of these mechanisms and how they inter-relate to complement and build technology transfer pathways from one another. Currently, there is a dearth of published information regarding the use of these mechanisms specifically to enhance Federal research and development, technology transfer, and commercialization goals. This presentation will build on prior studies that analyze process and outcomes of specific mechanisms published by various research institutes, including IDA, ITIF, and NAPA, among others.

ABSTRACT. This study identifies the characteristics and types of the regional innovation systems (RIS) of regions and cities in emerging economies in comparison to those in advanced economies. It uses the citation data of the US patents filed by 30 regions. Some RIS variables are newly developed, and they include intra-regional, inter-regional, and inter-national sourcing of knowledge and local ownership of innovation. The cluster analysis of these variables enables us to identify four major types of RIS around the world and link them to regional economic performance. The four types are, in the descending order of their per capita income levels, as follows: large, mature RIS characterized by a combination of long cycle technology specialization and high local ownership (Group 1), mixed RIS characterized by a long cycle and low local ownership (Group 2), “strong catch-up” characterized by short cycle and high local ownership (Group 3), and “weak catch-up” characterized by short cycle and low local ownership (Group 4). Groups 3 and 4 include only the regions in emerging world. They similarly specialize in the same short cycle time of technologies (CTT)-based sectors but show different records of economic performance. The key differentiating variable is the degree of local ownership of knowledge, which can be a basis for increasing domestic sourcing of knowledge and sustained catching up. Another important variable is decentralization, of which the level is lower in the strong catch-up group than in the weak catch-up group. In this Group 3, catching up is led by big businesses. Several cities experiencing upgrading, like Moscow, Beijing, and Shanghai, also show an increasing trend of local ownership and centralization.

ABSTRACT. Background and rationale

Access to funding is known to be critical for the emergence and consolidation of new innovative ventures. There are several studies mentioning funding as one of the pillars on which Innovation and Entrepreneurship Ecosystems (IEEs) are based, although there is still a lack of systemic knowledge about funding sources and their effects on ecosystems (Frimanslund et al., 2022). Although there are several types of private funding, such as venture capital and business angels, here focus on discussing the role of government funding for the development of ecosystems, since public funding dedicated to support scientific research - either applied research aiming at innovation directly, or more basic research - functions as trigger for innovation and the creation of new knowledge-intensive companies. Thus, the purpose of this article is to show how non-reimbursable funding from public sources affects the dynamics of IEEs, particularly in the context of emerging economies, where other sources of funding tend to be scarcer.

Literature Review

It is important to distinguish between the two groups of grants from public research support agencies that promote the development of IEEs. The first, direct, consists of programs aimed at applied research that focus on innovation, such as the Small Business Innovation Research (SBIR) in the US and Fapesp’s Innovative Research in Small Businesses (PIPE) in Brazil. Such initiatives are based on two fundamental principles: (i) the existence of market failures in the process of funding entrepreneurship and innovation; and (ii) the expectation of socialization and economic gains arising from the success of the companies supported. In these cases, public sources of funding not only generate liquidity for the ecosystem (Autio & Ranniko, 2016) but also act as indicators of the potential of selected projects. The relationship between knowledge-based firms and public research institutions can increase their innovative potential, which reinforces the role of public institutions as relevant agents in the entrepreneurial ecosystem. Also, public research funding improves the systemic conditions of IEEs, as it provides early-stage funding for technologies that will be the basis of new products and services. The second group, indirect, involves all support for scientific research, including basic research, which eventually may translate into the generation of innovation and new ventures. The prerogative of this group is supported by the knowledge spillover theory of entrepreneurship, which identifies the production and circulation of knowledge as assets that are strongly attached to territorial spaces and support the development of entrepreneurial and innovative activities. The approximation of new and existing enterprises with universities fosters gains in terms of technological capabilities (Audretsch et al., 2020). Furthermore, academic research itself can create opportunities for economic exploration, making universities relevant cradles for the emergence of new innovative ventures (Fischer et al., 2019). In both cases – direct and indirect – the role of public funding in ecosystems in incipient stages has a more relevant relative weight. This is due to the lack of maturity in the configuration of the IEEs, in which the presence of private capital still falls short in achieving traction.

Method

The methodology is divided into two fronts: 1) Literature review. The research envisages carrying out a literature review on public funding and the development of Innovation and Entrepreneurship Ecosystems (IEEs). The search was carried out in Scopus and Web of Science, selecting scientific articles from 2015 to 2022. The search occurred by topic, ordered by relevance. From the initial list of articles identified, the most relevant ones were selected according to the research objectives. 2) To advance in the development of methods, metrics, and indicators for the analysis of public funding in IEEs, we proceeded with the following steps. a. Develop an approach for the specific case of Unicamp. b. Work with Unicamp’s licensing and technology transfer data. Licensing cases: Connect companies with supporters by crossing the name of inventors/technologies/dates with Fapesp’s global base. Spin-offs: Connect companies with supporters by crossing the name of inventors/technologies/dates with Fapesp’s PIPE base. Data gathered from companies in the science park: Check the relationship between companies in Unicamp’s Science and Technology Park and Fapesp’s support. We carried out a survey and subsequent triangulation between the unstructured data provided by the Unicamp Innovation Agency (Inova) with the structured data provided by Fapesp. This enabled the analysis of the direct and indirect connections between public funding and IEEs.

Results

This article contributes to academic research and literature in two different ways. First, the confirmation of the gap identified in the literature concerning the lack of studies focused on the role of government funding for the development of innovation and entrepreneurship. Second, our analysis demonstrates the relative centrality of public funding within the context of an IEE embedded in a developing country. This will allow us to developed a more nuanced picture of the role played by public sources dedicated to research in shaping the dynamics of such ecosystems.

Implications & Significance

This article brings a few important lessons. First, we came to the conclusion that it is important to strengthen government funding to promote both applied research aiming at innovation and basic research capable of generating new ideas. Second, it is critical to assess the role of public funding in generating new ventures in selected IEEs through mechanisms capable of measuring this effect; for such, we recommend the assessment of licensing and technology transfer data, university spin-offs, and companies located in science parks.

REFERENCES

Audretsch, D. B., Belitski, M., & Cherkas, N. (2021). Entrepreneurial ecosystems in cities: The role of institutions. PLoS ONE, 16(3). doi: 10.1371/journal.pone.0247609 Autio, E., & Ranniko, H. (2016). Retaining Winners: Can Policy Boost High-Growth Entrepreneurship? Research Policy, 45(1), 42-55. doi:10.1111/caim.12338 Fischer, B. B., Moraes, G. H. S. M., & Schaeffer, P. R. (2019). Universities' institutional settings and academic entrepreneurship: Notes from a developing country. Technological Forecasting and Social Change, 147, 243-252. doi:10.1016/j.techfore.2019.07.009 Frimanslund, T., Kwiatkowski, G., & Oklevik, O. (2022). The role of finance in the literature of entrepreneurial ecosystems. European Planning Studies. doi:10.1080/09654313.2022.2055962

ABSTRACT. Background The emergence of artificial intelligence (AI) as new general-purpose technology is influencing how societies shape their development models for the next decade (Bresnahan & Trajtenberg, 1995; Klinger et al., 2018; Trajtenberg, 2018). Even though the concept of AI was conceived in 1956, it was not until the last decade that AI’s adoption scale, speed, and risks became a central policy challenge for governments (Taeihagh, 2021). Since the first national AI strategies publication in 2017, there has been an explosive increase to over 700 policy initiatives in more than 60 countries (OECD, 2021).

Recent studies and international discussions on AI governance have emphasized the need to build broad societal consensus around ethical principles and institutions (e.g., Calo, 2017; Gasser & Almeida, 2017; OECD, 2019; UNESCO, 2021). However, members of society with different imaginaries and expectations have different opportunities to frame what is relevant, urgent, possible, or inevitable in technology policy (Sand, 2019; Konrad & Boyle, 2019). Moreover, most AI policies have been developed by “domain experts,” leaving “lay people” on the margin, failing to incorporate multiple visions. The latter is problematic because multi-stakeholder processes are becoming ubiquitous amid crises of trust and social unrest that have proliferated during the last few years.

Countries face difficulties in implementing and ensuring the continuity of AI strategies due to changes in governing coalitions and competing social priorities. For example, Mexico and Argentina published their strategies in 2018 and 2019, in the last year of their governments, and were not implemented in depth nor continued by the next governing coalition (Gómez Mont et al., 2020). This manuscript discusses how a participatory approach to AI policymaking can enable policies to achieve higher consensus and resilience. To do so, we conducted an in-depth analysis of the building process of the Chilean National AI policy, in which approximately 10,000 people participated through different mechanisms and stages. This is an interesting case to study as, unlike other AI policies in the region, the Chilean AI policy achieved a high degree of consensus–as measured during a public consultation before the policy was enacted–and has been able to withstand changes in government coalitions to date.

Methods We undertook a longitudinal case study research approach, accompanying and participating in the development of the AI policy between 2019 and 2022. Utilizing interview data, official documents, and transcripts of grassroots discussions, we mapped the AI policy development utilizing process tracing techniques (Garud, Berends, & Tuertscher, 2018; Langley, 1999, 2007; Van de Ven & Huber, 1990; Van de Ven & Poole, 2005). We chose this design because case studies have been used for modeling and assessing complex causal relationships (George & Bennet, 2005) and have been found helpful in illuminating decisions (Yin, 2017), both of which enable an understanding of how technology and innovation policy is developed. Longitudinal case studies also enable us to explore AI's technological and social contexts, and the collection of qualitative evidence allows us to identify key process variables.

We divided the policy development process into four stages using an engineering systems architecture approach (Selva, Cameron & Crawley, 2015). In the “Conceive” stage, stakeholders address the challenges and opportunities of AI systems and think of possible solutions to them (e.g., regulations and strategies). In the “Design” stage, stakeholders design a roadmap and build the solution chosen during the Conceive stage. In the “Implement” stage, stakeholders start executing the actions developed in the Design stage. Finally, in the “Operate” stage, the solution owner monitors results, identifies future opportunities and challenges, and can decide to initiate a new policy process. We acknowledge that this framework can simplify uncertain and messy real-world politics. However, practitioners can use this framework to guide AI governance discussions.

Results and discussion Chile’s AI strategy was initially conceived as a top-down, expert-driven process. However, two external crises (Chile’s social riots of 2019 and the COVID-19 pandemic) lowered the barriers to adopting a bottom-up approach as there were mounting pressures from different stakeholders demanding involvement and requesting specific policy outcomes. The government responded by adapting the process and its governance to address the bottom-up pressure and navigate the conflicting demands of heterogeneous stakeholders. Furthermore, authorities agreed to foster self-convocated roundtables and organize regional discussions to gather information for the draft, leaving the expert committee only as a consultative group.

The constant relation between demands and responses shaped the AI development process and defined the level of overlap between the four stages (i.e., Conceive, Design, Implement, and Operate). To convince heterogeneous groups to participate, the government purposely developed tools to generate trust and lowered barriers to participation. For example, in the Design phase, there was an open deliberation period in which everyone could discuss and contribute content for the policy. Public officials developed the first draft with that deliberation input, which was later presented for consultation to the public. The option to review and comment on the draft was open to the public. This two-staged model gave the public more accountability and fostered trust in the AI strategy. Another trust-building example was when the government actively argued against the expert/non-expert dichotomy, responding to domain expert groups complaining because of the involvement of “lay people” in the process.

The process generates insights into how the intertwined nature of technology and development in emerging countries shapes public deliberation and moves processes beyond the expert/non-expert dichotomy. Deliberation during the process was usually framed based on Chile’s singularities, deficits (Pfotenhauer et al., 2019), and opportunities to address social goals, all of which directed the AI’s development (Schot & Steinmueller, 2018). Key initiatives in the policy’s action plan are a US $5 million grant for economic reactivation through AI, a public-private enterprise to foster AI and data science using Chile’s unique astronomical potential (see Guridi et al., 2020), and the prioritization of three industrial sectors for an AI Sandbox (i.e., healthcare, logistics, and fintech). Thus, the discussion focused on how AI contributed to the country's overall development, which contrasts with other policy narratives based on developing science and technology for its own sake.

The resulting policy achieved a high level of consensus and acceptance and has survived for more than a year since its publication. Nearly 90% of the people who participated in the consultation highly agreed or agreed with the proposed objective, and more than 80% with the topics and objectives proposed. Furthermore, the policy survived through a change in the country’s governing coalition, and to date, the new administration has continued implementing it and engaging in international outreach activities.

We contribute to technology and innovation policy literature by providing insights on how to enable participatory processes to build more resilient technology policies. Governments should acknowledge citizens’ reflexive agency to build democratic legitimacy in technology discussions (Biale & Liveriero, 2017). Furthermore, public discussion allows the creation of technology policy with a focus on anticipation, experimentation, participation, and directionality, following Schot & Steinmueller's (2018) transformative change framework. Finally, we show how crises and social unrest can be leveraged to foster participation and innovation when constructing technology policies.

ABSTRACT. Introduction Scientists were omnipresent in the media during the covid-19 pandemic. More than ever, the recent public health crisis required health scientists to become public spokespersons. In Flanders, the northern Dutch speaking region of Belgium, a few academics became known as “the virologists” in a very short period of time, and could count on various media bringing their up-to-date knowledge to the wider public. But not only academics from the medical sector were featured. The covid-19 pandemic became the new issue of choice for academics from various non-medical disciplines as well, with a number of them using the written press to voice their critique of the predominant medical discourse to the general public. While counting media coverage provides valuable insights into the "amount" of media coverage, the question of what discourses both medical scientists and non-medical scientists employed remains underexplored. That the public debate is also an arena where knowledge, ideas and interpretations from different scientific disciplines compete for attention – alongside “scholarly” circuits of communication – is often overlooked. This gap is likely informed by the persistent methodological problems we face when studying scientific communication beyond the academic sphere. The recently developed neural language processing tool at Sentometrics however, allows for a close analysis of the scientific discourse of academics in the written press regarding the covid-19 pandemic. Advanced textual indicators provide insight into the nature and relevance of these interactions, and improve understanding of the lexicon, sentiment, sequence and timing of scientists communicating during the recent public health crisis. Methods Researchers active at the five universities funded by the Flemish Community in Belgium were the focus of this study. Using data from Flanders Research Information Space (FRIS), an open source database operated by the Flemish government, active researchers (n = 35277) were used for analysis, supplemented with additional data on gender, career position, number of publications, sector and discipline. The names of active researchers were queried in the Belgian media database GoPress, collecting the occurrence of their names, after correction of namesakes (false positives). We excluded academics with a political mandate. We focused on Dutch speaking newspapers and magazines in Belgium, because of the highly concentrated Flemish media ecosystem. The time period of the newspaper articles was from Jan. 1, 2019 to Dec. 31, 2020. The year 2019 functioned as a control year, to analyze the distribution of media attention of academics in the written press before the covid-19 pandemic. In the near future, the analysis will be supplemented with data from the year 2021. The covid-19 pandemic was the overriding theme of 2020 and 2021 news in Belgium. First messages on covid-19 began to appear in January 2020 in Belgium, followed by a proliferation brought by the first casualties in March 2020. Quickly, the federal government sought assistance from several expert consortia, which included several academics. These quickly became the public face of the collective response to the pandemic. They consulted regularly to assess the public health crisis, advising the federal government to tighten, maintain or ease the measures, communicating publicly about them in written media. Preliminary results A first question presents itself as to which individual academics appeared in the written press. Only 7.6% of academics (2640 of 34669) appeared once or more in the Flemish written press of 2020. Moreover, five professors (four medical professors and one biostatistician) got 30.34% (6817 of 22467) of all media mentions that year for all academics at Flemish universities, pointing to high skewedness in the data. These “Big Five" became known as “the virologists” and took the lead in providing the public with actual information on the virus and its societal consequences. Their daily presence in several media rendered them “celebrity scientists” almost instantaneously, with initial praise and adulation from the public but, over time, also hatred and sometimes even death threats. In terms of sector - and disciplinary differences, academics from the medical sector evidently received relatively the most media attention in 2020 (49,92%; 11212 of 22462 articles). Whereas in 2019 it was social scientists who received the most media attention (41.0%; 5800 of 14130 articles), in absolute terms they surprisingly remained almost as prominent in 2020 (26,73%; 6004 of 22462), but lost media attention in relative terms. In relative terms, almost all non-medical disciplines lost media attention in 2020 compared to 2019, apart from academics from Psychology and cognitive sciences, who saw a very slight increase in relative popular attention (from 3,6% of articles in 2019 to 3,7% of articles in 2020). The covid-19 pandemic was therefore not just a medical affair; all kinds of academics brought their expertise into the public arena, interpreting, explaining, advocating, promoting the impact of the public health crisis on their area of expertise, some criticizing or expressing their dissatisfaction with the "purely medical" discourse of the virologists. A few academics took the lead in this, representing the “critical” opposing voice in the debate. Significance Results show how the written press provided an immediate opportunity for academics to publicly assert the policy relevance of their field and discipline, not “organically” from the traditional scientific publication cycle, but more directly from the stage they were given in mainstream media. While counting media coverage provides valuable insights into the "amount" of media coverage, the question of what discourses both medical scientists and non-medical scientists employed remains unanswered. Future analysis aim to unravel which context-specific lexicon academics from both medical and non-medical employed during the covid-19 pandemic, improving our understanding of the impact of proactive and reactive science communication of academics in the written press.

ABSTRACT. Background and rationale The demonstration of the societal and economic impact of a university remains challenging owing to a paucity of data that connect the education and research activities of the institution to short- and long-term outcomes. We have pursued the connection between education (PhD alumni) and economic (employment) outcomes and the connection between grant income (research) and patenting (innovation) to demonstrate how actionable insights can be developed for one institution (Cornell University).

Methods In this study, we have linked and enriched data available within Cornell University with datasets relating to career outcomes from an online social network and research publications to demonstrate the post-graduation career trajectories of doctoral holders. We have also linked research grants, publications, and patents awarded to Cornell researchers to explore gaps in translation activities from the funding inputs to the research and innovation outputs of the university more directly. This work was conducted under an approved IRB protocol requiring all personally identifiable information to be held securely and only aggregated deidentified results distributed beyond the authorized project team.

Results Using a cohort-based approach, we have determined the current employment of all PhD alumni from two Cornell colleges with a high degree of precision and recall. Across the cohorts studied (graduates from academic years 2000, 2005, 2010, 2015 and 2020) we were able to show that 79% could be unambiguously identified in a popular online social network using just their full name and known former affiliation to Cornell University. Of these, 1 in 10 alumni leave the US shortly after graduation while more than 78% remain in the US up to 22 years post-graduation. Of those that remain in the US, alumni are predominantly based in a small number of north-eastern and western states with high background rates of PhD employment. Half of Cornell PhD alumni are currently working in industry, and another one-third in education; the remainder are largely in government or non-profit organizations. For graduates working in industry, government and non-profit sectors, ‘scientist’, ‘engineer’ and ‘researcher’ roles are most common. Additionally, we anticipate that the exploratory work linking research grants to publications and patents will be able to help identify different patterns of funding sources tied to innovation outcomes based on research focus or department at Cornell. By comparing these links across departments at Cornell, we expect to uncover gaps between funding and levels of engagement in innovation activities and understand whether these differences can be partially explained by relative focus on basic to applied research activities.

Significance Our analysis of PhD alumni career outcomes indicates that the value of the research-based education and training that the higher degree reflects is retained in the wider economy irrespective of whether the individual pursues an academic career path or exits the sector. In the case of Cornell University, those that do leave academia tend to remain within the US, gravitate towards parts of the country where knowledge-based work is abundant, and occupy roles that capitalize on the knowledge and skills they developed during the course of the PhD. The anticipated findings from the work exploring linking Cornell grants to innovation outcomes will help identify existing practices within departments that could benefit from Cornell resource investment, enabling the university to focus on specific academic units in supporting translation activities for societal and economic impact.

Conclusion We have shown in this work that the societal and economic impact of an institution such as Cornell University can be demonstrated by combining data available within the university with other publicly- or commercially-available datasets. To that end, the economic benefit of PhD alumni who remain within the country and take up employment beyond academia that utilizes their doctoral training and skills can be effectively demonstrated, and the innovation outcomes of research-oriented grant income can be connected directly through advanced mapping approaches. Taken together, we suggest that this is the beginnings of a toolkit that other institutions may wish to develop and adopt for their own specific circumstances and that can inform and monitor university strategy and policy-making.

ABSTRACT. Adopted by the United Nations in 2015, the Sustainable Development Goals (SDGs) offer a globally shared opportunity to change the directions of science, technology and innovation (STI) to contribute to a better and more sustainable future for everyone. STI can help address many SDG challenges, for example, by increasing access to safe and nutritious food, improving per capita economic growth, or enhancing access to trans- port systems. However, in doing so, STI can also undermine progress towards some of the goals, for example, through carbon emissions or the pollution of water basins.

So how can we steer STI activities towards solving, rather than exacerbating, SDG challenges? Just doing more R&D will not contribute to achieving the SDGs. Depending on the directions of the associated STI, it can, in fact, undermine progress towards them. Determining how to invest in research and development for the SDGs is not a simple task. There is no single definitive perspective or STI direction for addressing any particular SDG. Each SDG challenge can be viewed differently, according to diverse and plural understandings, values, interests and STI priorities.

To help understand and better address the challenges of investing in STI for the SDGs, while embracing the complex relationship between STI and the SDGs, we carried out a major global study to determine how and to what extent the world’s STI priorities are aligned with the goals.

We analysed scientific publications and patents data to gather quantitative information about global research and innovation priorities, and how these align with SDG challenges. We conducted a global survey of stakeholders to explore views about what types of STI are needed in the future to help achieve the SDGs. This allowed us to consider the alignment between current and desired STI priorities. We interviewed local STI users, including fishers, farmers and researchers, to explore how different actors, each with their own priorities, are shaping local STI pathways to tackle specific sustainability challenges. We then appraised stakeholders’ views about how far each pathway aligns with sustainable development objectives. We produced data, mappings and case studies to gain a better understanding of STI priorities and to illustrate how such evidence and methods could be used in other contexts, according to plural interpretations of SDG challenges and STI pathways.

By combining these analyses, we gained deep insights into the way that particular STI priorities emerge both locally and globally, and how STI can be steered to improve alignment with the SDGs. Our results can help policymakers, research funders, academics, international organizations (INGOs) and aid organizations to make informed decisions about investing in research and innovation that will address the SDGs and ultimately create a positive impact on society.